Monitoring method and device based on deep learning

Abstract

A monitoring method and device based on deep learning, the method comprising: inputting two images captured by camera lenses into a tracking network; determining detection areas for each image; detecting whether a person enters / leaves the detection area in the two images and processing the detected areas; for an ID group containing multiple IDs, selecting one ID in the ID group as the person ID of the person corresponding to the ID group; when a person with an existing ID is detected to enter / leave the detection area again, assigning the person to the same ID and ID group; and counting the number of people appearing within a period of time. The present invention improves the accuracy of counting the number of people appearing in the camera image of a monitoring system based on a fisheye camera and deep learning. The implementation process does not involve redundant character feature extraction and comparison processes, which can significantly save software running time and storage space. Furthermore, the solution is insensitive to image distortion and is simple, easy to deploy, and easy to implement.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a monitoring method and device based on deep learning. Background Art

[0002] A fisheye lens is an extreme wide-angle lens with a focal length of 16mm or less and a viewing angle approaching or equal to 180°. To maximize the viewing angle, the front lens element of a fisheye lens is very short in diameter and bulges out in a parabolic shape, resembling a fish's eye, hence the name.

[0003] Fisheye camera is a camera with a fisheye lens. As mentioned above, its focal length is very short (16nm or less) and its viewing angle is close to or equal to 180°.

[0004] Deep learning, a type of machine learning, is an essential path to achieving artificial intelligence. Deep learning combines lower-level features to form more abstract higher-level representations to represent attribute categories or features, thereby discovering distributed feature representations of data.

[0005] Based on fisheye cameras and deep learning, it is possible to monitor people appearing in the camera image, for example, to count the flow of people.

[0006] However, in the existing technology, monitoring systems based on fisheye cameras and deep learning sometimes count the same person twice or even more times, which makes it impossible for the monitoring system to accurately count the number of people appearing in the camera image. Summary of the Invention

[0007] The technical problem solved by the present invention is how a monitoring system based on fisheye cameras and deep learning can accurately count the number of people appearing in the camera image.

[0008] To solve the above technical problems, an embodiment of the present invention provides a monitoring method based on deep learning, comprising:

[0009] Capturing images through a camera lens;

[0010] Input two images captured by the camera lens into the tracking network respectively;

[0011] Determine the detection area in the above two images respectively;

[0012] Detect whether a person enters / leaves the detection area in the above two images and process it;

[0013] For an ID group containing multiple IDs, one ID in the ID group is selected as the character ID of the character corresponding to the ID group;

[0014] When a person with an existing person ID is detected to enter / leave the detection area again, the person is assigned the same ID and ID group as before;

[0015] Count the number of characters that appear within a period of time.

[0016] Optionally, determining the detection areas in the two images respectively includes:

[0017] Determine the boundary area of ​​the above two images;

[0018] The boundary area is used as the detection area in the above two images.

[0019] Optionally, determining the boundary region between the two images includes: selecting the boundary region from the two images respectively in response to an operator's operation on an interactive interface.

[0020] Optionally, the method further includes: when it is detected that a person enters the detection area in the above two images, assigning an ID group to each person entering the detection area in the image.

[0021] Optionally, selecting an ID in the ID group as the character ID of the person corresponding to the ID group includes: selecting the smallest ID in the ID group as the character ID of the person corresponding to the ID group.

[0022] Optionally, the detection of whether a person enters / leaves the detection area in the above two images and the processing includes: when a person is detected entering the detection area in the previous image, recording the person ID and determining whether the ID belongs to any existing ID group; if so, it is included in the ID group; if not, a new ID group is created, and the ID belongs to the new ID group.

[0023] Optionally, the detecting whether a person enters / leaves the detection area in the above two images and processing includes: when it is detected that a person leaves the detection area in the previous image and enters the detection area in the latter image, the person ID in the latter image is counted into the ID group to which the person ID in the previous image belongs.

[0024] Optionally, the detecting whether a person enters / leaves the detection area in the two images and processing includes: when it is detected that a person leaves the detection area in the latter image, completing the generation of an ID group corresponding to the person.

[0025] Optionally, the method further includes: determining whether different ID groups contain any identical ID; and if so, merging the ID groups containing any identical ID.

[0026] Optionally, a smallest ID is selected from the merged ID group as the character ID of the character corresponding to the merged ID group.

[0027] Optionally, the camera lens is a fisheye camera lens.

[0028] In order to solve the above technical problems, an embodiment of the present invention further provides a monitoring device based on deep learning, comprising:

[0029] a processor adapted to load and execute instructions of a software program;

[0030] A memory adapted to store a software program comprising instructions for performing the following steps:

[0031] Capturing images through a camera lens;

[0032] Input two images captured by the camera lens into the tracking network respectively;

[0033] Determine the detection area in the above two images respectively;

[0034] Detect whether a person enters / leaves the detection area in the above two images and process it;

[0035] For an ID group containing multiple IDs, one ID in the ID group is selected as the character ID of the character corresponding to the ID group;

[0036] When a person with an existing person ID is detected to enter / leave the detection area again, the person is assigned the same ID and ID group as before;

[0037] Count the number of characters that appear within a period of time.

[0038] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0039] Two images captured by the camera lens are input into the tracking network respectively; the detection area is determined in the above two images respectively; whether a person enters / leaves the detection area in the above two images is detected and processed; for an ID group containing multiple IDs, one ID in the ID group is selected as the person ID of the person corresponding to the ID group; when a person with an existing person ID is detected to enter / leave the detection area again, the person is assigned to the same ID and ID group as before, and the number of people appearing in a period of time is counted, thereby improving the accuracy of counting the number of people appearing in the camera image of the monitoring system based on fisheye camera and deep learning.

[0040] Furthermore, there is no redundant character feature extraction and comparison process during the implementation process, which can significantly save software running time and storage space. At the same time, the solution is insensitive to image distortion. As long as the performance of the tracking network is good enough and the matching accuracy is high, good results can be achieved. It is simple, easy to deploy and implement, and is very suitable for applications where fisheye cameras are used as surveillance cameras. It is also suitable for multi-camera monitoring applications and other panoramic monitoring solutions.

[0041] Furthermore, a specific implementation method for detecting whether a person enters / leaves the detection area in the above two images and processing is disclosed, including: when a person is detected entering the detection area in the previous image, recording the person's ID and judging whether the ID belongs to any existing ID group; if so, it is counted into the ID group; if not, a new ID group is created, and the ID belongs to the new ID group; when a person is detected leaving the detection area in the previous image and entering the detection area in the next image, the person's ID in the next image is counted into the ID group to which the person's ID in the previous image belongs; when a person is detected leaving the detection area in the next image, the generation of the ID group corresponding to the person is completed. The method is simple and has a small amount of computation. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flow chart of a monitoring method based on deep learning in an embodiment of the present invention;

[0043] Figure 2 This is a flowchart of ID matching in an embodiment of the present invention;

[0044] Figure 3 Schematic diagram of fisheye lens data expansion in an embodiment of the present invention;

[0045] Figure 4 Schematic diagram of cross-lens between 2P images after fisheye lens data is expanded in an embodiment of the present invention;

[0046] Figure 5 Schematic diagram of cross-lens between the center image and the 2P image after fisheye lens data is expanded in an embodiment of the present invention;

[0047] Figure 6 This is a schematic diagram of ID merging in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] According to the analysis in the background technology section, based on fisheye cameras and deep learning, it is possible to monitor people appearing in the camera image, for example, to count the flow of people.

[0049] However, in the existing technology, monitoring systems based on fisheye cameras and deep learning sometimes count the same person twice or even more times, which makes it impossible for the monitoring system to accurately count the number of people appearing in the camera image.

[0050] After research, the inventor found that in the monitoring system based on fisheye camera and deep learning, the image data fed into the deep neural network needs to be expanded into a 2P graph (a 2P graph refers to, for example, Figure 2 As shown, for the image captured by the fisheye camera, the annular area excluding the central image is the 2P image of the fisheye camera. Dividing it into two equal parts is equivalent to dividing the annular area into two lens screen displays, and the two parts are respectively the upper 2P image and the lower 2P image), which is equivalent to the screen split-lens display, resulting in inconsistent IDs detected for the same person in different screens. This will lead to the above-mentioned defects in the prior art, that is, the monitoring system based on fisheye cameras and deep learning sometimes repeatedly counts the same person twice or even more times, which makes it impossible for the monitoring system to accurately count the number of detected people.

[0051] It can be seen that how a monitoring system based on fisheye cameras and deep learning can accurately count the number of people appearing in the camera image is an urgent problem that needs to be solved in this field.

[0052] Specifically, in the prior art, (refer to 201910826602.5), real-time video images from multiple cameras are first obtained, and multiple video images are output and displayed; then, a mask map is drawn for each of the real-time video images, and the video analysis area and the common area of ​​adjacent cameras are configured; then, based on the configured common area of ​​adjacent cameras, feature point matching is performed on the common area of ​​adjacent cameras, and the homography matrix of the adjacent cameras is calculated; then, through the interactive interface, a specific human target in the video image is selected, the ROI area of ​​the corresponding image is obtained, and a deep learning image algorithm is used to classify all ROI areas, and the ROI area with the highest score is obtained as the human ROI area; finally, the above-mentioned human ROI area is input into the tracking network (also known as the tracker) for cross-border head tracking, thereby realizing ID matching for cross-shot tracking.

[0053] The above-mentioned existing technical solution utilizes feature point matching and homography matrix, which is relatively complex and combined with deep learning methods, and there is no guarantee of processing speed. At the same time, this method is not accurate and is too dependent on the degree of camera distortion. If the distortion is too large, the accuracy will be significantly reduced.

[0054] As we all know, the extremely short focal length of fisheye cameras, while providing a wider viewing angle, also comes with certain drawbacks, such as barrel distortion. Barrel distortion refers to the lens-induced distortion that causes the image to appear barrel-shaped and bulging. Due to their extremely short focal length, fisheye cameras exhibit significant barrel distortion.

[0055] Therefore, the above-mentioned existing technical solutions are not applicable to the monitoring system based on fisheye cameras. In addition, edge engineering deployment will also have certain real-time obstacles.

[0056] In another existing technical solution, (reference 201910969309.4), a method for tracking suspicious persons across shots based on spatial constraints is used, which uses a deep learning algorithm to detect human targets in the video in real time. The provided interactive interface is used to select the selected suspicious person target of the user's interest, and single surveillance camera tracking of the selected suspicious person and cross-shot tracking based on the extraction of human features based on the deep learning algorithm as the basis for comparison are achieved. Based on the position information and motion information of the selected suspicious person target, global optimization is achieved within the local scene, which improves the efficiency of human retrieval and re-identification to a certain extent. However, in the implementation process of this existing technical solution, the training of the human feature extraction network and the consumption of feature storage are required, which is relatively complicated, relatively difficult to deploy at the edge, and has limitations on recognition speed.

[0057] Based on the above analysis, the inventors believe that existing monitoring solutions based on fisheye cameras and deep learning have at least the following two serious flaws:

[0058] 1) The integration of multiple methods makes the implementation process complex;

[0059] 2) Real-time performance cannot be guaranteed during actual use, and real-time performance will be greatly sacrificed to improve accuracy based on the deep learning model.

[0060] After further research, the inventor believes that the fisheye camera lens data expansion (such as Figure 1 The cross-border header after (as shown) includes at least the following two categories: (as shown) Figure 2 As shown) One is the cross-border header between 2P graphs; (as shown Figure 3 The second is the cross-border header between the central graph and the 2P graph. Specifically:

[0061] Regarding the cross-lens situation between 2P images (i.e., the first type of situation mentioned above), we believe that the cross-lens problem is that the deep learning-based tracking network does not associate different images when processing them. Therefore, the same person will be assigned an ID as a different person when appearing in different lens images, such as Figure 2As shown, the person on the 2P map originally has ID 1, and after leaving the 2P map and entering the 2P map, the ID becomes 2. However, this is actually the same person and should have the same ID. Secondly, under different lenses, a person's movement trajectory should be leaving from one lens and entering another lens, so there is an intersection area here, which is the "detection area" in the solution of the present invention, for example Figure 2 The area to the right of the black line in the middle 2P figure and the area to the left of the black line in the lower 2P figure.

[0062] For the cross-border header between the 2P graph and the central graph (i.e., the second type of case mentioned above), similarly, since the graphs are fed into the tracking network separately, the system considers the same person as a new person in each graph and assigns a different ID to him / her. For example, Figure 3 In the diagram, a person’s ID is 1 in the 2P diagram. After leaving the diagram and entering the center diagram, it becomes 2. Then, after entering the 2P diagram, it becomes 3. However, this is the same person and should have the same ID.

[0063] In the solution of the present invention, the character IDs that appear successively within a period are fixed to avoid counting the same character twice or even more times. Specifically:

[0064] The present invention inputs two images captured by camera lenses into a tracking network respectively; determines a detection area in the two images; detects whether a person enters or leaves the detection area in the two images and processes the detected area; for an ID group containing multiple IDs, selects one ID in the ID group as the person ID of the person corresponding to the ID group; when a person with an existing person ID is detected to enter or leave the detection area again, the person is assigned to the same ID and ID group as before, thereby improving the accuracy of counting the number of people appearing in the camera image by a monitoring system based on a fisheye camera and deep learning.

[0065] In order to enable those skilled in the art to better understand and implement the present invention, the present invention is described in detail below through specific embodiments with reference to the accompanying drawings.

[0066] Example 1

[0067] As described below, an embodiment of the present invention provides a monitoring method based on deep learning.

[0068] Reference Figure 1 The flowchart of the monitoring method based on deep learning is shown (also refer to Figure 2 The following is a detailed description of the specific steps:

[0069] S101, collecting images through a camera lens.

[0070] Wherein, the camera lens is a fisheye camera lens.

[0071] Regarding the defects of the existing technology in the error in the number of people counted, the inventor makes the following analysis: Figure 3 、 Figure 4 and Figure 5 , fisheye lens data expansion ( Figure 3 ) After that, cross-shot is divided into two cases: cross-shot between 2P pictures ( Figure 4 ); Cross-shot between the central image and the 2P image ( Figure 5 ).

[0072] For the cross-lens situation between 2P images (such as Figure 4 As shown in Figure 1): First, the cross-lens problem is that the deep learning-based tracker does not associate different images when processing them. Therefore, the same person will be assigned an ID as a different person when appearing in different lens images, such as Figure 4 As shown, the person on the 2P map originally has ID 1, and after leaving the 2P map and entering the 2P map, the ID becomes 2. However, this is actually the same person and should have the same ID. Secondly, the movement trajectory of a person in different shots should be leaving from one shot and entering another shot, so there is a boundary area here, which is the "detection area" in this embodiment. For example Figure 4 The area to the right of the black line in the middle 2P figure and the area to the left of the black line in the lower 2P figure.

[0073] For cross-shots between the 2P image and the central image (such as Figure 5 Similarly, because the images are fed into the tracker separately, the same person will be a new person in each image and will be assigned a different ID, e.g. Figure 5 In the diagram, a person’s ID is 1 in the 2P diagram. After leaving the diagram and entering the central diagram, it becomes 2. Then, after entering the 2P diagram, it becomes 3. However, this is the same person and should have the same ID.

[0074] The inventors overcome the above-mentioned defects in the prior art by the following methods: selecting the boundary area through an interactive interface or other methods; then assigning an ID group to each person who newly enters the boundary area, and dividing two (or more, due to the performance issues of the tracker, different detection IDs may appear for the same person in the same area) different IDs of the person into the same group based on the order in the time dimension, that is, a person who disappears at the end of an image must appear at the beginning of the next image. For example Figure 5 , there are three IDs in total, and these three should be one person, so they are divided into a group [1, 2, 3]. Finally, the ID with the smallest ID - 1 (the first ID to appear) is selected as the ID of this person, thus achieving the fixation of a person's ID across shots.

[0075] S102: Input two images captured by the camera lens into the tracking network respectively.

[0076] S103: Determine detection areas in the two images respectively.

[0077] Specifically, in some embodiments, determining the detection areas in the two images respectively includes:

[0078] Determine the boundary area of ​​the above two images;

[0079] The boundary area is used as the detection area in the above two images.

[0080] Furthermore, in some embodiments, determining the boundary region of the two images includes: selecting the boundary region from the two images respectively in response to an operator's operation on the interactive interface,

[0081] S104: Detect whether a person enters or leaves the detection area in the two images and process the result.

[0082] Specifically, in some embodiments, the detection of whether a person enters / leaves the detection area in the above two images and the processing include: when a person is detected entering the detection area in the previous image, recording the person's ID and determining whether the ID belongs to any existing ID group; if so, it is included in the ID group; if not, a new ID group is created, and the ID belongs to the new ID group.

[0083] In some embodiments, the detecting whether a person enters / leaves the detection area in the above two images and processing includes: when it is detected that a person leaves the detection area in the previous image and enters the detection area in the latter image, the person ID in the latter image is counted into the ID group to which the person ID in the previous image belongs.

[0084] In some embodiments, the process of detecting whether a person enters / leaves the detection area in the two images and processing the process includes: when a person is detected to have left the detection area in the latter image, generating an ID group corresponding to the person

[0085] For an ID group containing multiple IDs, one ID in the ID group is selected as the person ID of the person corresponding to the ID group;

[0086] When a person with an existing person ID is detected to enter / leave the detection area again, the person is assigned the same ID and ID group as before.

[0087] Furthermore, in some embodiments, selecting an ID in the ID group as the character ID of the person corresponding to the ID group includes: selecting the smallest ID in the ID group as the character ID of the person corresponding to the ID group.

[0088] In some embodiments, the method further includes: when it is detected that a person enters the detection area in the two images, assigning an ID group to each person entering the detection area in the image.

[0089] In some embodiments, the method further includes: determining whether different ID groups contain any identical ID; if so, merging the ID groups containing any identical ID.

[0090] Furthermore, in some embodiments, the smallest ID is selected from the merged ID group as the character ID of the character corresponding to the merged ID group.

[0091] S105, counting the number of characters that appear within a period of time.

[0092] From the above description of the technical solution, it can be seen that: in this embodiment, two images captured by the camera lens are respectively input into the tracking network; the detection area is respectively determined in the above two images; it is detected whether a person enters / leaves the detection area in the above two images and processed; for an ID group containing multiple IDs, one ID in the ID group is selected as the person ID of the person corresponding to the ID group; when a person with an existing person ID is detected to enter / leave the detection area again, the person is matched with the same ID and ID group as before, and the number of people appearing within a period of time is counted, thereby improving the accuracy of counting the number of people appearing in the camera screen of the monitoring system based on fisheye cameras and deep learning.

[0093] Furthermore, there is no redundant character feature extraction and comparison process during the implementation process, which can significantly save software running time and storage space. At the same time, the solution is insensitive to image distortion. As long as the performance of the tracking network is good enough and the matching accuracy is high, good results can be achieved. It is simple, easy to deploy and implement, and is very suitable for applications where fisheye cameras are used as surveillance cameras. It is also suitable for multi-camera monitoring applications and other panoramic monitoring solutions.

[0094] Furthermore, a specific implementation method for detecting whether a person enters / leaves the detection area in the above two images and processing is disclosed, including: when a person is detected entering the detection area in the previous image, recording the person's ID and judging whether the ID belongs to any existing ID group; if so, it is counted into the ID group; if not, a new ID group is created, and the ID belongs to the new ID group; when a person is detected leaving the detection area in the previous image and entering the detection area in the next image, the person's ID in the next image is counted into the ID group to which the person's ID in the previous image belongs; when a person is detected leaving the detection area in the next image, the generation of the ID group corresponding to the person is completed. The method is simple and has a small amount of computation.

[0095] That is to say, if Figure 2 As shown, this embodiment avoids the defect of incorrect character number counting in the prior art by:

[0096] 1) The two images captured by the fisheye camera are sent to the tracking network (tracker) respectively;

[0097] 2) Divide the detection area, that is, the intersection area of ​​the two images;

[0098] 3) When a person enters the detection area of ​​the first picture, record their ID. If they exist in the existing ID group, they are included in the ID group. If they do not exist in the ID group, it means they are different people, and a new ID group is created.

[0099] 4) When a person leaves the first image and enters the detection area of ​​the second image, their ID is counted into the ID group to which the ID in the first image belongs;

[0100] 5) When a person leaves the detection area of ​​the second image, a set of IDs is generated;

[0101] 6) When the same person returns to the detection area, the ID will not change and will be assigned to the previous ID group. Finally, one group of IDs represents one person.

[0102] 7) Repeat the above 5 steps for different people.

[0103] Among them, the key parts of the specific implementation are: the logic of ID grouping and the merging of duplicate IDs. In the implementation process, all IDs of a person from entering to leaving the detection area are grouped together; when the same person goes back and forth between the detection areas multiple times, different ID groups will be generated, but when entering the detection area, there may be a short jump, and then it will change back. At this time, because the IDs entering the area are different, a new ID group will be generated, and the existence of this ID group is unnecessary, so the existing ID groups are merged to filter out the redundant ID groups. The judgment condition is that if there is any identical ID in the ID group, it will be merged, because as long as there is an identical ID, it means it is the same person. For example Figure 6 In the example, a person starts from the bottom of the 2P graph, passes through the center graph, and arrives at the top of the 2P graph, forming an ID group [1, 2, 3]. Then, they leave the detection area on the top of the 2P graph and enter the detection area on the bottom of the 2P graph, generating a new ID group [3, 4]. In fact, these belong to the same person, so the two should be merged into a new ID group [1, 2, 3, 4], and 1 should be selected as the person's ID.

[0104] Example 2

[0105] As described below, an embodiment of the present invention provides a monitoring device based on deep learning.

[0106] The deep learning-based monitoring device includes:

[0107] a processor adapted to load and execute instructions of a software program;

[0108] A memory adapted to store a software program comprising instructions for performing the following steps:

[0109] Capturing images through a camera lens;

[0110] Input two images captured by the camera lens into the tracking network respectively;

[0111] Determine the detection area in the above two images respectively;

[0112] Detect whether a person enters / leaves the detection area in the above two images and process it;

[0113] For an ID group containing multiple IDs, one ID in the ID group is selected as the character ID of the character corresponding to the ID group;

[0114] When a person with an existing person ID is detected to enter / leave the detection area again, the person is assigned the same ID and ID group as before;

[0115] Count the number of characters that appear within a period of time.

[0116] From the above description of the technical solution, it can be seen that: in this embodiment, two images captured by the camera lens are respectively input into the tracking network; the detection area is respectively determined in the above two images; it is detected whether a person enters / leaves the detection area in the above two images and processed; for an ID group containing multiple IDs, one ID in the ID group is selected as the person ID of the person corresponding to the ID group; when a person with an existing person ID is detected to enter / leave the detection area again, the person is matched with the same ID and ID group as before, and the number of people appearing within a period of time is counted, thereby improving the accuracy of counting the number of people appearing in the camera screen of the monitoring system based on fisheye cameras and deep learning.

[0117] Those skilled in the art will understand that in the various methods of the above embodiments, all or part of the steps can be completed by hardware related to program instructions, and the program can be stored in a computer-readable storage medium, which may include: ROM, RAM, disk or CD, etc.

[0118] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

Claims

1. A monitoring method based on deep learning, characterized in that: include: Capture images through a fisheye camera lens; Expanding the image into a 2P image, wherein the 2P image includes two images; Input the above two images into the tracking network respectively; Determining detection areas in the two images respectively, including: determining a boundary area between the two images, and using the boundary area as the detection area in the two images; Detecting whether a person enters / leaves the detection area in the two images and processing the information, including: when a person is detected to have left the detection area in the first image and entered the detection area in the second image, adding the person ID in the second image to the ID group to which the person ID in the first image belongs; when a person is detected to have left the detection area in the second image, completing the generation of the ID group corresponding to the person; For an ID group containing multiple IDs, one ID in the ID group is selected as the character ID of the character corresponding to the ID group; When a person with an existing person ID is detected to enter / leave the detection area again, the person is assigned the same ID and ID group as before; Count the number of characters that appear within a period of time.

2. The deep learning-based monitoring method according to claim 1, wherein: Determining the boundary region between the two images includes: selecting the boundary region from the two images respectively in response to an operator's operation on an interactive interface.

3. The deep learning-based monitoring method according to claim 1, wherein: Also includes: When it is detected that a person has entered the detection area in the two images, an ID group is assigned to each person who has entered the detection area in the image.

4. The deep learning-based monitoring method according to claim 1, wherein: The selecting one ID in the ID group as the character ID of the person corresponding to the ID group includes: selecting the smallest ID in the ID group as the character ID of the person corresponding to the ID group.

5. The deep learning-based monitoring method according to claim 1, wherein: The detecting whether a person enters / leaves the detection area in the above two images and processing includes: when a person is detected to enter the detection area in the previous image, recording the person's ID and determining whether the ID belongs to any existing ID group; if so, counting it into the ID group; if not, creating a new ID group, and the ID belonging to the new ID group.

6. The deep learning-based monitoring method according to claim 1, wherein: Also includes: Determine whether different ID groups contain any identical IDs; If so, the ID groups containing any identical IDs are merged.

7. The deep learning-based monitoring method according to claim 6, wherein: The smallest ID is selected from the merged ID group as the person ID of the person corresponding to the merged ID group.

8. A monitoring device based on deep learning, characterized in that: include: a processor adapted to load and execute instructions of a software program; A memory adapted to store a software program comprising instructions for performing the following steps: Capture images through a fisheye camera lens; Expanding the image into a 2P image, wherein the 2P image includes two images; Input the above two images into the tracking network respectively; Determining detection areas in the two images respectively, including: determining a boundary area between the two images, and using the boundary area as the detection area in the two images; Detecting whether a person enters / leaves the detection area in the two images and processing the information, including: when a person is detected to have left the detection area in the first image and entered the detection area in the second image, adding the person ID in the second image to the ID group to which the person ID in the first image belongs; when a person is detected to have left the detection area in the second image, completing the generation of the ID group corresponding to the person; For an ID group containing multiple IDs, one ID in the ID group is selected as the character ID of the character corresponding to the ID group; When a person with an existing person ID is detected to enter / leave the detection area again, the person is assigned the same ID and ID group as before; Count the number of characters that appear within a period of time.

Images