Intelligent monitoring method and system based on face recognition, medium and program product

Abstract

The invention discloses an intelligent monitoring method and system based on face recognition, a medium and a program product, and the method comprises the steps: determining a dynamic coordinate of a target person; determining the advancing direction of the target person according to the dynamic coordinates; when it is determined that the target person leaves the first monitoring area, determining a leaving frame; judging whether the leaving frame contains a preset exit or not; if the preset exit is not included, determining a second monitoring picture of the second monitoring area according to the leaving frame; determining an entering frame in the second monitoring picture according to the advancing direction; detecting face data entering the second monitoring area from the entering frame in real time; executing the step of determining the dynamic coordinates of the target person under the condition that the target person is determined to be detected according to the face data; and if the preset exit is included, ending monitoring of the target person. The method and the device are used for continuously monitoring target personnel and improving the integrity of the monitored target personnel, so that the safety of a monitored area is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent monitoring, and in particular to an intelligent monitoring method, system, medium and program product based on face recognition. Background Art

[0002] With the acceleration of urbanization and the improvement of public safety awareness, the demand for effective security monitoring systems is increasing. Face recognition can quickly identify individuals and improve the response speed and accuracy of security systems. The rapid development of computer vision and machine learning, especially the breakthrough of deep learning technology, has greatly promoted the accuracy and application scope of face recognition technology.

[0003] In the related technology, a real-time video stream captured by a surveillance camera is obtained; the real-time video stream is parsed to obtain the characteristic values ​​of the face; the face is tracked based on the characteristic values ​​of the face, and the face image of the tracked face is continuously captured; when the tracked face enters a preset electronic fence area, the best face image is selected for face recognition to obtain face identity information; authority authentication and identification are performed based on the face identity information, and different markings are performed on the image; when a human body is detected entering the controlled area, the personnel access authority determination is triggered.

[0004] However, the execution device of the above technical solution is only a monitoring device. When a person leaves the current monitoring screen, it is difficult to continue to monitor the whereabouts of the blacklisted person or the graylisted person, which reduces the integrity of the monitoring and further reduces the security of the monitoring area. Summary of the invention

[0005] The present application provides an intelligent monitoring method, system, medium and program product based on face recognition, which are used to continuously monitor target personnel, improve the integrity of monitored target personnel, and thus improve the security of the monitored area.

[0006] In a first aspect, the present application provides an intelligent monitoring method based on face recognition, which determines the dynamic coordinates of the target person when it is determined that the target person appears in a first monitoring area; Determine the target person's direction of travel based on dynamic coordinates; When it is determined that the target person leaves the first monitoring area, determining a leaving frame, where the leaving frame is a frame that overlaps with the first monitoring screen when the target person leaves the first monitoring area; Determine whether the exit frame contains a preset exit; If the preset exit is not included, determining the second monitoring screen of the second monitoring area according to leaving the frame; Determine the entry frame in the second monitoring screen according to the direction of travel; Real-time detection of face data entering the second monitoring area from the border; In the case where it is determined that a target person is detected according to the face data, a step of determining the dynamic coordinates of the target person is performed; If the preset exit is included, the monitoring of the target person ends.

[0007] By adopting the above technical solution, when it is determined that the target person leaves the first monitoring area, it is determined whether the exit frame contains a preset exit, so that the target person's moving trajectory can be continuously tracked. If the exit frame does not contain an exit, it means that the target person is likely to have entered the adjacent second monitoring area. At this time, the system can quickly lock the entry frame of the second area according to the exit frame of the first area and the direction of travel of the target person, and detect the face in real time here until the target person is recaptured. Therefore, once the target appears in any monitoring area, the system can always grasp its dynamic position through relay cross-region tracking, and achieve real-time monitoring without dead angles and full coverage. The spatial scale of target tracking is greatly expanded. At the same time, if the target person leaves from the preset exit, the system can terminate the tracking in time to avoid unnecessary waste of resources. Continuous monitoring of target personnel is achieved, the integrity of monitoring target personnel is improved, and the security of the monitoring area is thereby improved.

[0008] In combination with some embodiments of the first aspect, in some embodiments, when it is determined that a target person appears in the first monitoring area, determining the dynamic coordinates of the target person specifically includes: Receiving a first monitoring video stream corresponding to a first monitoring area in real time; capturing facial data in a first surveillance video stream using a face detection algorithm; Extracting facial features from facial data and matching the facial features in a preset facial database; When it is determined that the matched facial data is the target person, the target facial data of the target person is annotated using the facial tracking technology; The dynamic coordinates of the target person are determined using the annotation box based on the target face data.

[0009] By adopting the above technical solution, the face detection algorithm is used to capture faces in the video stream, and key features are extracted to match with the preset face library. The system can quickly determine whether the target person appears in the monitoring screen. Once the match is confirmed, the face tracking technology can be used to lock the target and mark its dynamic position in real time. Since facial features are unique and stable, using them as the basis for target identification can greatly improve the pertinence and accuracy of monitoring and effectively avoid various interferences in complex environments. At the same time, the establishment of a preset face library also provides the system with samples of target persons for comparison, making it possible to achieve directional monitoring. In addition, the ability of face tracking technology to mark positions in real time lays the foundation for the subsequent generation of dynamic coordinates.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, determining the dynamic coordinates of the target person using a labeling box according to the target face data specifically includes: Determine a monitoring screen according to the first monitoring video stream; A temporary coordinate system is established in the monitoring screen, the origin of the temporary coordinate system is the lower left vertex of the monitoring screen, the horizontal coordinate of the temporary coordinate system is the lower border, and the vertical coordinate is the left border; Mark the center point of the annotation box in the temporary coordinate system to obtain the dynamic coordinates of the target person.

[0011] By adopting the above technical solution, on the basis of determining the position of the target person, the system does not directly use physical parameters such as pixel coordinates, but establishes a virtual logical coordinate system in the monitoring screen. The coordinate system takes the lower left corner of the screen as the origin, and the lower frame and left frame as the horizontal and vertical coordinate axes, respectively, to form a relative reference system independent of the actual physical size. The system corresponds the geometric center of the face annotation frame to the coordinate system, thereby generating dynamic coordinates that can reflect the relative position of the target person in real time. This novel parameter representation method has many unique advantages. First, the logical coordinate avoids the difference in imaging scale between different cameras, has universality and portability, and is conducive to the integration and sharing of monitoring information. Secondly, with the frame as the coordinate axis, a closed relative reference system independent of the outside world is formed, which simplifies the calculation and use of coordinates. In addition, the logical properties of the reference system are also consistent with the image processing algorithm, which is conducive to the digital realization of the software system. Finally, the dynamic coordinates clearly reflect the position change of the target person relative to the boundary of the monitoring area, which is convenient for judging the entry and exit status of the target.

[0012] In combination with some embodiments of the first aspect, in some embodiments, the moving direction of the target person is determined according to the dynamic coordinates, and the method further includes: Perform a smoothing operation on the dynamic coordinates to obtain smoothed dynamic coordinates; Calculate the target person's moving vector based on the smoothed dynamic coordinates; The direction of movement of the target person is calculated using an angle calculation algorithm based on the movement vector.

[0013] By adopting the above technical solution, on the basis of generating the dynamic coordinates of the target person, the system first smoothes the coordinate sequence to eliminate the instantaneous noise introduced by factors such as face detection error or camera jitter, and improve the stability of the data. Then, by calculating the moving vector, the system can accurately capture the displacement trend of the target person. Finally, by using the angle calculation algorithm, the displacement vector is mapped to the logical coordinate system to obtain the real-time moving direction of the target person. Compared with the traditional mobile target detection solution, the advantage of this solution is that it makes full use of the rich information of the previous face tracking and dynamic coordinate representation, and on this basis, through mathematical methods such as smoothing, vector calculation, and angle estimation, it finally realizes the accurate determination of the moving direction. At the same time, unifying the representation of the moving direction into the logical coordinate system also makes cross-region target tracking possible. Once the target person leaves the current monitoring area, the system can directly predict the next possible area based on its last moving direction, and focus on the border in this direction, so as to find the target in the first time and realize seamless relay monitoring.

[0014] In combination with some embodiments of the first aspect, in some embodiments, determining the entry frame in the second monitoring screen according to the traveling direction specifically includes: Determine the direction of travel according to the direction of travel; Determine a first intersecting frame in the second monitoring picture that intersects with the traveling direction, the first intersecting frame being the first frame where the traveling direction intersects with the second monitoring picture; The first intersecting bounding box is determined as the entering bounding box.

[0015] By adopting the above technical solution, when the target person leaves the current area, the system first determines the next area he may enter based on his final direction of travel. By calculating the intersection of the orientation and the border, the target person's contact point in the next area can be inferred, and this point is determined as the entry border. This prediction mechanism based on the movement trend can greatly narrow the search range of the next area. At the same time, since the determination of the border depends entirely on the logical orientation relationship and does not require the actual geographical location, it is suitable for camera networks of any topological structure. In addition, since the position of the entry border has been accurately inferred, the face retrieval of the next area will be concentrated in a highly locked border area, thereby significantly improving the efficiency and success rate of target matching and recognition.

[0016] In combination with some embodiments of the first aspect, in some embodiments, after the monitoring of the target person ends if the preset exit is included, the method further includes: When it is determined that the target person leaves the first monitoring area, the timer starts; When it is determined that the target person has entered the second monitoring area, the timing is ended to obtain the monitoring vacuum time; Determine whether the monitoring vacuum time is greater than the preset time; If not, the step of determining the dynamic coordinates of the target person is executed; If it is greater, a warning message is sent to the monitoring terminal, and the warning message includes the location information of the target person.

[0017] By adopting the above technical solution, when the target person leaves the current area, the system starts timing until the target is recaptured in the next area. The time difference thus obtained is the monitoring vacuum period. If the duration exceeds the preset threshold, it means that the target person disappears abnormally during the movement between adjacent areas, and abnormal behavior or safety hazards are very likely to occur. At this time, the system will immediately send a warning message to the monitoring terminal, and attach the latest location of the target person for security personnel to check and deal with. In this way, the system can not only detect the abnormal movements of the target person in time, but also accurately locate the target position in the first time. In addition, since the sending of warning information is strictly based on the objective indicator of the monitoring vacuum duration, it can avoid interference from human factors to the greatest extent and improve the accuracy and reliability of the early warning.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, when it is determined that the target person leaves the first monitoring area, after starting the timing, the method further includes: When it is determined that the timing time is greater than the preset time, determining whether the target person has entered the second monitoring area; If the user does not enter the second monitoring area, then retrieve several adjacent monitoring videos of several monitoring areas adjacent to the first monitoring area; Determine whether there is a target person in several adjacent surveillance videos; If so, the step of determining the dynamic coordinates of the target person is performed; If not, a danger message will be sent to the monitoring terminal, and the import and export controllers will be controlled to close the import and export; If the second monitoring area is entered, a step of sending a warning message to the monitoring terminal is executed.

[0019] By adopting the above technical solution, once the monitoring vacuum times out, the system first determines whether the target person has entered the next area originally planned. If the answer is no, it means that the target is likely to be staying inside a blind spot. At this time, the system will intelligently retrieve all surveillance videos adjacent to the current area, and use the face recognition algorithm to search for the target person in these videos. Once the target is found, the system immediately locks its position and continues to track it, thereby minimizing the scope of the blind spot and achieving no-dead-angle monitoring. However, if the target person is never found, the system will further increase the warning level, not only sending danger information to the monitoring terminal, but also directly controlling all entrances and exits to prohibit passage. This closed-control emergency response can effectively control the target person's range of activities in the first time and minimize safety hazards.

[0020] In a second aspect, an embodiment of the present application provides an intelligent monitoring system based on face recognition, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and one or more processors call the computer instructions to enable x to execute the method described in the first aspect and any possible implementation method of the first aspect.

[0021] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a system, causes the system to execute the method described in the first aspect and any possible implementation of the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer program product, characterized in that when the computer program product runs on a system, the system executes the method described in any possible implementation manner in the first aspect.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The present application provides an intelligent monitoring method based on face recognition. When determining that the target person leaves the first monitoring area, it is determined whether the exit frame contains a preset exit, so as to realize continuous tracking of the target person's moving trajectory. If the exit frame does not contain an exit, it means that the target person is likely to have entered the adjacent second monitoring area. At this time, the system can quickly lock the entry frame of the second area according to the exit frame of the first area and the direction of travel of the target person, and detect the face in real time here until the target person is recaptured. Therefore, once the target appears in any monitoring area, the system can always grasp its dynamic position through relay cross-region tracking, and achieve real-time monitoring without dead angles and full coverage. The spatial scale of target tracking is greatly expanded. At the same time, if the target person leaves from the preset exit, the system can terminate the tracking in time to avoid unnecessary waste of resources. Continuous monitoring of target personnel is achieved, the integrity of monitoring target personnel is improved, and the security of the monitoring area is thereby improved.

[0024] 2. The present application provides an intelligent monitoring method based on face recognition. When the target person leaves the current area, the system first determines the next area he may enter based on his final direction of travel. By calculating the intersection of the orientation and the border, the target person's contact point in the next area can be inferred, and this point is determined as the entry border. This prediction mechanism based on the movement trend can greatly narrow the search range of the next area. At the same time, since the determination of the border depends entirely on the logical orientation relationship and does not require the actual geographical location, it is suitable for camera networks of arbitrary topology. In addition, since the position of the entry border has been accurately inferred, the face retrieval in the next area will be concentrated in a highly locked border area, thereby significantly improving the efficiency and success rate of target matching and recognition.

[0025] 3. The present application provides an intelligent monitoring method based on face recognition. Once the monitoring vacuum times out, the system first determines whether the target person has entered the original next area. If the answer is no, it means that the target is likely to be staying inside a blind spot. At this time, the system will intelligently retrieve all surveillance videos adjacent to the current area, and use the face recognition algorithm to search for the target person in these videos. Once the target is found, the system immediately locks its position and continues to track, thereby minimizing the scope of the blind spot and achieving no-dead-angle monitoring. However, if the target person is never found, the system will further increase the warning level, not only sending danger information to the monitoring terminal, but also directly controlling all entrances and exits to prohibit passage. This closed-control emergency response can effectively control the target person's range of activities in the first time and minimize safety hazards. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1It is a flow chart of an intelligent monitoring method based on face recognition in an embodiment of the present application.

[0027] Figure 2 This is another flow chart of an intelligent monitoring method based on face recognition in an embodiment of the present application.

[0028] Figure 3 It is a schematic diagram of the physical device structure of an intelligent monitoring system based on face recognition provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more listed items.

[0030] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0031] An application scenario of an embodiment of the present application is described below: In a bustling commercial district, high-rise buildings stand side by side and people are bustling about. In order to ensure public safety in the area, advanced facial recognition monitoring systems are installed at major intersections and key areas. The system uses high-definition cameras to capture pedestrian facial features in real time and compares them with a preset facial database to quickly identify potential safety hazards.

[0032] One day, a red warning light suddenly lit up on the large screen of the monitoring center. It turned out that the system had identified a target person at a busy intersection. The staff on duty quickly contacted a nearby patrol car and transmitted the target person's high-definition facial image and location information to the staff's handheld terminal. While the patrol car rushed to the scene, the monitoring center was also closely monitoring the target person's movements.

[0033] However, what the staff did not expect was that the target person seemed to notice something unusual, suddenly quickened his pace, and turned into a narrow alley. Due to the limited coverage of the camera, the target person quickly disappeared from the surveillance screen. In desperation, the staff on duty tried to call other surveillance equipment near the alley, hoping to re-locate the target person's location, but due to the lack of effective information sharing and linkage mechanism between various surveillance systems, this effort did not achieve the expected effect.

[0034] At the same time, the target person had taken advantage of the chaos to escape into a nearby large shopping mall. Due to the dense flow of people and complex environment in the mall, it was difficult for the staff to determine the specific whereabouts of the target person based on the surveillance footage of a certain entrance and exit. The surveillance system inside the mall, which belonged to a different management department, could not connect with the staff's system for real-time data, allowing the target person to move freely in the blank area of ​​the surveillance.

[0035] After nearly an hour of searching, the staff finally found the target person in a remote corner of the mall. However, this incident also exposed many deficiencies in the current monitoring system: although the intelligence level of a single monitoring point is high, due to the lack of unified coordination and data sharing, it is difficult to achieve continuous tracking and effective control of the target person, which poses a hidden danger to public safety. In order to solve the above technical problems, this application provides an intelligent monitoring method based on face recognition, which is used to continuously monitor the target person, improve the integrity of the monitored target person, and thus improve the safety of the monitored area.

[0036] Combine the following Figure 1 , an intelligent monitoring method based on face recognition in an embodiment of the present application is described: See also Figure 1 , which is a flow chart of an intelligent monitoring method based on face recognition in an embodiment of the present application.

[0037] S101, when it is determined that a target person appears in the first monitoring area, determining the dynamic coordinates of the target person; When the system determines that the target person appears in the first monitoring area, it determines the dynamic coordinates of the target person. The specific implementation method of this step will be described below and will not be repeated here.

[0038] S102, determining the moving direction of the target person according to the dynamic coordinates; The system determines the moving direction of the target person based on the dynamic coordinates, and then smoothes the dynamic coordinates to obtain smoothed dynamic coordinates; Calculate the target person's moving vector based on the smoothed dynamic coordinates; The direction of movement of the target person is calculated using an angle calculation algorithm based on the movement vector.

[0039] In this step, the system first smoothes the dynamic coordinates of the target person. Smoothing refers to correcting and optimizing the original coordinate data through a certain mathematical algorithm, thereby eliminating coordinate jitter or jump caused by measurement errors, random noise and other factors. Common smoothing algorithms include moving average method, Kalman filter, etc. For example, suppose that the horizontal coordinates of a person in 5 consecutive frames are 100, 98, 102, 101, 99, and the vertical coordinates are 200, 202, 198, 201, 203. Through smoothing, these coordinates may be adjusted to (100, 200), (99, 201), (100, 200), (101, 201), (100, 202), so that the movement trajectory of the person is more stable and coherent.

[0040] After obtaining the smoothed dynamic coordinates, the system will further calculate the character's movement vector. The movement vector describes the displacement direction and distance of the character in a unit of time, and it can be obtained by the difference between the character's coordinates in the previous and next frames. For example, if the character's coordinates in the t-1 frame and the t frame are (100, 200) and (105, 198) respectively, then its movement vector is (5, -2), indicating that the character has moved 5 pixels in the horizontal direction and -2 pixels in the vertical direction.

[0041] Finally, the system uses the motion vector to calculate the real-time direction of the person through trigonometric functions. Specifically, the horizontal and vertical components of the motion vector can be regarded as the two right-angled sides of a right triangle, and then the inverse tangent function is used to find the angle between the vector and the horizontal direction to determine the direction of travel. For example, for the above-mentioned motion vector (5, -2), the angle between it and the horizontal direction is arctan (-2 / 5) ≈ -21.8°, from which it can be judged that the person is moving to the right front. By continuously updating the motion vector and the angle, the system can grasp the direction of travel of the target person in real time, providing a key basis for subsequent cross-regional tracking.

[0042] S103, when it is determined that the target person leaves the first monitoring area, determining to leave the frame; When it is determined that the target person leaves the first monitoring area, the system determines a leaving frame, where the leaving frame is a frame that overlaps with the first monitoring screen when the target person leaves the first monitoring area.

[0043] When the system detects that the target person has left the first monitoring area, it will automatically determine the specific location where the target person has left the area, that is, the exit frame. The so-called exit frame refers to the boundary line where the target person last appeared in the first monitoring screen. This boundary line may be any of the four edges of the screen, or it may be a virtual dividing line preset inside the screen.

[0044] For example, assuming that the screen size of the first monitoring area is 1000×800 pixels, and the system sets a vertical virtual boundary line at the position (900, 400). If the target person enters the screen from the left side and is continuously monitored at positions (250, 600), (500, 580), (750, 590), etc., and leaves the screen after the last appearance at position (910, 410), it can be determined that the person leaves the frame at the right boundary, that is, the line segment of x=1000.

[0045] The significance of determining the exit frame is that it provides an important reference for predicting the subsequent whereabouts of the target person. Generally speaking, the direction and position of the person leaving the screen largely determines the range of his activities in the adjacent area. For example, a person leaving from the right border is very likely to appear in the adjacent monitoring area on the right. In this way, when tracking across regions, the system can focus on those areas adjacent to the exit frame, thereby narrowing the search range and improving tracking efficiency.

[0046] S104, determining whether the exit frame includes a preset exit; After determining the target person's exit frame, the system needs to further determine whether there is a preset exit on the frame. The so-called preset exit refers to the pre-calibrated entrance and exit positions in the monitoring area that allow people to pass normally. These locations usually correspond to physical facilities such as doors, passages, gates, or virtual electronic fences.

[0047] For example, in a closed campus, the main entrance, side door, fire escape, etc. of a building can all be considered as preset exits. In an open square, for security management purposes, the system may set up several virtual gates at the edge of the screen as entrances and exits. When someone passes through these designated locations, it usually means that their behavior is legal and compliant, and no special attention is needed.

[0048] Therefore, during the tracking process, if the system finds that the target person is leaving from a preset exit, it can reasonably infer that he is about to leave the entire monitoring area, and then take certain targeted measures, such as recording his departure time, capturing his facial features, and lifting the key monitoring of him. On the contrary, if the target person leaves the screen from other non-exit locations, it may mean that he has abnormal or suspicious behavior, which needs to attract the system's high attention.

[0049] S105, determining a second monitoring screen of a second monitoring area according to leaving the frame; If the preset exit is not included, the system determines the second monitoring screen of the second monitoring area based on the exit frame. Once the system determines that the target person has left the first monitoring area from a non-exit location, it is necessary to further determine the second monitoring area where he may appear in order to continue tracking. The key to this process is to infer the target person's activity range in the adjacent area based on the exit frame, and then lock the corresponding monitoring screen.

[0050] In the specific implementation, the system first needs to establish a global monitoring area topology map. This map uses each monitoring area as a node and the boundaries between adjacent areas as connections to describe the spatial layout of the entire monitoring network. In this map, each node should also contain detailed attribute information of the area, such as the corresponding camera number, screen coordinates, monitoring direction, etc.

[0051] When the target person leaves a border of the first monitoring area, the system will search for all nodes adjacent to the border in the topology map, and determine the area that is most likely to cover the target person based on the orientation information of each node. For example, if the target person leaves the right border of the first area, he or she is most likely to appear in the second area adjacent to the right. Therefore, the system can directly use the monitoring screen corresponding to the second area as the tracking target without traversing other areas.

[0052] In actual applications, due to the influence of factors such as shooting angle and picture distortion, it may be difficult to accurately locate the whereabouts of the target person based solely on topological relationships. Therefore, the system can also take into account other factors, such as the departure direction, movement speed, historical trajectory, etc. of the target person, and dynamically adjust the tracking area. For example, if the system finds that the target person has been moving quickly to the right in the first area, even if he eventually leaves from the upper border, there is a high probability that he will appear in the second area in the upper right corner. For another example, if the target person has traveled back and forth between two areas many times before, when he leaves one of the areas again, the system can prioritize the other area as the tracking focus.

[0053] S106, determining an entry frame in the second monitoring picture according to the traveling direction; The system determines the entry frame in the second monitoring screen according to the traveling direction, specifically: determining the traveling direction according to the traveling direction; Determine a first intersecting frame in the second monitoring picture that intersects with the traveling direction, the first intersecting frame being the first frame where the traveling direction intersects with the second monitoring picture; The first intersecting bounding box is determined as the entering bounding box.

[0054] After the system determines the second monitoring area that the target person may enter based on the target person's exit frame, it needs to further predict the specific range of the target person's activities in the area in order to quickly lock and track the target. This requires inferring the target person's entry frame in the second area, that is, the most likely boundary line, based on the target person's direction of travel in the first area.

[0055] In specific implementation, the system first needs to quantify the direction of travel of the target person. As mentioned earlier, the direction of travel can be represented by the angle of the moving vector. For the convenience of calculation, the system usually divides each direction into several intervals and identifies them with specific symbols or values. For example, the full range from 0° to 360° can be divided into 8 intervals, each corresponding to a direction: east, southeast, south, southwest, west, northwest, north, and northeast. In this way, the direction of travel of the target person can be represented by a simple direction value, such as east, southwest, etc.

[0056] After obtaining the direction of the target person when leaving the first area, the system will automatically analyze the relationship between the surveillance image of the second area and the direction, and select the border that best matches it as the entry border. The specific algorithm is as follows: The four borders (top, bottom, left, and right) of the second monitoring picture are extracted, and their respective orientation intervals are calculated.

[0057] Determine which frame's orientation interval the target person's moving direction belongs to. If it belongs to multiple intervals at the same time, select the frame with the smallest angle between the midpoint of the interval and the moving direction.

[0058] The above border is marked as an entry border, and is used as a basis to narrow the subsequent person search range.

[0059] For example, suppose the target person leaves from the right border of the first area and moves in the northeast direction. In the surveillance image of the second area, the direction interval of the upper border is from northwest to northeast, and the direction interval of the right border is from northeast to southeast. Obviously, the target person's moving direction belongs to both intervals. However, further calculations show that the angle between the northeast direction and the midpoint of the upper border interval due north is 45°, and the angle between the northeast direction and the midpoint of the right border interval due east is 135°. Therefore, the system will mark the upper border as an entry border and focus on searching the area close to the upper border.

[0060] S107, detecting face data entering the second monitoring area from the entry frame in real time; When the system predicts the target person's entry into the second monitoring area based on his / her direction, it can collect and analyze the monitoring data of the area in a targeted manner to achieve continuous and efficient tracking. The core of this process is to detect and compare the face data of the person entering the area near the border in real time.

[0061] S108, determining that a target person is detected according to the face data; When it is determined that the target person is detected based on the face data, the step of determining the dynamic coordinates of the target person is performed (step S101), that is, the above steps are looped until the target person leaves the monitoring area.

[0062] S109, end monitoring of the target person.

[0063] If the preset exit is included, the monitoring of the target person ends.

[0064] In addition, there may be blind spots in the monitoring area, so it is necessary to pay special attention to the blind spots. When it is determined that the target person leaves the first monitoring area, the timer starts; When it is determined that the target person has entered the second monitoring area, the timing is ended to obtain the monitoring vacuum time; Determine whether the monitoring vacuum time is greater than the preset time; If not, the step of determining the dynamic coordinates of the target person is executed; If it is greater, a warning message is sent to the monitoring terminal, and the warning message includes the location information of the target person.

[0065] When it is determined that the target person leaves the first monitoring area, after starting the timer, the method further includes: When it is determined that the timing time is greater than the preset time, determining whether the target person has entered the second monitoring area; If the user does not enter the second monitoring area, then retrieve several adjacent monitoring videos of several monitoring areas adjacent to the first monitoring area; Determine whether there is a target person in several adjacent surveillance videos; If so, the step of determining the dynamic coordinates of the target person is performed; If not, a danger message will be sent to the monitoring terminal, and the import and export controllers will be controlled to close the import and export; If the second monitoring area is entered, a step of sending a warning message to the monitoring terminal is executed.

[0066] The above embodiments have the following beneficial effects: When it is determined that the target person leaves the first monitoring area, it is determined whether the exit frame contains the preset exit, so that the target person's moving trajectory can be continuously tracked. If the exit frame does not contain the exit, it means that the target person is likely to have entered the adjacent second monitoring area. At this time, the system can quickly lock the entry frame of the second area according to the exit frame of the first area and the direction of travel of the target person, and detect the face in real time here until the target person is recaptured. Therefore, once the target appears in any monitoring area, the system can always grasp its dynamic position through relay cross-region tracking, and achieve real-time monitoring without dead angles and full coverage. The spatial scale of target tracking is greatly expanded. At the same time, if the target person leaves from the preset exit, the system can terminate the tracking in time to avoid unnecessary waste of resources. Continuous monitoring of target personnel is achieved, the integrity of monitoring target personnel is improved, and the security of the monitoring area is thereby improved.

[0067] On the basis of generating the dynamic coordinates of the target person, the system first smoothes the coordinate sequence to eliminate the instantaneous noise introduced by factors such as face detection error or camera jitter, and improve the stability of the data. Then, by calculating the moving vector, the system can accurately capture the displacement trend of the target person. Finally, by using the angle calculation algorithm, the displacement vector is mapped to the logical coordinate system to obtain the real-time moving direction of the target person. Compared with the traditional mobile target detection scheme, the advantage of this scheme is that it makes full use of the rich information of the previous face tracking and dynamic coordinate representation, and finally realizes the accurate determination of the moving direction through mathematical methods such as smoothing, vector calculation, and angle estimation. At the same time, unifying the representation of the moving direction into the logical coordinate system also makes cross-region target tracking possible. Once the target person leaves the current monitoring area, the system can directly predict the next possible area based on its last moving direction, and focus on the border in that direction, so as to find the target in the first time and realize seamless relay monitoring.

[0068] When the target person leaves the current area, the system first determines the next area he may enter based on his final direction of travel. By calculating the intersection of the orientation and the border, the target person's contact point in the next area can be inferred, and this point is determined as the entry border. This prediction mechanism based on the movement trend can greatly narrow the search range of the next area. At the same time, since the determination of the border depends entirely on the logical orientation relationship and does not require the actual geographical location, it is suitable for camera networks of any topological structure. In addition, since the position of the entry border has been accurately inferred, the face retrieval of the next area will be concentrated in a highly locked border area, thereby significantly improving the efficiency and success rate of target matching and recognition.

[0069] When the target person leaves the current area, the system starts timing until the target is recaptured in the next area. The time difference obtained is the monitoring vacuum period. If the duration exceeds the preset threshold, it means that the target person disappears abnormally during the movement between adjacent areas, and abnormal behavior or safety hazards are very likely to occur. At this time, the system will immediately send a warning message to the monitoring terminal, and attach the latest location of the target person for security personnel to check and deal with. In this way, the system can not only detect the abnormal movements of the target person in time, but also accurately locate the target position in the first time. In addition, since the sending of warning information is strictly based on the objective indicator of the monitoring vacuum duration, it can avoid interference from human factors to the greatest extent and improve the accuracy and reliability of the early warning.

[0070] Once the monitoring vacuum times out, the system first determines whether the target person has entered the next area originally planned. If the answer is no, it means that the target is likely to be staying inside a blind spot. At this time, the system will intelligently retrieve all surveillance videos adjacent to the current area and use the face recognition algorithm to search for the target person in these videos. Once the target is found, the system immediately locks its position and continues to track it, thereby minimizing the scope of the blind spot and achieving no-dead-angle monitoring. However, if the target person is never found, the system will further increase the warning level, not only sending danger information to the monitoring terminal, but also directly controlling all entrances and exits to prohibit passage. This closed-off emergency response can effectively control the target person's range of activities in the first time and minimize safety hazards.

[0071] The following describes step S101 with another embodiment. Figure 2 , another intelligent monitoring method based on face recognition in an embodiment of the present application is described: See also Figure 2 , is another flow chart of an intelligent monitoring method based on face recognition in an embodiment of the present application.

[0072] S201, receiving a first monitoring video stream corresponding to a first monitoring area in real time; In intelligent monitoring systems, the first monitoring area usually refers to the area of ​​initial concern, such as building entrances, parking lot entrances and exits, etc. The system needs to obtain video data in this area in real time as input for subsequent face recognition and target tracking. This process is usually achieved with the help of high-definition cameras and video transmission equipment.

[0073] For example, a 4K high-definition camera is installed at the entrance of an office building lobby and connected to the monitoring center via a wired network. The camera captures images at a speed of 30 frames per second and transmits the video stream to the central server in real time. The server uses a video decoder to convert the data stream into a series of frame images, each with a resolution of 3840×2160 pixels and a color depth of 24 bits. These high-quality images provide data guarantee for subsequent face detection.

[0074] S202, using a face detection algorithm to capture face data in the first monitoring video stream; After obtaining the video stream, the system needs to quickly and accurately locate the face area from each frame of the image to prepare for subsequent feature extraction. Currently, commonly used face detection algorithms include Haar cascade classifiers, HOG+SVM, and MTCNN based on deep learning. These algorithms can detect multiple faces of different postures and sizes in real time under complex backgrounds.

[0075] For example, the system uses the MTCNN algorithm in the 100th frame of the video and detects three faces within 50 milliseconds, located in the upper left, upper right, and center areas of the image, with sizes ranging from 100×100 to 300×300 pixels. For subsequent feature extraction and comparison, the system saves these three face areas as separate image files and records their position coordinates in the original image, such as (100, 200), (500, 100), (320, 310), etc.

[0076] S203, extracting facial features from the facial data, and matching the facial features in a preset facial database; After detecting a face, the system needs to extract key features that can represent the identity, match it in the pre-established face database, and determine whether it is the target person of interest. Commonly used face feature extraction algorithms include LBPH, Eigenface, Fisherface, etc. These algorithms can obtain unique texture, shape, structure and other information from face images to form a feature vector of fixed dimension. The system calculates the similarity between this vector and the feature vectors of each person stored in the face database to find the identity with the highest matching degree.

[0077] Assume that three faces are detected in step S202, and the corresponding feature vectors are: A: (0.12, 0.33, 0.08, 0.26, ..., 0.91) B: (0.52, 0.17, 0.23, 0.09, ..., 0.71) C: (0.66, 0.45, 0.23, 0.81, ..., 0.04) In the face database of the system, the feature vector of the target person X is: (0.59, 0.42, 0.26, 0.88, ..., 0.11). By calculating the Euclidean distance, it is found that vector C is closest to X, and the difference between the two is only 0.15. Therefore, the system preliminarily determines that face C is the target person X. If the threshold is set to 0.2, then the match is successful, and the appearance of X is captured by the monitoring system.

[0078] S204, when it is determined that the matched facial data is a target person, labeling the target facial data of the target person using face tracking technology; After the match is successful, in order to continue to track the target person, the system needs to uniquely mark the face and distinguish it from other faces. This process is achieved by adding a special annotation box (such as a red rectangle) around the target face. At the same time, the system will also generate a unique tracking ID for the annotation box to identify the same target under different times and different cameras.

[0079] Continuing with the previous example, suppose face C is identified as target X. The system draws a red rectangular frame around it and saves its size, color, line width and other parameters. In addition, the system automatically generates a tracking ID "20220616093205_CAM001_X" based on the current timestamp 20220616093205 and camera number CAM001", and assigns it to the annotation frame. In this way, when target X moves to other locations in the video or appears under another camera, it can easily maintain the tracking status and will not be confused with other targets.

[0080] S205: Determine the dynamic coordinates of the target person using a labeling box according to the target face data.

[0081] The system determines the dynamic coordinates of the target person using the annotation box according to the target face data, specifically: determining the monitoring screen according to the first monitoring video stream; A temporary coordinate system is established in the monitoring screen, the origin of the temporary coordinate system is the lower left vertex of the monitoring screen, the horizontal coordinate of the temporary coordinate system is the lower border, and the vertical coordinate is the left border; Mark the center point of the annotation box in the temporary coordinate system to obtain the dynamic coordinates of the target person.

[0082] After marking the target face, the system needs to obtain its accurate coordinates in the picture in real time to provide a basis for subsequent behavior analysis, trajectory prediction and other tasks. The usual practice is to establish a temporary coordinate system based on the edge of the monitoring screen and dynamically record the horizontal and vertical coordinate values ​​of the center point of the annotation box.

[0083] For example, the image size captured by the camera is 1920×1080, and the system establishes a two-dimensional coordinate system with the lower left corner as the origin (0, 0). The center of the rectangular frame surrounding the target face is located at (960, 250) of the image, so the coordinate of the target person X is recorded as (960, 250) at this moment. Assume that after 0.1 seconds, X moves 50 pixels to the right and the center of the annotation box comes to (1010, 250), then the dynamic coordinate of X is updated to (1010, 250) and saved together with the timestamp. By continuously recording these dynamic coordinate points, the movement trajectory of the target person in the monitoring area can be drawn for more in-depth analysis.

[0084] The above embodiments have the following beneficial effects: By using the face detection algorithm to capture faces in the video stream and extracting key features to match with the preset face library, the system can quickly determine whether the target person appears in the monitoring screen. Once the match is confirmed, the face tracking technology can be used to lock the target and mark its dynamic position in real time. Since facial features are unique and stable, using them as the basis for target identification can greatly improve the pertinence and accuracy of monitoring and effectively avoid various interferences in complex environments. At the same time, the establishment of a preset face library also provides the system with samples of target persons for comparison, making it possible to achieve directional monitoring. In addition, the ability of face tracking technology to mark positions in real time lays the foundation for the subsequent generation of dynamic coordinates.

[0085] On the basis of determining the position of the target person, the system does not directly use physical parameters such as pixel coordinates, but establishes a virtual logical coordinate system in the monitoring screen. The coordinate system takes the lower left corner of the screen as the origin, and the lower frame and left frame as the horizontal and vertical coordinate axes, respectively, forming a relative reference system independent of the actual physical size. The system corresponds the geometric center of the face annotation frame to the coordinate system, thereby generating dynamic coordinates that can reflect the relative position of the target person in real time. This novel parameter representation method has many unique advantages. First, the logical coordinate avoids the difference in imaging scale between different cameras, has universality and portability, and is conducive to the integration and sharing of monitoring information. Secondly, with the frame as the coordinate axis, a closed relative reference system independent of the outside world is formed, which simplifies the calculation and use of coordinates. In addition, the logical properties of the reference system are also consistent with the image processing algorithm, which is conducive to the digital realization of the software system. Finally, the dynamic coordinates clearly reflect the position change of the target person relative to the boundary of the monitoring area, which is convenient for judging the entry and exit status of the target.

[0086] The following describes the system in the embodiment of the present invention from the perspective of hardware processing. Figure 3 , is a schematic diagram of the physical device structure of an intelligent monitoring system based on face recognition provided in an embodiment of the present application.

[0087] It should be noted that Figure 3 The structure of the system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0088] like Figure 3 As shown, the system includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage part 308 to the random access memory (RAM) 303, such as executing the method in the above embodiment. In RAM 303, various programs and data required for system operation are also stored. CPU 301, ROM 302 and RAM 303 are connected to each other through a bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0089] The following components are connected to the I / O interface 305: an input section 306 including a camera, an infrared sensor, etc.; an output section 307 including a liquid crystal display (LCD) and a speaker, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed so that a computer program read therefrom is installed into the storage section 308 as needed.

[0090] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 309, and / or installed from a removable medium 311. When the computer program is executed by the central processing unit (CPU) 301, various functions defined in the present invention are performed.

[0091] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.

[0092] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Among them, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0093] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the system described in the above embodiment; or may exist independently without being assembled into the system. The above storage medium carries one or more computer programs, and when the above one or more computer programs are executed by a processor of a system, the system implements the method provided in the above embodiment.

[0094] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0095] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0096] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0097] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. An intelligent monitoring method based on face recognition, characterized in that: include: When it is determined that a target person appears in the first monitoring area, determining the dynamic coordinates of the target person; Determining the moving direction of the target person according to the dynamic coordinates; When it is determined that the target person leaves the first monitoring area, determining a leaving frame, where the leaving frame is a frame that overlaps with the first monitoring screen when the target person leaves the first monitoring area; Determine whether the exit frame includes a preset exit; If the preset exit is not included, determining a second monitoring screen of the second monitoring area according to the exit frame; Determining an entry frame in the second monitoring picture according to the traveling direction; Detecting in real time the face data of a person entering the second monitoring area from the entry frame; In the case where the target person is detected according to the face data, executing the step of determining the dynamic coordinates of the target person; If the preset exit is included, the monitoring of the target person is terminated.

2. The method according to claim 1, characterized in that In the case where it is determined that a target person appears in the first monitoring area, determining the dynamic coordinates of the target person specifically includes: Receiving a first monitoring video stream corresponding to the first monitoring area in real time; Using a face detection algorithm to capture face data in the first surveillance video stream; Extracting facial features from the facial data and matching the facial features in a preset facial database; When it is determined that the matched facial data is a target person, the target facial data of the target person is annotated using face tracking technology; The dynamic coordinates of the target person are determined using a marking frame according to the target face data.

3. The method according to claim 2, characterized in that The step of determining the dynamic coordinates of the target person using a labeling frame according to the target face data specifically includes: Determine a monitoring screen according to the first monitoring video stream; Establishing a temporary coordinate system in the monitoring screen, wherein the origin of the temporary coordinate system is the lower left vertex of the monitoring screen, the horizontal coordinate of the temporary coordinate system is the lower border, and the vertical coordinate is the left border; The center point of the marking frame is marked in the temporary coordinate system to obtain the dynamic coordinates of the target person.

4. The method according to claim 1, characterized in that The method further comprises: determining the moving direction of the target person according to the dynamic coordinates; Performing a smoothing operation on the dynamic coordinates to obtain smoothed dynamic coordinates; Calculating a moving vector of the target person according to the smoothed dynamic coordinates; The moving direction of the target person is calculated using an angle calculation algorithm according to the moving vector.

5. The method according to claim 1, characterized in that The step of determining an entry frame in the second monitoring picture according to the traveling direction specifically includes: determining a travel direction according to the travel direction; Determine a first intersecting frame in the second monitoring picture that intersects with the traveling direction, wherein the first intersecting frame is a first frame where the traveling direction intersects with the second monitoring picture; The first intersecting border is determined as the entering border.

6. The method according to claim 1, characterized in that After the monitoring of the target person ends if the preset exit is included, the method further includes: When it is determined that the target person leaves the first monitoring area, starting the timer; When it is determined that the target person enters the second monitoring area, the timing is ended to obtain the monitoring vacuum time length; Determining whether the monitoring vacuum duration is greater than a preset duration; If not, executing the step of determining the dynamic coordinates of the target person; If it is greater, a warning message is sent to the monitoring terminal, and the warning message includes the location information of the target person.

7. The method according to claim 6, characterized in that In the case where it is determined that the target person leaves the first monitoring area, after starting the timing, the method further includes: If it is determined that the timing time is greater than the preset time, determining whether the target person enters the second monitoring area; If the user does not enter the second monitoring area, then retrieve several adjacent monitoring videos of several monitoring areas adjacent to the first monitoring area; Determining whether the target person exists in the plurality of adjacent surveillance videos; If so, executing the step of determining the dynamic coordinates of the target person; If not, send danger information to the monitoring terminal and control the import and export controller to close the import and export; If the second monitoring area is entered, the step of sending a warning message to the monitoring terminal is executed.

8. An intelligent monitoring system based on face recognition, characterized in that: The system comprises: One or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the system to execute the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a system, the system is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on a system, the system is caused to execute the method according to any one of claims 1 to 7.

Images