Method and device for automatically tracking and identifying personnel violation, and medium

Abstract

The invention discloses a method for automatically tracking and identifying personnel violation. The method comprises the following steps: acquiring video stream data of a camera, and extracting video frames according to a preset frame rate; and identifying the human body target in each video frame by using the human body identification model to obtain an ID and a target frame corresponding to the human body target. Video frames are selected according to the number of interval frames, human body targets in the video frames are cut for violation classification, and whether violation exists or not is judged. And comparing the center point coordinates of different frames, and filtering the static target. If yes, the movement distance of the human body target in the preset time period is calculated, then a prediction target frame is calculated, and the prediction target frame is sent to a camera focusing interface to reduce the focal length. And after the focal length is shortened, human body target violation behavior identification is executed again so as to realize automatic tracking and identification of violation of the human body target. Therefore, the illegal behavior can be detected more accurately, and the probability of false alarm is greatly reduced.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent detection, and in particular relates to a method, device and medium for automatically tracking and identifying personnel violations. Background Art

[0002] AI target detection and camera automatic tracking technology, as an important part of modern intelligent systems, integrates advanced technologies in multiple fields such as deep learning, computer vision and automatic control. It can not only quickly identify specific targets such as pedestrians, vehicles or specific objects from complex scenes, but also improve the accuracy and robustness of recognition through continuous optimization of deep learning models.

[0003] In terms of automatic camera tracking, the technology uses advanced image processing algorithms to analyze the position, speed and movement trajectory of the target in real time, and automatically adjusts the camera's viewing angle and focal length to ensure that the target always remains within the field of view. Even in challenging environments such as lighting changes, occlusion or fast-moving targets, the system can maintain stable and efficient tracking performance.

[0004] AI target detection and camera automatic tracking technology have shown broad application prospects in many fields such as security monitoring, intelligent traffic management, drone tracking, sports game analysis, etc., providing strong technical support for the intelligence and automation of social life.

[0005] However, in some complex scenarios, various false alarms are likely to occur when identifying people wearing clothing violations (such as not wearing a helmet or reflective clothing), and the accuracy of the alarm needs to be confirmed by moving the lens and zooming in. Summary of the invention

[0006] The technical purpose of the present invention is to provide a method, device and medium for automatically tracking and identifying violations of personnel, so as to solve the problem of false alarms easily occurring for violations of human targets in complex environments.

[0007] To solve the above problems, the technical solution of the present invention is: A method for automatically tracking and identifying personnel violations includes the following steps: Get the video stream data from the camera and extract video frames according to the preset frame rate; The human target in each video frame is identified by using the human recognition model, and the IDs corresponding to all human targets are tracked and obtained by using the ByteTrack algorithm, and the target frame corresponding to each human target is recorded; the video frame is selected according to the first preset interval frame number, and the human targets in the video frame are cropped for violation classification, and the number of violations within the preset time period is counted. If the number exceeds the preset number, it is determined that the human target has violated the rules and the next step is entered. Otherwise, the above steps are repeated; Compare the center point coordinates of the same human target separated by a second preset interval frame number to determine whether it exceeds the preset minimum moving distance, if so, retain it, otherwise filter out the human target; After classification, if the human target has any violation, the center point of the target frame corresponding to the human target with the violation is obtained, and the moving distance of the human target within the preset time period is calculated. The predicted target frame is calculated based on the moving distance, and the predicted target frame is sent to the camera focus interface to shorten the focal length. After zooming in, repeat the above video frame extraction and human target violation identification. If there is any violation, face recognition is performed. If a face is detected, the face database is compared and the alarm person information is reported. If no face is detected, the current frame image is directly reported to the alarm, and the camera is reset after the alarm is completed.

[0008] Among them, for the acquired video stream data, video frames are extracted at a frame rate of 25.

[0009] Specifically, in the human recognition model, the confidence level of human targets is set to 0.8, and human targets with a confidence level less than 0.8 are automatically filtered out.

[0010] Among them, when performing the first human target violation detection, the target box corresponding to each human target is recorded as [x1, y1, x2, y2]; the first preset interval frame number is selected as 5, and the human target is cropped and classified for violation every 5 frames. The number of violations in 10 consecutive detections is counted. If the number of violations is greater than 7 preset times, it is determined that the human target has violated the regulations.

[0011] Specifically, when filtering out misidentified stationary targets, the center point of the target box of the detected human target in the first frame of the video is calculated, and then the center point of the target box of the same human target in the eleventh frame of the video is calculated after an interval of 10 frames. The displacement of the above two center points is compared to determine whether it exceeds the preset minimum moving distance. If so, it is retained, otherwise the human target is filtered out.

[0012] Among them, in the second human target violation detection, the center point of the target frame corresponding to the human target with violation behavior is obtained, the center point of the target frame in the first frame of the video frame (x0, y0) is selected, and then the center point of the target frame of the same human target in the fiftieth frame of the video frame (x49, y49) is selected, and the moving distance of the selected human target in the fifty frames is calculated as distance_x = x49 - x0, distance_y = y49 - y0; (x49+distance_x*time, y49+distance_y*time) is used as the center point of the predicted target frame, time is the camera focusing time, which is set to 2s, and then the coordinates of the predicted target frame are calculated as (x1, y1, x2, y2), and the coordinates of the predicted target frame are sent to the camera focusing interface to zoom in; no detection is performed during the zooming process until the picture is stable.

[0013] Specifically, after zooming in, determine frame by frame whether the predicted target frame is less than 300 pixels from the nearest scene frame. If it is less than 300 pixels, call the camera interface to move once in the direction of the nearest edge.

[0014] Specifically, after zooming in, if no human target is detected or the human target is lost for more than 7 seconds, the camera interface is called to reset the camera.

[0015] An electronic device includes a processor and a memory. The memory stores computer instructions that can be run on the processor. The processor is used to call the computer instructions in the memory to execute the above-mentioned method for automatically tracking and identifying personnel violations.

[0016] A storage medium is used to store computer instructions, wherein when the computer instructions are executed, the device where the storage medium is located is controlled to execute the above-mentioned method for automatically tracking and identifying personnel violations.

[0017] Due to the adoption of the above technical solution, the present invention has the following advantages and positive effects compared with the prior art: The present invention designs a set of corresponding control logic for personnel violation detection, which successively acquires video stream data and extracts frames, identifies and tracks human targets, detects violations, filters stationary targets, tracks and predicts violation targets, and focuses and adjusts cameras, thereby achieving more accurate detection of violations and greatly reducing the probability of false alarms. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.

[0019] Figure 1 The present invention is a flow chart of a method for automatically tracking and identifying personnel violations. DETAILED DESCRIPTION

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings and other implementation methods can be obtained based on these drawings without creative work.

[0021] In order to simplify the drawings, only the parts related to the present invention are schematically shown in each figure, and they do not represent the actual structure of the product. In addition, in order to simplify the drawings and facilitate understanding, in some figures, only one of the parts with the same structure or function is schematically drawn or marked. In this article, "one" not only means "only one", but also means "more than one".

[0022] The following is a further detailed description of a method, device and medium for automatically tracking and identifying personnel violations proposed by the present invention in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description and claims.

[0023] Example See also Figure 1 This embodiment provides a method for automatically tracking and identifying personnel violations, including the following steps: First, based on the ffmepg engine service, the video stream data of the camera is pulled, and video frames are extracted according to the preset frame rate. In this embodiment, the video frames are extracted at a frame rate of 25 as an example.

[0024] Then, the first detection of violations of human targets is performed, which can also be understood as preliminary detection. In this step, the human target in each video frame is identified using the human recognition model, and the confidence of the human target is set to 0.8, and the human targets less than 0.8 are automatically filtered and removed. Among them, the human recognition model is a Tensorrt model trained based on YOLOv5, with a model size of [1, 3, 544, 960] and input uint8 rgb image data. The ByteTrack algorithm is used to track and obtain the IDs (id1, id2, id3...) corresponding to all human targets, and the coordinates of the target box corresponding to each human target are recorded ([x1, y1, x2, y2]), that is, all recognized human targets are recorded. After that, video frames are selected at intervals of 5 frames, and the human targets in the selected video frames are cropped for violation classification, that is, whether they meet the violation standards. In this embodiment, for the application level of detecting whether port staff have violated regulations, not wearing a safety helmet and not wearing a life jacket are included in the violation behaviors; since the port camera is generally set at a high position and the scene to be monitored is complex, a secondary violation confirmation is required later. The number of violations is counted for 10 consecutive detections (a total of 50 frames, when the frame rate is 25, it takes 2 seconds), and if the number of violations is greater than 7 preset times, it is determined that the human target has violated regulations.

[0025] In order to improve the accuracy of detection, it is better to filter out the misidentified static targets among all human targets. Specifically, take one of the human targets as a column, calculate the center point of the target frame of the human target in the first frame of the video frame, and then calculate the center point of the target frame of the same human target in the eleventh frame of the video frame after an interval of 10 frames. Compare the displacement of the above two center points to determine whether it exceeds the preset minimum moving distance. If it exceeds, it will be retained, otherwise the human target will be filtered out. Until all human targets are judged, if all human targets are filtered out, return to the initial step, otherwise, go to the next step.

[0026] After the classification is completed, the second human target violation detection is performed to obtain the target frame corresponding to the human target with violation behavior, select the center point of the target frame in the first frame of the video frame (x0, y0), and then select the center point of the target frame of the same human target in the 50th frame of the video frame (x49, y49), and calculate the moving distance of the selected human target in the 50 frames distance_x = x49 - x0, distance_y = y49 - y0. Then (x49+distance_x*time, y49+distance_y*time) is used as the center point of the predicted target frame, and time is the camera focus time, which is preferably set to 2s. w,h are the width and height of the predicted target frame respectively (the data obtained by debugging w,h are set to 300 pixels), and then the coordinates of the predicted target frame are calculated as (x1, y1, x2, y2), and the coordinates of the predicted target frame are sent to the camera focus interface NET_DVR_PTZSelZoomIn to zoom in. It is worth mentioning that no detection is performed during the zooming process until the picture is stable. Among them, the NET_DVR_PTZSelZoomIn interface is the interface for adjusting the lens focal length of the existing Hikvision dome camera SDK.

[0027] Finally, after zooming in, repeat the above video frame extraction and human target violation identification. If there is a violation, perform face recognition. If a face is detected, compare it with the face database and report the alarm personnel information. If no face is detected, directly report the alarm current frame picture. After the alarm is completed, call the camera interface NET_DVR_PTZPreset_Other to reset the camera. If no human target is detected or the human target is lost for more than 7s, call the camera interface NET_DVR_PTZPreset_Other to reset the camera. Preferably, in this step, it is also judged frame by frame whether the predicted target frame is less than 300 pixels from the nearest scene frame. If it is less than 300 pixels, call the camera interface to move once in the direction of the nearest edge. Among them, the NET_DVR_PTZPreset_Other interface is the interface of the existing Hikvision ball camera SDK to reset the camera to the preset point.

[0028] Preferably, this embodiment also provides an electronic device, which includes a processor and a memory, wherein the memory stores computer instructions that can be executed on the processor, and the processor is used to call the computer instructions in the memory to execute the method for automatic tracking and identification of personnel violations as described above.

[0029] Further preferably, the present embodiment also provides a storage medium, the storage medium is used to store computer instructions, wherein when the computer instructions are executed, the device where the storage medium is located is controlled to execute the above-mentioned method for automatic tracking and identification of personnel violations.

[0030] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they still fall within the protection scope of the present invention.

Claims

1. A method for automatically tracking and identifying personnel violations, characterized in that: The steps include: Get the video stream data from the camera and extract video frames according to the preset frame rate; The human target in each of the video frames is identified by using a human recognition model, and the IDs corresponding to all human targets are tracked and obtained by using a ByteTrack algorithm, and the target frame corresponding to each human target is recorded; video frames are selected according to a first preset interval frame number, and the human targets in the video frames are cropped for violation classification, and the number of violations within a preset time period is counted. If the number exceeds the preset number, it is determined that the human target has violated the rules and the next step is entered, otherwise the above steps are repeated; Compare the center point coordinates of the same human target separated by a second preset interval frame number to determine whether it exceeds the preset minimum moving distance, if so, retain it, otherwise filter out the human target; After classification, if the human target has any violation, the center point of the target frame corresponding to the human target with the violation is obtained, and the moving distance of the human target within the preset time period is calculated. The predicted target frame is calculated based on the moving distance, and the predicted target frame is sent to the camera focus interface to shorten the focal length. After zooming in, repeat the above video frame extraction and human target violation identification. If there is any violation, face recognition is performed. If a face is detected, the face database is compared and the alarm person information is reported. If no face is detected, the current frame image is directly reported to the alarm, and the camera is reset after the alarm is completed.

2. The method for automatically tracking and identifying personnel violations according to claim 1 is characterized in that: For the acquired video stream data, video frames are extracted at a frame rate of 25.

3. The method for automatically tracking and identifying personnel violations according to claim 1 is characterized in that: In the human recognition model, the confidence level of human targets is set to 0.8, and human targets with a confidence level less than 0.8 are automatically filtered out.

4. The method for automatically tracking and identifying personnel violations according to claim 2 is characterized in that: When performing the first human target violation detection, the target frame corresponding to each human target is recorded as [x1, y1, x2, y2]; the first preset interval frame number is selected as 5, and the human target is cropped and classified for violation every 5 frames. The number of violations for 10 consecutive detections is counted. If the number of violations is greater than 7 preset times, it is determined that the human target has violated the regulations.

5. The method for automatically tracking and identifying personnel violations according to claim 1 is characterized in that: When filtering out misidentified stationary targets, calculate the center point of the target frame of the detected human target in the first frame of the video, and then calculate the center point of the target frame of the same human target in the eleventh frame of the video after an interval of 10 frames. Compare the displacement of the above two center points to determine whether it exceeds the preset minimum moving distance. If so, retain it, otherwise filter out the human target.

6. The method for automatically tracking and identifying personnel violations according to claim 4, characterized in that: In the second human target violation detection, obtain the target frame corresponding to the human target with violation behavior, select the center point (x0, y0) of the target frame in the first frame of the video, and then select the center point (x49, y49) of the target frame of the same human target in the fiftieth frame of the video, and calculate the moving distance distance_x = x49 - x0, distance_y = y49 - y0 of the selected human target in the fifty frames; take (x49 + distance_x*time, y49 + distance_y*time) as the center point of the predicted target frame, time is the camera focusing time, set to 2s, and then calculate the coordinates of the predicted target frame as (x1, y1, x2, y2), and send the coordinates of the predicted target frame to the camera focusing interface to zoom in; no detection is performed during the zooming process until the picture is stable.

7. The method for automatically tracking and identifying personnel violations according to claim 6, characterized in that: After zooming in, determine frame by frame whether the predicted target frame is less than 300 pixels from the nearest scene frame. If it is less than 300 pixels, call the camera interface to move once in the direction of the nearest edge.

8. The method for automatically tracking and identifying personnel violations according to claim 1, characterized in that: After zooming in, if no human target is detected or the human target is lost for more than 7 seconds, the camera interface is called to reset the camera.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores computer instructions that can be executed on the processor, and the processor is used to call the computer instructions in the memory to execute the method for automatic tracking and identification of personnel violations as described in any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium is used to store computer instructions, wherein when the computer instructions are executed, the device where the storage medium is located is controlled to execute the method for automatic tracking and identification of personnel violations as described in any one of claims 1 to 7.

Images