Biorder matching-based off-post detection method and system, and readable medium
By incorporating IOU matching and Hungarian matching algorithms into the target tracking method, the problem of pedestrian ID drift in complex backgrounds was solved, achieving highly accurate off-duty detection and ensuring accurate identification and alarm of on-duty personnel.
Patent Information
- Application Number
- CN202511629490.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-07
- Publication Date
- 2026-02-27
AI Technical Summary
In situations with complex image backgrounds, existing technologies suffer from frequent target ID changes, leading to unstable tracking and incorrect judgments of staff leaving their posts, thus failing to effectively ensure the on-duty status of staff.
A leave-of-absence detection method based on two-order matching is adopted. By combining IOU matching and Hungarian matching algorithms with the DeepSORT target tracking algorithm of deep learning, the drift of pedestrian coordinate boxes and IDs is prevented, thereby improving the accuracy of leave-of-absence detection.
It significantly improved the accuracy of off-duty detection, prevented false detections caused by pedestrian ID drift, and ensured accurate identification and alarm of on-duty personnel.
Smart Images
Figure CN121582864A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method, system, and readable medium for detecting absence from duty based on two-order matching, belonging to the field of intelligent image processing technology. Background Technology
[0002] With the widespread application of video surveillance systems in numerous fields, many monitoring scenarios still rely on manual supervision of personnel's on-duty status. In work environments such as duty rooms, control rooms, security rooms, and production lines, personnel are typically required to remain on duty continuously. However, real-time monitoring becomes difficult to guarantee once abnormal absence or dereliction of duty occurs. Furthermore, prolonged staring at screens can easily lead to fatigue for monitoring personnel, resulting in safety hazards or economic risks. Therefore, there is an urgent need for a technology capable of automatically detecting personnel absence in real time to ensure the safe and stable operation of the production environment.
[0003] Current industry-standard object detection methods are generally built upon existing object detection and tracking frameworks, acquiring personnel bounding boxes and further determining whether they have left their posts. Although existing deep learning-based object tracking algorithms are relatively mature, in complex image backgrounds, frequent changes in object IDs can lead to tracking instability, resulting in incorrect post-departure judgments. Summary of the Invention
[0004] To address the aforementioned problems, the purpose of this invention is to provide a method, system, and readable medium for detecting absence from work based on two-order matching, which prevents pedestrian coordinate frame and ID drift and improves the accuracy of absence detection.
[0005] To achieve the above objectives, this invention proposes the following technical solution: a method for detecting absence from duty based on two-order matching, comprising the following steps: acquiring images of a selected area of video; performing target detection and real-time tracking of pedestrians through the images to obtain an initial trajectory; performing IOU matching between the position of the pedestrian in the current frame and the initial trajectory, and determining whether the pedestrian is in a duty state based on the IOU matching result; if the pedestrian is in a duty state, then using a Hungarian matching algorithm to determine whether the pedestrian has any intersection with other pedestrians in the image; based on the determination result, determining that the pedestrian is in an absence from duty state. For pedestrians in a duty state, a cost matrix is calculated based on the IOU matching of the detection boxes in the current frame and the previous frame, and then the Hungarian matching algorithm is used to obtain matched and unmatched pedestrians. Unmatched pedestrians in the current frame have no intersection with anyone in the image. Finally, combining the number of unmatched instances and the unmatched time threshold, it is further determined that the pedestrian is in an absence from duty state.
[0006] Furthermore, the image is obtained from a selected area of video from a camera via a real-time streaming protocol.
[0007] Furthermore, deepsort is used to track pedestrians in real time, and the trajectory obtained from the tracking result of deepsort is used as the initial trajectory.
[0008] Furthermore, the method for determining whether the pedestrian is on duty based on the IOU matching result is as follows: determine the undetermined on-duty trajectory based on the IOU matching result; count the number of times the undetermined on-duty trajectory occurs; when the number of undetermined on-duty trajectories exceeds a threshold, the pedestrian is considered to be on duty.
[0009] Furthermore, the method for determining the undetermined on-duty trajectory based on the IOU matching result is as follows: calculate the IOU matching result; if the IOU matching result is greater than a threshold, the matching is considered successful; if the matching is successful, the pedestrian's initial trajectory is taken as the undetermined on-duty trajectory.
[0010] Furthermore, the pedestrian's on-duty trajectory is compared with the on-duty trajectories of other pedestrians using a Hungarian matching algorithm. If the two on-duty trajectories intersect, it is confirmed that the pedestrian is on duty; otherwise, it is determined whether the pedestrian is off duty.
[0011] Furthermore, based on the judgment result, the method for determining that the pedestrian is in an off-duty state is as follows: if the pedestrian does not intersect with other pedestrians, the pedestrian is considered to be in an uncertain off-duty state; the number of uncertain off-duty states is counted; when the number of uncertain off-duty trajectories exceeds a threshold, the pedestrian is considered to be off-duty; if the time the pedestrian is off-duty exceeds a preset time, the pedestrian is considered to be in an off-duty state.
[0012] Furthermore, if the pedestrian is off-duty, an alarm will be raised.
[0013] This invention also discloses a leave-of-post detection system based on two-order matching, comprising: an image acquisition module for acquiring images of a selected area of video; an initial track acquisition module for performing target detection and real-time tracking of pedestrians through the images to obtain an initial track; an IOU matching module for performing IOU matching between the position of a pedestrian in the current frame and the initial track, and determining whether the pedestrian is in an on-duty state based on the result of the IOU matching; and a Hungarian matching algorithm matching module for determining whether there is an intersection between the pedestrian and other pedestrians in the image when the pedestrian is in an on-duty state, using the Hungarian matching algorithm; and determining whether the pedestrian is in an off-duty state based on the determination result.
[0014] The present invention also discloses a computer-readable storage medium storing a computer program, which is executed by a processor to implement the off-duty detection method based on two-order matching as described in any of the preceding claims.
[0015] The technical solution of the present invention has at least the following technical effects or advantages: Compared with the direct use of existing target tracking methods, the present invention solves the problem of the offset of the off-duty alarm coordinates relative to the workstation by adding IOU matching to the target tracking method; Hungarian matching is performed on on-duty personnel to effectively prevent false detection of personnel leaving the post after pedestrian ID drift, and significantly improve the accuracy of off-duty detection. Attached Figure Description
[0016] Figure 1 This is a flowchart of a leave-of-post detection method based on two-order matching in one embodiment of the present invention. Detailed Implementation
[0017] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention is described in detail through specific embodiments. However, it should be understood that the specific embodiments are provided only for a better understanding of the present invention and should not be construed as limiting the present invention. In the description of the present invention, it should be understood that the terminology used is for descriptive purposes only and should not be construed as indicating or implying relative importance.
[0018] To address the problem in existing technologies where frequent target ID changes in complex image backgrounds lead to tracking instability and subsequent errors in off-duty detection, this invention proposes an off-duty detection method, system, and readable medium based on two-order matching. The method involves acquiring images from a selected area of video; performing target detection and real-time tracking of pedestrians within the images to obtain an initial trajectory; performing IOU matching between the pedestrian's position in the current frame and the initial trajectory to determine if the pedestrian is on duty; if the pedestrian is on duty, using a Hungarian matching algorithm to determine if there is any intersection between the pedestrian and other pedestrians in the image; and finally, determining if the pedestrian is off duty based on the result. This invention addresses the issue of off-duty alarm coordinates shifting relative to the workstation by adding IOU matching to the target tracking method; and by performing Hungarian matching on on-duty personnel, it effectively prevents false detections of off-duty personnel due to pedestrian ID drift, significantly improving the accuracy of off-duty detection.
[0019] Example 1 This embodiment discloses a leave-of-absence detection method based on two-order matching, such as... Figure 1 As shown, it includes the following steps: S1 acquires images of the selected area of video.
[0020] Images are transmitted via the Real Time Streaming Protocol. RTSPThe system acquires video from a selected area via a camera. RTSP is a network application layer protocol designed for controlling real-time audio and video streaming. It uses VCR (Videocorder)-style commands to control media stream playback, pause, fast forward, and other functions. It is typically used in conjunction with RTP (Real-time Transport Protocol) and other protocols. TCP ( RTP The Control Protocol (CP) is used to complete the streaming data transmission.
[0021] S2 uses images to detect and track pedestrians in real time to obtain an initial trajectory.
[0022] Real-time pedestrian tracking is achieved using deepsort, with the trajectory obtained from deepsort serving as the initial trajectory. DeepSORT is a deep learning-based multi-object tracking algorithm that introduces appearance feature learning into the traditional SORT (Simple Online and Realtime Tracking) algorithm to improve the accuracy and robustness of target tracking. This algorithm predicts the target's trajectory using a Kalman filter and extracts the target's appearance features using a deep neural network, thus maintaining consistency in the target's identity even when it is occluded or temporarily disappears.
[0023] S3 performs IOU matching between the current frame's pedestrian position and the initial track, and determines whether the pedestrian is on duty based on the IOU matching result.
[0024] The method for determining whether a pedestrian is on duty based on the results of IOU matching is as follows: The uncertain on-duty trajectory is determined based on the IOU matching result. Specifically, the IOU matching result is calculated; if the IOU matching result is greater than the threshold, the match is considered successful; if the match is successful, the pedestrian's initial trajectory is taken as the uncertain on-duty trajectory.
[0025] The number of times an employee's on-duty location was not determined was recorded.
[0026] If the number of times the pedestrian's on-duty trajectory is not determined exceeds a threshold, the pedestrian is considered to be on-duty.
[0027] S4 If the pedestrian is on duty, the Hungarian matching algorithm is used to determine whether there is any intersection between the pedestrian and other pedestrians in the image; based on the determination result, it is determined that the pedestrian is off duty.
[0028] The pedestrian's on-duty trajectory is used to perform a Hungarian matching algorithm with the on-duty trajectories of other pedestrians. If the two on-duty trajectories intersect, it is confirmed that the pedestrian is on duty; otherwise, it is determined whether the pedestrian is off duty.
[0029] Based on the judgment results, the method for determining whether a pedestrian is in an off-duty state is as follows: if the pedestrian does not intersect with other pedestrians, the pedestrian is considered to be in an uncertain off-duty state; count the number of uncertain off-duty states; when the number of uncertain off-duty trajectories exceeds a threshold, the pedestrian is considered to be off-duty; if the time the pedestrian is off-duty exceeds a preset time, the pedestrian is considered to be in an off-duty state.
[0030] If the pedestrian is off duty, report the incident to the police.
[0031] Example 2 Based on the same inventive concept, this embodiment discloses a leave-of-post detection system based on two-order matching, comprising: The image acquisition module is used to acquire images of a selected area of video. The initial trajectory acquisition module is used to perform target detection and real-time tracking of pedestrians through images to obtain an initial trajectory. The IOU matching module is used to perform IOU matching between the position of the pedestrian in the current frame and the initial track, and to determine whether the pedestrian is on duty based on the IOU matching result. The Hungarian matching algorithm module is used to determine whether a pedestrian has any overlap with other pedestrians in the image when the pedestrian is on duty; based on the determination result, it is determined whether the pedestrian is off duty.
[0032] Example 3 Based on the same inventive concept, this embodiment discloses a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to implement the off-duty detection method based on two-order matching as described above.
[0033] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0034] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0035] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0036] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0037] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific embodiments of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention. The above content is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.
Claims
1. A method for detecting absence from work based on two-order matching, characterized in that, Includes the following steps: Acquire images from a selected area of video; The image is used to perform target detection and real-time tracking of pedestrians to obtain an initial trajectory; The position of the pedestrian in the current frame is matched with the initial track using an IOU, and the result of the IOU matching is used to determine whether the pedestrian is in a working state. If the pedestrian is on duty, the Hungarian matching algorithm is used to determine whether the pedestrian has any overlap with other pedestrians in the image; Based on the assessment results, it was determined that the pedestrian was off duty.
2. The off-duty detection method based on two-order matching as described in claim 1, characterized in that, The image is obtained from a selected area of video from a camera via a real-time streaming protocol.
3. The off-duty detection method based on two-order matching as described in claim 1, characterized in that, The target tracking method using deepsort is used to track pedestrians in real time, and the trajectory obtained from the tracking result of deepsort is used as the initial trajectory.
4. The off-duty detection method based on two-order matching as described in claim 1, characterized in that, The method for determining whether the pedestrian is on duty based on the IOU matching result is as follows: The undetermined on-duty trajectory is determined based on the IOU matching results; Count the number of times the on-duty location was not determined; If the number of times the undetermined on-duty trajectory exceeds a threshold, the pedestrian is considered to be on duty.
5. The off-duty detection method based on two-order matching as described in claim 4, characterized in that, The method for determining the undetermined on-duty trajectory based on the IOU matching results is as follows: Calculate the results of IOU matching; If the IOU matching result is greater than the threshold, the match is considered successful; If a match is successful, the pedestrian's initial trajectory will be taken as the undetermined on-duty trajectory.
6. The off-duty detection method based on two-order matching as described in claim 1, characterized in that, The pedestrian's on-duty trajectory is used to perform a Hungarian matching algorithm with the on-duty trajectories of other pedestrians. If the two on-duty trajectories intersect, it is confirmed that the pedestrian is on duty; otherwise, it is determined whether the pedestrian is off duty.
7. The off-duty detection method based on two-order matching as described in claim 6, characterized in that, The method for determining that the pedestrian is off-duty based on the judgment result is as follows: If the pedestrian does not interact with any other pedestrians, the pedestrian is considered to be in an uncertain off-duty state. Count the number of times the undetermined absence status was recorded; If the number of times the undetermined off-duty trajectory exceeds a threshold, the pedestrian is considered to be off-duty. If the pedestrian is away from their post for more than a preset time, then the pedestrian is considered to be in a state of being away from their post.
8. The method for detecting absence from duty based on two-order matching as described in any one of claims 1-7, characterized in that, If the pedestrian is off duty, an alarm should be raised.
9. A leave-of-post detection system based on two-order matching, characterized in that, include: The image acquisition module is used to acquire images of a selected area of video. The initial trajectory acquisition module is used to perform target detection and real-time tracking of pedestrians through the image to obtain an initial trajectory; The IOU matching module is used to perform IOU matching between the position of the pedestrian in the current frame and the initial track, and to determine whether the pedestrian is in a working state based on the IOU matching result. The Hungarian matching algorithm module is used to determine whether there is any overlap between the pedestrian and other pedestrians in the image when the pedestrian is in a working state, using the Hungarian matching algorithm. Based on the assessment results, it was determined that the pedestrian was off duty.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is executed by a processor to implement the off-duty detection method based on two-order matching as described in any one of claims 1-8.