A method and system for identifying shoulder flash lights based on video classification

Abstract

The present invention discloses a method and system for identifying shoulder flashlights based on video classification, which obtains a video stream and obtains the position coordinates of a human body in each frame of data of the video stream; inputs the position coordinates of a human body in each frame of data into a tracker to obtain the position coordinate information of each person; continuously records the human body position coordinate information of each video frame to identify the moving direction of each person and judge the orientation of the human body; inputs N consecutive human body screenshots of each person in two opposite moving directions into a video classification model to obtain the classification results of each orientation respectively, and triggers an alarm when the classification results of the two directions are both that the shoulder flashlight is not worn. By identifying whether the operator is wearing a shoulder flashlight through continuous frames, the dynamic change characteristics of the shoulder flashlight over time are captured, and the video classification model can realize automatic monitoring, reduce manual intervention, and improve supervision efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent monitoring and security systems, and in particular relates to a method and system for identifying shoulder flash lights based on video classification. Background Art

[0002] Shoulder flash lights can provide a clear warning effect to surrounding people and vehicles at night or in poor visibility by flashing red and blue lights alternately, reminding them to pay attention and avoid, thereby reducing the occurrence of accidents. This is especially important for the safety of night operations. Wearing shoulder flash lights can improve the visibility of operators, allowing passing vehicles and pedestrians to notice the presence of operators in time, thereby avoiding safety accidents such as collisions or falls, and protecting the safety of operators.

[0003] At present, workers mainly rely on personal consciousness to wear shoulder flashlights, and lack an effective supervision mechanism. This practice of relying on personal consciousness may lead to a low wearing rate of shoulder flashlights in some cases, which in turn affects the safety of workers. In order to improve the safety of workers, a supervision method is needed. This patent proposes a shoulder flashlight recognition algorithm, which determines whether a person is wearing a shoulder flashlight through video surveillance, which is of great help to improve the safety awareness of workers.

[0004] Because in a single-frame image, there are some white or bright feature areas on the human body that interfere with the judgment of shoulder flashes, such as reflective strips, mobile phone screens, reflections from helmets, car lights in the background, etc., it is difficult to make an accurate judgment on whether a person is wearing shoulder flashes based on a single-frame image. Summary of the invention

[0005] To solve the above problems, the purpose of the present invention is to provide a method and system for identifying shoulder flash lights based on video classification. Whether the operator is wearing shoulder flash lights can be identified through continuous frames, and the dynamic change characteristics of the shoulder flash lights over time can be captured. The video classification model can realize automated monitoring, reduce manual intervention, and improve supervision efficiency.

[0006] The technical solution provided by the present invention is: a method for identifying shoulder flash lights based on video classification, comprising the following steps:

[0007] Obtaining a video stream and obtaining the position coordinates of a human body in each frame of data of the video stream;

[0008] Input the position coordinates of the human body in each frame of data into the tracker to obtain the position coordinate information of each person;

[0009] Continuously record the human body position coordinate information in each video frame to identify each person's movement direction and determine their body orientation;

[0010] N consecutive human screenshots of each person moving in two opposite directions are input into the video classification model to obtain the classification results for each direction respectively. When the classification results of the identification in both directions are that the shoulder flash light is not worn, an alarm is triggered.

[0011] Preferably, obtaining the video stream and obtaining the position coordinates of the human body in each frame of data of the video stream further includes:

[0012] Get the video stream and decode it to extract frames to obtain video frame data;

[0013] The video frame data is input into a target detection model to obtain the position coordinates of the human body in each frame of data.

[0014] Preferably, the target detection model adopts the yolov5 target detection model. Before acquiring the video stream data, video data of people in the construction or working area is collected, and the yolov5 target detection model is labeled and trained.

[0015] Preferably, continuously recording the human body position coordinate information of each video frame to identify the moving direction of each person and determine the human body orientation further includes:

[0016] The moving direction of the human body along the y-axis is continuously recorded, and the human body in the video at this time is defined as visible surface A. When the moving direction of the y-axis is opposite to the previous record, the human body in the video at this time is defined as visible surface B.

[0017] Preferably, before inputting N consecutive human body screenshots of each person in two opposite moving directions into the video classification model:

[0018] N consecutive human body images are scaled to 256×128. If both visible surface A and visible surface B are present, the video classification model is used to identify whether a shoulder flash is worn based on visible surface A and visible surface B. If only visible surface A or visible surface B is present, human target detection is continued on the newly acquired video data.

[0019] Preferably, inputting N consecutive human screenshots of each person in two opposite moving directions into the video classification model to obtain classification results for each orientation respectively, and triggering an alarm when the classification results of the two directions are both that the person is not wearing a shoulder flash light further includes:

[0020] The video classification model is a deep learning algorithm based on the TSM algorithm. It moves some channels in the time dimension to promote information exchange between adjacent frames. It analyzes the classification results of two opposite visible surfaces based on N consecutive human body screenshots. When the classification results of the two opposite visible surfaces are both that the shoulder flash lights are not worn, an alarm is triggered. When the classification result of one visible surface is that the shoulder flash lights are worn, no alarm is triggered.

[0021] Preferably, before acquiring the video stream data, the video classification model is trained:

[0022] Collect single-person continuous human body image data in the construction or work area, divide the single-person continuous human body image data into a training set and a test set, annotate the training set and train the video classification model, and use the test set to test whether the training meets the accuracy requirement.

[0023] Based on the same concept, the present invention also provides a shoulder flash light recognition system based on video classification, comprising:

[0024] A data acquisition unit, used to acquire a video stream and obtain the position coordinates of a human body in each frame of data of the video stream;

[0025] The target recognition unit is used to input the position coordinates of the human body in each frame of data into the tracker to obtain the position coordinate information of each person;

[0026] The target analysis unit is used to continuously record the human body position coordinate information of each video frame to identify the movement direction of each person and determine the orientation of the body;

[0027] The classification and recognition unit is used to input N consecutive human body screenshots of each person in two opposite moving directions into the video classification model to obtain the classification results for each direction respectively. When the classification results of the recognition in two directions are both that the shoulder flash lights are not worn, an alarm is triggered.

[0028] Based on the same concept, the present invention also provides an electronic device, comprising:

[0029] A memory, the memory being used to store a processing program;

[0030] A processor, when executing the processing program, implements any one of the above-mentioned methods for identifying shoulder flash lights based on video classification.

[0031] Based on the same concept, the present invention also provides a readable storage medium, on which a processing program is stored. When the processing program is executed by a processor, any one of the above-mentioned shoulder flash light recognition methods based on video classification is implemented.

[0032] Due to the adoption of the above technical solution, the present invention has the following advantages and positive effects compared with the prior art:

[0033] The technical solution of the present invention acquires the video stream in real time and analyzes the position coordinates of the human body frame by frame. The movement direction and orientation of each person are continuously recorded through a tracker. By inputting N consecutive human body screenshots into a video classification model for classification, this method can accurately determine the wearing status of each person's shoulder flash lights in different orientations. When the recognition results of two opposite directions are both not worn, the system will trigger an alarm. This double verification mechanism improves the accuracy and robustness of recognition. In situations where shoulder flash lights need to be worn to improve visibility and safety (such as night work, road construction, etc.), this method can promptly detect people who are not wearing shoulder flash lights, so that timely measures can be taken to avoid potential safety risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The specific embodiments of the present invention are further described in detail below with reference to the accompanying drawings, wherein:

[0035] Figure 1 This is a flow chart of the shoulder flash light recognition method based on video classification of the present invention;

[0036] Figure 2 Schematic diagram of the human body movement direction analysis process of the present invention;

[0037] Figure 3 This is an example of a continuous frame of a human body. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description and claims. It should be noted that the drawings are all in a very simplified form and use non-precise ratios, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0039] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0040] In the prior art, if a single-frame image is used to identify shoulder flashes, the image feature is only a bright spot on the body, and it is easily affected by reflective strips, helmet reflections, background lights, etc., causing the model to mistakenly identify irrelevant bright lights as shoulder flashes. When identifying based on continuous video frames, the feature of shoulder flashes becomes a light point with a fixed strobe interval at a fixed position on the human body, which is a stable and unique feature. Therefore, in this scenario, video classification stability is better than image classification.

[0041] First embodiment

[0042] like Figure 1As shown, this embodiment provides a method for identifying shoulder flash lights based on video classification, comprising the following steps:

[0043] Obtaining a video stream and obtaining the position coordinates of a human body in each frame of data of the video stream;

[0044] Input the position coordinates of the human body in each frame of data into the tracker to obtain the position coordinate information of each person;

[0045] Continuously record the human body position coordinate information in each video frame to identify each person's movement direction and determine their body orientation;

[0046] N consecutive human screenshots of each person moving in two opposite directions are input into the video classification model to obtain the classification results for each direction respectively. When the classification results of the identification in both directions are that the shoulder flash light is not worn, an alarm is triggered.

[0047] The technical solution of this embodiment acquires the video stream in real time and analyzes the position coordinates of the human body frame by frame. The moving direction and orientation of each person are continuously recorded through the tracker. By inputting N consecutive human screenshots into the video classification model for classification, this method can accurately determine the wearing status of each person's shoulder flash lights in different orientations. When the recognition results of two opposite directions are both not wearing, the system will trigger an alarm. This double verification mechanism improves the accuracy and robustness of recognition. In situations where shoulder flash lights need to be worn to improve visibility and safety (such as night work, road construction, etc.), this method can promptly detect personnel who are not wearing shoulder flash lights, so as to take timely measures to avoid potential safety risks. Compared with single image classification, video classification has temporal continuity and correlation, and the features in a single frame image are not sufficient to describe a dynamically changing event. Image classification algorithms mainly focus on the spatial features of static images, while videos are composed of a series of continuous frames, and there is a temporal correlation between these frames.

[0048] Preferably, obtaining the video stream and obtaining the position coordinates of the human body in each frame of data of the video stream further includes:

[0049] Get the video stream and decode it to extract frames to obtain video frame data;

[0050] The video frame data is input into a target detection model to obtain the position coordinates of the human body in each frame of data.

[0051] Since it is impossible to determine whether a person is wearing a shoulder flash based on a single picture under different lighting and angles, this embodiment obtains the video stream in real time and analyzes the position coordinates of the human body frame by frame. The method based on continuous video frames can better capture the changing characteristics of the shoulder flash at different times, which is better than the method based on single-frame judgment. It can automatically monitor the entire process, reduce manual intervention, and improve supervision efficiency. The video classification model has extremely high generalization and recognition accuracy.

[0052] Specifically, the target detection model adopts the yolov5 target detection model. Before acquiring the video stream data, video data of people in the construction or working area is collected, and the yolov5 target detection model is labeled and trained.

[0053] Preferably, see Figure 2 , Figure 3 , continuously recording the human body position coordinate information of each video frame to identify the moving direction of each person and determine the direction of the human body further includes:

[0054] The moving direction of the human body along the y-axis is continuously recorded, and the human body in the video at this time is defined as visible surface A. When the moving direction of the y-axis is opposite to the previous record, the human body in the video at this time is defined as visible surface B.

[0055] The human body is divided into visible and invisible sides in the video. When the human body moves, its y-axis moving direction (upward or downward) is continuously recorded. At this time, the human body in the video is visible side A. When the y-axis moving direction is opposite to the previous record, the human body in the video is visible side B. When both sides of the human body are visible, the next step of analysis can be performed. When only one side of the human body is visible, it is impossible to determine whether the flash light is blocked.

[0056] Preferably, before inputting N consecutive human body screenshots of each person in two opposite moving directions into the video classification model:

[0057] N consecutive human body images are scaled to 256×128. If both visible surface A and visible surface B are present, the video classification model is used to identify whether a shoulder flash is worn based on visible surface A and visible surface B. If only visible surface A or visible surface B is present, human target detection is continued on the newly acquired video data.

[0058] Preferably, inputting N consecutive human screenshots of each person in two opposite moving directions into the video classification model to obtain classification results for each orientation respectively, and triggering an alarm when the classification results of the two directions are both that the person is not wearing a shoulder flash light further includes:

[0059] The video classification model is a deep learning algorithm based on the TSM algorithm. It moves some channels in the time dimension to promote information exchange between adjacent frames. It analyzes the classification results of two opposite visible surfaces based on N consecutive human body screenshots. When the classification results of the two opposite visible surfaces are both that the shoulder flash lights are not worn, an alarm is triggered. When the classification result of one visible surface is that the shoulder flash lights are worn, no alarm is triggered.

[0060] TSM (Temporal Shift Module) promotes information exchange between adjacent frames by moving some channels in the temporal dimension. This design allows the model to capture the temporal dynamic features in video data while maintaining the computational efficiency of 2D CNN. Its advantages are: 1. Efficient temporal modeling: TSM effectively models temporal information in video frame sequences by moving channels in the temporal dimension. This method does not require additional computation to achieve temporal information interaction, allowing the model to use long-term spatiotemporal contextual relationships for video understanding; 2. Low-latency online video recognition: TSM is particularly suitable for online video stream recognition tasks. Due to its small network structure and relatively small computational complexity compared to the transformer model, it can give real-time predictions with very low latency; 3. Hardware-friendly: TSM is very concise in implementation and does not require changing the existing 2D CNN architecture. It only requires inserting the TSM module in the convolutional layer, which makes it very friendly in hardware deployment and easy to implement and optimize. When inputting the model, we choose to scale 8 consecutive human body images to 256×128 input models. The model will input human body input as one of wearing shoulder flash, not wearing shoulder flash, and non-human. When the classification results of both sides of the human body are that no flashing lights are worn, an alarm is triggered.

[0061] Preferably, before acquiring the video stream data, the video classification model is trained:

[0062] Collect single-person continuous human body image data in the construction or work area, divide the single-person continuous human body image data into a training set and a test set, annotate the training set and train the video classification model, and use the test set to test whether the training meets the accuracy requirement.

[0063] The video classification model is trained to identify whether the operator is wearing a shoulder flashlight through continuous frames, capturing the dynamic change characteristics of the shoulder flashlight over time. Traditional shoulder flashlight wearing supervision relies on manual inspection, while the video classification model can realize automatic monitoring, reduce manual intervention, and improve supervision efficiency. The video classification deep learning model has good generalization ability and can adapt to different lighting conditions and background noise, improving the recognition rate of shoulder flashlights in complex environments.

[0064] This application aims to identify the shoulder flashlights worn by workers and proposes a shoulder flashlight recognition algorithm based on deep learning video classification. This algorithm realizes automatic monitoring through video recognition, which not only significantly reduces the cost and inefficiency of manual supervision, but also greatly improves the stability and accuracy of recognition. Compared with the traditional classification algorithm based on single-frame images, this patent has the following significant advantages:

[0065] 1. High stability: Through the analysis of continuous video frames, the algorithm can capture the dynamic change characteristics of shoulder flash lights at different times, effectively avoiding misjudgment caused by factors such as lighting and angle in a single frame image.

[0066] 2. High accuracy: The video classification model has excellent generalization ability and can adapt to various complex environmental conditions, including different lighting conditions and background noise, thereby improving the accuracy of recognition.

[0067] 3. Efficiency: Automated monitoring reduces the need for manual intervention, improves supervision efficiency, and ensures that the safety of operators is protected in a timely manner. The model requires less computing power, is easy to deploy, and has a fast inference speed.

[0068] Based on the same concept, the present invention also provides a shoulder flash light recognition system based on video classification, comprising:

[0069] A data acquisition unit, used to acquire a video stream and obtain the position coordinates of a human body in each frame of data of the video stream;

[0070] The target recognition unit is used to input the position coordinates of the human body in each frame of data into the tracker to obtain the position coordinate information of each person;

[0071] The target analysis unit is used to continuously record the human body position coordinate information of each video frame to identify the movement direction of each person and determine the orientation of the body;

[0072] The classification and recognition unit is used to input N consecutive human body screenshots of each person in two opposite moving directions into the video classification model to obtain the classification results for each direction respectively. When the classification results of the recognition in two directions are both that the shoulder flash lights are not worn, an alarm is triggered.

[0073] The technical solution of this embodiment acquires the video stream in real time and analyzes the position coordinates of the human body frame by frame. The movement direction and orientation of each person are continuously recorded through the tracker. By inputting N consecutive human body screenshots into the video classification model for classification, this method can accurately determine the wearing status of each person's shoulder flash lights in different orientations. When the recognition results of two opposite directions are both not worn, the system will trigger an alarm. This double verification mechanism improves the accuracy and robustness of recognition. In situations where shoulder flash lights need to be worn to improve visibility and safety (such as night work, road construction, etc.), this method can promptly detect people who are not wearing shoulder flash lights, so that timely measures can be taken to avoid potential safety risks.

[0074] Based on the same concept, the present invention also provides an electronic device, comprising:

[0075] A memory, the memory being used to store a processing program;

[0076] A processor, when executing the processing program, implements any one of the above-mentioned methods for identifying shoulder flash lights based on video classification.

[0077] Based on the same concept, the present invention also provides a readable storage medium, on which a processing program is stored. When the processing program is executed by a processor, any one of the above-mentioned shoulder flash light recognition methods based on video classification is implemented.

[0078] If the shoulder flash light recognition method based on video classification is implemented in the form of program instructions and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of software. The computer software is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0079] Those skilled in the art can clearly understand that, for the sake of convenience and brevity in description, the identification content specifically executed by the above-described system and device can refer to the corresponding process in the aforementioned method embodiment.

[0080] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they still fall within the protection scope of the present invention.

Claims

1. A shoulder flash light recognition method based on video classification, characterized in that: The following steps are involved: Obtaining a video stream and obtaining the position coordinates of a human body in each frame of data of the video stream; Input the position coordinates of the human body in each frame of data into the tracker to obtain the position coordinate information of each person; Continuously record the human body position coordinate information in each video frame to identify each person's movement direction and determine their body orientation; N consecutive human screenshots of each person moving in two opposite directions are input into the video classification model to obtain the classification results for each direction respectively. When the classification results of the identification in both directions are that the shoulder flash light is not worn, an alarm is triggered.

2. The method for identifying shoulder flash lights based on video classification according to claim 1, characterized in that: Acquiring a video stream and acquiring the position coordinates of a human body in each frame of data of the video stream further includes: Get the video stream and decode it to extract frames to obtain video frame data; The video frame data is input into a target detection model to obtain the position coordinates of the human body in each frame of data.

3. The method for identifying shoulder flash lights based on video classification according to claim 2, characterized in that: The target detection model adopts the yolov5 target detection model. Before acquiring the video stream data, video data of people in the construction or working area is collected, and the yolov5 target detection model is labeled and trained.

4. The method for identifying shoulder flash lights based on video classification according to claim 1, characterized in that: Continuously recording the human body position coordinate information of each video frame to identify the moving direction of each person and determine the orientation of the human body further includes: The moving direction of the human body along the y-axis is continuously recorded, and the human body in the video at this time is defined as visible surface A. When the moving direction of the y-axis is opposite to the previous record, the human body in the video at this time is defined as visible surface B.

5. The method for identifying shoulder flash lights based on video classification according to claim 4, characterized in that: Before feeding N consecutive human screenshots of each person moving in two opposite directions into the video classification model: N consecutive human body images are scaled to 256×128. If both visible surface A and visible surface B are present, the video classification model is used to identify whether a shoulder flash is worn based on visible surface A and visible surface B. If only visible surface A or visible surface B is present, human target detection is continued on the newly acquired video data.

6. The method for identifying shoulder flash lights based on video classification according to claim 1, characterized in that: Inputting N consecutive human screenshots of each person in two opposite moving directions into the video classification model to obtain classification results for each direction respectively. When the classification results of the two directions are both that the shoulder flash light is not worn, the alarm is triggered, further including: The video classification model is a deep learning algorithm based on the TSM algorithm. It moves some channels in the time dimension to promote information exchange between adjacent frames. It analyzes the classification results of two opposite visible surfaces based on N consecutive human body screenshots. When the classification results of the two opposite visible surfaces are both that the shoulder flash lights are not worn, an alarm is triggered. When the classification result of one visible surface is that the shoulder flash lights are worn, no alarm is triggered.

7. The method for identifying shoulder flash lights based on video classification according to claim 1, characterized in that: Before acquiring the video stream data, the video classification model is trained: Collect single-person continuous human body image data in the construction or work area, divide the single-person continuous human body image data into a training set and a test set, annotate the training set and train the video classification model, and use the test set to test whether the training meets the accuracy requirement.

8. A shoulder flash light recognition system based on video classification, characterized in that: include: A data acquisition unit, used to acquire a video stream and obtain the position coordinates of a human body in each frame of data of the video stream; The target recognition unit is used to input the position coordinates of the human body in each frame of data into the tracker to obtain the position coordinate information of each person; The target analysis unit is used to continuously record the human body position coordinate information of each video frame to identify the movement direction of each person and determine the orientation of the body; The classification and recognition unit is used to input N consecutive human body screenshots of each person in two opposite moving directions into the video classification model to obtain the classification results for each direction respectively. When the classification results of the recognition in two directions are both that the shoulder flash lights are not worn, an alarm is triggered.

9. An electronic device, characterized in that: include: A memory, the memory being used to store a processing program; A processor, wherein when executing the processing program, the processor implements the shoulder flash light recognition method based on video classification as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a processing program, and when the processing program is executed by the processor, the video classification-based shoulder flash light recognition method as described in any one of claims 1 to 7 is implemented.

Images