Human body shoulder flashing light detection alarm method and device based on deep learning

Abstract

According to the human body shoulder flashing light detection and alarm method and device based on deep learning, a video stream containing a portrait is acquired in real time, and a video frame is preprocessed; based on the preprocessed video frame, human body detection and human body key point detection are sequentially carried out through a target detection model, and a shoulder region of interest corresponding to a human body is extracted based on key point coordinates; respectively extracting a time sequence feature, a brightness change feature and a frequency feature of the extracted shoulder region-of-interest image sequence; and fusing the time sequence characteristics, the brightness change characteristics and the frequency characteristics, classifying the states of the shoulder flashing light, judging the turn-on or turn-off state of the shoulder flashing light, and outputting a state classification result. The target recognition performance is improved by optimizing the detection network, so that the accuracy is improved, and the tracking stability is improved by improving the tracking algorithm such as feature extraction optimization and multi-target association enhancement, so that the state judgment is more accurate.

Description

Technical Field

[0001] This invention belongs to the field of intelligent detection technology, and in particular relates to a method and device for detecting and alarming human shoulder flashlights based on deep learning. Background Technology

[0002] For the recognition of human shoulder flashlights, traditional visual methods mainly fall into two categories: one is image processing-based methods, whose main drawbacks include poor environmental adaptability, high light sensitivity, and a high false detection rate; the other is motion detection-based methods, which are easily affected by interference, have poor real-time performance, and unstable accuracy. Summary of the Invention

[0003] To address the aforementioned issues, the present invention aims to provide a deep learning-based method and apparatus for detecting and alerting human shoulder flashlights. By optimizing the detection network (enhanced attention mechanisms and multi-scale fusion), the accuracy of target recognition is improved. Furthermore, improved tracking algorithms, such as optimized feature extraction and enhanced multi-target association, enhance tracking stability, resulting in more accurate state determination. In addition, system innovations improve real-time processing capabilities and resource utilization, leading to better scalability, practicality, and reliability.

[0004] The technical solution provided by this invention is: a deep learning-based method for detecting and alarming human shoulder flashlights, comprising the following steps: Real-time acquisition of video streams containing human images, and preprocessing of video frames; Based on the preprocessed video frames, human body detection and human body key point detection are performed sequentially through the target detection model, and the corresponding shoulder region of interest is extracted based on the key point coordinates. For the extracted image sequence of the shoulder region of interest, its temporal features, brightness variation features, and frequency features are extracted respectively; By integrating the aforementioned timing features, brightness variation features, and frequency features, the state of the shoulder flash is classified, its on or off state is determined, and the state classification result is output.

[0005] Preferably, the video data acquisition and preprocessing steps specifically include: Collect raw video data including shoulder flash under different lighting conditions, distances, and complex backgrounds; The original video data is subjected to data augmentation processing, which includes brightness adjustment, contrast adjustment, noise enhancement, and applying a random mask to the lower body region. The positions and on / off states of the shoulder flashlights in the enhanced video data are labeled, and the data is divided into training and validation sets according to a preset ratio. The input video frame size is adjusted to the preset size, and the pixel values ​​are normalized and converted into tensor format.

[0006] Preferably, in the target detection and shoulder region of interest extraction steps, the target detection model used is based on the YOLOv5 network structure, and the channel and spatial attention module CBAM is integrated in its detection head to enhance the ability to perceive light sources.

[0007] Preferably, the target detection and shoulder region of interest extraction steps specifically include: The preprocessed video frames are input into the human detection model to obtain the bounding box coordinates and confidence scores of each human body in the image. The same preprocessed video frame is input into the human key point detection model to obtain the coordinates of the left and right shoulder key points corresponding to the human detection box. Based on the coordinates of the left and right shoulder key points, the corresponding shoulder region of interest image is extracted from the original video frame; The shoulder region of interest image is input into the shoulder flash detection model to obtain the detection box coordinates and confidence level of the shoulder flash within the shoulder region.

[0008] Preferably, after the target detection and shoulder region of interest extraction steps, a multi-target tracking step is also included: The DeepSORT algorithm is used to track detected human targets, and a unique ID is assigned to each target; Use Kalman filtering to predict the motion state of the tracked target in the next frame; Calculate the correlation matrix between the human detection box in the current frame and the existing tracking trajectory. The correlation matrix integrates the detection box intersection-union ratio cost, appearance feature cost, and motion cost. The Hungarian algorithm is used to associate the detection box with the trajectory, and the corresponding tracking trajectory is updated, initialized or deleted accordingly.

[0009] Preferably, in the shoulder region of interest sequence analysis step, extracting the brightness change features specifically includes: calculating the brightness difference between adjacent frames in the shoulder region of interest image sequence; extracting the frequency features specifically includes: performing time-frequency analysis on the shoulder region of interest image sequence to statistically obtain the flicker frequency features of the shoulder flashlight.

[0010] Preferably, the state classification and output steps specifically include: The fused features are compared with a preset threshold to determine whether the shoulder flash is on or off. When it is determined that the shoulder flash light is abnormally off, an alarm message is generated and reported. Following the state classification and output steps, a real-time visualization step is also included: Draw the human body bounding box, tracking ID, and shoulder flash status information on the video frame; Visualize human movement trajectories; Generate and save a CSV format record file containing complete trajectory information; It supports outputting the processing results as MP4 video files or RTSP video streams.

[0011] Based on the same concept, the present invention also provides a deep learning-based human shoulder flash light detection and alarm device, comprising: The data acquisition module is used to acquire video streams containing human images in real time and preprocess the video frames; The detection module is used to perform human body detection and human key point detection sequentially through the target detection model based on the preprocessed video frames, and extract the corresponding shoulder region of interest based on the key point coordinates. The information extraction module is used to extract temporal features, brightness variation features, and frequency features from the extracted image sequence of the shoulder region of interest. The judgment module is used to integrate the timing features, brightness change features and frequency features to classify the state of the shoulder flash, determine whether it is on or off, and output the state classification result.

[0012] Based on the same concept, the present invention also provides an electronic device, comprising: a memory for storing a processing program; and a processor, wherein the processor, when executing the processing program, implements the deep learning-based human shoulder flash light detection and alarm method described above.

[0013] Based on the same concept, the present invention also provides a readable storage medium storing a processing program, which, when executed by a processor, implements the deep learning-based human shoulder flash light detection and alarm method described above.

[0014] Because the present invention adopts the above technical solution, it has the following advantages and positive effects compared with the prior art: 1. High-precision and robust detection capabilities: Through the cascaded localization strategy of "human body-key point-shoulder ROI", the detection of small targets is transformed into local salient region analysis. Combined with the YOLOv5 model with integrated CBAM attention mechanism, the ability to perceive shoulder flash lights is significantly improved, and interference from complex backgrounds, lighting changes and target scale is effectively overcome.

[0015] 2. Stable and Consistent Multi-Target Tracking and State Analysis: The DeepSORT algorithm is introduced, fusing motion and appearance features to assign and maintain a unique ID for each person, ensuring identity consistency across frames. Through multi-dimensional fusion analysis of the shoulder ROI sequence based on temporal, brightness differences, and frequency characteristics, the flickering dynamics can be accurately captured, effectively distinguishing between genuine shoulder flashlights and interference from ambient light variations and noise, achieving highly reliable state classification.

[0016] 3. End-to-end proactive alarm and visualization output: The system transforms analysis results into automated decisions, reporting alarms in real time for abnormal shutdown states. Simultaneously, by overlaying visualizations of detection boxes, IDs, statuses, and motion trajectories, and outputting structured data and video streams, it provides intuitive situational awareness, post-event audit evidence, and supports seamless integration with existing monitoring platforms, forming a complete closed-loop security management solution. Attached Figure Description

[0017] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the human shoulder flash light detection and alarm method based on deep learning according to the present invention. Detailed Implementation

[0018] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer from the following description and claims. It should be noted that the drawings are all in a very simplified form and use non-precise ratios, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0019] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0020] Example This embodiment provides a deep learning-based method for detecting and alarming human shoulder flashlights, including the following steps: Real-time acquisition of video streams containing human images, and preprocessing of video frames; Based on the preprocessed video frames, human body detection and human body key point detection are performed sequentially through the target detection model, and the corresponding shoulder region of interest is extracted based on the key point coordinates. For the extracted image sequence of the shoulder region of interest, its temporal features, brightness variation features, and frequency features are extracted respectively; By integrating the aforementioned timing features, brightness variation features, and frequency features, the state of the shoulder flash is classified, its on or off state is determined, and the state classification result is output.

[0021] This embodiment does not directly search for the shoulder flashlight in the entire image. Instead, it first identifies the subject as the "person" and then accurately calculates the spatial position of the shoulder using key points (usually shoulder key points, such as the shoulder and neck). The step of "extracting the region of interest in the shoulder" transforms the complex global image search problem into a well-defined local region analysis problem. This not only narrows the processing scope and improves speed, but more importantly, it eliminates interference from other luminous areas in the image (such as headlights, screens, and window reflections), reducing the possibility of false alarms at the source. Multi-dimensional feature fusion enhances the robustness of the judgment, making the state judgment more reliable, adaptable to various complex scenarios, and significantly reducing the false alarm rate. It achieves dynamic perception and classification of state (on / off). Going beyond simple "presence / absence detection," it achieves monitoring of the device's working status, resulting in higher application value. Traditional methods may only detect "whether the shoulder flashlight exists," but the core of this solution is to determine its "on or off state." This is crucial for safety alarm applications. For example, in scenarios such as mines and construction sites, where personnel are required to turn on their shoulder flashlights, this system can detect violations such as personnel being on duty but not having their shoulder flashlights turned on, and immediately issue an alarm. This forms a complete end-to-end automated process, achieving full automation from raw video stream to final alarm decision, requiring no manual intervention and resulting in high efficiency. The solution starts with "video stream input," proceeding through preprocessing, target detection, region extraction, feature extraction, fusion classification, and finally "outputting status results," forming a complete closed loop. This automated processing capability enables its application in 24 / 7 real-time monitoring systems, significantly improving safety management efficiency. The technical solution, through precise region positioning and multi-dimensional feature fusion, effectively solves the challenge of status recognition for small but critical lighting devices in complex and dynamic real-world scenarios.

[0022] Preferably, the video data acquisition and preprocessing steps specifically include: Collect raw video data including shoulder flash under different lighting conditions, distances, and complex backgrounds; The original video data is subjected to data augmentation processing, which includes brightness adjustment, contrast adjustment, noise enhancement, and applying a random mask to the lower body region. The positions and on / off states of the shoulder flashlights in the enhanced video data are labeled, and the data is divided into training and validation sets according to a preset ratio. The input video frame size is adjusted to the preset size, and the pixel values ​​are normalized and converted into tensor format.

[0023] This embodiment constructs a dataset covering the variability of real-world scenes, enabling reliable operation in new and unseen environments. It simulates real-world challenges and forces the model to focus on the shoulders, making it insensitive to lighting, noise, and occlusion, resulting in a low false alarm rate. It provides "on / off" state supervision signals, enabling core "state monitoring and alerting" functions. It provides stable and standardized model inputs, leading to faster model convergence and a more stable training process.

[0024] Preferably, in the target detection and shoulder region of interest extraction steps, the target detection model used is based on the YOLOv5 network structure, and the channel and spatial attention module CBAM is integrated in its detection head to enhance the ability to perceive light sources.

[0025] This embodiment surgically enhances YOLOv5 by introducing the sophisticated attention mechanism CBAM, making it particularly adept at capturing and locating small targets with strong signal characteristics, such as shoulder flash lights, in complex environments. It is a prime example of achieving maximum performance improvement with minimal cost, directly enhancing the reliability and accuracy of the entire system's front-end detection process.

[0026] Preferably, the target detection and shoulder region of interest extraction steps specifically include: The preprocessed video frames are input into the human detection model to obtain the bounding box coordinates and confidence scores of each human body in the image. The same preprocessed video frame is input into the human key point detection model to obtain the coordinates of the left and right shoulder key points corresponding to the human detection box. Based on the coordinates of the left and right shoulder key points, the corresponding shoulder region of interest image is extracted from the original video frame; The shoulder region of interest image is input into the shoulder flash detection model to obtain the detection box coordinates and confidence level of the shoulder flash within the shoulder region.

[0027] This solution employs a three-level localization approach: "human body → shoulder → flashlight," narrowing the search range to more relevant areas at each step. Ultimately, in the shoulder ROI image, the initially minute shoulder flashlights become relatively prominent targets with clearer features, enabling the final shoulder flashlight detection model to easily and accurately complete the localization task. By first detecting the human body, then detecting the shoulder keypoints of that body, and finally detecting the shoulder flashlights near those keypoints, a rigorous logical chain is formed. This ensures that even if there are multiple people and multiple lights in the image, the system can clearly identify "which light belongs to which person," which is crucial for subsequent individual-based status alerts. The final shoulder flashlight detection model is a high-precision, specialized model, potentially computationally intensive. Running it directly across the entire image to find minute lights would be a significant waste of computation. This solution first uses a relatively efficient human detector and keypoint model to quickly identify candidate regions, then allows the complex shoulder flashlight model to perform "precision calculations" only on these small, high-value ROIs. This embodiment is a typical example of a cascaded classifier, achieving optimal computational efficiency. While the ROI extraction solution doesn't solve the problem all at once with a single end-to-end model, it breaks down a highly challenging real-world problem into several mature and solvable sub-problems through a carefully designed sequential process. It sacrifices some theoretical simplicity in exchange for a comprehensive improvement in accuracy, robustness, efficiency, and multi-objective processing capabilities, making it a very practical, efficient, and powerful engineering solution.

[0028] Preferably, after the target detection and shoulder region of interest extraction steps, a multi-target tracking step is also included: The DeepSORT algorithm is used to track detected human targets, and a unique ID is assigned to each target; Use Kalman filtering to predict the motion state of the tracked target in the next frame; Calculate the correlation matrix between the human detection box in the current frame and the existing tracking trajectory. The correlation matrix integrates the detection box intersection-union ratio cost, appearance feature cost, and motion cost. The Hungarian algorithm is used to associate the detection box with the trajectory, and the corresponding tracking trajectory is updated, initialized or deleted accordingly.

[0029] Introducing Multi-Object Tracking (MOT) is a key upgrade for building a robust and usable real-time system. It's more than just "giving an ID"; it fundamentally improves the coherence, accuracy, and usability of the entire solution. Unique ID assignment is the foundation for all subsequent advanced features. It enables the system to perform subsequent analysis and alerts based on "individuals" rather than "intra-frame detection boxes." Kalman filter prediction uses a motion model to predict the target's position in the next frame. It is resilient to brief occlusion / missed detections; when a target is briefly occluded or occasionally missed by the model, the tracker can maintain its presence based on the predicted trajectory and correctly associate it when the target reappears, avoiding frequent ID jumps. Limiting the search range to the vicinity of the predicted position significantly reduces computation and improves matching accuracy, guiding data association. It effectively eliminates false alarms caused by single-frame detection jitter, making status judgments smoother and more reliable. In complex scenarios such as crowd intersections and occlusions, it can maximize the preservation of IDs without loss or confusion. With stable IDs, the system can record the status history of each specific person's shoulder flashlight. For example: "When personnel ID002 first entered the monitored area, their shoulder flashlights were on. After moving to a certain location, they turned off, which lasted for 30 frames (1 second), triggering an alarm." This upgrades the system from "frame analysis" to "personnel tracking and monitoring," making the business logic more complete.

[0030] Preferably, in the shoulder region of interest sequence analysis step, extracting the brightness change features specifically includes: calculating the brightness difference between adjacent frames in the shoulder region of interest image sequence; extracting the frequency features specifically includes: performing time-frequency analysis on the shoulder region of interest image sequence to statistically obtain the flicker frequency features of the shoulder flashlight.

[0031] This embodiment, based on a deep understanding of the business scenario (flashing light), has performed precise feature engineering. By extracting highly discriminative physical features, it constructs a powerful yet transparent judgment logic, ultimately ensuring extremely high accuracy and reliability of the shoulder flashing light status classification task in the complex real world.

[0032] Preferably, the state classification and output steps specifically include: The fused features are compared with a preset threshold to determine whether the shoulder flash is on or off. When it is determined that the shoulder flash light is abnormally off, an alarm message is generated and reported. Following the state classification and output steps, a real-time visualization step is also included: Draw the human body bounding box, tracking ID, and shoulder flash status information on the video frame; Visualize human movement trajectories; Generate and save a CSV format record file containing complete trajectory information; It supports outputting the processing results as MP4 video files or RTSP video streams.

[0033] Compared to complex classifiers (such as another neural network), the thresholding method has extremely low computational overhead, making it ideal for real-time systems. Furthermore, its clear decision boundaries allow engineers to adjust sensitivity based on the specific scenario (e.g., lowering the alarm threshold in more dangerous areas). This is the core value output of the entire system. It's not a passive analysis tool, but an active safety sentinel. Once an "abnormal shutdown" is detected, the system immediately generates and reports an alarm, potentially preventing a safety incident and achieving a closed loop from "perception" to "intervention." Alarm information can be rapidly sent to the central monitoring platform or on-site personnel's mobile devices via lightweight network protocols (such as MQTT and HTTP POST), achieving near real-time safety response. Low-latency status assessment and alarm triggering meet the stringent timeliness requirements of industrial monitoring. This final design step reflects a strong product-centric and user-centric approach. It ensures that this advanced algorithm is not merely a precise figure in the laboratory, but a powerful, reliable, usable, and auditable security tool that is visible, trustworthy, usable, and auditable in the real world. It completes the crucial leap from "technical feasibility verification" to "industrial application."

[0034] Based on the same concept, the present invention also provides a deep learning-based human shoulder flash light detection and alarm device, comprising: The data acquisition module is used to acquire video streams containing human images in real time and preprocess the video frames; The detection module is used to perform human body detection and human key point detection sequentially through the target detection model based on the preprocessed video frames, and extract the corresponding shoulder region of interest based on the key point coordinates. The information extraction module is used to extract temporal features, brightness variation features, and frequency features from the extracted image sequence of the shoulder region of interest. The judgment module is used to integrate the timing features, brightness change features and frequency features to classify the state of the shoulder flash, determine whether it is on or off, and output the state classification result.

[0035] Based on the same concept, the present invention also provides an electronic device, comprising: a memory for storing a processing program; and a processor, wherein the processor, when executing the processing program, implements the deep learning-based human shoulder flash light detection and alarm method described above.

[0036] Based on the same concept, the present invention also provides a readable storage medium storing a processing program, which, when executed by a processor, implements the deep learning-based human shoulder flash light detection and alarm method described above.

[0037] If the deep learning-based human shoulder flashing light detection and alarm method is implemented as a program instruction and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in software. This computer software is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0038] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific identification content executed by the system and device described above can be referred to the corresponding process in the foregoing method embodiments.

[0039] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they shall still fall within the protection scope of the present invention.

Claims

1. A deep learning-based method for detecting and alarming human shoulder flashlights, characterized in that, Includes the following steps: Real-time acquisition of video streams containing human images, and preprocessing of video frames; Based on the preprocessed video frames, human body detection and human body key point detection are performed sequentially through the target detection model, and the corresponding shoulder region of interest is extracted based on the key point coordinates. For the extracted image sequence of the shoulder region of interest, its temporal features, brightness variation features, and frequency features are extracted respectively; By integrating the aforementioned timing features, brightness variation features, and frequency features, the state of the shoulder flash is classified, its on or off state is determined, and the state classification result is output.

2. The deep learning-based human shoulder flash light detection and alarm method according to claim 1, characterized in that, The video data acquisition and preprocessing steps specifically include: Collect raw video data including shoulder flash under different lighting conditions, distances, and complex backgrounds; The original video data is subjected to data augmentation processing, which includes brightness adjustment, contrast adjustment, noise enhancement, and applying a random mask to the lower body region. The positions and on / off states of the shoulder flashlights in the enhanced video data are labeled, and the data is divided into training and validation sets according to a preset ratio. The input video frame size is adjusted to the preset size, and the pixel values ​​are normalized and converted into tensor format.

3. The deep learning-based human shoulder flash light detection and alarm method according to claim 1, characterized in that, In the target detection and shoulder region of interest extraction steps, the target detection model used is based on the YOLOv5 network structure, and the channel and spatial attention module CBAM is integrated into its detection head to enhance the ability to perceive light sources.

4. The deep learning-based human shoulder flash light detection and alarm method according to claim 1, characterized in that, The specific steps for target detection and shoulder region of interest extraction are as follows: The preprocessed video frames are input into the human detection model to obtain the bounding box coordinates and confidence scores of each human body in the image. The same preprocessed video frame is input into the human key point detection model to obtain the coordinates of the left and right shoulder key points corresponding to the human detection box. Based on the coordinates of the left and right shoulder key points, the corresponding shoulder region of interest image is extracted from the original video frame; The shoulder region of interest image is input into the shoulder flash detection model to obtain the detection box coordinates and confidence level of the shoulder flash within the shoulder region.

5. The deep learning-based human shoulder flash light detection and alarm method according to claim 1, characterized in that, Following the target detection and shoulder region of interest extraction steps, a multi-target tracking step is also included: The DeepSORT algorithm is used to track detected human targets, and a unique ID is assigned to each target; Use Kalman filtering to predict the motion state of the tracked target in the next frame; Calculate the correlation matrix between the human detection box in the current frame and the existing tracking trajectory. The correlation matrix integrates the detection box intersection-union ratio cost, appearance feature cost, and motion cost. The Hungarian algorithm is used to associate the detection box with the trajectory, and the corresponding tracking trajectory is updated, initialized or deleted accordingly.

6. The deep learning-based human shoulder flash light detection and alarm method according to claim 1, characterized in that, In the shoulder region of interest sequence analysis step, extracting the brightness change features specifically includes: calculating the brightness difference between adjacent frames in the shoulder region of interest image sequence; extracting the frequency features specifically includes: performing time-frequency analysis on the shoulder region of interest image sequence to statistically obtain the flicker frequency features of the shoulder flashlight.

7. The deep learning-based human shoulder flash light detection and alarm method according to claim 1, characterized in that: The state classification and output steps specifically include: The fused features are compared with a preset threshold to determine whether the shoulder flash is on or off. When it is determined that the shoulder flash light is abnormally off, an alarm message is generated and reported. Following the state classification and output steps, a real-time visualization step is also included: Draw the human body bounding box, tracking ID, and shoulder flash status information on the video frame; Visualize human movement trajectories; Generate and save a CSV format record file containing complete trajectory information; It supports outputting the processing results as MP4 video files or RTSP video streams.

8. A deep learning-based human shoulder flash light detection and alarm device, characterized in that, include: The data acquisition module is used to acquire video streams containing human images in real time and preprocess the video frames; The detection module is used to perform human body detection and human key point detection sequentially through the target detection model based on the preprocessed video frames, and extract the corresponding shoulder region of interest based on the key point coordinates. The information extraction module is used to extract temporal features, brightness variation features, and frequency features from the extracted image sequence of the shoulder region of interest. The judgment module is used to integrate the timing features, brightness change features and frequency features to classify the state of the shoulder flash, determine whether it is on or off, and output the state classification result.

9. An electronic device, characterized in that, include: The memory is used to store the processing program; A processor, which, when executing the processing program, implements the deep learning-based human shoulder flash light detection and alarm method as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that, The readable storage medium stores a processing program, which, when executed by a processor, implements the deep learning-based human shoulder flash light detection and alarm method as described in any one of claims 1 to 7.

Images