SlowFast-based high-altitude operation site safety belt detection method
By using a safety belt detection method for high-altitude operations based on the SlowFast framework and employing a SlowFast dual-stream neural network and an improved YOLOv8 target detection model, the existing technologies are able to distinguish between workers and non-workers and lack dynamic perception. This method achieves accurate safety belt detection at high-altitude work sites, improving the reliability and safety of the system.
Patent Information
- Application Number
- CN202511832431.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-08
- Publication Date
- 2026-02-24
AI Technical Summary
Existing safety belt detection technology for high-altitude operations cannot effectively distinguish between workers and non-workers, and lacks the ability to dynamically perceive work behavior, resulting in frequent false alarms and reducing the reliability and practicality of the system.
A safety belt detection method for high-altitude operations based on the SlowFast framework is adopted. The method distinguishes the personnel status through behavior detection, and uses the SlowFast dual-stream neural network and the improved YOLOv8 target detection model, combined with the convolutional attention module and Wise-IoU-v3 loss function, to achieve accurate safety belt detection for high-altitude workers.
It enables precise detection of safety belts for workers at heights, reduces false alarms, improves the accuracy of safety belt detection and the reliability of the system, and ensures the safety of working at heights.
Smart Images
Figure CN121564658A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of power supervision technology, specifically relating to a method for detecting safety belts at high-altitude work sites based on SlowFast. Background Technology
[0002] High-altitude power operations are a core component of current power transmission, transformation, and distribution work. Failure to wear safety belts or improper wearing of them can easily lead to falls, seriously threatening personnel lives. In the existing regulatory system, manual inspections and video surveillance are the main methods, with visual tools such as surveillance cameras and fixed cameras widely used for on-site safety monitoring. Currently, the AI algorithms integrated into the equipment of State Grid Hunan Electric Power Co., Ltd. are mostly based on target detection technology, enabling the recognition of static or instantaneous behaviors such as wearing safety helmets, smoking, and using mobile phones. Some equipment also supports the detection of protective equipment such as safety belts, work clothes, and insulated gloves.
[0003] However, existing seatbelt recognition schemes based on object detection have significant limitations, mainly in the following aspects:
[0004] 1. Inability to effectively distinguish between workers and non-workers. In complex actual work sites, in addition to workers at heights, there are usually roles such as "work supervisors" and "safety monitors" on the ground. Simple target detection models will treat all detected personnel as warning targets, causing the system to issue false alarms for "not wearing safety belts" to ground-based monitoring personnel who do not need to wear safety belts.
[0005] Second, there is a lack of dynamic perception capabilities for work-related behaviors. Existing methods can only determine the presence of a safety belt in a single frame of an image, but cannot correlate this with the personnel's behavioral state. Frequent false alarms not only severely interfere with effective on-site safety supervision, but also reduce the reliability and practicality of the early warning system, failing to provide accurate reminders for the core violation of workers not wearing safety belts at heights.
[0006] Therefore, the industry urgently needs a technology that can intelligently distinguish the working status, so as to accurately lock the safety belt detection target onto the personnel who are actually working at height. Summary of the Invention
[0007] To address the shortcomings of existing technologies, the present invention aims to provide a high-altitude work site safety belt detection method based on SlowFast. This method improves upon the SlowFast framework by using behavior detection to distinguish personnel status, thereby achieving accurate safety belt detection.
[0008] This invention provides a method for detecting safety belts at high-altitude work sites based on SlowFast, comprising the following steps:
[0009] S1. Collect monitoring videos of high-altitude operations to obtain a video dataset;
[0010] S2. Extract video clips of personnel wearing safety belts and those not wearing safety belts during high-altitude operations from the video dataset, label the personnel not wearing safety belts, and obtain the training dataset;
[0011] S3. Construct an initial seatbelt detection model based on the SlowFast dual-stream neural network behavior recognition model;
[0012] S4. Train the initial seat belt detection model to obtain the seat belt monitoring model;
[0013] S5. Use a safety belt monitoring model to detect violations in surveillance videos of high-altitude work sites.
[0014] Step S1 includes the following steps:
[0015] The system collects surveillance videos of workers performing high-altitude operations; the surveillance videos include segments showing both workers wearing safety belts and workers not wearing safety belts during the high-altitude operations.
[0016] The original video was cleaned and preprocessed to remove invalid segments, resulting in a video dataset.
[0017] Step S2 includes the following steps:
[0018] Extract key climbing actions from the video dataset; the high-altitude operation sites include substations, power line towers, etc.
[0019] The video clips were labeled frame by frame to determine whether the workers were wearing safety belts. The labeling information included the human body bounding box, the presence of the safety belt, and the location of key points.
[0020] Store the labeled data in COCO format to construct the labeled dataset;
[0021] The training dataset is composed of video clips corresponding to key actions of working at height in various scenarios and whether or not a safety belt is worn.
[0022] In step S3, based on the standard SlowFast two-stream neural network behavior recognition model, the target detection module Detectron2 is replaced with the seat belt detection module to construct an initial seat belt detection model; the seat belt detection module is specifically based on the YOLOv8 target detection model, with a convolutional attention module introduced into its backbone network, and the bounding box regression loss function is modified to Wise-IoU-v3.
[0023] The initial safety belt detection model first processes the input video clip data into the safety belt detection module to obtain the position information of the worker and the safety belt, the safety belt detection confidence score, and then crops out the image region corresponding to the worker and inputs it into the SlowFast dual-stream neural network. The SlowFast dual-stream neural network extracts spatiotemporal features from the input data to identify the worker's behavior patterns and outputs the action classification probability. Finally, multimodal fusion judgment is performed based on the safety belt detection confidence score and the action classification probability to obtain the safety belt detection result at the high-altitude work site.
[0024] The seatbelt detection module includes a backbone network, a neck network, and a detection head module connected in sequence; the structure of the neck network and the detection head module is the same as that of the YOLOv8 target detection model.
[0025] A Convolutional Attention Module (CBAM) is introduced into the backbone network; the backbone network consists of a first CBS module, a second CBS module, a first C2f module, a third CBS module, a second C2f module, a first CBAM module, a fourth CBS module, a third C2f module, a second CBAM module, a fifth CBS module, a fourth C2f module, and an SPPF module, which are connected in series.
[0026] The first and second CBAM modules have the same structure, including a spatial attention submodule and a channel attention submodule; the spatial attention submodule performs max pooling and average pooling on the input feature map F in the channel dimension, respectively, to obtain two... The feature descriptions are then concatenated along the channel dimension. Convolution and Sigmoid activation function generate spatial attention weights The channel attention submodule performs global average pooling and max pooling on the feature map F in the spatial dimension, respectively, to obtain two... The feature vectors are then used to generate channel attention weights via MLP and Sigmoid activation functions. Finally, the spatial attention weights output by the spatial attention submodule and the channel attention submodule are compared. With channel attention weights The core network internal network layer output weights are merged and updated.
[0027] In step S4, the seat belt detection module is first trained using the training dataset obtained in step S2; then, based on the open-source database, the initial seat belt detection model is trained, and the dual-stream path structure is used to capture human movements in the video, so that the initial seat belt detection model can recognize the behavioral information of high-altitude workers.
[0028] The action information includes "stand", "bend", "walk", "jump", "carry", "hold", and "climb".
[0029] 1. In the training of the seatbelt detection module, the bounding box regression loss function was modified to Wise-IoU-v3. First, the IoU value and the moving average were calculated. The ratio of the two values gives the outlier degree. Then, a focusing mechanism is constructed based on outlier degree, expressed using the following formula: ;in, This is the first hyperparameter; This is the second hyperparameter; This is a dynamic scaling factor;
[0030] Finally, the Wise-IoU-v3 loss function is calculated. Use the following formula to calculate: ;in, It is a dynamic weighting factor, and ; This is the base loss that includes the distance attention mechanism;
[0031] The specific training parameters are as follows: the SGD optimizer is used, the initial learning rate is set to 0.01, and the cosine annealing scheduler is used; the momentum is set to 0.937, the weight decay is 0.0005; the batch size is set to 16; the model is trained for a total of 300 epochs, and a validation set is divided in the training dataset. The optimal weights are selected from the validation set based on the average accuracy (mAP@0.5).
[0032] In the training of the initial seat belt detection model, the sampling time step of the Slow path frame is set to 16, the sampling time step of the Fast path frame is set to 2, and the number of channels used in the Fast path is 1 / 8 of that in the Slow path.
[0033] Step S5 includes the following steps:
[0034] Cameras are installed at the high-altitude work site, covering the entire work area without obstruction for the workers;
[0035] The video clips captured by the camera are input into the seat belt detection model to obtain the seat belt detection confidence, seat belt detection frame, worker detection frame, and worker's action information.
[0036] Based on the preset judgment strategy, the safety belt detection at the high-altitude work site is completed using the safety belt detection confidence level, the safety belt detection frame, the worker detection frame, and the worker's action information.
[0037] The preset judgment strategy is as follows: when the confidence level of the safety belt detection is greater than 0.7, the center coordinates of the safety belt detection frame are within the range of the operator's detection frame, and the action information is "hold" or "climb", the operation is judged to be compliant; if the confidence level of the safety belt detection is not greater than 0.7 or the action information is abnormal, an early warning mechanism is triggered, and alarm information is pushed to the supervision platform in real time to ensure timely intervention of risks.
[0038] This invention discloses a safety belt detection method for high-altitude operations based on SlowFast. It improves upon the SlowFast framework by using behavior detection to distinguish personnel status, thereby achieving accurate safety belt detection. Attached Figure Description
[0039] Figure 1 This is a schematic flowchart of the method of the present invention;
[0040] Figure 2 This is a schematic diagram of the seat belt detection module in an embodiment of the present invention;
[0041] Figure 3 This is a schematic diagram of the results obtained by model detection in an embodiment of the present invention. Detailed Implementation
[0042] This invention provides a method for detecting safety belts at high-altitude work sites based on SlowFast, the flowchart of which is shown below. Figure 1 As shown, it includes the following steps:
[0043] S1. Collect monitoring videos of high-altitude operations to obtain a video dataset;
[0044] Step S1 includes the following steps:
[0045] The system collects surveillance videos of workers performing high-altitude operations; the surveillance videos include segments showing both workers wearing safety belts and workers not wearing safety belts during the high-altitude operations.
[0046] The original video was cleaned and preprocessed to remove invalid segments, resulting in a video dataset.
[0047] By using surveillance videos with different weather conditions, time periods, and perspectives, the generalization ability of subsequent models can be increased.
[0048] S2. Extract video clips of personnel wearing safety belts and those not wearing safety belts during high-altitude operations from the video dataset, label the personnel not wearing safety belts, and obtain the training dataset;
[0049] Step S2 includes the following steps:
[0050] Extract key climbing actions from the video dataset; the high-altitude operation sites include substations, power line towers, etc.
[0051] The video clips were labeled frame by frame to determine whether the workers were wearing safety belts. The labeling information included the human body bounding box, the presence of the safety belt, and the location of key points.
[0052] Store the labeled data in COCO format to construct the labeled dataset;
[0053] The training dataset is composed of video clips corresponding to key actions of working at height in various scenarios and whether or not a safety belt is worn.
[0054] S3. Construct an initial seatbelt detection model based on the SlowFast dual-stream neural network behavior recognition model;
[0055] In step S3, based on the standard SlowFast two-stream neural network behavior recognition model, the target detection module Detectron2 is replaced with a seatbelt detection module to construct an initial seatbelt detection model. Specifically, the seatbelt detection module is based on the YOLOv8 target detection model, with a convolutional attention module introduced into its backbone network. Simultaneously, the bounding box regression loss function is modified to Wise-IoU-v3. Its structural diagram is shown below. Figure 2 As shown;
[0056] The initial safety belt detection model first processes the input video clip data into the safety belt detection module to obtain the position information of the worker and the safety belt, the safety belt detection confidence score, and then crops out the image region corresponding to the worker and inputs it into the SlowFast dual-stream neural network. The SlowFast dual-stream neural network extracts spatiotemporal features from the input data to identify the worker's behavior patterns and outputs the action classification probability. Finally, multimodal fusion judgment is performed based on the safety belt detection confidence score and the action classification probability to obtain the safety belt detection result at the high-altitude work site.
[0057] The seatbelt detection module includes a backbone network, a neck network, and a detection head module connected in sequence; the structure of the neck network and the detection head module is the same as that of the YOLOv8 target detection model.
[0058] A Convolutional Attention Module (CBAM) is introduced into the backbone network; the backbone network consists of a first CBS module, a second CBS module, a first C2f module, a third CBS module, a second C2f module, a first CBAM module, a fourth CBS module, a third C2f module, a second CBAM module, a fifth CBS module, a fourth C2f module, and an SPPF module, which are connected in series.
[0059] The first and second CBAM modules have the same structure, including a spatial attention submodule and a channel attention submodule; the spatial attention submodule performs max pooling and average pooling on the input feature map F in the channel dimension, respectively, to obtain two... The feature descriptions are then concatenated along the channel dimension. Convolution and Sigmoid activation function generate spatial attention weights The channel attention submodule performs global average pooling and max pooling on the feature map F in the spatial dimension, respectively, to obtain two... The feature vectors are then used to generate channel attention weights via MLP and Sigmoid activation functions. Finally, the spatial attention weights output by the spatial attention submodule and the channel attention submodule are compared. With channel attention weights By merging the data and updating the output weights of the internal network layers of the backbone network, the model's ability to detect small targets such as seat belts is enhanced.
[0060] S4. Train the initial seat belt detection model to obtain the seat belt monitoring model;
[0061] In step S4, the seat belt detection module is first trained using the training dataset obtained in step S2; then, based on the open-source database, the initial seat belt detection model is trained, and the dual-stream path structure is used to capture human movements in the video, so that the initial seat belt detection model can recognize the behavioral information of high-altitude workers.
[0062] The action information includes "stand", "bend", "walk", "jump", "carry", "hold", and "climb".
[0063] In the training of the seatbelt detection module, the bounding box regression loss function was modified to Wise-IoU-v3. First, the IoU value and the moving average were calculated. The ratio of the two values gives the outlier degree. Then, a focusing mechanism is constructed based on outlier degree, expressed using the following formula: ;in, This is the first hyperparameter; This is the second hyperparameter; This is a dynamic scaling factor;
[0064] Finally, the Wise-IoU-v3 loss function is calculated. Use the following formula to calculate: ;in, It is a dynamic weighting factor, and ; The base loss incorporates a distance attention mechanism; through a dynamic non-monotonic focusing mechanism, it effectively solves the gradient imbalance problem between high-quality and low-quality anchor boxes during training, and effectively improves the localization accuracy of small targets.
[0065] The specific training parameters are as follows: the SGD optimizer is used, the initial learning rate is set to 0.01, and the cosine annealing scheduler is used; the momentum is set to 0.937, the weight decay is 0.0005; the batch size is set to 16; the model is trained for a total of 300 epochs, and a validation set is divided in the training dataset. The optimal weights are selected from the validation set based on the average accuracy (mAP@0.5).
[0066] In the training of the initial seat belt detection model, the sampling time step of the Slow path frame is set to 16, the sampling time step of the Fast path frame is set to 2, and the number of channels used in the Fast path is 1 / 8 of that in the Slow path.
[0067] S5. Use a safety belt monitoring model to detect violations in surveillance videos of high-altitude work sites.
[0068] Step S5 includes the following steps:
[0069] Cameras are installed at the high-altitude work site, covering the entire work area without obstruction for the workers;
[0070] The video clips captured by the camera are input into the seat belt detection model to obtain the seat belt detection confidence score, seat belt detection frame, worker detection frame, and worker action information. A schematic diagram of the results is shown below. Figure 3 As shown;
[0071] Based on the preset judgment strategy, the safety belt detection at the high-altitude work site is completed using the safety belt detection confidence level, the safety belt detection frame, the worker detection frame, and the worker's action information.
[0072] The preset judgment strategy is as follows: when the confidence level of the safety belt detection is greater than 0.7, the center coordinates of the safety belt detection frame are within the range of the operator's detection frame, and the action information is "hold" or "climb", the operation is judged to be compliant; if the confidence level of the safety belt detection is not greater than 0.7 or the action information is abnormal, an early warning mechanism is triggered, and alarm information is pushed to the supervision platform in real time to ensure timely intervention of risks.
Claims
1. A method for detecting safety belts at high-altitude work sites based on SlowFast, characterized in that, Includes the following steps: S1. Collect monitoring videos of high-altitude operations to obtain a video dataset; S2. Extract video clips of personnel wearing safety belts and those not wearing safety belts during high-altitude operations from the video dataset, label the personnel not wearing safety belts, and obtain the training dataset; S3. Construct an initial seatbelt detection model based on the SlowFast dual-stream neural network behavior recognition model; S4. Train the initial seat belt detection model to obtain the seat belt monitoring model; S5. Use a safety belt monitoring model to detect violations in surveillance videos of high-altitude work sites.
2. The method for detecting safety belts at high-altitude work sites based on SlowFast as described in claim 1, characterized in that, Step S1 includes the following steps: The system collects surveillance videos of workers performing high-altitude operations; the surveillance videos include segments showing both workers wearing safety belts and workers not wearing safety belts during the high-altitude operations. The original video was cleaned and preprocessed to remove invalid segments, resulting in a video dataset.
3. The method for detecting safety belts at high-altitude work sites based on SlowFast according to claim 1, characterized in that, Step S2 includes the following steps: Extract key climbing actions from the video dataset; the high-altitude operation sites include substations and power line towers; The video clips were labeled frame by frame to determine whether the workers were wearing safety belts. The labeling information included the human body bounding box, the presence of the safety belt, and the location of key points. Store the labeled data in COCO format to construct the labeled dataset; The training dataset is composed of video clips corresponding to key actions of working at height in various scenarios and whether or not a safety belt is worn.
4. The method for detecting safety belts at high-altitude work sites based on SlowFast as described in claim 1, characterized in that, In step S3, based on the standard SlowFast two-stream neural network behavior recognition model, the target detection module Detectron2 is replaced with the seat belt detection module to construct an initial seat belt detection model; the seat belt detection module is specifically based on the YOLOv8 target detection model, with a convolutional attention module introduced into its backbone network, and the bounding box regression loss function is modified to Wise-IoU-v3. The initial safety belt detection model first processes the input video clip data into the safety belt detection module to obtain the position information of the worker and the safety belt, the safety belt detection confidence score, and then crops out the image region corresponding to the worker and inputs it into the SlowFast dual-stream neural network. The SlowFast dual-stream neural network extracts spatiotemporal features from the input data to identify the worker's behavior patterns and outputs the action classification probability. Finally, multimodal fusion judgment is performed based on the safety belt detection confidence score and the action classification probability to obtain the safety belt detection result at the high-altitude work site.
5. The method for detecting safety belts at high-altitude work sites based on SlowFast according to claim 4, characterized in that, The seatbelt detection module includes a backbone network, a neck network, and a detection head module connected in sequence; the structure of the neck network and the detection head module is the same as that of the YOLOv8 target detection model. A Convolutional Attention Module (CBAM) is introduced into the backbone network; the backbone network consists of a first CBS module, a second CBS module, a first C2f module, a third CBS module, a second C2f module, a first CBAM module, a fourth CBS module, a third C2f module, a second CBAM module, a fifth CBS module, a fourth C2f module, and an SPPF module, which are connected in series. The seat belt detection module performs feature extraction and target detection on the input image to obtain the seat belt detection confidence score, the seat belt detection box, and the worker detection box.
6. The method for detecting safety belts at high-altitude work sites based on SlowFast according to claim 5, characterized in that, The first and second CBAM modules have the same structure, including a spatial attention submodule and a channel attention submodule; the spatial attention submodule performs max pooling and average pooling on the input feature map F in the channel dimension, respectively, to obtain two... The feature descriptions are then concatenated along the channel dimension. Convolution and Sigmoid activation function generate spatial attention weights The channel attention submodule performs global average pooling and max pooling on the feature map F in the spatial dimension, respectively, to obtain two... The feature vectors are then used to generate channel attention weights via MLP and Sigmoid activation functions. Finally, the spatial attention weights output by the spatial attention submodule and the channel attention submodule are compared. With channel attention weights The core network internal network layer output weights are merged and updated.
7. The method for detecting safety belts at high-altitude work sites based on SlowFast according to claim 1, characterized in that, In step S4, the seat belt detection module is first trained using the training dataset obtained in step S2; then, an initial seat belt detection model is trained based on an open-source database, and human movements in the video are captured using a dual-stream path structure, so that the initial seat belt detection model can identify the behavior information of high-altitude workers; the action information includes "stand", "bend", "walk", "jump", "carry", "hold" and "climb".
8. The method for detecting safety belts at high-altitude work sites based on SlowFast according to claim 7, characterized in that, In the training of the seatbelt detection module, the bounding box regression loss function was modified to Wise-IoU-v3, specifically: first, the IoU value and the moving average were calculated. The ratio of the two values gives the outlier degree. ; Then, a focusing mechanism is constructed based on outlier degree, expressed by the following formula: ;in, This is the first hyperparameter; This is the second hyperparameter; This is a dynamic scaling factor; Finally, the Wise-IoU-v3 loss function is calculated. Use the following formula to calculate: ;in, It is a dynamic weighting factor, and ; This is the base loss that includes the distance attention mechanism; The specific training parameters are as follows: the SGD optimizer is used, the initial learning rate is set to 0.01, and the cosine annealing scheduler is used; the momentum is set to 0.937, the weight decay is 0.0005; the batch size is set to 16; the model is trained for a total of 300 epochs, and a validation set is divided in the training dataset. The best weights are selected from the validation set based on the average accuracy. In the training of the initial seat belt detection model, the sampling time step of the Slow path frame is set to 16, the sampling time step of the Fast path frame is set to 2, and the number of channels used in the Fast path is 1 / 8 of that in the Slow path.
9. The method for detecting safety belts at high-altitude work sites based on SlowFast according to claim 1, characterized in that, Step S5 includes the following steps: Cameras are installed at the high-altitude work site, covering the entire work area without obstruction for the workers; The video clips captured by the camera are input into the seat belt detection model to obtain the seat belt detection confidence, seat belt detection frame, worker detection frame, and worker's action information. Based on the preset judgment strategy, the safety belt detection at the high-altitude work site is completed using the safety belt detection confidence level, the safety belt detection frame, the worker detection frame, and the worker's action information.
10. The method for detecting safety belts at high-altitude work sites based on SlowFast according to claim 9, characterized in that, The preset judgment strategy is as follows: when the confidence level of the safety belt detection is greater than 0.7, the center coordinates of the safety belt detection frame are within the range of the operator's detection frame, and the action information is "hold" or "climb", the operation is judged to be compliant; if the confidence level of the safety belt detection is not greater than 0.7 or the action information is abnormal, an early warning mechanism is triggered, and alarm information is pushed to the supervision platform in real time to ensure timely intervention of risks.