Lightweight ShuffleNet-YOLO construction wearing real-time detection method

Abstract

According to the lightweight ShuffleNet-YOLO construction wearing real-time detection method provided by the invention, a lightweight network ShuffleNet v2 and an attention mechanism GAM are fused, so that a model has the characteristic of high detection speed of the lightweight network and also has the advantage of accuracy of the recognition rate of a target detection algorithm, and in order to further improve the prediction precision, a WIoU and Soft-NMS fusion improvement strategy is also adopted, so that the prediction accuracy is improved, and the detection accuracy is improved. A final ShuffleNetv2-YOLOv5s.60 (ShuffleNet-YOLO) network model is constructed, the average precision mAP at50 of the model reaches 88.6%, and the reasoning speed of edge equipment is larger than or equal to 30 FPS. According to the invention, real-time detection of the wearing of the constructor is realized through testing, the safety supervision of the constructor is enhanced, the manual intervention is reduced, the cost is reduced, the method has the advantages of rapid judgment, high precision, convenient deployment and the like, safety accidents are reduced, and the development of an intelligent construction site is promoted.

Description

Technical Field

[0001] The present invention belongs to the technical field of real-time monitoring of smart construction sites, and specifically relates to a lightweight ShuffleNet-YOLO (ShuffleNetv2-YOLOv5s.60) construction wear real-time detection method, which is achieved by integrating the lightweight network ShuffleNetv2, the attention mechanism GAM, the weighted intersection-over-union loss function WIoU, and the flexible non-maximum suppression Soft-NMS optimization model. Background Art

[0002] In the field of smart construction sites, ensuring the safety of construction site personnel has become one of the focuses of core technology applications. Detecting whether construction workers are wearing safety helmets is a key measure to prevent injuries from falling objects and ensure the safety of personnel's heads. At the same time, identifying whether reflective clothing is worn can effectively improve personnel visibility in poorly lit construction environments and avoid accidental injuries. In addition, continuous monitoring of personnel in designated safety hazard prohibited areas is also crucial to prevent unauthorized entry and reduce the risk of accidents. However, most construction sites currently still rely on traditional manual supervision methods to check the wearing of safety helmets, reflective clothing, and personnel entering prohibited areas. This method is not only inefficient, time-consuming and labor-intensive, but also difficult to achieve all-weather, real-time monitoring without blind spots. Especially in large or complex construction projects, it is difficult for managers to cover all corners with their line of sight, which can easily lead to loopholes in safety supervision.

[0003] To address this issue, smart construction sites urgently need to adopt more advanced and efficient technologies. The application and optimization of construction safety datasets is particularly crucial. Leveraging AI technologies such as deep learning, real-time analysis of surveillance video can automatically identify construction workers' safety gear and activities in restricted areas, enabling intelligent safety oversight. However, achieving this goal requires overcoming multiple challenges. First, the complex and ever-changing construction site environment, where factors such as lighting and obstructions can affect recognition, requires algorithms with strong adaptability and robustness. Second, the diversity and quality of the dataset directly impact the model's training effectiveness and generalization capabilities. High-quality datasets covering a wide range of construction scenarios, lighting conditions, and personnel status are essential. Furthermore, the real-time performance of the model is crucial, especially when deployed on resource-constrained devices such as mobile devices. Models must be streamlined and optimized to ensure rapid response and accurate recognition.

[0004] In recent years, deep learning technology, particularly innovations based on convolutional neural networks (CNNs), has injected unprecedented vitality into the intelligent analysis of construction worker safety datasets. While early models such as R-CNN, SPP-Net, and Fast R-CNN achieved significant progress in improving recognition accuracy, their limited detection speed made them unable to meet the urgent need for real-time monitoring on construction sites, thus limiting their practical application. A turning point came with the emergence of the YOLO (You Only Look Once) family of algorithms, particularly the release of YOLOv1 in 2015, which completely overturned the speed bottleneck of traditional object detection. Through its unique single-pass forward propagation mechanism, YOLO enables real-time and dynamic monitoring of safety hazards on construction sites, significantly improving detection efficiency. Even more impressively, YOLO excels in reducing background false detection rates to approximately half that of Fast R-CNN, while also demonstrating strong generalization capabilities, laying a solid foundation for the model's flexible application in diverse construction scenarios and even across domains. As the YOLO series continues to evolve, subsequent versions have been deeply optimized to address initial issues such as insufficient positioning accuracy and challenges in detecting small objects, further consolidating its leading position in the field of intelligent construction safety and protection. These improvements not only enhance detection accuracy but also strengthen the model's robustness and practicality.

[0005] At the same time, researchers have developed lightweight network architectures such as SqueezeNet, Xception, MobileNet, and ShuffleNet to meet the diverse needs of model deployment, particularly for hardware-constrained PC and mobile devices. These networks are ingeniously designed to significantly reduce model size and accelerate detection while maintaining high recognition accuracy, providing strong technical support for the widespread deployment of construction safety protection systems. This trend not only advances the intelligentization of construction safety protection but also opens up broad opportunities for the application of deep learning technology in a wider range of practical scenarios.

[0006] In summary, this research focuses on the following areas: First, we will strengthen the construction and sharing of datasets. By collecting diverse construction scenario data, we will improve the adaptability and generalization capabilities of the model. To this end, we have constructed datasets for hard hats, reflective clothing, and construction workers that are suitable for industrial scenarios. Second, we will continuously optimize the algorithm architecture to improve recognition accuracy and speed to meet real-time requirements. Third, we will explore model compression and deployment technologies to enable deep learning models to run efficiently on resource-constrained devices such as mobile devices. To address these issues, we have integrated a lightweight ShuffleNetV2 network into the YOLOv5 algorithm to replace the backbone network and added a GAM attention mechanism to avoid significant precision degradation, enhance the capture of target semantic information and feature expression capabilities, and improve detection accuracy. Fourth, we will promote the deep integration of technology with actual construction processes to achieve safety supervision and optimization of the entire smart construction site chain. Specifically, we will conduct extensive experiments to verify the effectiveness of the improved algorithm proposed in this paper. Through these efforts, we will promote the development of smart construction sites towards greater intelligence, efficiency, and safety. Summary of the Invention

[0007] Based on the above problems, the purpose of the present invention is to provide a lightweight ShuffleNet-YOLO (Shufflenetv2-YOLOv5s.60) real-time detection method for construction wear, which can accurately and efficiently identify detection targets and achieve the purpose of real-time dynamic monitoring.

[0008] A lightweight ShuffleNet-YOLO real-time detection method for construction wear, including:

[0009] Step 1: Collect image data of different types of safety helmets, reflective vests, and construction workers;

[0010] Step 2: Preprocess the collected data to construct a dataset of construction workers’ wearables; the preprocessing includes rotation, scaling, cropping, occlusion, color change, and data annotation;

[0011] Step 3: Improve the target detection algorithm model of YOLOv5s.60 ​​based on the lightweight network ShuffleNetv2, attention mechanism GAM, weighted intersection over Union loss function (WIoU), and soft non-maximum suppression (Soft-NMS), and build a new target detection network model ShuffleNetv2-YOLOv5s.60;

[0012] Step 4: Use the preprocessed data set to train the ShuffleNetv2-YOLOv5s.60 ​​network model and continuously optimize the parameters to achieve the best detection effect;

[0013] Step 5: Deploy the model on the video surveillance end to test whether it can identify construction workers, safety helmets, and reflective clothing;

[0014] The specific structure of the target detection network model ShuffleNetv2-YOLOv5s.60 ​​in step 3 is as follows: first, the feature extraction backbone network (Backbone) in the YOLOv5s.60 ​​model is replaced as a whole by the CBRM, Shuffle_block module and attention mechanism GAM in the lightweight network ShuffleNetv2; then, GAM is added between the neck network (Neck) and the prediction end (Prediction) in the YOLOv5s.60 ​​model; finally, the weighted intersection over Union loss function (Weighted Intersection over Union, WIoU) and flexible non-maximum suppression (Soft Non-Maximum Suppression, Soft-NMS) are used at the prediction end in the YOLOv5s.60 ​​model to replace the complete loss function (Complete IoU Loss, CIoU) and non-maximum suppression (NMS) in YOLOv5s.60.

[0015] The step 4 comprises:

[0016] Step 4.1: Use the preprocessed image dataset as the input of the improved ShuffleNetv2-YOLOv5s.60 ​​model, and perform mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling in sequence;

[0017] Step 4.2: In the feature extraction backbone network (Backbone), features are extracted through the CBRM module, {Shuffle_block*2, GAM}*3 module in sequence;

[0018] Step 4.3: In the neck module, a combination of Feature Pyramid Network (FPN) and Path Aggregation Network (PAN) is used to perform multi-scale fusion of different feature maps;

[0019] Step 4.4: Extract image features through the GAM module, that is, the channel attention submodule and the spatial attention submodule;

[0020] Step 4.5: Use WIoU to calculate the score of the target proposal box;

[0021] Step 4.6: Use Soft-NMS to suppress the suggestion boxes smaller than the threshold and retain the suggestion boxes larger than the threshold;

[0022] Step 4.7: When the maximum number of training rounds is reached, display indicators such as accuracy, adjust training parameters, and repeat the above steps until the optimal value is reached.

[0023] The beneficial effects of the present invention are:

[0024] 1: This paper constructs a dataset of construction workers wearing smart construction site equipment, which includes not only the recognition and anti-recognition of safety helmets and reflective clothing, but also the detection of construction workers, with a total of five major categories;

[0025] 2: This paper constructs a lightweight safety wear detection model, integrates the lightweight network ShuffleNetv2 and the attention mechanism GAM, and adds them to the target detection algorithm YOLOv5s.60. It also uses WIoU and Soft-NMS fusion to improve the model prediction accuracy, so that the overall model has the advantages of fast detection speed of lightweight network and recognition rate accuracy of target detection algorithm;

[0026] 3: The present invention realizes real-time automatic detection of construction workers' on-site safety wear, which helps to strengthen the safety prevention and control of smart construction sites, reduce costs, has the advantage of good universality, and promotes the construction of smart construction sites. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of a lightweight ShuffleNet-YOLO (ShuffleNetv2-YOLOv5s.60) construction wear real-time detection method in the present invention;

[0028] Figure 2 The target detection data in the present invention;

[0029] Among them, (a) is the data image of different safety helmets, (b) is the data image of different reflective clothing, and (c) is the data image of different construction workers in different states;

[0030] Figure 3 Preprocessing of target detection data in the present invention;

[0031] Among them, (a) is rotation, (b) is scaling, (c) is cropping, and (d) is occlusion;

[0032] Figure 4 It is the annotation of the data set in this invention;

[0033] Among them, (a) is the initial interface screenshot, (b) is the new project screenshot, (c) is the dataset annotation screenshot, and (d) is the save and export screenshot;

[0034] Figure 5 This is the YOLOv5s.60 ​​network structure diagram in the present invention;

[0035] Figure 6 This is the network unit structure diagram of the lightweight network ShuffleNetv2 in the present invention:

[0036] Figure 7 Schematic diagram of the GAM structure of the attention mechanism in the present invention;

[0037] Figure 8 This is the improved network structure diagram of the present invention;

[0038] Figure 9 This is the final model training result diagram in the present invention;

[0039] Among them, (a) confusion matrix (b) label information (c) the relationship between the horizontal and vertical coordinates of the center point and the height and width of the box (d) visualization of training results analysis (e) precision P value, (f) mean average precision mAP

[0040] Figure 10 Designed for monitoring interface;

[0041] Among them, (a) image recognition function interface, (b) camera / video monitoring function interface;

[0042] Figure 11 This is a diagram showing the effect of real-time detection of construction workers wearing the device in the present invention;

[0043] Among them, (a) is a schematic diagram of automatic image detection, and (b) is a schematic diagram of real-time camera monitoring. DETAILED DESCRIPTION

[0044] The invention will be further described below with reference to the accompanying drawings and specific implementation examples.

[0045] like Figure 1 As shown in FIG, a lightweight ShuffleNet-YOLO (ShuffleNetv2-YOLOv5s.60) construction wear real-time detection method includes the following steps:

[0046] Step 1: Collect different image data of helmets, reflective clothing, and people walking;

[0047] In this implementation, the data set includes three parts of data: whether the helmet is worn, whether the reflective clothing is worn, and the number of construction personnel. Figure 2 .

[0048] Step 2: Preprocess the collected data to construct a dataset of construction workers’ wearable devices; the preprocessing includes rotation, scaling, cropping, occlusion, and data annotation;

[0049] The following five random combination operations are performed on the wearing and not wearing of helmets (hat, nohat), whether wearing reflective clothing (Refrelective-clothse, other-clothse), and personnel (Person). Through rotation, scaling, cropping, and blocking, see Figure 3 ; Dataset annotation: labelimg is an open source image annotation tool, see Figure 4 .

[0050] Step 3: Improve the target detection algorithm model of YOLOv5s.60 ​​based on the lightweight network ShuffleNetv2, attention mechanism GAM, weighted intersection over Union loss function (WIoU), and soft non-maximum suppression (Soft-NMS), and build a new target detection network model ShuffleNetv2-YOLOv5s.60;

[0051] YOLOv5s.60 ​​model, including input, backbone, neck, and prediction output, see Figure 5 . Use the CBAM and Shuffle_block structures in the lightweight network ShuffleNetv2, see Figure 6 And the attention mechanism GAM, see Figure 7 Replace the feature extraction backbone network (Backbone) in the YOLOv5s.60 ​​model, add GAM between the neck network (Neck) and the output end (Prediction) of predicting small, medium, and large targets to improve feature extraction, and finally, replace the complete loss function (Complete IoU Loss, CIoU) and non-maximum suppression (NMS) in YOLOv5s.60 ​​with WIoU and Soft-NMS on the prediction end. The improved network structure is shown in the figure. Figure 8 .

[0052] Step 4: Use the preprocessed dataset to train the ShuffleNetv2-YOLOv5s.60 ​​network model; including:

[0053] The step 4 comprises:

[0054] Step 4.1: Use the preprocessed image dataset as the input of the improved ShuffleNetv2-YOLOv5s.60 ​​model, and perform mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling in sequence;

[0055] Step 4.2: In the feature extraction backbone network (Backbone), features are extracted through the CBRM module, {Shuffle_block*2, GAM}*3 module in sequence;

[0056] Step 4.3: In the neck module, a combination of Feature Pyramid Network (FPN) and Path Aggregation Network (PAN) is used to perform multi-scale fusion of different feature maps;

[0057] Step 4.4: Extract image features through the GAM module, that is, the channel attention submodule and the spatial attention submodule;

[0058] Step 4.5: Use WIoU to calculate the score of the target proposal box;

[0059] Step 4.6: Use Soft-NMS to suppress the suggestion boxes smaller than the threshold and retain the suggestion boxes larger than the threshold;

[0060] Step 4.7: When the maximum number of training rounds is reached, display the accuracy and other indicators, adjust the training parameters, and repeat the above steps until the optimal value is reached. The final model training results are shown in Figure 9 .

[0061] Step 5: Deploy the model on the video surveillance end to test the target;

[0062] Finally, the model is deployed on the computer side, and users can upload videos or pictures for detection; including: interface design, see Figure 10 As shown. Real-time detection, see Figure 11 ,where, (a) is a schematic diagram of automatic image detection, and (b) is a schematic diagram of real-time camera monitoring.

Claims

1. A lightweight ShuffleNet-YOLO (ShuffleNetv2-YOLOv5s.60) construction wear real-time detection method, characterized by: include: Step 1: Collect image data of different helmets, reflective vests, and construction workers; Step 2: Preprocess the collected data on safety helmets, reflective clothing, and construction workers to build a dataset of construction workers wearing equipment on smart construction sites. Step 3: Improve the target detection algorithm model of YOLOv5s.60 ​​based on the lightweight network ShuffleNetv2, attention mechanism GAM, weighted intersection over Union loss function (WIoU), and soft non-maximum suppression (Soft-NMS), and build a new target detection network model ShuffleNetv2-YOLOv5s.60; Step 4: Use the preprocessed dataset to train the ShuffleNetv2-YOLOv5s.60 ​​network model; Step 5: Deploy the model on the video surveillance end to test whether the target is detected.

2. A lightweight ShuffleNet-YOLO construction wear real-time detection method according to claim 1, characterized in that: The preprocessing includes rotation, scaling, cropping, occlusion, color shading change and data labeling.

3. A lightweight ShuffleNet-YOLO construction wear real-time detection method according to claim 1, characterized in that: The specific structure of the target detection network model YOLOv5s.60-ShuffleNetv2 in step 3 is as follows: first, the feature extraction backbone network (Backbone) in the YOLOv5s.60 ​​model is replaced as a whole by the CBRM, Shuffle_block module and attention mechanism GAM in the lightweight network ShuffleNetv2; then, GAM is added between the neck network (Neck) and the prediction end (Prediction) in the YOLOv5s.60 ​​model; finally, the weighted intersection over Union loss function (Weighted Intersection over Union, WIoU) and flexible non-maximum suppression (Soft Non-Maximum Suppression, Soft-NMS) are used to replace the complete loss function (Complete IoU Loss, CIoU) and non-maximum suppression (NMS) in YOLOv5s.60 ​​at the prediction end in the YOLOv5s.60 ​​model.

4. A lightweight ShuffleNet-YOLO construction wear real-time detection method according to claim 1, characterized in that: The step 4 comprises: Step 4.1: Use the preprocessed image dataset as the input of the improved ShuffleNet-YOLO model, and perform mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling in sequence; Step 4.2: In the feature extraction backbone network Backbone, pass through the CBRM module, {Shuffle_block*2, GAM}*3 modules are used to extract features; Step 4.3: In the neck network Neck module, a combination of feature pyramid FPN and path aggregation network PAN is used to perform multi-scale fusion of different feature maps; Step 4.4: Extract image features through the GAM module, that is, the channel attention submodule and the spatial attention submodule; Step 4.5: Use WIoU to calculate the score of the target proposal box; Step 4.6: Use Soft-NMS to suppress the suggestion boxes smaller than the threshold and retain the suggestion boxes larger than the threshold; Step 4.7: When the maximum number of training rounds is reached, display indicators such as accuracy, adjust training parameters, and repeat the above steps until the optimal value is reached.

Images