A construction site safety helmet detection method, computer equipment and storage medium

Abstract

The present invention belongs to the technical field of construction site safety helmet detection, and discloses a construction site safety helmet detection method, computer equipment and storage medium. The present invention comprises the following steps: building a safety helmet detection model based on an improved YOLOV10 network structure, wherein the improved YOLOV10 is obtained by making the following improvements on the basis of the original network structure: replacing the fusion module of the original feature pyramid structure with a weighted fusion module having a bidirectional weighted feature pyramid structure; replacing the original downsampling module with a downsampling HWD module based on wavelet transform; replacing the multi-head attention in some self-attention modules with CBMA modules; training the safety helmet detection model using a training data set, and using the trained safety helmet detection model to monitor the safety helmets at the construction site in real time. The present invention can perform fast and accurate detection with fewer model parameters.

Description

Technical Field

[0001] The invention belongs to the technical field of construction site safety helmet detection, and in particular relates to a construction site safety helmet detection method, computer equipment and storage medium. Background Art

[0002] Traditional methods of detecting hard hats on construction sites, such as manual inspections and video surveillance, generally have problems such as low detection efficiency, easy missed detection, and false detection. They are very dependent on experienced on-site managers to check one by one. To play a protective role, hard hats must be worn correctly (that is, the hat must be worn straight, and the bottom tie of the hard hat must be buckled under the chin). Therefore, it is necessary to inspect the workers' hard hat wearing all the time, which is time-consuming and laborious, and it is difficult to meet the needs of modern construction management. Therefore, in recent years, target detection algorithms based on deep learning have gradually become an effective way to solve this problem and have achieved rapid development.

[0003] However, when general target detection technology is used in construction sites, it is difficult to quickly and accurately identify people due to the complex background environment of the site, the dense distribution of people, occlusion, and the small size of the human targets captured by the camera, which may cause missed detection or wrong detection. On the basis of the existing hard hat detection technology, there are higher requirements for detection accuracy, and the classification of danger warnings also requires more detailed. At present, the challenges faced by hard hat wearing detection technology are: small hard hat targets may lead to wrong detection and missed detection, the detection speed is too slow, and the requirements for the edge devices where the system is to be deployed are too high (too much memory is occupied).

[0004] Currently, commonly used target detection algorithms are divided into two categories: one is a two-stage algorithm, the most representative of which is R-CNN and its improved algorithms Faster R-CNN, Mask R-CNN, etc. This type of algorithm is generally divided into two steps. First, some candidate regions are generated, and then these candidate regions are classified and regressed. The advantages of this algorithm are obvious. The accuracy of target detection is very high, but the speed is very slow and a large number of candidate regions need to be generated, which increases the complexity of calculation and time.

[0005] The other type is the one-stage target detection algorithm, represented by the YOLO series, SSD, RetinaNet, etc. This type of method generates several detection boxes for a target through regression, and then outputs only one detection box through post-processing such as NMS (non-maximum suppression). Obviously, the one-stage detection speed is faster than the two-stage detection algorithm, and the detection accuracy also meets the actual requirements. However, the recent improvements are all sub-optimization, and the speed and accuracy can be further improved.

[0006] In summary, although the two-stage detection algorithm has high accuracy, its detection speed is low, so it is not suitable for actual construction site application. The speed and accuracy of the one-stage detection algorithm can be used in practical applications, but there is still room for improvement in performance. Summary of the invention

[0007] The purpose of the present invention is to propose a method for detecting safety helmets at construction sites. The method performs safety helmet detection at construction sites based on an improved YOLOV10 model, and can perform fast and accurate detection with fewer parameter models.

[0008] In order to achieve the above object, the present invention adopts the following technical scheme:

[0009] A construction site safety helmet detection method comprises the following steps:

[0010] Step 1. Build a helmet detection model based on the improved YOLOV10 network structure;

[0011] The improved YOLOV10 network structure includes a backbone network, a feature fusion network and a detection head. The network structure is improved on the basis of the original YOLOV10 network structure by: replacing the fusion module of the original feature pyramid structure with a weighted fusion module with a bidirectional weighted feature pyramid structure; replacing the original downsampling module with a downsampling HWD module based on wavelet transform; replacing the multi-head attention MHSA in some self-attention modules PSA with CBMA modules;

[0012] Step 2. Use the training data set to train the helmet detection model.

[0013] Step 3. Use the trained hard hat detection model to monitor hard hats at the construction site in real time, obtain hard hat detection results, and issue graded warnings based on the hard hat detection results.

[0014] In addition, based on the above-mentioned construction site safety helmet detection method, the present invention also proposes a computer device, which includes a memory and one or more processors. An executable code is stored in the memory. When the processor executes the executable code, the steps of the above-mentioned construction site safety helmet detection method are implemented.

[0015] In addition, based on the above-mentioned construction site safety helmet detection method, the present invention also proposes a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, it is used to implement the steps of the above-mentioned construction site safety helmet detection method.

[0016] The present invention has the following advantages:

[0017] As described above, the present invention relates to a method for detecting safety helmets at construction sites, computer equipment, and storage media. The method of the present invention is an improvement based on the YOLOV10 network structure. Specifically, on the basis of the original YOLOV10 network structure, a HWD downsampling module is used to replace the convolution and SCDown downsampling modules with a step size of 2 in the original YOLOV10 network structure. The HWD module is a downsampling module based on Haar wavelets. The high-frequency and low-frequency information of the feature map is fully extracted using Haar wavelets. The size of the feature map is reduced, but the original spatial information is transferred to the channel through the HWD module, and important information is preserved, retaining rich detail feature information, which is important for the key point detection of the safety helmet, and improves the detection effect. In addition, the feature fusion module constructed based on PAN and FPN in the original YOLOV10 model is replaced by a weighted fusion module with a bidirectional weighted feature pyramid structure. The difference between the bidirectional weighted feature pyramid and the previous one is that it adopts a weighted feature fusion mechanism to combine features of different levels, and the fusion weights are learned during the training process to ensure the best feature integration. The bidirectional connections in the bidirectional weighted feature pyramid help to better acquire features at different scales, improving the network's ability to handle objects of different sizes and complexities. This is particularly important in target detection tasks because the sizes in images may vary significantly. In addition, the MHSA multi-head self-attention mechanism in some self-attention modules PSA is replaced with CBMA (Convolutional Block Attention Module). The purpose of CBMA is to overcome the limitations of traditional convolutional neural networks in processing information of different scales, shapes, and directions. To this end, CBMA introduces two attention mechanisms: channel attention and spatial attention. Channel attention is conducive to enhancing the feature representation of different channels, and spatial attention is conducive to extracting key information at different positions in space. By combining the two, the feature expression ability of the network can be improved, thereby improving the detection accuracy. Moreover, this is a lightweight attention mechanism. Through the above improvements, the helmet detection model built based on the improved YOLOV10 network structure can perform fast and accurate detection with fewer model parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Flow chart of a method for detecting safety helmets at a construction site according to an embodiment of the present invention;

[0019] Figure 2 A simplified model architecture diagram of the improved YOLOV10 network structure in an embodiment of the present invention;

[0020] Figure 3 A network structure diagram of a backbone network in an embodiment of the present invention;

[0021] Figure 4 It is a network structure diagram of a feature fusion network and a detection head in an embodiment of the present invention;

[0022] Figure 5 : is a network structure diagram of C2f and Bottleneck modules in an embodiment of the present invention; wherein Figure 5 (a) in the figure is the C2f network structure diagram. Figure 5 (b) and (c) are the Bottleneck network structures with and without residual connections, respectively;

[0023] Figure 6 This is a network structure diagram of C2fCIB in an embodiment of the present invention;

[0024] Figure 7 A network structure diagram of a CIB in an embodiment of the present invention;

[0025] Figure 8 This is a network structure diagram of SPPF in an embodiment of the present invention;

[0026] Fig. 9 This is a network structure diagram of PSCBMA in an embodiment of the present invention;

[0027] Fig.10 A network structure diagram of CBMA in an embodiment of the present invention;

[0028] Fig.11 A network structure diagram of HWD in an embodiment of the present invention;

[0029] Fig.12 Schematic diagram of a simple principle of a weighted fusion module in an embodiment of the present invention;

[0030] Fig.13 Schematic diagram of the working process of the construction site safety helmet detection method in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0032] Example 1

[0033] This embodiment 1 describes a method for detecting safety helmets at construction sites. The method is an improved safety helmet wearing detection algorithm for construction sites based on the improved YOLOV10 (full name: You only look once version 10) network structure. The general process includes the following parts: data acquisition, which is used to collect image data of the construction site; a data set preparation part, which is used to annotate the collected data and then divide it into a training set, a verification set, and a test set; and an image detection part, which is used to send the collected data to a pre-trained safety helmet detection model for detection to obtain a detection result.

[0034] Compared with the previous generation YOLO algorithm, the YOLOV10 network reconsiders the trade-off between efficiency and performance, introduces Consistent Dual Assignments, removes the reliance on NMS (non-maximum suppression), and redesigns the basic modules based on the principle of improving efficiency and accuracy, further improving the efficiency and accuracy of detection. Specifically, in the training phase, the one-to-many allocation strategy (assigning multiple positive samples to each true bounding box) is combined to provide rich supervision signals to help the model learn better and the one-to-one allocation strategy (assigning one positive sample to each true bounding box) ensures that the model can learn to select the best predicted bounding box during training. In the actual detection phase, only the one-to-one allocation strategy is used, which further improves the speed of detection. It is a real-time end-to-end detection algorithm that is very suitable for deployment on edge devices for real-time detection. In addition, the performance and effect of detection are further improved by introducing the PSA and CIB modules. In terms of the detection head, the YOLOV10 network found through a series of research and investigation that in terms of the detection head, the regression branch and the classification branch have the same structure, but the contribution of the regression branch to the detection effect is far greater than that of the classification branch. Therefore, the use of deep separable convolution instead of ordinary convolution reduces the number of parameters and improves the detection efficiency without sacrificing the detection effect.

[0035] The construction site safety helmet detection method proposed in the present invention is improved based on YOLOV10. Compared with the previous generation YOLO algorithm, YOLOV10 reconsiders the trade-off between efficiency and performance. Compared with the previous YOLO series algorithms, YOLOV10 uses fewer parameters and inference time, and can also achieve good detection effects.

[0036] like Figure 1 As shown, the construction site safety helmet detection method in this embodiment 1 includes the following steps:

[0037] Step 1. Build a helmet detection model based on the improved YOLOV10 network structure.

[0038] The improved YOLOV10 network structure includes a backbone network, a feature fusion network, and a detection head. Figure 2 As shown in the figure. The backbone network mainly extracts features, the feature fusion network performs feature fusion, and the detection head classifies and regresses the feature map to output the detection results. There are three detection heads, which are obtained by making the following improvements on the original YOLOV10 network structure: replacing the original fusion module of the feature pyramid structure based on PAN and FPN with a weighted fusion module with a bidirectional weighted feature pyramid structure to improve the efficiency and effect of feature fusion; using the downsampling HWD module based on wavelet transform to replace the original downsampling module, which can save more abundant information after downsampling to ensure the detection effect; replacing the multi-head attention MHSA in some self-attention modules PSA with CBMA modules to further save parameters and computing power while ensuring the effect.

[0039] like Figure 3 As shown in the figure, the backbone network includes three convolution modules, three cross-stage local fusion C2f modules, two downsampling HWD modules based on Haar wavelet transform, one cross-stage local fusion convolution interaction C2fCIB module, a (fast version of) spatial feature pyramid SPPF module and a partial convolution attention mechanism PSCBMA module.

[0040] The three convolution modules are defined as Conv-0, Conv-1 and Conv-3 modules respectively, the three cross-stage local fusion C2f modules are defined as C2f-2, C2f-4 and C2f-6 modules respectively, the two downsampling HWD modules based on Haar wavelet transform are defined as HWD-5 and HWD-7 modules respectively, and the spatial feature pyramid SPPF module is defined as SPPF-9 module.

[0041] The signal processing flow in the backbone network is as follows:

[0042] The input image is continuously passed through the Conv-0 and Conv-1 modules for feature extraction. The size of the feature map is reduced by half after each convolution module, and the size of the feature map becomes 1 / 4 of the input image after two convolution modules; then the feature map is refined through the C2f-2 module; specifically, the C2f module divides the input data into two branches for processing, one of which is directly passed to the output, and the other branch is processed by multiple BottleNeck modules and passed to the output and added to the output of the first branch. The C2f module improves the feature expression ability, and such branch design also improves the nonlinear representation ability of the model; then the Conv-3 module extracts features and reduces the feature map to half the size of the feature map input to this module, and then refines the features through the C2f-4 module; then The HWD-5 module is used to downsample the feature map to reduce the size of the feature map by half; the C2f-6 module is used to enhance the expressiveness of the feature map; the HWD-7 module is used for downsampling, and the C2fCIB module, which combines C2f and CIB, is used to further enhance the expressiveness of the feature map. Compared with the C2f structure, C2fCIB introduces the convolutional interaction module CIB to enhance the interaction between each branch; the SPPF-9 module is used to solve the limitation that the network input must be of a fixed size, which is conducive to the fusion of local features and global features; finally, the PSCBMA module is used to combine channel attention and spatial attention, so that the network can pay attention to the position of the feature channel and the image space at the same time, which can improve the model's distinction and positioning capabilities and improve the overall performance of the model.

[0043] like Figure 4 As shown in the figure, the feature fusion network includes five cross-stage local fusion convolution interaction C2fCIB modules, two upsampling Unsample modules, three downsampling HWD modules based on Haar wavelet transform, and multiple weighted fusion modules with a bidirectional weighted feature pyramid structure.

[0044] The signal processing flow in the feature fusion network is as follows:

[0045] The input of the feature fusion network is the output of the backbone network, that is, the output of the PSCBMA module. First, the feature map is upsampled and enlarged by two times through an upsampling Unsample module; then a weighted fusion module is used to dynamically weight the upsampling result and the output of the C2f-6 module in the backbone network, and the result is sent to a C2fCIB module (i.e., the first C2fCIB module) for feature refinement; then an upsampling Upsample module is used to upsample the feature map by two times and send it to a weighted fusion module, and dynamically weighted merge it with the output of the Conv-3 module in the backbone network; then it is sent to a C2fCIB module (i.e., the second C2fCIB module) for feature refinement; the result is input into a weighted fusion module, and merged with the output of the Conv-3 module in the backbone network and the output of the HWD downsampling of the C2f-2 module in the backbone network (i.e., the result of downsampling the C2f-2 module through a HWD module), and then the result is input into A C2fCIB module (i.e., the third C2fCIB module) is used to refine the features, and the first detection head connected to the third C2fCIB module is used to output a result. The output of the third C2fCIB module is downsampled by a HWD module to reduce the size of the feature map, which is input into the weighted fusion module and merged with the output of the first C2fCIB module in the feature fusion network and the output of the C2f-6 module in the backbone network into a C2fCIB module (i.e., the fourth C2fCIB module), and the second detection head connected to the fourth C2fCIB module is used to output a result. The output of the fourth C2fCIB module is downsampled by a HWD module to reduce the feature map and the output is dynamically weighted merged with the output of the backbone network (i.e., the output of the PSCBMA module) using a weighted fusion module, and input into a C2fCIB module (i.e., the fifth C2fCIB module) to refine the features, and then sent to the last detection head connected to the fifth C2fCIB module to output the result.

[0046] The usual detection head is composed of a classification head and a regression head, which use the same architecture. However, in terms of actual computational overhead, the classification head is more than twice that of the regression head, and the regression head is more important to the overall performance of the model. Therefore, YOLO V10 uses a lightweight architecture for the classification head, which is composed of deep separable convolutions. The previous YOLO algorithm generated several prediction boxes for an object as positive samples, providing the model with rich supervision signals for training, which promoted the improvement of model performance. In actual reasoning, redundant candidate boxes were removed by non-maximum suppression, which affected the detection speed in actual reasoning applications. In YOLO V10 training, one-to-many allocation and one-to-one allocation are used at the same time. In actual reasoning, only one-to-one allocation is performed, avoiding non-maximum suppression operations and saving time costs. This not only provides rich supervision information in training, but also guides the model to select the label closest to the actual value, improving the detection performance and detection speed of the model.

[0047] like Figure 5 In (a), the cross-stage local fusion C2f module mainly includes the convolution module Conv and the bottleneck module, which integrates the advantages of the cross-stage partial network (CSP) and the bottleneck module, and can make the extracted feature information richer.

[0048] First, the C2f module divides the input image into two parts in the Channel dimension. One part passes through N Bottleneck modules (two are taken as an example in the figure), and each Bottleneck module directly outputs the result to the tensor merging module. Each Bottleneck module contains multiple Conv; the other part is directly merged with the output of Bottleneck in the channel dimension and finally output through a convolution.

[0049] Bottleneck modules are divided into two types: one with residual connection and one without residual connection, such as Figure 5 As shown in (b) and (c) of . Among them, shortcut (residual connection) = False / True, which means without residual connection or with residual connection respectively.

[0050] The size of the feature map does not change after passing through the C2f module.

[0051] The cross-stage local fusion C2f module splits the input image in the channel dimension and then merges it. This can provide a richer information flow, reduce redundant calculations, and improve the expressiveness of features.

[0052] like Figure 6As shown in Figure 1, the Cross Stage Partial Fusion Convolution Interactive Block (C2fCIB) is a new module introduced in YOLOV10. It uses the C2f structure to replace Bottleneck with the convolutional interactive CIB module structure. The CIB module consists of depth-separable convolutions, as shown in Figure 1. Figure 7 shown.

[0053] The depthwise separable convolution is composed of depthwise convolution DW and pointwise convolution ConV. The depthwise convolution uses a convolution operation for each input channel, and then a 1×1 convolution adjusts the number of output channels.

[0054] This decomposition method reduces the amount of calculation from (The complexity of standard convolution) is reduced to (Complexity of depthwise separable convolution).

[0055] in is the size of the convolution kernel, M and N are the number of input and output channels respectively, is the size of the feature map.

[0056] Obviously, the computing resource requirements are greatly reduced.

[0057] By utilizing the characteristics of depthwise separable convolution, the C2fCIB module further reduces redundant calculations and improves the efficiency of feature extraction. Such convolution operations and changes in channel interaction make the feature information richer.

[0058] like Figure 8As shown in the figure, the fast version of Spatial Pyramid Pooling–Fast (SPPF) is improved based on SPP. It consists of the Maxpool module and the Conv module, and is nearly 2.5 times faster than SPP. SPP is mainly used to solve problems such as image distortion caused by operations such as image cropping and scaling, as well as the problem of repeated extraction of image-related features by convolutional neural networks. It greatly improves the speed of generating candidate boxes and saves computational costs. SPPF performs pooling on feature maps at different scales to ensure that more spatial information is retained while reducing the amount of calculation. The general processing process of the SPPF module is: the input feature map first passes through the Conv module and then through the cascaded Maxpool module, and the output of each pooling result and the convolution result are merged, and then the convolution operation is performed after merging. Through multi-scale pooling, SPPF can capture features and information of different sizes, making the model more sensitive to the target. Compared with SPP, the SPPF module reduces computational complexity and resource consumption and improves the running speed of the model.

[0059] like Fig. 9 As shown, the partial convolutional attention mechanism PSCBMA module includes a convolutional module Conv and a convolutional attention mechanism CBMA module; there are four convolutional modules Conv, which are defined as the first, second, third, and fourth convolutional modules Conv respectively.

[0060] The signal processing flow in the PSCBMA module is as follows:

[0061] The input feature map first passes through the first convolution module Conv to extract features, and then is sent to the CBMA module to use the spatial attention mechanism and channel attention mechanism to focus on more important features. The output of the CBMA module is merged with the input of CBMA and then passes through the second and third convolution modules Conv continuously. It is then merged with the output of CBMA and sent to the tensor merging module, merged with the output of the first convolution module Conv in the PSCBMA module, and sent to the fourth convolution module Conv to finally output the result.

[0062] In this paper, a new attention module is proposed by introducing the partial convolutional attention mechanism PSCBMA module and combining the PSA and CBMA modules, which further optimizes the detection effect and efficiency of the entire model.

[0063] like Fig.10As shown in the figure, the attention mechanism module (Convolutional Block Attention Module, CBMA) is an attention mechanism module used to enhance the performance of convolutional neural networks. It consists of a channel attention module and a spatial attention module. The CBMA module improves the perception ability of the model by introducing channel attention and spatial attention, thereby improving performance without increasing the complexity of the network. By introducing the CBMA module, the detection effect of the model is optimized. CBMA multiplies the output features of the channel attention module and the spatial attention module element by element to obtain the final attention-enhanced feature. This enhanced feature will be used as the input of subsequent network layers to suppress noise and irrelevant information while retaining key information.

[0064] The channel attention module first performs global maximum pooling and global average pooling operations on each channel, calculates the maximum eigenvalue and average eigenvalue on each channel, generates two vectors containing the number of channels, representing the global maximum feature and average feature of each channel respectively, and inputs the feature vectors after global maximum pooling and average pooling into a shared fully connected layer. This fully connected layer is used to learn the attention weights of each channel. Through learning, the network can adaptively decide which channels are more important for the current task. The global maximum feature and the global average feature are combined and added to obtain the final attention weight vector. To ensure that the attention weight is between 0 and 1, the Sigmoid activation function is applied to generate the channel attention weight and applied to each channel of the original feature map. The obtained attention weight is multiplied by each channel of the original feature map to obtain the attention-weighted channel feature map, which will emphasize the channels that are helpful to the current task and suppress irrelevant channels. The formula is as follows:

[0065] ;

[0066] in, represents the sigmoid activation function, F is the input feature map, MaxPool is the global maximum pooling and AvgPool is the global average pooling, and MLP represents the multi-layer perceptron composed of convolutional modules.

[0067] The spatial attention module performs maximum pooling and average pooling operations along the channel dimension on the input feature map to generate features of different context scales; then the features after maximum pooling and average pooling are connected (concatenated) along the channel dimension to obtain a feature map with context information of different scales. This feature map is then processed through a convolutional layer to generate spatial attention weights. Similar to the channel attention module, the Sigmoid activation function is applied to the generated spatial attention weights to limit the weights between 0 and 1. The obtained spatial attention weights are applied to the original feature map to weight the features of each spatial position, which can highlight important image areas and reduce the influence of unimportant areas. The formula is as follows:

[0068] M S ( F ) = σ ( f ( [ A v g P o o l ( F ) ; M a x P o o l ( F ) ] ) ） ;

[0069] Among them, f represents the convolution operation.

[0070] like Fig.11 As shown in the figure, the Haar Wavelet Downsampling (HWD) module is a downsampling module based on Haar wavelet transform. Downsampling operations, such as maximum pooling or strided convolution with a step size other than 1, are widely used in convolutional neural networks. Their function is to aggregate local features and expand the receptive field. However, some information will inevitably be lost. This information is very important for detecting some key points of helmets and faces. Therefore, HWD is used to replace the downsampling module in the original model. Hear wavelet is a simple and effective wavelet transform method that can better retain the information in the original image while reducing the size of the feature map. This is very important for the detection of key points.

[0071] The HWD module consists of a lossless feature encoding block and a feature representation learning block, where the number of output channels is adjusted by the feature representation learning block; the lossless feature encoding block uses the Haar wavelet transform layer to effectively reduce the spatial resolution of the feature map while retaining all information. When the Haar wavelet is applied to the image, four components are obtained, and the spatial resolution of each component is only half of the original signal. It can be seen that the low-pass and high-pass decomposition filters are used to extract low-frequency and high-frequency information from the image: low-frequency components (A), and high-frequency components in the horizontal direction (H), vertical direction (V), and diagonal direction (D). The size of each component is H / 2×W / 2, and the number of channels C has increased fourfold to 4C, which can be seen as the information is transferred from the spatial dimension to the channel dimension, retaining the original information. The number of channels is then adjusted through the convolution module Conv.

[0072] The feature representation learning block is composed of convolution modules, and the activation function is the ReLU function. It performs well: either make the parameters vanish or let them pass. ReLU alleviates the vanishing gradient problem of neural networks.

[0073] The traditional downsampling module reduces the resolution of the feature map by convolution or pooling, which inevitably loses some key details. The HWD downsampling module provided by the present invention uses Haar wavelet transform to extract key high-frequency information and low-frequency information in the feature map. The feature map after wavelet transform will generate four components. The size of the four component feature maps is half of the original feature map, but the number of channels becomes four times the original. It can be understood that the original spatial information is transferred to the channel information. The number of channels of the final output can be adjusted in the feature learning block.

[0074] like Fig.12 As shown in the figure, the bidirectional weighted feature pyramid introduces a bidirectional structure based on the feature pyramid network, that is, in the process of information transmission, in addition to the top-down transmission, the bottom-up transmission is also introduced, so that the features can be perceived more comprehensively. The bidirectional weighted feature pyramid introduces learnable weights to learn the importance of different input features during feature fusion, which helps the model identify input features of different resolutions that contribute unequally to the output features. In addition, the bidirectional weighted feature pyramid optimizes cross-scale connections and enhances feature fusion by removing nodes with only a single input edge, adding additional edges from the original input to the output node (if they are at the same level), and treating each bidirectional path as a feature network layer.

[0075] like Fig.12 The figure shows multiple feature maps to be weighted fused, including feature Figure 1 ,feature Figure 2 ,…, feature map n, and the corresponding weights of these feature maps are w1, w2,…, wn respectively.

[0076] The upsampling module is used to enlarge the size of the feature map. Common methods include nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation. The computational efficiency of these three methods increases in turn, and the upsampling effect decreases in turn. In the model built in this patent, the nearest neighbor interpolation method is used to reduce the amount of calculation.

[0077] Non-Maximum Suppression: NMS is a technique used to reduce the number of overlapping bounding boxes or regions in object detection algorithms. It sorts the detection boxes of each category by confidence, then traverses them one by one, retains the boxes with the highest confidence, and suppresses the overlapping boxes with low confidence, so as to obtain more accurate and concise detection results. However, NMS needs to set an overlap threshold. The choice of this parameter has a great impact on the final result. Too high or too low a threshold will affect the detection effect. In large-scale detection tasks, the computational complexity of NMS may affect the overall reasoning speed. For some complex scenes, such as crowded people or dense targets, NMS may not be able to effectively distinguish multiple adjacent targets.

[0078] Therefore, NMS was cancelled in YOLOV10, which improved the detection speed. A consistent dual allocation strategy was used in YOLOV10, combining the one-to-many and one-to-one allocation strategies in the training phase. The former allocates multiple prediction boxes as positive samples for each true target box. Assume that 10 prediction boxes are generated. If the IOU value of 6 of them with the true box is greater than a certain threshold, these 6 prediction boxes will participate in the training as positive samples to help the model learn better; the latter allocates one prediction box as a positive sample for each true target box. In actual application detection, only the one-to-one allocation strategy is used, which saves NMS processing, avoids redundant detection, and improves detection speed.

[0079] Step 2. Use the training data set to train the built helmet detection model.

[0080] Before training the model, let’s first introduce the process of building the training data set, as follows:

[0081] First, the head and helmet image data are collected through the construction site video monitoring equipment, and then packaged and sorted.

[0082] Use LabelImg (a labeling tool software) to label the helmet and face in the collected head helmet image, mark the key points of the front brim of the helmet and the buckles at both ends, and the key points of the jaw of the face. Prepare image data of helmets with cracks and extrusion deformation, and label the damaged cracked parts. Input image data, the label of the output data includes the detection category, the coordinates of the detection box, the coordinates of the key points, and whether the key points are visible.

[0083] The specific marking methods for the safety helmet and the face are: safety helmet: hat; front brim of the safety helmet: hat_front; left strap of the safety helmet: left; right strap of the safety helmet: right; lower jaw of the face: face_down, damage crack: break.

[0084] After production is completed, it is divided into the training set, validation set and test set required by the model in a ratio of 8:1:1.

[0085] The training set and validation set data are sent to the model for training and validation. The validation set is used for validation in each round of training. After a certain number of training times, based on the results, if the weights obtained through training are tested using the test set and the effect does not meet the requirements for use, the training parameters are further adjusted and retrained until the results meet the use standards.

[0086] Step 3. Finally, the application is deployed. The trained helmet detection model, that is, the model with optimal parameters, is used to monitor the helmets on the construction site in real time, output the helmet detection results, and design a helmet classification warning method.

[0087] After the model training process, select a suitable edge device for deployment, such as the Jetson Orin series edge device.

[0088] After the model is deployed, at the construction site, you can use a camera that can upload images to the cloud to transmit the collected data to the cloud server in real time, and the edge device can obtain the collected images from the cloud server in real time, or use a drone to take pictures or a USB camera that can be directly connected to the edge device to collect the construction site hard hat data and pass it to the edge device. After pre-processing, the image is input into the trained hard hat detection model, and the hard hat detection model outputs the detection results.

[0089] like Fig.13 As shown in the figure, the data to be tested is sent to the trained helmet detection model to detect whether the helmet has defects and the key points marked, and a graded warning is made according to the test results. The graded warning process is as follows:

[0090] The first level of detection is to determine whether the safety helmet is detected. If no safety helmet is detected, a first-level warning is issued; the second level of detection is to determine whether the surface of the safety helmet is damaged or cracked. If the surface of the safety helmet is damaged or cracked, a second-level warning is issued; the third level of detection is to determine whether the front brim of the safety helmet is detected. If no front brim of the safety helmet is detected, a third-level warning is issued; the fourth level of detection is to determine whether the detected safety helmet strap is buckled below the chin key point. If not detected, a fourth-level warning is issued; if all four levels of detection are passed, it is safe to wear and no warning is issued.

[0091] The method of the present invention is based on the improved construction site helmet wearing detection method of the YOLOV10 model, and the efficiency and accuracy are significantly improved compared with the previous YOLOV algorithm. On the basis of the original V10 model, the feature fusion part of the model is further improved, and a weighted feature fusion module is used instead, and a learnable weight is introduced to more efficiently perform cross-scale feature fusion under the premise of ensuring accuracy. The HWD downsampling module is introduced. The downsampling module based on Haar wavelet can retain feature information more completely when downsampling compared with the original downsampling module, and can convert feature information from space to channel dimension, thereby improving the detection accuracy and being more helpful for the detection of small targets such as helmets and heads. The MHSA multi-head attention mechanism in the PSA module is replaced by the CBMA module. The CBMA module can improve the feature expression ability of the convolutional neural network by adaptively learning channel and spatial attention weights, so that the model can pay more attention to the relevant features of the helmet and improve the detection effect.

[0092] Example 2

[0093] This embodiment 2 describes a computer device, which includes a memory and one or more processors. The memory stores executable code. When the processor executes the executable code, it is used to implement the steps of the construction site safety helmet detection method in the above embodiment 1.

[0094] In this embodiment, the computer device is any device or apparatus with data processing capability, which will not be described in detail here.

[0095] Example 3

[0096] This embodiment 3 describes a computer-readable storage medium on which a program is stored. When the program is executed by a processor, it is used to implement the steps of the construction site safety helmet detection method in the above-mentioned embodiment 1.

[0097] The computer-readable storage medium may be an internal storage unit of any device or apparatus with data processing capabilities, such as a hard disk or memory, or an external storage device of any device with data processing capabilities, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc., equipped on the device.

[0098] Of course, the above description is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiments. It should be noted that all equivalent substitutions and obvious deformation forms made by any technician familiar with the field under the guidance of this specification fall within the essential scope of this specification and should be protected by the present invention.

Claims

1. A construction site safety helmet detection method, characterized in that: The steps include: Step 1. Build a helmet detection model based on the improved YOLOV10 network structure; The improved YOLOV10 network structure includes a backbone network, a feature fusion network and a detection head. The network structure is improved on the basis of the original YOLOV10 network structure by: replacing the fusion module of the original feature pyramid structure with a weighted fusion module with a bidirectional weighted feature pyramid structure; replacing the original downsampling module with a downsampling HWD module based on wavelet transform; replacing the multi-head attention MHSA in some self-attention modules PSA with CBMA modules; Step 2. Use the training data set to train the built helmet detection model; Step 3. Use the trained helmet detection model to monitor the helmets on the construction site in real time, obtain the helmet detection results, and make graded warnings based on the helmet detection results; The backbone network includes three convolution modules, three cross-stage local fusion C2f modules, two downsampling HWD modules based on Haar wavelet transform, a cross-stage local fusion convolution interaction C2fCIB module, a spatial feature pyramid SPPF module and a partial convolution attention mechanism PSCBMA module; The three convolution modules are defined as Conv-0, Conv-1 and Conv-3 modules respectively, the three cross-stage local fusion C2f modules are defined as C2f-2, C2f-4 and C2f-6 modules respectively, the two downsampling HWD modules based on Haar wavelet transform are defined as HWD-5 and HWD-7 modules respectively, and the spatial feature pyramid SPPF module is defined as SPPF-9 module; The signal processing flow in the backbone network is as follows: the input image is continuously extracted through the Conv-0 and Conv-1 modules, and then refined through the C2f-2 module, and then through the Conv-3 module to extract features and reduce the feature map, and then through the C2f-4 module to refine the features, and then through the HWD-5 module to downsample the feature map, and then through the C2f-6 module to enhance the expression ability of the feature map, and then use the HWD-7 module for downsampling, and then through the C2fCIB module that combines the C2f and CIB modules to further enhance the expression ability of the feature map, and then through the SPPF-9 module to solve the limitation that the network input must be of a fixed size, which is conducive to the fusion of local features and global features; finally, the PSCBMA module combines channel attention and spatial attention, so that the network can pay attention to the position of the feature channel and image space at the same time.

2. The construction site safety helmet detection method according to claim 1, characterized in that: The feature fusion network includes five cross-stage local fusion convolution interaction C2fCIB modules, two upsampling Unsample modules, three downsampling HWD modules based on Haar wavelet transform, and multiple weighted fusion modules with bidirectional weighted feature pyramid structures; the signal processing flow in the feature fusion network is as follows: The input of the feature fusion network is the output of the backbone network, that is, the output of the PSCBMA module. First, the feature map is upsampled and enlarged by two times through an upsampling Unsample module, and then a weighted fusion module is used to dynamically weight the upsampling result and the output of the C2f-6 module in the backbone network. The result is sent to the first C2fCIB module for feature refinement, and then an upsampling Upsample module is used to upsample the feature map by two times and send it to a weighted fusion module. It is dynamically weighted and merged with the output of the Conv-3 module in the backbone network, and then sent to a C2fCIB module for feature refinement. The result is input into a weighted fusion module and combined with the output of the Conv-3 module in the backbone network and with The results of the HWD downsampling outputs of the C2f-2 modules in the backbone network are merged, and then the results are input into a C2fCIB module to refine the features. The first detection head is used to output a result, and then the feature map is downsampled through an HWD module to reduce the size of the feature map, which is input into a weighted fusion module and merged with the output of the first C2fCIB module in the feature fusion network and the output of the C2f-6 module in the backbone network into a C2fCIB module. The second detection head is used to output a result, and then an HWD module is used to downsample and reduce the feature map. The output is dynamically weighted and merged with the output of the backbone network using a weighted fusion module, and then input into a C2fCIB module to refine the features and send it to the last detection head to output the result.

3. The construction site safety helmet detection method according to claim 2, characterized in that: The cross-stage local fusion convolution interaction C2fCIB module adopts the C2f structure and replaces the bottleneck module with a convolution interaction CIB module structure. The CIB module is composed of depth-separable convolutions. The depthwise separable convolution consists of depthwise convolution and pointwise convolution. The depthwise convolution uses a convolution operation for each input channel, and then a 1×1 convolution adjusts the number of output channels.

4. The construction site safety helmet detection method according to claim 1, characterized in that: The partial convolutional attention mechanism PSCBMA module includes a convolutional module Conv and a convolutional attention mechanism CBMA module; wherein there are four convolutional modules Conv, which are defined as the first, second, third and fourth convolutional modules Conv respectively; The signal processing flow in the PSCBMA module is as follows: The input feature map first passes through the first convolution module Conv to extract features, and then is sent to the CBMA module to use the spatial attention mechanism and channel attention mechanism to focus on more important features. The output of the CBMA module is merged with the input of the CBMA and then passes through the second and third convolution modules Conv continuously. Then, it is merged with the output of the CBMA and sent to the tensor merging module, and then merged with the output of the first convolution module Conv in the PSCBMA module and sent to the fourth convolution module Conv to finally output the result; The attention mechanism CBMA module consists of a channel attention module and a spatial attention module; The channel attention module first performs global maximum pooling and global average pooling operations on each channel, calculates the maximum eigenvalue and average eigenvalue on each channel, generates two vectors containing channel numbers, representing the global maximum feature and global average feature of each channel respectively, and inputs the features after global maximum pooling and average pooling into a shared fully connected layer; merges the global maximum feature and the global average feature to obtain the final attention weight vector; to ensure that the attention weight is between 0 and 1, the Sigmoid activation function is applied to generate the channel attention weight and apply it to each channel of the original feature map; the obtained attention weight is multiplied by each channel of the original feature map to obtain the attention-weighted channel feature map; The spatial attention module performs maximum pooling and average pooling operations on the input feature map along the channel dimension to generate features of different context scales; then the features after maximum pooling and average pooling are connected along the channel dimension to obtain a feature map with context information of different scales; the feature map is then processed through a convolutional layer to generate spatial attention weights; the Sigmoid activation function is applied to the generated spatial attention weights to limit the weights between 0 and 1; the obtained spatial attention weights are applied to the original feature map to weight the features at each spatial position.

5. The construction site safety helmet detection method according to claim 2, characterized in that: The HWD module consists of a lossless feature encoding block and a feature representation learning block, wherein the number of output channels is adjusted by the feature representation learning block; the lossless feature encoding block utilizes the Haar wavelet transform layer to effectively reduce the spatial resolution of the feature map while retaining all information. When the Haar wavelet is applied to the image, four components are obtained, and the spatial resolution of each component is only half of the original signal; the feature representation learning block is composed of a convolution module, and the activation function is the ReLU function.

6. The construction site safety helmet detection method according to claim 2, characterized in that: In step 2, the construction process of the training data set is as follows: Use LabelImg to label the helmet and face in the collected head helmet image, marking the key points of the front brim of the helmet and the buckles at both ends, as well as the jaw key points of the face; We also prepared image data of helmets with cracks and extrusion deformation, and labeled the damaged and cracked parts; the data was divided into training set, validation set, and test set in a ratio of 8:1:1; Among them, the specific marking methods for safety helmets and faces are: safety helmet: hat; front brim of safety helmet: hat_front; left strap of safety helmet: left; right strap of safety helmet: right; lower jaw of face: face_down, crack damage: break.

7. The construction site safety helmet detection method according to claim 1, characterized in that: In step 3, the data to be tested is sent to the trained helmet detection model to detect whether the helmet has defects and the marked key points, and a graded warning is performed according to the test results. The graded warning process is as follows: The first level detection determines whether a safety helmet is detected. If no safety helmet is detected, a first level warning is issued; The second level detection is to determine whether the surface of the helmet is damaged or cracked. If the surface of the helmet is damaged or cracked, a second level warning will be issued; The third level detection determines whether the front brim of the helmet is detected. If the front brim of the helmet is not detected, a third level warning is issued; The fourth level detection determines whether the helmet strap is buckled below the chin key point. If it is not detected, a fourth level warning is issued; If the above four levels of testing are passed, it is safe to wear and no warning is given.

8. A computer device comprising a memory and one or more processors; an executable code is stored in the memory; characterized in that: When the processor executes the executable code, it is used to implement the steps of the construction site safety helmet detection method described in any one of claims 1 to 7.

9. A computer-readable storage medium having a program stored thereon; characterized in that: When the program is executed by a processor, it is used to implement the steps of the construction site safety helmet detection method described in any one of claims 1 to 7.

Images