A target detection method of adaptive label assignment

Abstract

The present application is directed to the complex scene in the practical application of smart construction site, and provides a safety helmet detection method of self-adaptive label allocation. The present application proposes an auxiliary correction mechanism, which places an auxiliary network in the backbone network to supplement the scene detail information. At the same time, a feature guiding module is designed, which filters the redundant information in the shallow layer through the attention mechanism, and further strengthens the feature representation in the deep layer with the removed redundant information, so that the feature representation is more comprehensive, and different sizes of targets can be fully dealt with. In addition, the present application uses dynamic weights to adaptively determine positive and negative samples, and improves the safety helmet detection accuracy in the complex construction site from the perspective of label classification.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of target detection, and relates to a safety helmet detection method based on adaptive label assignment. BACKGROUND

[0002] With the continuous maturity of deep learning technology, target detection technology has also developed rapidly. In the safety helmet detection method based on a convolutional neural network, it is necessary to judge whether a certain position is a positive sample or a negative sample when calculating the loss function, and a suitable label is assigned to the corresponding position, so that the loss function can gradually learn this behavior. This process is called label assignment.

[0003] At present, there are some differences between the detection methods with anchor boxes and without anchor boxes in the entire framework. The detection method based on anchor boxes usually actively calculates the IoU of anchor boxes and real boxes on the feature map, selects a suitable threshold range according to human experience, divides the anchor boxes with an IoU greater than the upper limit of the threshold into positive samples, divides the anchor boxes with an IoU less than the lower limit of the threshold into negative samples, and divides the anchor boxes within the range into ignored samples. The one-stage SSD method is based on this idea, but the anchor box and real box matching strategy is slightly different, resulting in a number of positive samples much less than that of negative samples. The classic detection method FCOS based on anchor-free determines positive and negative samples according to the target center region and its scale, maps the coordinates in the feature map back to the original image, and if the mapping result is within the real box and the class is consistent, it is considered as a positive sample, otherwise it is considered as a negative sample. The above are relatively classic fixed label assignment strategies. The anchor boxes, thresholds and the like in these strategies are set according to human experience values, and the model obtained by training may have some difference from the best result. With the progress of science and technology, the recent detectors introduce adaptive mechanism label assignment, so that the network can learn the label assignment process as much as possible, and try to eliminate the influence of human experience values, and realize the best detection of images in different scenes.

[0004] In view of the above problems existing in the prior art, it is necessary to study a safety helmet detection method based on adaptive label assignment, which can realize the effect that auxiliary network information flow is supplemented to the backbone network and is transmitted layer by layer in the backbone network, and the interaction between deep information and shallow features is completed by using the mutual guidance of the deep information and the shallow features. The technology has an auxiliary correction mechanism, and simultaneously guides the matching operation of anchor boxes and real boxes in the process of network training by using the adaptive label assignment strategy. SUMMARY

[0005] The purpose of the present application is to provide a safety helmet detection method based on adaptive label assignment to solve the label assignment problem mentioned in the background art.

[0006] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0007] A safety helmet detection method of adaptive label assignment, comprising the following steps:

[0008] Step (1): constructing an image dataset of wearing a safety helmet and not wearing a safety helmet, preprocessing the dataset, and dividing the dataset into a training set and a validation set;

[0009] Step (2): constructing a safety helmet detection model based on a DLANet network, and adding an auxiliary correction mechanism and a feature guiding module to improve the safety helmet detection model;

[0010] Step (3): inputting the training dataset into the improved safety helmet detection model for training, inputting the validation dataset into the safety helmet detection model generated in each iteration for validation, and selecting the model with the highest validation accuracy from all the safety helmet detection models as the optimal safety helmet detection model;

[0011] Step (4): saving the optimal safety helmet detection model, inputting the image to be detected into the model, and outputting the target detection result of the detection image.

[0012] The specific operation of preprocessing in step (1) is: after cropping the image to a specified size, performing multi-scale random scaling, image flipping and image pixel normalization to increase the performance of the model.

[0013] The safety helmet detection network in step (2) is composed of three modules: the first module is a basic network module, which uses a DLANet network without a classification layer, extracts features from the training image through the basic network model, and extracts effective features through the convolution kernel and the input picture; the second module is an auxiliary correction mechanism, and the basic network of this module is VGG16; the auxiliary correction mechanism is used to fuse the information extracted by the auxiliary network and the information extracted by the backbone network; after the auxiliary network features are supplemented to the backbone network, the convolution under different inflation rates and scales is used to learn the large receptive field in complex scenes, and then the fused information is corrected to improve the feature representation of the backbone network; the third module is a feature guiding module, which adopts mutual feedback learning, guides the shallow layer to filter out redundant information and retain more details to improve small target prediction through the attention mechanism on the effect of deep features, and feeds back the filtered redundant information to the deep features to supplement more semantic information for predicting medium and large targets.

[0014] In step (3), the improved safety helmet detection model is inputted for training, and the adaptive label assignment is optimized in the training process to realize automatic label assignment.

[0015] When the detection result of the output image is that the safety helmet is not worn, the human body region image of the person not wearing the safety helmet and a reminding signal are sent to the terminal, and an alarm sound is further issued to remind the person not wearing the safety helmet.

[0016] The safety helmet detection method with adaptive label assignment mainly comprises an auxiliary correction mechanism (ACM) and a feature guide module (FGM), realizes the effect that auxiliary network information flow is supplemented to the backbone network and is transmitted layer by layer in the backbone network, and completes the interaction between deep information and shallow features by mutual guidance. The auxiliary correction mechanism comprises two parts: a backbone network and an auxiliary network. The mechanism takes VGG-16 as the backbone network, processes the result of the auxiliary network acting on the backbone network through a dense hollow spatial pyramid pooling module, and then gradually penetrates into the subsequent features of the backbone network; then, the feature guide module is used to perform feature screening and fusion between deep and shallow layers. Meanwhile, in the process of network training, the matching operation of anchor boxes and real boxes is performed under the guidance of the adaptive label assignment strategy. The safety helmet detection method with adaptive label assignment has the beneficial effects that the adaptive label assignment is optimized, the feature guide module and the multi-scale target fusion network are proposed, and the accuracy of safety helmet detection is improved. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 FIG. 1 is a flow chart of a safety helmet detection method with adaptive label assignment according to the present application;

[0018] Figure 2 FIG. 2 is a multi-scale target detection network framework according to the present application;

[0019] Figure 3 FIG. 3 is a dense hollow spatial pyramid pooling according to the present application; DETAILED DESCRIPTION

[0020] The embodiments of the present application will be described below in conjunction with the drawings and examples.

[0021] Example 1

[0022] As shown in FIG. 1, the safety helmet detection method with adaptive label assignment disclosed in the present embodiment is suitable for safety helmet detection of workers at a construction site, and specifically comprises the following steps: Figure 1 Step (1): constructing image data sets of wearing safety helmets and not wearing safety helmets, pre-processing the picture data set, and dividing it into a training set and a validation set;

[0023]

[0024] ​Step (2): a safety helmet detection model is constructed, and the safety helmet detection model is improved, including a correction mechanism based on an auxiliary network, a feature guiding module, and an improvement of a label assignment method;

[0025] Step (3): the training data set is input into the improved safety helmet detection model for training, the verification data set is input into the safety helmet detection model generated in each iteration for verification, and the model with the highest verification accuracy is selected from all the safety helmet detection models as the optimal safety helmet detection model;

[0026] Step (4): after obtaining the optimal safety helmet detection model, the optimal detection model is initialized by using a deep learning related framework. The required detection image is input into the model, and the target detection result of the detection image is output.

[0027] In step (1), the data set is preprocessed, specifically: the image is cropped to a specified size and subjected to multi-scale random scaling, image flipping and image pixel normalization. In the specific implementation process, the construction site image is directly obtained, or the real-time construction site image is captured from the construction site monitoring camera, and the labelImg labeling tool is used to label the to-be-detected target in the construction site image, that is, the safety helmet region. The type label corresponding to the image of wearing a safety helmet is 1, and the type label corresponding to the image of not wearing a safety helmet is 0. The labeled image is cropped, subjected to multi-scale random scaling, image flipping and image pixel normalization and other preprocessing, and the training image set and the verification image set are collected in a ratio of 7 to 3. The preprocessing process not only ensures that the size and other parameters of the training image meet the requirements, but also performs data enhancement, which is beneficial to the robustness enhancement of the target detection model.

[0028] In step (2), the backbone network of the safety helmet detection model removes the fully connected layers fc_6, fc_7 and fc_8 in the VGG-16 according to the SSD backbone network, wherein the fc_6 and fc_7 are replaced by the convolutional conv6 and conv7, and four additional convolutional layers are added for layer-by-layer feature extraction. The entire safety helmet detection model includes three parts, which are: a correction mechanism based on an auxiliary network, a feature guiding module, and a self-adaptive label assignment. The first part: the correction mechanism based on the auxiliary network includes two branches, the first branch is an auxiliary network built by a convolutional group and two down-sampling modules, and the second branch is the backbone network. The two branches are in parallel and cascaded, use different dilation rates in the atrial convolution to generate multiple scale features with different receptive fields, encourage the neurons on the intermediate feature map to encode semantic content at multiple scales, and thus capture more comprehensive feature information in complex scenes. The second part: the feature guiding module, the module has two inputs (corresponding Figure 2The deep features (P1 and P2) are separated from the shallow features by a scale feature. Therefore, when the deep features are upsampled to keep them at the same scale as the shallow features, a four-fold upsampling operation should be performed, i.e., two 2x upsampling operations. Compared to shallow features, which are suitable for predicting small targets, the results from upsampling deep features (corresponding to...) Figure 2 The P3 algorithm removes redundant information other than small targets, retaining more semantic information and highlighting the main features of small targets. Simultaneously, an attention mechanism is introduced to optimize the upsampled feature map (corresponding to...). Figure 2 The training focuses on the focal region in P3 (a key area on the map), significantly reflecting the large target situation. The third part: a dynamically adjusted label assignment principle, where IoU is the proportion of cross-reference information between the anchor box and the ground truth box, and IoU_regress is the IoU value between the model's output helmet bounding box and the ground truth box. The proportion between IoU and IoU_regress is used as weights, and these weights are further used to adaptively fuse IoU and IoU_regress, avoiding initial misjudgments. As training progresses, the network can dynamically adjust the proportion of both according to the framework requirements, adaptively achieving label assignment for helmet detection.

[0029] In step (3), the improved safety helmet detection model is input for training. During the training process, adaptive label allocation is optimized to achieve automatic label allocation. Specifically, the operation is as follows: First, the number of positive samples allocated based on IoU is denoted as n, and the number of ignored samples is denoted as m. m and n change continuously as training progresses. Then, based on the optimized... The anchor boxes are sorted by their values. The first n samples are considered positive, the next m are ignored, and the rest are negative. Based on this labeling strategy, the localization capability gradually improves during training, and the coordinates of the bounding boxes gradually converge towards the coordinates of the ground truth boxes. The number of samples will gradually increase. In this process, not only can anchor boxes that are more sensitive to the target object be captured, but also the number of samples will always be kept within a balanced range based on the dynamic changes of m and n, avoiding the problem of too many positive samples caused by manual thresholding and ensuring the stable training of the network.

[0030] In step (4), the original images of the construction workers at the construction site are collected and input into the optimal detection model in step (3). When the model detects a worker, it is determined whether the worker is wearing a safety helmet in the head area. If so, it is considered that the worker is wearing a safety helmet normally. If not, the worker is not wearing a safety helmet. The image of the worker is collected and an alarm is issued.

[0031] Comparative experiment:

[0032] Training and testing process: The training and testing of the model were performed by using two NVIDIA RTX A4000 GPUs in the Pytorch framework. The weight decay coefficient was 0.0005, the momentum was 0.9, the initial learning rate was 0.001, and then it was gradually reduced to 0 with the number of iterations, Smooth L1 and Focal Loss were used as the loss function of the network framework. Through the images collected by the real-time monitoring system, a data set of 1600 images was constructed. 1500 images were labeled and trained, and the remaining 100 images were used for safety helmet detection experiments.

[0033] Table 1 shows the performance results of multiple algorithms. It can be seen that whether it is a one-stage or two-stage algorithm, it is difficult to get a larger improvement, and the detection accuracy of most algorithms is maintained at about 80.0%. Compared with other methods, the method proposed in the present application achieves an average accuracy of 82.0% under the same conditions, i.e., the backbone network is VGG-16 and the input resolution is 300x300, which surpasses other methods in the table.

[0034] Table 1 shows the test and detection results

[0035]

Claims

1. A method for detecting safety helmets with adaptive label assignment, characterized in that... Includes the following steps: Step (1): Construct an image dataset of people wearing safety helmets and not wearing safety helmets, preprocess the dataset, and divide the dataset into a training set and a validation set; Step (2): Construct a helmet detection model based on the DLANet network, and add an auxiliary correction mechanism and a feature guidance module to improve the helmet detection model. The helmet detection model consists of three modules: The first module is the basic network module, which uses the DLANet network with the classification layer removed. The basic network model extracts features from the training image and extracts effective features by convolving the input image with the convolution kernel in the network. The second module is the auxiliary correction mechanism. The basic network of this module is VGG16. The auxiliary correction mechanism is used to fuse the information extracted by the auxiliary network and the information extracted by the backbone network. After the auxiliary network features are added to the backbone network, the receptive field of a large range in complex scenes is learned by convolution under different dilation rates and scales, thereby correcting the fused information and improving the feature representation of the backbone network. The third module is the feature guidance module. This module adopts mutual feedback learning. The attention mechanism guides the shallow layer to filter out redundant information and retain more details to improve the prediction of small targets. The filtered redundant information is fed back to the deep features to supplement more semantic information to predict medium and large targets. Step (3): Input the training dataset into the improved helmet detection model for training, input the validation dataset into the helmet detection model generated in each iteration for validation, and select the model with the highest validation accuracy from all the helmet detection models obtained as the optimal helmet detection model; Step (4): Save the optimal safety helmet detection model, input the required detection image into the model, and output the target detection result of the detection image.

2. The helmet detection method with adaptive tag allocation according to claim 1, characterized in that: The specific preprocessing operations in step (1) are as follows: after cropping the image to the specified size, perform multi-scale random scaling, image flipping, and normalization of image pixels to increase the performance of the model.

3. The helmet detection method with adaptive tag allocation according to claim 1, characterized in that: In step (3), the improved safety helmet detection model is input for training, and adaptive label allocation is optimized during the training process to achieve automatic label allocation.

4. The helmet detection method with adaptive tag allocation according to claim 1, characterized in that: In step (4), when the detection result of the output image is that no safety helmet is being worn, the image of the human body area without a safety helmet and the reminder signal are sent to the terminal, and then an alarm sound is issued to remind the person who is not wearing a safety helmet.

Images