A workshop safety helmet detection method based on feature adaptive fusion

Abstract

The application relates to a workshop safety helmet detection method based on feature adaptive fusion, which comprises the following steps: obtaining to-be-detected data and performing pretreatment, inputting a pre-trained safety helmet detection model, performing NMS post-processing based on the number of prediction frames in a neighborhood based on the output of the safety helmet detection model, and obtaining a detection result, wherein the safety helmet detection model comprises a multi-scale feature adaptive fusion network and a target detection network which are connected with each other, the multi-scale feature adaptive fusion network is used for obtaining a target feature map used for prediction based on a plurality of input feature maps of the target detection network and corresponding spatial weights and channel weights. Compared with an existing safety helmet wearing detection method, the application can more accurately detect ordinary hats and improve the detection capability of small targets in a workshop scene, and can effectively reduce accidental false detection in a continuous frame detection process.

Description

Technical Field

[0001] The present invention relates to the technical field of target data monitoring, and in particular to a workshop safety helmet detection method based on feature adaptive fusion. Background Art

[0002] Hardhats are a crucial piece of protective gear in industrial production environments, effectively preventing or reducing accidental head injuries. With the continuous improvement in the performance of deep learning-based object detection algorithms, the use of intelligent recognition technology to replace manual supervision is a major trend. Existing hardhat detection methods are mostly targeted at outdoor or construction scenarios; currently, there are no hardhat detection methods specifically designed for use within workshops. Compared to samples to be detected in outdoor or construction scenarios, targets within workshops are smaller and have lower image quality. Furthermore, workshop production environments have more obstructions, making the identification of these samples more difficult. Existing hardhat detection methods still need to improve their ability to detect hardhat targets within workshops.

[0003] (1) The samples of existing hard hat detection technologies are all from general outdoor or construction scenes, and there are no data samples that can be used specifically for workshop hard hat detection.

[0004] (2) The existing safety helmet detection model has a low recognition accuracy for small-scale targets to be detected in the workshop environment.

[0005] (3) Most existing helmet wearing detection models are trained using single-frame images. However, in actual applications, the models usually detect continuous surveillance video images. This can lead to occasional false detections when detecting continuous frames. Summary of the Invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a workshop helmet detection method based on feature adaptive fusion. By assigning channel spatial weights and channel weights through multiple feature maps, multi-scale features can be adaptively fused to avoid small target features being submerged in multiple fusion processes, thereby achieving the purpose of improving the detection accuracy of small targets.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] The present invention provides a method for detecting safety helmets in a workshop based on feature adaptive fusion, comprising the following steps:

[0009] The data to be tested is obtained and pre-processed, and then input into the pre-trained helmet detection model. Based on the output of the helmet detection model, NMS post-processing based on the number of prediction boxes in the neighborhood is performed to obtain the detection result.

[0010] Among them, the helmet detection model includes an interconnected multi-scale feature adaptive fusion network and a target detection network. The multi-scale feature adaptive fusion network is used to obtain multiple input feature maps output by the target detection network and spatial weights and channel weights adaptively determined based on the input feature maps, and obtain target feature maps as the output of the helmet detection model.

[0011] As a preferred technical solution, the target detection network is a YOLOv5 network, and the target detection network is used to obtain the input feature map.

[0012] As a preferred technical solution, the acquisition of the target feature map for prediction includes the following steps:

[0013] Performing preprocessing including scaling and dimensionality reduction on the multiple input feature maps to obtain preprocessed feature maps;

[0014] Based on the preprocessed feature map, obtaining the spatial weight corresponding to the spatial description branch and the channel weight corresponding to the channel description branch, wherein the spatial description branch is set in parallel with the spatial description branch;

[0015] The target feature map is obtained based on the preprocessed feature map and the spatial weight and channel weight.

[0016] As a preferred technical solution, the spatial weight is obtained using the following formula:

[0017]

[0018] in, represents the feature map divided by channel dimension, σ 7×7 represents a convolution layer with a convolution kernel of 7×7, MP() represents global average pooling, MP() represents global maximum pooling, μ k ,k∈{1,2,3,4,5,6} is the spatial weight, F f is the feature map after preprocessing.

[0019] As a preferred technical solution, the spatial weight is obtained using the following formula:

[0020]

[0021] in, represents the feature map divided by channel dimension, σ represents the convolution layer, AP() represents the global average pooling, MP() represents the global maximum pooling, μ is the spatial weight, F f is the feature map after preprocessing.

[0022] As a preferred technical solution, the channel weight is obtained using the following formula:

[0023]

[0024] Among them, Sigmoid represents the Sigmoid activation function, σ represents the convolution layer, Relu represents the Relu activation function, α represents the channel weight, F f is the preprocessed feature map, AP() represents global average pooling, Represents the feature map divided by channel dimension.

[0025] As a preferred technical solution, the output of the spatial description branch is:

[0026]

[0027] in, is the value at layer n (x, y), X represents the input feature map, m is the input feature map channel, Represents the spatial weight of each feature map, where c represents the channel of the spatial description feature.

[0028] As a preferred technical solution, the output of the channel description branch is:

[0029]

[0030] in, The value of the channel description branch output at layer n (x, y) is Represents the channel weight.

[0031] As a preferred technical solution, the NMS post-processing based on the statistics of the number of prediction boxes in the neighborhood includes the following steps:

[0032] Determine the target bounding box with the highest degree of overlap with the true target, determine the number of prediction boxes contained in the neighborhood, and obtain the prediction box set and the corresponding confidence set;

[0033] According to the predicted frame set and the corresponding confidence set, NMS processing is performed to obtain the predicted frame set after removing redundant frames.

[0034] As a preferred technical solution, the acquisition of the number of prediction frames is implemented using the following formula:

[0035]

[0036] In the formula, C represents the number of prediction boxes contained in the neighborhood, N represents the number of prediction boxes of similar targets in the graph, and x i ,y iRepresents the center point coordinates of each prediction box, τ represents the counting function, that is, if the distance between the two center points is less than the neighborhood radius, then C increases by 1, is the neighborhood radius, λ represents the neighborhood range adjustment parameter, and w and h represent the size of the target bounding box.

[0037] As a preferred technical solution, for the helmet detection model, the process of obtaining a training set for training and a test set for verification includes the following steps:

[0038] Obtain images of the interior of the workshop and at the entrance of the workshop, including images of people wearing ordinary hats and people without helmets, and obtain a sample set by annotating the images;

[0039] Based on the sample set, the training set and the test set are obtained by data set division.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] (1) High small target detection accuracy: The present invention provides a helmet detection model, including a multi-scale feature adaptive fusion network and a target detection network that are interconnected. The multi-scale feature adaptive fusion network can obtain a target feature map for prediction based on multiple input feature maps of the target detection network and the corresponding spatial weights and channel weights. Finally, the model output is subjected to NMS post-processing based on the statistics of the number of prediction boxes in the neighborhood to obtain the detection result. This method assigns channel spatial weights and channel weights through multiple feature maps, thereby realizing multi-scale feature adaptive fusion, avoiding small target features from being submerged in multiple fusion processes, and achieving the purpose of improving small target detection accuracy.

[0042] (2) Low false detection rate: An NMS post-processing algorithm is provided to count the number of prediction boxes in the neighborhood. By counting the number of prediction boxes in the neighborhood of the highest confidence prediction box, it is determined whether to retain the prediction box, thereby suppressing the occasional false detection of the model during the continuous frame detection process.

[0043] (3) High robustness: We collected samples of helmet wearing detection in the workshop production environment, and introduced difficult-to-detect samples such as wearing ordinary hats, holding helmets, and head occlusion, forming a sample set dedicated to workshop helmet detection, which can more accurately detect the wearing of ordinary hats. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is the implementation process of a workshop helmet detection method based on feature adaptive fusion in Example 1;

[0045] Figure 2 Schematic diagram of the structure of the multi-scale feature adaptive fusion network;

[0046] Figure 3 A schematic diagram of the neighborhood range;

[0047] Figure 4 This is a schematic diagram of the application of the helmet detection method;

[0048] Figure 5 Schematic diagram of the detection effect during the application process. DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0050] Example 1

[0051] This embodiment provides a workshop hardhat detection method based on feature adaptive fusion. It collects data samples from within the workshop and introduces a variety of difficult-to-detect samples, such as those wearing standard hats and those with occluded heads. To address the inadequacy of existing hardhat detection models for detecting small targets in industrial production environments and the tendency for false detection during continuous frame detection, a multi-scale feature adaptive fusion network and a non-maximum suppression (NMS) post-processing algorithm based on the number of prediction boxes within a neighborhood are proposed. These algorithms are then integrated with the commonly used hardhat inspection model YOLOv5, ultimately forming an improved YOLOv5+ model that can accurately identify whether workers in the workshop are wearing hardhats.

[0052] The process of obtaining a sample set includes the following steps:

[0053] The open helmet wearing detection dataset SHWD is introduced, which contains a total of 7581 images. Since most of the samples in the existing helmet wearing detection dataset are from outdoors and there are almost no samples that are difficult to identify, such as wearing ordinary hats, holding helmets, or with occlusion, the present invention obtained 4100 images of the interior of the workshop through both network collection and offline collection. The images were then annotated using the labelImg tool, and the sample label for wearing a helmet was defined as 1, and the sample label for not wearing a helmet was defined as 0. Taking into account that in the actual production process, there are fewer samples of not wearing a helmet and wearing ordinary hats, the present invention collected data samples such as not wearing a helmet and wearing an ordinary hat from the entrance of the workshop, totaling 2200 images.

[0054] like Figure 2(a) is a schematic diagram of the structure of the multi-scale feature adaptive fusion network provided by the present invention, Figure 2 (b) in the Figure 2 Schematic diagram of the structure of the adaptive feature fusion module in (a), where global maximum pooling and global average pooling are used in the spatial description branch to compress the channel dimension and jointly aggregate the spatial information of the features. In the channel description branch, global average pooling is used to compress the spatial dimension to reflect the response of each channel.

[0055] In order to obtain higher-level semantic information, existing target detection models usually perform multiple downsampling operations on the input image, which can easily lead to the loss of small target features. Although structures such as NAS-PAN or Bi-PAN can be used to enhance multi-scale feature expression, thereby reducing the loss of small target features, mechanically adding features of different scales directly can easily lead to the masking of small target features during the fusion process. In order to avoid the small target features being submerged in the feature fusion process, such as Figure 2 As shown in the figure, the present invention designs a multi-scale feature adaptive fusion network and combines it with the YOLOv5 target detection model. By assigning channel spatial weights and channel weights to the six feature maps at three scales, multi-scale features are adaptively fused to avoid small target features being submerged in multiple fusion processes, thereby achieving the purpose of improving small target detection accuracy. The specific method is as follows:

[0056] The input of the multi-scale feature adaptive fusion module is defined as The inputs are unified to an intermediate scale by bilinear interpolation or maximum pooling, and then a convolution layer with a convolution kernel of 1×1 is used to reduce the dimension of the input features to obtain F. f ∈R 40×40×6 Then, the adaptive fusion weight is determined through two parallel branches: spatial description and channel description.

[0057] Define space description branch output is the value at (x,y) on the nth layer. It can be calculated by the following formula (1), where X represents the input feature map, m is the input feature map channel, Represents the spatial weight of each feature map, where c represents the channel of the spatial description feature.

[0058]

[0059] Spatial weight μ k ,k∈{1,2,3,4,5,6}:

[0060]

[0061] in σ 7×7 ,AP(),MP() respectively represent the division of feature maps by channel dimension, convolution layer with 7×7 convolution kernel, global average pooling, and global maximum pooling.

[0062] definition Output the value of the channel description branch at layer n (x, y), so the channel description output can be expressed as:

[0063]

[0064] in Indicates the channel description.

[0065]

[0066] Among them, Sigmoid,σ 3×3 , Relu represent the Sigmoid activation function, the convolution layer with a convolution kernel of 3×3, and the Relu activation function respectively.

[0067] Formula (1) and formula (3) can be used to assign adaptive fusion weights based on channel description and adaptive fusion weights based on spatial description to the six feature maps at three scales. Then, formula (5) can be used to adaptively fuse features F1 and F6, F2 and F5, and F3 and F4 at the same time, and finally output the feature map P that can be used for prediction. z ,z∈{1,2,3}. P z :

[0068] P z =δ n +η n (5)

[0069] The NMS post-processing algorithm for counting the number of prediction boxes in the neighborhood is as follows:

[0070] When detecting consecutive frames, there are typically multiple prediction boxes surrounding a correctly predicted bounding box, while only a few prediction boxes surround an incorrectly predicted bounding box. The number of prediction boxes actually reflects the richness of valid features at the current target location. Therefore, to effectively reduce occasional false detections during consecutive frame detection, this paper designs an NMS post-processing algorithm based on the statistics of the number of adjacent prediction boxes.

[0071] like Figure 3 As shown, the present invention describes the bounding box with the highest degree of overlap with the true target as box:

[0072] box={x c, y c ,w,h} (6)

[0073] where x c ,y c ,w,h represent the center point coordinates of the box and the size of the bounding box respectively.

[0074] Then define the neighborhood radius d:

[0075]

[0076] Where λ represents the neighborhood range adjustment parameter. Then, as shown in the following formula (8), the number of prediction boxes C contained in the neighborhood range is counted.

[0077]

[0078] Where N represents the number of prediction boxes of this type of target in the image, x i ,y i Represent the center point coordinates of each prediction box, and τ represents the counting function, that is, if the distance between the two center points is less than the neighborhood radius, C increases by 1.

[0079] In summary, the NMS post-processing algorithm based on the statistics of the number of prediction boxes in the neighborhood is as described in Table 1:

[0080] Table 1 NMS post-processing algorithm based on the statistics of the number of prediction boxes in the neighborhood

[0081]

[0082] Among them, B in the algorithm input represents the set of all predicted boxes, S represents the corresponding confidence set, and N t Indicates the NMS threshold, N p Indicates the threshold value of the number of prediction boxes in the neighborhood. DIoU() is a calculation method that represents the degree of overlap between bounding boxes. The D in the algorithm output represents the set of prediction boxes after removing redundant boxes, and S represents the corresponding confidence set. The present invention sets λ = 0.6, N t =0.4, N p =6.

[0083] This paper designs a multi-scale feature adaptive fusion network structure, generating adaptively fused channel description weights and spatial description weights for features at different scales. It also provides an NMS post-processing algorithm for counting prediction boxes within a neighborhood. By counting the number of prediction boxes within the neighborhood of the highest-confidence prediction box, it determines whether to retain that prediction box, thereby suppressing occasional false detections in the model during continuous frame detection. Samples of helmet wearing detection were collected in a workshop production environment. Furthermore, samples with difficult detection characteristics, such as those wearing ordinary hats, holding helmets, and with occluded heads, were introduced to form a dataset specifically for workshop helmet detection.

[0084] Compared with the existing helmet wearing detection method, the present invention can more accurately detect the wearing of ordinary hats and improve the detection ability of small targets in workshop scenes, and can effectively reduce accidental false detections that occur during continuous frame detection.

[0085] Example 2

[0086] like Figure 1 As described, this embodiment provides an application of the safety helmet detection method described in Example 1. First, after completing the collection of the workshop safety helmet data set, the LabelImg tool is used to perform data annotation to form a data set that can be used specifically for workshop safety helmets. The data set is then divided into a test set and a training set, of which the test set accounts for 30% and the training set accounts for 70%. Then, the code of the workshop safety helmet detection model is implemented based on the PyTorch framework, and the model is trained using a single NVIDIA3090Ti GPU. During the training process, data enhancement methods such as random rotation and mosaic are introduced, and a multi-scale training method is adopted. The initial learning rate is set to 0.00001 and the batch size is 16. The learning rate is updated using the cosine annealing method, and a total of 200 rounds of training are performed. After the training is completed, the model is verified using the test set, and the hyperparameters are reset until an optimal detection model is trained. Finally, the operating environment and code required for the model are packaged using Docker technology and deployed on a GPU server.

[0087] like Figure 4 As described, first, workshop video data is collected through video surveillance equipment, and then the data is sent to the GPU server. The data is framed on the GPU server, and the workshop safety helmet detection model deployed on the GPU is run to detect in real time whether the workshop workers are wearing safety helmets, and the detection results are sent to the data storage and management server. If there are samples without wearing safety helmets, a warning picture is generated.

[0088] like Figure 5 The above is a schematic diagram of the detection effect during application. Compared with the original YOLOv5 model (Examples 1-3), the present invention (Examples 4-6) effectively avoids occasional false detections (Example 2) during continuous frame detection. Compared with the original YOLOv5 model (Example 7), the present invention (Example 8) also has better detection results for small targets far from the camera.

[0089] Example 3

[0090] This embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for the workshop safety helmet detection method based on feature adaptive fusion as described in Example 1.

[0091] Example 4

[0092] This embodiment provides a computer-readable storage medium, including one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the workshop safety helmet detection method based on feature adaptive fusion as described in Example 1.

[0093] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A workshop helmet detection method based on feature adaptive fusion, characterized in that: The steps include: The data to be tested is obtained and pre-processed, and then input into the pre-trained helmet detection model. Based on the output of the helmet detection model, NMS post-processing based on the number of prediction boxes in the neighborhood is performed to obtain the detection result. The helmet detection model includes an interconnected multi-scale feature adaptive fusion network and a target detection network. The multi-scale feature adaptive fusion network is used to obtain multiple input feature maps output by the target detection network and spatial weights and channel weights adaptively determined based on the input feature maps, and obtain a target feature map as the output of the helmet detection model. The spatial weight is obtained using the following formula: in, represents the feature map divided by channel dimension, represents the convolutional layer, represents global average pooling, represents global maximum pooling, is the spatial weight, is the preprocessed feature map, The channel weight is obtained using the following formula: in, Sigmoid represents the Sigmoid activation function, represents the convolutional layer, Relu represents the Relu activation function, represents the channel weight, is the preprocessed feature map, represents global average pooling, Represents the feature map divided by channel dimension.

2. The method for detecting safety helmets in workshops based on feature adaptive fusion according to claim 1, characterized in that: The target detection network is a YOLOv5 network, and the target detection network is used to obtain the input feature map.

3. The method for detecting safety helmets in workshops based on feature adaptive fusion according to claim 1, characterized in that: The acquisition of the target feature map for prediction includes the following steps: Performing preprocessing including scaling and dimensionality reduction on the multiple input feature maps to obtain preprocessed feature maps; Based on the preprocessed feature map, obtaining the spatial weight corresponding to the spatial description branch and the channel weight corresponding to the channel description branch, wherein the spatial description branch is set in parallel with the spatial description branch; The target feature map is obtained based on the preprocessed feature map and the spatial weight and channel weight.

4. The method for detecting safety helmets in workshops based on feature adaptive fusion according to claim 3, characterized in that: The output of the spatial description branch is: in, is the value at layer n (x, y), X represents the input feature map, m is the input feature map channel, Represents the spatial weight of each feature map, where c represents the channel of the spatial description feature.

5. The method for detecting safety helmets in workshops based on feature adaptive fusion according to claim 3, characterized in that: The output of the channel description branch is: in, The channel description branch outputs the value at layer n (x, y), Represents the channel weight.

6. The method for detecting safety helmets in workshops based on feature adaptive fusion according to claim 1, characterized in that: The NMS post-processing based on the statistics of the number of prediction boxes in the neighborhood includes the following steps: Determine the target bounding box with the highest degree of overlap with the true target, determine the number of prediction boxes contained in the neighborhood, and obtain the prediction box set and the corresponding confidence set; According to the predicted frame set and the corresponding confidence set, NMS processing is performed to obtain the predicted frame set after removing redundant frames.

7. The method for detecting safety helmets in workshops based on feature adaptive fusion according to claim 6, characterized in that: The acquisition of the number of prediction frames is achieved using the following formula: Where, Indicates the number of prediction boxes contained in the neighborhood, N indicates the number of prediction boxes of similar targets in the graph, , Represent the center point coordinates of each prediction box, Represents the counting function, that is, if the distance between the two center points is less than the neighborhood radius, then Add 1, is the neighborhood radius, , represents the neighborhood adjustment parameter, Indicates the size of the target bounding box.

8. The method for detecting safety helmets in workshops based on feature adaptive fusion according to claim 1, characterized in that: For the helmet detection model, the process of obtaining a training set for training and a test set for verification includes the following steps: Obtain images of the interior of the workshop and at the entrance of the workshop, including images of people wearing ordinary hats and people without helmets, and obtain a sample set by annotating the images; Based on the sample set, the training set and the test set are obtained by data set division.

Images