A method for detecting safety helmets and reflective clothing based on drone images
Through the use of lightweight MobileNetV3 backbone networks, design small target detection layers, introducing EMA attention mechanisms and BIFPN in the drone The test of the problem improves the accuracy and robustness of the detection.
Patent Information
- Application Number
- CN202410326065.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-21
- Publication Date
- 2025-01-24
- Estimated Expiration
- 2044-03-21
AI Technical Summary
The aerial images of drones have large changes in size, small target size, complex scenes and changeable external factors, which lead to problems such as missed and missed inspections in safety helmets and reflective clothing detection.
The lightweight MOBILENETV3 backbone network is used to replace the CSPDARKNET53 backbone network, design a small target detection layer, introduce the EMA attention mechanism, and use the two -way characteristic pyramid network BIFPN in the NECK part to improve the speed and generalization capabilities of the model.
Effectively reduce the impact of changes in scale and complex scenarios on the test results, improve the generalization ability and robustness of the model, and reduce the incidence of missed inspection and misunderstanding.
Smart Images
Figure CN118230197B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for detecting a safety helmet and a reflective clothing, and belongs to the technical field of construction safety detection. Background Art
[0002] Road reconstruction and expansion and major and medium-sized repair projects need to reasonably guarantee road traffic. Many construction projects are in a state of half construction and half traffic, and there are many safety risks and hidden dangers on site and in the surrounding environment. Workers work in high-risk areas for a long time, and wearing safety helmets and reflective clothing can play an important role in preventing safety accidents. Due to the weak safety awareness of employees, they ignore the safety hazards of the construction site and take off their safety helmets and reflective clothing, resulting in various safety accidents. Traditional safety detection mainly relies on manual and monitoring equipment, and cannot achieve full coverage and real-time monitoring. With the rapid development of drone technology and computer vision technology, drones equipped with deep learning technology are widely used in practical application fields such as monitoring climate change, assisting search and rescue operations, and maintenance in the construction industry. Due to factors such as the uncertain height of drone aerial photography and the complex and changeable construction environment, drone visual target detection technology faces many challenges such as large image scale changes, small target size, complex scenes, and changeable external factors. Summary of the invention
[0003] The present invention aims to solve the problems of missed detection and false detection of people in target detection due to large scale variation, small target size, complex scenes and changeable external factors in drone aerial images, and further proposes a helmet and reflective clothing detection method based on drone images.
[0004] The technical solution adopted by the present invention to solve the above-mentioned problem is: the steps of the present invention include:
[0005] Step 1: Replace the CSPDarknet53 backbone network with the lightweight MobileNetV3 backbone network to reduce the number of network parameters and calculations and improve the inference speed of the model;
[0006] Step 2: Design a small target detection layer to improve the network's ability to capture target object information;
[0007] Step 3: Introduce the EMA attention mechanism in the C2f module to improve the ability to capture important features and suppress redundant features;
[0008] Step 4: Use the bidirectional feature pyramid network BiFPN in the Neck part to effectively reduce the impact of scale changes and complex scenes on the detection results and improve the generalization ability of the model.
[0009] Furthermore, step 1 specifically includes:
[0010] Step 101, pad the input image through 1×1 convolution to increase the number of channels; secondly, apply deep convolution in high-dimensional space;
[0011] Step 102: Optimize feature map data using SE attention mechanism;
[0012] Step 103: Use 1×1 convolution to reduce the number of channels. When the stride is 1 and the input and output feature shapes are equal, use residual connection. When the stride is 2, directly output the reduced feature map.
[0013] Furthermore, in step 3, an attention mechanism is introduced into the first three C2f modules of the neck network.
[0014] The beneficial effects of the present invention are:
[0015] 1. The present invention replaces the CSPDarknet53 backbone network with a lightweight MobileNetV3 backbone network, reducing the number of network parameters and computational complexity, and improving the model's reasoning speed. Secondly, in order to enhance the network's ability to extract features of multi-scale targets, the algorithm designs a small target detection layer to enhance the network's ability to capture target object information; at the same time, the EMA attention mechanism is introduced in the C2f module to improve the ability to capture important features and suppress redundant features; finally, the bidirectional feature pyramid network BiFPN is used in the Neck part to effectively reduce the impact of scale changes and complex scenes on detection results, and improve the generalization ability of the model;
[0016] 2. The present invention introduces BiFPN learnable weights to learn the importance of different input features, and repeatedly applies bottom-up and top-down multi-scale feature fusion;
[0017] 3. The present invention effectively solves the problem of information loss and excess when extracting features of different scales by introducing a bidirectional flow of feature information;
[0018] 4. The BiFPN of the present invention fuses the up- and down-sampled feature maps layer by layer, and introduces horizontal and vertical connections, so that features of different scales can be better integrated and utilized, thereby having strong robustness when processing complex scenes such as scale changes and occlusions;
[0019] 5. The feature grouping and multi-scale structure of the present invention effectively establish short-term and long-term dependencies to obtain better performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 This is the Mobilenet V3block structure diagram;
[0021] Figure 2 It is a schematic diagram of SE attention mechanism;
[0022] Figure 3 This is a schematic diagram of adding a small target detection layer;
[0023] Figure 4 is a schematic diagram of feature network design;
[0024] Figure 5 It is an EMA structure chart;
[0025] Figure 6 This is the structure diagram of the C2f module adding the EMA attention mechanism;
[0026] Figure 7 This is a schematic diagram of the improved YOLOv8 network structure. DETAILED DESCRIPTION
[0027] Specific implementation method 1: Combination Figures 1 to 6 This embodiment describes a method for detecting helmets and reflective clothing based on drone images, which includes the following steps:
[0028] Step 1: Replace the CSPDarknet53 backbone network with the lightweight MobileNetV3 backbone network to reduce the number of network parameters and calculations and improve the inference speed of the model;
[0029] Step 2: Design a small target detection layer to improve the network's ability to capture target object information;
[0030] Step 3: Introduce the EMA attention mechanism in the C2f module to improve the ability to capture important features and suppress redundant features;
[0031] Step 4: Use the bidirectional feature pyramid network BiFPN in the Neck part to effectively reduce the impact of scale changes and complex scenes on the detection results and improve the generalization ability of the model.
[0032] In this embodiment, the attention mechanism first performs global average pooling on the feature map. The relationship between the number of channels in the feature map and the pooling result (one-dimensional vector) is: [h, w, c] ==> [None, c]; then the output vector is obtained through two fully connected layers, the number of output channels of the first fully connected layer is equal to 1 / 4 of the number of channels of the original input feature map, and the number of output channels of the second fully connected layer is equal to the number of channels in the original input feature map; that is, the dimension is reduced first and then increased; the output vector of the fully connected layer can be considered as each element of the vector representing the weight relationship derived from the analysis of each feature map; more important feature maps are given greater weights, that is, their vector elements have larger values; on the contrary, less important feature maps correspond to smaller weight values; the first fully connected layer uses the ReLU activation function, and the second fully connected layer uses the hard_sigmoid activation function; after two fully connected layers, a vector of channel elements is obtained, each element is the weight of each channel, and the weight is multiplied by the corresponding original feature map to obtain new feature map data.
[0033] The original YOLOv8 network structure designs three feature maps with different downsampling scales, which are used to detect small targets, medium targets and large targets respectively. As the network depth increases, the feature map becomes smaller, more abstract, has a larger receptive field and contains more semantic information, so the small-sized feature map is usually used to detect large targets. On the contrary, the target position data in the large-scale feature map is more accurate and more suitable for detecting small targets. Therefore, in order to improve the network's detection ability for tiny targets, a larger-scale feature map needs to be added to the neck network FPN+PAN structure. The optimized network structure is as follows: Figure 5 shown.
[0034] like Figure 4 As shown in the network model structure, the pink circle represents a micro-small target detector, the orange circle represents a small target detector, the blue circle represents a medium-sized target detector, and the green circle represents a large target detector; Figure 4 (a) The feature pyramid network FPN introduces a bottom-up path to fuse multi-scale features from level 2 to level 5 (P2-P5), thereby improving the detector's ability to detect objects of different scales. However, it is computationally intensive and takes a long time to train and infer, and is limited by unidirectional information flow. To solve this problem, PAN adds an additional top-down path aggregation network on top of FPN, which can better preserve the details in the low-resolution feature map, thereby improving the detection accuracy. However, it also increases the computational complexity, such as Figure 4 (b) Figure 4(c) YOLOv8 borrows the idea of PAN. In order to simplify the network and improve the detection speed, YOLOv8 optimizes the feature pyramid network and removes nodes without feature fusion. However, the above feature fusion methods all have the problem of weak small target positioning and recognition capabilities. This is because in the feature extraction process, small targets are easily disturbed by normal-sized targets and the network will delete the unclear information, which makes the small target information continue to decrease, resulting in the small target detection effect failing to meet the expected requirements. BiFPN introduces learnable weights to learn the importance of different input features, and repeatedly applies bottom-up and top-down multi-scale feature fusion. By introducing the bidirectional flow of feature information, it effectively solves the problem of information loss and excess when extracting features of different scales. BiFPN
[31] fuses the up- and down-sampled feature maps layer by layer, and introduces horizontal and vertical connections to better integrate and utilize features of different scales, so that it has strong robustness when dealing with complex scenes such as scale changes and occlusions. Figure 4 (d).
[0035] The feature enhancement module of the present invention is composed of a C2f module of a neck structure and an attention mechanism EMA. The C2f structure is shown in the following figure: Figure 1 As shown in the figure, C2f here is the embodiment of the Bottleneck module; the C2f structure consists of two residual network structures, which has better fitting classification function, can obtain higher classification accuracy, and can be optimized and trained as the number of network layers increases. This is also the reason why the C2f module is selected for feature enhancement; based on the C2f structure, the feature enhancement module structure diagram embedded with the EMA attention mechanism is shown in Figure 6 As shown in the figure, it is also composed of two nested residual modules. By embedding the EMA module into the second residual block of the C2f module, features can be better extracted. The specific operation process is similar to that of C2f, with an additional step of attention mechanism, and the weight extraction and allocation process, which is more friendly to the learning of small targets.
[0036] In this implementation, EMA is an efficient multi-scale attention mechanism that preserves information and reduces computational cost without reducing the channel dimension. Figure 5 As shown in the figure, the adopted parallel substructure avoids the sequential processing of the network, while the convolution operation produces effective channel description and better pixel-level attention for high-level feature maps. Specifically, 1x1 convolution is extracted from the CA module for shared components to form a 1x1 branch. 3x3 kernels are placed in parallel for fast multi-scale spatial structure information aggregation to form a 3x3 branch.
[0037] Specific implementation method 2: Combination Figures 1 to 6To illustrate this embodiment, step 1 of a method for detecting helmets and reflective clothing based on drone images described in this embodiment specifically includes:
[0038] Step 101, pad the input image through 1×1 convolution to increase the number of channels; secondly, apply deep convolution in high-dimensional space;
[0039] Step 102: Optimize feature map data using SE attention mechanism;
[0040] Step 103: Use 1×1 convolution to reduce the number of channels. When the stride is 1 and the input and output feature shapes are equal, use residual connection. When the stride is 2, directly output the reduced feature map.
[0041] Specific implementation method three: Combination Figures 1 to 6 This embodiment is described. In step 3 of a method for detecting helmets and reflective clothing based on drone images described in this embodiment, an attention mechanism is introduced into the first three C2f modules of the neck network.
[0042] C2f refers to CSP Bottleneck with 2convolutions: it extracts features from the image by using multiple convolutional layers and pooling layers, and passes these features to the fully connected layer for classification and positioning; its core idea is to fuse different feature maps, use depthwise separable convolutional layers to reduce the amount of parameters and computation, and use upsampling layers to increase the resolution of feature maps, thereby improving detection accuracy and speed.
[0043] BiFPN refers to Bi-directional Feature Pyramid Network: BiFPN is the abbreviation of bidirectional feature pyramid network, which is a neural network architecture used for target detection and segmentation tasks in computer vision. It fuses the up and down sampled feature maps layer by layer, and introduces horizontal and vertical connections, so that features of different scales can be better integrated and utilized.
[0044] YOLOv8 means You Only Look Once Version 8: YOLOv8 is the latest version of Ultralytics' YOLO object recognition and image segmentation model, introducing new features and improvements to enhance performance and flexibility.
[0045] EMA stands for Excitation and Modulation Attention: an attention mechanism for neural networks. Its main principle is to improve the performance of the model by focusing the network's attention on the most important part of the current task through weighted processing of the input data.
[0046] P2 is equivalent to performing only two downsampling operations in the backbone network. The two P2 layer features obtained from top to bottom and bottom to top in the neck network are fused with the same-scale features in the backbone network in the form of concat to derive a new detection head.
[0047] CSPDarknet53 refers to Cross Stage Partial Network: CSPDarknet53 is a deep convolutional neural network model for image recognition and classification tasks; it is based on the Darknet53 model and improves the accuracy and efficiency of the model by using the CSP (CrossStage Partial) structure.
[0048] SE stands for Squeeze-and-Excitation: The SE attention mechanism is based on a supervised learning method in reinforcement learning, which is used to calculate the correlation between observed features and potential targets; it constructs attention weights through learned content, which are used to adjust the importance of different features, so that the model can focus on key information.
[0049] FPN refers to Feature Pyramid Network, which is mainly proposed for the multi-scale characteristics of objects in images.
[0050] PAN refers to Pyramid Attention Network: It combines the attention mechanism with the spatial pyramid to more accurately extract dense features and improve the accuracy of semantic segmentation.
[0051] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.
Claims
1. A method for detecting helmets and reflective clothing based on drone images, characterized in that: The steps of the method for detecting helmets and reflective clothing based on drone images include: Step 1: Replace the CSPDarknet53 backbone network with the lightweight MobileNetV3 backbone network to reduce the number of network parameters and calculations and improve the inference speed of the model; Step 2: Design a small target detection layer to improve the network's ability to capture target object information; Step 3: Introduce the EMA attention mechanism in the C2f module to improve the ability to capture important features and suppress redundant features; Step 4: Use the bidirectional feature pyramid network BiFPN in the Neck part to effectively reduce the impact of scale changes and complex scenes on the detection results and improve the generalization ability of the model; The attention mechanism first performs global average pooling on the feature map. The relationship between the number of channels in the feature map and the pooling result is: [h,w,c] ==> [None,c]. Then, the output vector is obtained through two fully connected layers. The number of output channels of the first fully connected layer is equal to 1 / 4 of the number of channels of the original input feature map, and the number of output channels of the second fully connected layer is equal to the number of channels in the original input feature map, that is, the dimension is reduced first and then increased. The first fully connected layer uses the ReLU activation function, and the second fully connected layer uses the hard_sigmoid activation function. After two fully connected layers, a vector of channel elements is obtained, each element is the weight of each channel, and the weight is multiplied by the corresponding weight of the original feature map to obtain the new feature map data. The feature map is designed by the original YOLOv8 network structure, which has three sampling scales, which are used to detect small targets, medium targets and large targets respectively; a large-scale feature map is added to the neck network FPN+PAN structure; The feature pyramid network FPN introduces a bottom-up path to fuse multi-scale features from level 2 to level 5. PAN adds an additional top-down path aggregation network on top of FPN. YOLOv8 optimizes the feature pyramid network and removes nodes without feature fusion. BiFPN fuses the up- and down-sampled feature maps layer by layer, and introduces horizontal and vertical connections to fuse and utilize features of different scales, making it robust when dealing with scale changes and complex occlusion scenes. The feature enhancement module is composed of the C2f module of the neck structure and the attention mechanism EMA. C2f is the concretization of the Bottleneck module. The C2f structure consists of two residual network structures. Based on the C2f structure, the feature enhancement module of the EMA attention mechanism is embedded, and the EMA module is embedded in the second residual module of the C2f module. The 1x1 convolution is extracted from the CA module for sharing components to form a 1x1 branch; the 3x3 kernels are placed in parallel for fast multi-scale spatial structure information aggregation to form a 3x3 branch.
2. The method for detecting safety helmets and reflective clothing based on drone images according to claim 1, characterized in that: Step 1 specifically includes: Step 101, pad the input image through 1×1 convolution to increase the number of channels; secondly, apply deep convolution in high-dimensional space; Step 102: Optimize feature map data using SE attention mechanism; Step 103: Use 1×1 convolution to reduce the number of channels. When the stride is 1 and the input and output feature shapes are equal, use residual connection. When the stride is 2, directly output the reduced feature map.
3. The method for detecting safety helmets and reflective clothing based on drone images according to claim 1, characterized in that: In step 3, the attention mechanism is introduced in the first three C2f modules of the neck network.