An improved YOLOv5-based unmanned aerial vehicle capture scene target detection method

Abstract

The application discloses a kind of based on improved YOLOv5's unmanned plane capture scene target detection method, the invention can in the case where guaranteeing accuracy and completeness, efficiently to the small target of unmanned plane capture image is positioned and classified.For the problem that the detection effect is limited by more distribution dense and occluded small target in the image captured by unmanned plane, a hybrid Transformer module is introduced to improve the detection head;According to the characteristics of the target in the unmanned plane image, the attention module uses CBAM for adaptive feature refinement;And introduce CBAM module in the feature fusion structure to improve the feature fusion efficiency of small target;Then the improved feature fusion structure is combined with the Transformer structure, so that the model has better feature fusion capability, reduces the loss of small target features.

Description

Technical Field

[0001] This invention relates to a method for target detection in drone capture scenes using the YOLOv5 deep learning target detection framework, and belongs to the field of computer vision applications. Background Technology

[0002] Object detection is an important research direction in computer vision. Its purpose is to estimate the location and classification information of objects in a corresponding image through a single image. Object detection technology in drone capture scenarios has a wide range of applications in transportation, national defense, wildlife protection, plant protection and many other fields.

[0003] While existing object detection algorithms achieve good results in everyday images, drone scenarios present more densely distributed and occluded small targets, limiting the effectiveness of existing methods. Furthermore, these methods suffer from difficulties in network convergence, slow learning speeds, and high time and resource consumption. This invention primarily utilizes the YOLO series of object detection frameworks and proposes a drone-captured object detection method based on YOLOv5. Summary of the Invention

[0004] The purpose of this invention is to propose a target detection method for drone-captured scenes based on an improved YOLOv5, which can efficiently locate and classify targets in drone-captured images while ensuring accuracy and completeness.

[0005] To address the limitation in detection performance caused by densely distributed and occluded small targets in drone-captured images, this invention introduces a hybrid Transformer module to improve the detection head. Considering the characteristics of targets in drone images, the attention module employs CBAM for adaptive feature refinement. Furthermore, the CBAM module is introduced into the feature fusion structure to improve the feature fusion efficiency for small targets. Finally, the improved feature fusion structure is integrated with the hybrid Transformer structure, giving the model better feature fusion capabilities and reducing the loss of small target features. This invention solves the problem of existing methods neglecting small target features and global and local features, resulting in better performance in small target detection in drone scenarios. The technical solution adopted in this invention is as follows: Step 1: Collect captured images in the drone scene to form sample images; label the target location and classification information on the sample images to form labeled images; the sample images and labeled images form a training set; Step 2: The sample image is first input into the feature extraction module. The first layer, the Focus module, segments and connects the input image while preserving the original information. Layers 2 to 13 consist of consecutive Conv modules, C3 units, and a Convolutional Attention Module (CBAM). The Conv module uses a 3×3 convolutional kernel with a stride of 2; the C3 unit uses a 1×1 convolutional kernel with a stride of 1; the 15th layer is a CA proposed in this invention. The Transformer unit is an improved version of the CBAM module, the feature fusion module, and the hybrid Transformer module. It is responsible for giving the model better feature fusion capabilities and reducing the loss of small target features. Step 3: Layers 16 to 30 consist of consecutive Conv modules, Upsample modules, and CA modules. It consists of an FF module, a C3 unit, and a CBAM module. The Conv operation uses a 1×1 convolutional kernel with a stride of 1; Upsample provides an upsampling operation, doubling the size of the feature map with each upsampling iteration; CA... The FF module is a convolutional attention fusion module proposed in this invention. It is an improved version of the CBAM module and the feature fusion module, responsible for improving the feature fusion efficiency of small targets. The input of this part is processed by CA. The feature map processed by the Transformer unit; after the first upsampling, it is fused with the feature map output from layer 9; after the second upsampling, it is fused with the feature map output from layer 6; after the third upsampling, it is fused with the feature map output from layer 3. Step 4: Layers 31 to 41 consist of consecutive Conv modules, C3 units, CBAM modules, and multiple Concat operations. The Conv operations use 3×3 convolutional kernels with a stride of 2. The input to this part is the feature map processed by the CBAM module in layer 30. The first Concat merges features from layers 31 and 26; the second Concat merges features from layers 35 and 21; and the third Concat merges features from layers 39 and 16. Step 5: Layer 42 is the Detect layer, which is responsible for predicting the processed features. This model contains 4 detection heads, which detect features from layers 41, 37, 33 and 29 respectively. This model introduces a hybrid Transformer module in this layer to improve the detection heads and form a hybrid attention detection head, so that the model has stronger feature representation. Step 6: Train the target detection network and use the trained target detection network for target detection tasks in actual UAV scenarios. Attached Figure Description

[0006] Figure 1 This is an overall schematic diagram of a drone-based scene target detection method based on the improved YOLOv5.

[0007] Figure 2 This is a schematic diagram of the hybrid Transformer module structure.

[0008] Figure 3 It is the convolutional attention feature fusion module (CA) FF structure diagram.

[0009] Figure 4 It is CA Transformer module structure diagram. Detailed Implementation

[0010] This invention proposes an improved YOLOv5-based target detection network for drone capture scenes. This network primarily improves the detection accuracy and speed of small targets in captured images from drone scenes. The purpose of this invention is to improve the following aspects: 1. Based on the overall model design, the original YOLOv5 network design is improved. To enhance the efficiency of feature refinement and feature fusion for small targets, a CBAM module is introduced, and an improved attention fusion module CA is proposed. FF. CA The FF module refines features from both channel and spatial dimensions. The feature fusion process for feature maps at different scales can be represented as follows: Where Z is the feature map after feature fusion, X is the low-level high-resolution feature map in the feature pyramid, Y is the feature map after upsampling of the high-level high semantic feature map, and CBAM(X ⊕ Y) represents the attention weight matrix obtained after element-wise addition of X and Y and passing through the CBAM module. 2. Improved detection head for both global and local features. The improved detection head connects to the original YOLOv5 detection head via a hybrid Transformer module. The hybrid Transformer module consists of two sub-layers; the first layer utilizes a multi-head Self... The Attention structure captures the global context, introduces convolutional layers to extract the local context, and then aggregates the global and local contexts to obtain a stronger feature representation. The second layer is a feedforward neural network, mainly composed of a multilayer perceptron (MLP). LayerNorm is applied before each sub-layer, and DropPath is applied after each sub-layer. In the hybrid Transformer module, multi-head Self The process of calculating the self-attention weights in the Attention structure is as follows: Where Z represents the self-attention weights, X represents the image patch input sequence, and QKT is used to calculate the correlation between each image patch and other image patches. The scaling factor is used to obtain the weight coefficients through softmax normalization. Finally, the weight coefficients and V are weighted and summed to obtain the self-attention weight matrix for each image patch. In equation (2), Q, K, and V are respectively of dimension... , , The three matrices (generally set) = = The input sequence X is multiplied by three randomly initialized matrices. , , The calculation is as follows:

[0011] 3. To improve the model's feature fusion capabilities and reduce the loss of small target features, CA is proposed. The Transformer module replaces the feature fusion module at the residual connection point in the Hybrid Transformer with CA. FF is composed of.

[0012] Step 1: Collect captured images in the drone scene to form sample images; label the target location and classification information on the sample images to form labeled images; the sample images and labeled images form a training set; Step 2: The sample image is first input into the feature extraction module. The first layer, the Focus module, segments and connects the input image while preserving the original information. Layers 2 to 13 consist of consecutive Conv modules, C3 units, and a Convolutional Attention Module (CBAM). The Conv module uses a 3×3 convolutional kernel with a stride of 2; the C3 unit uses a 1×1 convolutional kernel with a stride of 1; the 15th layer is a CA proposed in this invention. The Transformer unit is an improved version of the CBAM module, the feature fusion module, and the hybrid Transformer module. It is responsible for giving the model better feature fusion capabilities and reducing the loss of small target features. Step 3: Layers 16 to 30 consist of consecutive Conv modules, Upsample modules, and CA modules. It consists of an FF module, a C3 unit, and a CBAM module. The Conv operation uses a 1×1 convolutional kernel with a stride of 1; Upsample provides an upsampling operation, doubling the size of the feature map with each upsampling iteration; CA... The FF module is a convolutional attention fusion module proposed in this invention. It is an improved version of the CBAM module and the feature fusion module, responsible for improving the feature fusion efficiency of small targets. The input of this part is processed by CA. The feature map processed by the Transformer unit; after the first upsampling, it is fused with the feature map output from layer 9; after the second upsampling, it is fused with the feature map output from layer 6; after the third upsampling, it is fused with the feature map output from layer 3. Step 4: Layers 31 to 41 consist of consecutive Conv modules, C3 units, CBAM modules, and multiple Concat operations. The Conv operations use 3×3 convolutional kernels with a stride of 2. The input to this part is the feature map processed by the CBAM module in layer 30. The first Concat merges features from layers 31 and 26; the second Concat merges features from layers 35 and 21; and the third Concat merges features from layers 39 and 16. Step 5: Layer 42 is the Detect layer, which is responsible for predicting the processed features. This model contains 4 detector heads, which detect features from layers 41, 37, 33 and 29 respectively. This model introduces a hybrid Transformer module in this layer to improve the detector heads and form a hybrid attention detector head, so that the model has stronger feature representation. Step 6: Train the target detection network and use the trained target detection network for target detection tasks in actual UAV scenarios.

Claims

1. A target detection method for drone-captured scenes based on an improved YOLOv5 can efficiently locate and classify targets in drone-captured images while ensuring accuracy and completeness. Its features include... The following steps are required: Step 1: Collect captured images in the drone scene to form sample images; label the target location and classification information on the sample images to form labeled images; the sample images and labeled images form a training set; Step 2: Input the sample image into the feature extraction module. The first layer is the Focus module, which segments and connects the input image while preserving the original information. The second to thirteenth layers are consecutive Conv modules, C3 units, and convolutional attention modules (CBAM). The Conv module uses a 3×3 convolutional kernel with a stride of 2; the C3 unit uses a 1×1 convolutional kernel with a stride of 1; the 15th layer is a CA. The Transformer unit, an improved version of the CBAM module, feature fusion module, and hybrid Transformer module, is responsible for enhancing the model's feature fusion capabilities and reducing feature loss for small targets. The improved detection head is connected to the original YOLOv5 detection head via the hybrid Transformer module. The hybrid Transformer module consists of two sub-layers; the first layer utilizes multi-head Self... The Attention structure captures the global context, introduces convolutional layers to extract the local context, and then aggregates the global and local contexts to obtain a stronger feature representation. The second layer is a feedforward neural network, mainly composed of a multilayer perceptron (MLP). LayerNorm is applied before each sub-layer, and DropPath is applied after each sub-layer. Step 3: Layers 16 to 30 consist of consecutive Conv modules, Upsample modules, and CA modules. It consists of an FF module, a C3 unit, and a CBAM module; The Conv operation uses a 1×1 convolutional kernel with a stride of 1; Upsample provides an upsampling operation, doubling the size of the feature map with each upsampling iteration; CA The FF module is a convolutional attention fusion module, which is an improved version of the CBAM module and the feature fusion module. It is responsible for improving the feature fusion efficiency for small targets. The input of this part is processed by CA. Feature maps processed by the Transformer unit; the first upsampled feature map is fused with the feature map output from layer 9; the second upsampled feature map is fused with the feature map output from layer 6; the third upsampled feature map is fused with the feature map output from layer 3; CA The FF module refines features from both channel and spatial dimensions. The feature fusion process for feature maps at different scales can be represented as: Z = CBAM(X Y) X+(1-CBAM(X Y)) Y, where Z is the feature map after feature fusion, X is the low-level high-resolution feature map in the feature pyramid, Y is the feature map after upsampling of the high-level high semantic feature map, and CBAM(X ⊕ Y) represents the attention weight matrix obtained after element-wise addition of X and Y and passing through the CBAM module. Step 4: Layers 31 to 41 consist of consecutive Conv modules, C3 units, CBAM modules, and multiple Concat operations; the Conv operations use 3×3 convolutional kernels with a stride of 2; the input to this part is the feature map processed by the CBAM module in layer 30; the first Concat merges the features from layers 31 and 26; the second Concat merges the features from layers 35 and 21. The third concat merges the features from layer 39 and layer 16; Step 5: Layer 42 is the Detect layer, which is responsible for predicting the processed features. This model contains four detection heads, which detect features from layers 41, 37, 33 and 29 respectively. This model introduces a hybrid Transformer module in this layer to improve the detection heads and form a hybrid attention detection head, so that the model has stronger feature representation. Step 6: Train the target detection network and use the trained target detection network for target detection tasks in actual UAV scenarios.

Images