Unmanned aerial vehicle image small target detection method based on improved YOLOv8

Abstract

The invention relates to the field of computer vision target detection, in particular to an unmanned aerial vehicle image small target detection method based on improved YOLOv8. The method comprises the steps of preparing an unmanned aerial vehicle captured image target detection data set and performing preprocessing, constructing a YOLOv8 reference network model and setting training parameters; improving a backbone network of the YOLOv8 reference network model by using a feature extraction module RFN based on a receptive field attention mechanism; a feature fusion module SAF based on a scale attention mechanism is added in front of a detection head of the YOLOv8 reference network model; and inputting a training image into the improved network model for training, and verifying a result. The application of the invention effectively improves the efficiency and precision of detecting the small target in the image captured by the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the field of computer vision target detection, and in particular to a method for detecting small targets in unmanned aerial vehicle images based on improved YOLOv8. Background Art

[0002] Object detection is a fundamental task in computer vision and is widely used in military operations, autonomous driving, security monitoring, remote sensing and other fields. In recent years, object detection in images captured by drones has become an indispensable application in many fields due to the large coverage and high resolution of drone images. In these applications, detecting small objects such as vehicles, pedestrians and various infrastructures is key to assessing the state of the environment, understanding human activities and ensuring the safety of important facilities. However, accurately detecting small objects in drone remote sensing images is extremely challenging.

[0003] Due to the rapid development of deep learning technology and the continuous innovation of researchers around the world, object detection has made significant progress. These developments have not only improved the accuracy and efficiency of object detection systems, but also improved their applicability in various real-world scenarios. At present, the mainstream deep learning-based object detection methods include the R-CNN series, SSD, YOLO series, etc. YOLO is a single-stage object detection method. Compared with traditional two-stage methods (such as the R-CNN series), the design concept of YOLO is to regard object detection as a regression problem, directly predicting the bounding box and its category probability from the input image. Therefore, the YOLO series of algorithms have higher real-time performance and lower latency while maintaining high detection accuracy.

[0004] Although existing object detection algorithms perform well in most vision tasks, they still face considerable challenges in detecting small objects in drone-captured images. This is mainly due to the low resolution of small objects and blurred object details caused by the high-altitude operation of drones. In addition, the sizes of objects in drone images vary, and they are often partially occluded, making detection more complicated. At the same time, the detection head of the YOLOv8 baseline network model lacks global contextual information, which affects the accuracy and recall of the prediction. Therefore, achieving more accurate and efficient detection of small objects in drone-captured images remains a pressing problem. Summary of the invention

[0005] In order to solve the problem that the existing target detection algorithm has low accuracy and efficiency when detecting small targets in drone captured images, the present invention provides a drone image small target detection method based on improved YOLOv8, thereby achieving the purpose of effectively improving the efficiency and accuracy of detecting small targets in drone captured images.

[0006] The present invention provides a method for detecting small targets in drone images based on improved YOLOv8, which is characterized by comprising the following steps: Step 1: Obtain and preprocess the drone captured image dataset, convert the annotation format into YOLO format, and divide it into training set, validation set, and test set; then build the YOLOv8 benchmark network model and adjust the training parameters; Step 2, construct a feature extraction module RFN based on the receptive field attention mechanism, and use the feature extraction module RFN to improve the backbone network of the YOLOv8 benchmark network model; Step 3: construct a feature fusion module SAF based on the scale attention mechanism, and add the feature fusion module SAF before the detection head of the YOLOv8 benchmark network model; Step 4: Input the images of the training set constructed in step 1 into the improved network model for training and verify the results.

[0007] Furthermore, in step 1, YOLOv8 is selected as the improved baseline network model. YOLOv8 is divided into the following three parts: a backbone network. The backbone network of YOLOv8 is a key component for extracting features from the input image. It consists of a CBS module and a C2f module, wherein the CBS module is used for downsampling operations and the C2f module is used for feature extraction; a neck network. The neck network of the YOLOv8 baseline network model adopts a path aggregation network + feature pyramid network structure, which allows secondary fusion of low-level features with high-level features, and helps capture targets of different scales; a detection head. The detection head is responsible for generating the final prediction output, including the location and size of the bounding box, and the category probability of each box.

[0008] Furthermore, in step 2, the construction process of the feature extraction module RFN based on the receptive field attention mechanism is as follows: The feature extraction module RFN consists of two parts: residual structure and receptive field attention; the feature extraction module RFN receives a shape of Original feature map ,in is the number of channels of the feature map, and are the height and width of the feature map respectively; In the residual structure, the original feature map After a convolution kernel size of Partial convolution is used to extract the features of the first quarter channel, and the output shape is Feature map ; Use a convolution kernel size of The standard convolution transforms the feature map The number of channels is expanded to twice the original number and batch normalization and Activation function, the output shape is Feature map ; Use a convolution kernel size of The standard convolution transforms the feature map The number of channels is reduced to , the output shape is Feature map ; The feature map With the input original feature map Add each element to get the output feature map of the residual structure ; The calculation formula of the residual structure is, in, Represents the original feature map received by the feature extraction module RFN; The convolution kernel is represented by Partial convolution; The convolution kernel is represented by Standard convolution of and Represent batch normalization operation and activation function respectively; Represents the feature map after partial convolution; Represents the first Feature map after standard convolution; Represents the second Feature map after convolution; Output feature map representing the residual structure; In the receptive field attention, for the output feature map of the residual structure , the convolution kernel is applied as The grouped convolution expands the number of channels and generates a shape of Receptive field spatial feature map ; Output feature map for the residual structure , through average pooling and convolution kernel is The grouped convolution increases the depth of the feature map while keeping its resolution unchanged, and uses The activation function normalizes the weights of each position in the feature map to obtain a shape of Weight graph ; Represents a set of convolution kernels. Each element in the feature map of the receptive field space Multiply the elements at the corresponding positions to get the weighted feature map ; The weighted feature map Shaped as The shape of the convolution kernel is The standard convolution of reduces the resolution to produce a shape of Feature map ; The calculation formula for receptive field attention is, in, is the output feature map of the residual structure; The convolution kernel is represented by Grouped convolution; , They represent average pooling, Activation function; The representative shape is Receptive field spatial feature map; The representative shape is Weight map; Representative feature map Reshape Shape; Symbol represents element-wise multiplication, Each element in The weight graph Each element in And the receptive field space feature map The elements at the corresponding position obtained by multiplication; It is the feature map finally output by the feature extraction module RFN.

[0009] Furthermore, in step 2, the process of using the feature extraction module RFN to improve the backbone network of the YOLOv8 benchmark network model is as follows: The backbone network of the YOLOv8 benchmark network model is divided into four consecutive stages, each stage contains a downsampling layer and multiple feature extraction modules RFN; the downsampling multiple of the first stage is 1 / 4, and the number of feature extraction modules RFN is 2; the downsampling multiple of the second stage is 1 / 2, and the number of feature extraction modules RFN is 2; the downsampling multiple of the third stage is 1 / 2, and the number of feature extraction modules RFN is 8; the downsampling multiple of the fourth stage is 1 / 2, and the number of feature extraction modules RFN is 2.

[0010] Furthermore, in step 3, the construction process of the feature fusion module SAF based on the scale attention mechanism is as follows: The feature fusion module SAF receives three feature maps of different scales from the neck network, namely, feature maps , feature map , feature map , use convolution operation to keep the same number of channels for three feature maps of different shapes; apply scale attention mechanism for weighted fusion; For the lowest resolution feature map , for the feature map Downsample to make its resolution equal to the feature map Keep consistent; for the downsampled feature map Use the scale attention mechanism to weight; the weighted feature map With feature map Add and divide by 2 to get the fused feature map , and fuse the feature map Added to the output list of the feature fusion module SAF, the calculation formula is, For the feature map with the highest resolution , using bilinear interpolation from the feature map Upsample to make its resolution equal to the feature map Keep consistent; for the upsampled feature map Use the scale attention mechanism to weight; the weighted feature map With feature map Add and divide by 2 to get the fused feature map , and fuse the feature map Added to the output list of the feature fusion module SAF, the calculation formula is, For the intermediate layer feature map , using bilinear interpolation from the feature map Upsample to make its resolution equal to the feature map Keep consistent and upsample the feature map Use scale attention mechanism to weight feature maps; Downsample to make its resolution equal to the feature map Keep consistent and downsample the feature map Use the scale attention mechanism to weight; the weighted feature map , weighted feature map And the intermediate layer feature map Add and divide by 3 to get the fused feature map , the fusion feature map Added to the output list of the feature fusion module SAF, the calculation formula is, in, For the Feature map after layer fusion; is the scale attention mechanism; represents downsampling; Indicates upsampling.

[0011] Furthermore, in step 3, the weighting process of the scale attention mechanism is as follows: Input feature map for scale attention mechanism , use average pooling to extract input feature maps The global information in the output shape is Feature map ; For feature map Apply the convolution kernel as Standard convolution of ; after the convolution operation, apply Activation function and Activation function generates scalar weights ; Use scalar weights For the input feature map Weight each element of to obtain the weighted feature map ; The weighted calculation formula using the scale attention mechanism is, in, It is the input feature map of the scale attention mechanism; It is the feature map weighted by the scale attention mechanism; Represents the scalar weight generated by the scale attention mechanism; and Respectively represent Activation function and Activation function.

[0012] The present invention provides a method for detecting small targets in drone images based on improved YOLOv8. The method first prepares a drone captured image target detection dataset and performs preprocessing, constructs a YOLOv8 benchmark network model and sets training parameters; then uses a feature extraction module RFN based on a receptive field attention mechanism to improve the backbone network of the YOLOv8 benchmark network model; then, adds a feature fusion module SAF based on a scale attention mechanism before the detection head of the YOLOv8 benchmark network model; finally, inputs the training image into the improved network model for training and verifies the result, and obtains a good model weight file for detecting various targets in drone captured images. In summary, the present invention has the positive effect of effectively improving the efficiency and accuracy of small target detection in drone captured images. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a flow chart for realizing the present invention; Figure 2 The architecture diagram of the feature extraction module RFN constructed for the present invention; Figure 3 The architecture diagram of the backbone network improved by using the feature graph module RFN in the present invention; Figure 4 The architecture diagram of the feature fusion module SAF constructed for the present invention. DETAILED DESCRIPTION

[0014] The present invention provides a method for detecting small targets in drone images based on improved YOLOv8, and the implementation steps are as follows: Figure 1 As shown, it is mainly achieved through the following process.

[0015] Step 1: Download the drone-captured image dataset VisDrone2019 for target detection tasks from the official website of the VisDrone2019 challenge; use Python to write a script to convert the XML format annotation file into YOLO format; divide the dataset into 6471 training images, 548 validation images, and 1610 test images.

[0016] Step 2: Build the YOLOv8 benchmark network model. The YOLOv8 benchmark network model has a simple structure and stable detection accuracy. It mainly consists of three main parts: (1) Backbone network. The backbone network of YOLOv8 is the key component for extracting features from the input image. It consists of a CBS module and a C2f module. The CBS module is used for downsampling operations and the C2f module is used for feature extraction. (2) Neck network. The neck network of YOLOv8 adopts the structure of path aggregation network + feature pyramid network, which allows low-level features to be fused with high-level features twice, which helps to capture objects of different scales. (3) Detection head. The detection head is responsible for generating the final prediction output, which includes the location and size of the bounding box and the category probability of each box. Set the resolution of the input image to 640×640, the training round to 300, and enable the SGD optimizer and automatic mixed precision training.

[0017] Step 3: Construct a feature extraction module RFN based on the receptive field attention mechanism. Figure 2 As shown, the feature extraction module RFN consists of two parts: residual structure and receptive field attention; the feature extraction module RFN receives a shape of Original feature map ,in is the number of channels of the feature map, and are the height and width of the feature map respectively; In the residual structure, the original feature map After a convolution kernel size of Partial convolution is used to extract the features of the first quarter channel, and the output shape is Feature map ; Use a convolution kernel size of The standard convolution transforms the feature map The number of channels is expanded to twice the original number and batch normalization and Activation function, the output shape is Feature map ; Use a convolution kernel size of The standard convolution transforms the feature map The number of channels is reduced to , the output shape is Feature map ; The feature map With the input original feature map Add each element to get the output feature map of the residual structure ; The calculation formula of the residual structure is, in, Represents the original feature map received by the feature extraction module RFN; The convolution kernel is represented by Partial convolution; The convolution kernel is represented by Standard convolution of and Represent batch normalization operation and activation function respectively; Represents the feature map after partial convolution; Represents the first Feature map after standard convolution; Represents the second Feature map after convolution; Output feature map representing the residual structure; In the receptive field attention, for the output feature map of the residual structure , the convolution kernel is applied as The grouped convolution expands the number of channels and generates a shape of Receptive field spatial feature map ; Output feature map for the residual structure , through average pooling and convolution kernel is The grouped convolution increases the depth of the feature map while keeping its resolution unchanged, and uses The activation function normalizes the weights of each position in the feature map to obtain a shape of Weight graph ; Represents a set of convolution kernels. Each element in the feature map of the receptive field space Multiply the elements at the corresponding positions to get the weighted feature map ; The weighted feature map Shaped as The shape of the convolution kernel is The standard convolution of reduces the resolution to produce a shape of Feature map ; The calculation formula for receptive field attention is, in, is the output feature map of the residual structure; The convolution kernel is represented by Grouped convolution; , They represent average pooling, Activation function; The representative shape is Receptive field spatial feature map; The representative shape is Weight map; Representative feature map Reshape Shape; Symbol represents element-wise multiplication, Each element in The weight graph Each element in And the receptive field space feature map The elements at the corresponding position obtained by multiplication; It is the feature map finally output by the feature extraction module RFN.

[0018] Step 4: Use the feature extraction module RFN to improve the backbone network of the YOLOv8 baseline network model. Figure 3 As shown in the figure, the improved backbone network is divided into four consecutive stages, each stage contains a downsampling layer and multiple feature extraction modules RFN; the downsampling multiple of the first stage is 1 / 4, and the number of feature extraction modules RFN is 2; the downsampling multiple of the second stage is 1 / 2, and the number of feature extraction modules RFN is 2; the downsampling multiple of the third stage is 1 / 2, and the number of feature extraction modules RFN is 8; the downsampling multiple of the fourth stage is 1 / 2, and the number of feature extraction modules RFN is 2.

[0019] Step 5: Construct a feature fusion module SAF based on the scale attention mechanism. Figure 4 As shown, the feature fusion module SAF receives three feature maps of different scales from the neck network, namely, feature maps , feature map , feature map , use convolution operation to keep the same number of channels for three feature maps of different shapes; apply scale attention mechanism for weighted fusion; For the lowest resolution feature map , for the feature map Downsample to make its resolution equal to the feature map Keep consistent; for the downsampled feature map Use the scale attention mechanism to weight; the weighted feature map With feature map Add and divide by 2 to get the fused feature map , and fuse the feature map Added to the output list of the feature fusion module SAF, the calculation formula is, For the feature map with the highest resolution , using bilinear interpolation from the feature map Upsample to make its resolution equal to the feature map Keep consistent; for the upsampled feature map Use the scale attention mechanism to weight; the weighted feature map With feature map Add and divide by 2 to get the fused feature map , and fuse the feature map Added to the output list of the feature fusion module SAF, the calculation formula is, For the intermediate layer feature map , using bilinear interpolation from the feature map Upsample to make its resolution equal to the feature map Keep consistent and upsample the feature map Use scale attention mechanism to weight feature maps; Downsample to make its resolution equal to the feature map Keep consistent and downsample the feature map Use the scale attention mechanism to weight; the weighted feature map , weighted feature map And the intermediate layer feature map Add and divide by 3 to get the fused feature map , the fusion feature map Added to the output list of the feature fusion module SAF, the calculation formula is, in, For the Feature map after layer fusion; is the scale attention mechanism; represents downsampling; Indicates upsampling.

[0020] Specifically, the weighting process of the scale attention mechanism in step 5 is as follows: Input feature map for scale attention mechanism , use average pooling to extract input feature maps The global information in the output shape is Feature map ; For feature map Apply the convolution kernel as Standard convolution of ; after the convolution operation, apply Activation function and Activation function generates scalar weights ; Use scalar weights For the input feature map Weight each element of to obtain the weighted feature map ; The weighted calculation formula using the scale attention mechanism is, in, It is the input feature map of the scale attention mechanism; It is the feature map weighted by the scale attention mechanism; Represents the scalar weight generated by the scale attention mechanism; and Respectively represent Activation function and Activation function.

[0021] Step 6: Input the training data set constructed in step 1 into the improved network model for training, and use the verification data set constructed in step 1 to verify and evaluate the training results.

[0022] Specifically, in order to evaluate the effectiveness of the improved model and compare it with other mainstream target detection models, training and verification were carried out in the same experimental environment. The GPU used was NVIDIA RTX 2060 with a video memory capacity of 6G, the operating system was Ubuntu 22.04, and the software environment was Python 3.8, Pytorch 1.13.1, and CUDA 11.7.

[0023] The average indicators of the verification results include precision, recall, and average accuracy (map), where map50 represents the average precision calculated when the intersection-over-union threshold is 0.5, and map50-95 represents the average precision calculated within the range of the intersection-over-union threshold from 0.5 to 0.95 with a step size of 0.05 each time (i.e. 0.5, 0.55, 0.6, ..., 0.95). The experimental data are shown in Table 1 below: Table 1 Comparative experimental data of the present invention and other mainstream target detection models It can be seen from the comparative data of various models in Table 1 that for small targets in the images captured by drones, the model provided by the present invention has achieved good results in various evaluation indicators. Therefore, the model provided by the present invention is more suitable for the detection of images captured by drones.

Claims

1. A method for detecting small targets in drone images based on improved YOLOv8, characterized in that: The following steps are included: Step 1: Obtain and preprocess the drone captured image dataset, convert the annotation format into YOLO format, and divide it into training set, validation set, and test set; then build the YOLOv8 benchmark network model and adjust the training parameters; Step 2, construct a feature extraction module RFN based on the receptive field attention mechanism, and use the feature extraction module RFN to improve the backbone network of the YOLOv8 benchmark network model; Step 3: construct a feature fusion module SAF based on the scale attention mechanism, and add the feature fusion module SAF before the detection head of the YOLOv8 benchmark network model; Step 4: Input the images of the training set constructed in step 1 into the improved network model for training and verify the results.

2. The method for detecting small targets in drone images based on improved YOLOv8 according to claim 1, characterized in that: In step 2, the construction process of the feature extraction module RFN is as follows: The feature extraction module RFN consists of two parts: residual structure and receptive field attention; the feature extraction module RFN receives a shape of The original feature map ,in is the number of channels of the feature map, and are the height and width of the feature map respectively; In the residual structure, the original feature map After a convolution kernel size of Partial convolution is used to extract the features of the first quarter channel, and the output shape is Feature map ; Use a convolution kernel size of The standard convolution transforms the feature map The number of channels is expanded to twice the original number and batch normalization and Activation function, the output shape is Feature map ; Use a convolution kernel size of The standard convolution transforms the feature map The number of channels is reduced to , the output shape is Feature map ; The feature map With the input original feature map Add each element to get the output feature map of the residual structure ; The calculation formula of the residual structure is, in, Represents the original feature map received by the feature extraction module RFN; The convolution kernel is represented by Partial convolution; The convolution kernel is represented by Standard convolution of and Represent batch normalization operation and activation function respectively; Represents the feature map after partial convolution; Represents the first Feature map after standard convolution; Represents the second Feature map after convolution; Output feature map representing the residual structure; In the receptive field attention, for the output feature map of the residual structure , the convolution kernel is applied as The grouped convolution expands the number of channels and generates a shape of Receptive field spatial feature map ; Output feature map for the residual structure , through average pooling and convolution kernel is The grouped convolution increases the depth of the feature map while keeping its resolution unchanged, and uses The activation function normalizes the weights of each position in the feature map to obtain a shape of The weight graph ; Represents a set of convolution kernels. Each element in the receptive field space feature map Multiply the elements at the corresponding positions to get the weighted feature map ; The weighted feature map Shaped as The shape of the convolution kernel is The standard convolution of reduces the resolution to produce a shape of Feature map ; The calculation formula for receptive field attention is, in, is the output feature map of the residual structure; The convolution kernel is represented by Grouped convolution; , They represent average pooling, Activation function; The representative shape is Receptive field spatial feature map; The representative shape is Weight map; Representative feature map Reshape Shape; Symbol represents element-wise multiplication, Each element in is the weight graph A. And the receptive field space feature map The elements at the corresponding position obtained by multiplication; It is the feature map finally output by the feature extraction module RFN.

3. The method for detecting small targets in drone images based on improved YOLOv8 according to claim 2, characterized in that: In step 2, the process of using the feature extraction module RFN to improve the backbone network of the YOLOv8 baseline network model is as follows, The backbone network of the YOLOv8 benchmark network model is divided into four consecutive stages, each stage contains a downsampling layer and multiple feature extraction modules RFN; the downsampling multiple of the first stage is 1 / 4, and the number of feature extraction modules RFN is 2; the downsampling multiple of the second stage is 1 / 2, and the number of feature extraction modules RFN is 2; the downsampling multiple of the third stage is 1 / 2, and the number of feature extraction modules RFN is 8; the downsampling multiple of the fourth stage is 1 / 2, and the number of feature extraction modules RFN is 2.

4. The method for detecting small targets in drone images based on improved YOLOv8 according to claim 1, characterized in that: In step 3, the construction process of the feature fusion module SAF is as follows: The feature fusion module SAF receives three feature maps of different scales from the neck network, namely, feature maps , feature map , feature map , convolution operations are used to keep the same number of channels for the three feature maps of different shapes; Apply scale attention mechanism for weighted fusion; For the lowest resolution feature map , for the feature map Downsample to make its resolution equal to the feature map Keep consistent; for the downsampled feature map Use the scale attention mechanism to weight; the weighted feature map With feature map Add and divide by 2 to get the fused feature map , and fuse the feature map Added to the output list of the feature fusion module SAF, the calculation formula is, For the feature map with the highest resolution , using bilinear interpolation from the feature map Upsample to make its resolution equal to the feature map Keep consistent; for the upsampled feature map Use the scale attention mechanism to weight; the weighted feature map With feature map Add and divide by 2 to get the fused feature map , and fuse the feature map Added to the output list of the feature fusion module SAF, the calculation formula is, For the intermediate layer feature map , using bilinear interpolation from the feature map Upsample to make its resolution equal to the feature map Keep consistent and upsample the feature map Use scale attention mechanism to weight feature maps; Downsample to make its resolution equal to the feature map Keep consistent and downsample the feature map Use the scale attention mechanism to weight; the weighted feature map , weighted feature map And the intermediate layer feature map Add and divide by 3 to get the fused feature map , the fusion feature map Added to the output list of the feature fusion module SAF, the calculation formula is, in, For the Feature map after layer fusion; is the scale attention mechanism; represents downsampling; Indicates upsampling.

5. The method for detecting small targets in drone images based on improved YOLOv8 according to claim 4, characterized in that: The weighting process of the scale attention mechanism is as follows, Input feature map for scale attention mechanism , use average pooling to extract input feature maps The global information in the output shape is Feature map ; For feature map Apply the convolution kernel as Standard convolution of ; after the convolution operation, apply Activation function and Activation function generates scalar weights ; Use scalar weights For the input feature map Weight each element of to obtain the weighted feature map ; The weighted calculation formula using the scale attention mechanism is, in, It is the input feature map of the scale attention mechanism; It is the feature map weighted by the scale attention mechanism; Represents the scalar weight generated by the scale attention mechanism; and Respectively represent Activation function and Activation function.

Images