Unmanned aerial vehicle aerial image target detection method based on improved YOLOv8

Abstract

The invention belongs to the field of computer vision and image processing, and particularly relates to an unmanned aerial vehicle aerial image target detection method based on improved YOLOv8, and the method comprises the steps: collecting an unmanned aerial vehicle aerial image, and carrying out the preprocessing of the image, and obtaining a preprocessing image set; adding a small target detection branch and a special small target detection head in the YOLOv8 network, and replacing a C2F module in the YOLOv8 network with a T-C2F structure module to obtain a target detection model; inputting the preprocessed image into a target detection model to obtain a preliminary detection result, and post-processing the preliminary detection result to obtain a detection result; training a target detection model by adopting the preprocessed image set; calculating loss according to a detection result and carrying out back propagation on training model parameters; performing unmanned aerial vehicle aerial image target detection by adopting the target detection model; according to the invention, the small target detection performance is improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and image processing, and in particular relates to a method for detecting targets in unmanned aerial vehicle aerial images based on improved YOLOv8. Background Art

[0002] With the rapid development of drone technology, drone aerial photography is increasingly being used in various fields, such as agricultural plant protection, disaster relief, military reconnaissance, etc. In these applications, it is crucial to quickly, accurately, and in real time detect targets in drone aerial images. Object detection is an important research direction in the field of computer vision, and its application in drone aerial images is particularly important. Real-time and accurate detection of targets in drone aerial images is the basis for many practical applications, such as automatic tracking and obstacle avoidance. However, due to the dynamics of drone aerial images, changes in viewing angles, and changes in target size, target detection faces many challenges.

[0003] At present, in the field of target detection, target detection methods based on deep learning dominate, and the YOLO series of algorithms have attracted widespread attention for their high efficiency and accuracy. In particular, although the existing YOLOv8-based methods have achieved efficient target detection in drone aerial images, the target detection task becomes particularly difficult in the face of acute problems such as target occlusion, truncation, blur, and high density in complex scenes, and it is easy to miss and misdetect. Therefore, how to further optimize the YOLO series-based drone aerial image target detection method and improve its accuracy and real-time performance is still a hot topic and difficulty in current research. Summary of the invention

[0004] The present invention provides a target detection method for unmanned aerial image based on improved YOLOv8, aiming to solve the problems of missed detection of incomplete targets and difficulty in detecting small targets in the prior art.

[0005] Specific solutions include:

[0006] S1. Collect drone aerial images and preprocess them to obtain a preprocessed image set;

[0007] S2. Introduce the BiFPN network into the YOLOv8 network, add a small target detection branch and a dedicated small target detection head, and use the T-C2F structure module to replace the C2F module in the YOLOv8 network to obtain the target detection model;

[0008] S3. Input the preprocessed image into the target detection model to obtain a preliminary detection result, and post-process the preliminary detection result to obtain a detection result; use the preprocessed image set to train the target detection model;

[0009] S4. Calculate the loss based on the detection results and back-propagate the training model parameters;

[0010] S5. Use the target detection model obtained in S4 to perform target detection in UAV aerial images.

[0011] Furthermore, the T-C2F structure module includes a cascaded first CBS layer, a Split layer, a Bottle module, a Concat layer, a Transformer layer and a second CBS layer. The Bottle module includes N cascaded Bottleneck layers; the first CBS layer, the Split layer and the N Bottleneck layers are all residually connected to the Concat layer.

[0012] Furthermore, the Transformer layer includes two feature extraction modules and one Contraction layer; each feature extraction module includes a cascaded Contraction layer, an MHSA layer, and an Expansion layer, and the input of the feature extraction module is concatenated and fused with the output of the Expansion layer to form the output of the feature extraction module.

[0013] Furthermore, the target detection model includes a Backbone module, a Neck module and a Head module, wherein:

[0014] The Backbone module includes a first Conv layer, a second Conv layer, a first T-C2F structure module, a third Conv layer, a second T-C2F structure module, a fourth Conv layer, a third T-C2F structure module, a fifth Conv layer, a fourth T-C2F structure module and an SPPF layer, which are connected layer by layer;

[0015] The Neck module uses the BiFPN network and adds a small target detection branch to the BiFPN network through a bidirectional cross-layer direct connection to further introduce shallower features;

[0016] The Head module includes detection head 1, detection head 2, detection head 3 and a special small target detection head.

[0017] Further, the Neck module includes a first upsampling layer, a first Concat layer, a fifth T-C2F structure module, a second upsampling layer, a second Concat layer, a sixth T-C2F structure module, a third upsampling layer, a third Concat layer, a seventh T-C2F structure module, a sixth Conv layer, a fourth Concat layer, an eighth T-C2F structure module, a seventh Conv layer, a fifth Concat layer, a ninth T-C2F structure module, an eighth Conv layer, a sixth Concat layer, and a tenth T-C2F structure module, which are connected layer by layer;

[0018] Among them, the fifth T-C2F structure module is connected to the fifth Concat layer, and the sixth T-C2F structure module is connected to the fourth Concat layer; the second T-C2F structure module is jump-connected to the fourth Concat layer, and the third T-C2F structure module is jump-connected to the fifth Concat layer.

[0019] Furthermore, the dedicated small object detection head includes a cascaded attention mechanism layer, an anchor box rotation layer, a deformable convolution layer and a 1×1 convolution layer.

[0020] Furthermore, the attention mechanism layer includes a cascaded GAP layer, a first Conv2d layer, a first BN+Sigmoid layer, an upsample layer, a second Conv2d layer and a second BN+Sigmoid layer; the input of the attention mechanism layer and the output of the second BN+Sigmoid layer are concatenated and fused to form the output of the attention mechanism layer.

[0021] Furthermore, the preprocessing process includes:

[0022] S11. De-noising the drone aerial image by using a median filter to obtain a de-noised image;

[0023] S12. performing contrast enhancement on the denoised image using histogram equalization to obtain an enhanced image;

[0024] S13. Convert the enhanced image from RGB image format to YUV color space to obtain a preprocessed image.

[0025] Beneficial effects of the present invention:

[0026] The present invention introduces the Transformer encoder to construct a deep semantic understanding unit, so that feature points can capture long-distance feature dependencies, thereby obtaining rich contextual information. This enhances the network's understanding of incomplete targets in complex scenes and effectively improves the detection effect in situations such as target occlusion and blur.

[0027] The present invention adds skip connections on the basis of a simplified PANet scheme and integrates more features, so that the network can capture targets of different sizes at the same time and improves the robustness to target scale changes.

[0028] In order to further improve the algorithm's ability to detect small-scale targets, this paper adds a small target branch and designs a dedicated small target detection head. This detection head suppresses background information through the attention mechanism, and more accurately describes the target position information through the anchor frame rotation network and deformable convolution. At the same time, the decoupled detection head is used to decouple the classification task and the regression task, effectively improving the small target detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a flow chart of the method of the present invention;

[0030] Figure 2 This is a structural diagram of the deep semantic understanding unit of the Transformer encoder of the present invention;

[0031] Figure 3 This is the Transformer layer structure diagram of the present invention;

[0032] Figure 4 This is a diagram of the MHSA layer structure of the present invention;

[0033] Figure 5 Schematic diagram of the BiFPN network structure of the present invention;

[0034] Figure 6 This is a structural diagram of a small target detection head dedicated to the present invention;

[0035] Figure 7 This is the structure diagram of the attention mechanism layer of the present invention;

[0036] Figure 8 This is the overall structure diagram of the target detection model of the present invention;

[0037] Fig. 9 This is a structural diagram of the decoupling detection head of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The present invention provides a method for detecting targets in drone aerial images based on improved YOLOv8. Figure 1 As shown, the following steps are included:

[0040] S1. Collect UAV aerial images and preprocess them to obtain a preprocessed image set.

[0041] Specifically, the preprocessing operation includes denoising, contrast enhancement, and color space conversion. The preprocessing process of the embodiment of the present invention includes:

[0042] S11. De-noising the drone aerial image by using a median filter to obtain a de-noised image;

[0043] S12. performing contrast enhancement on the denoised image using histogram equalization to obtain an enhanced image;

[0044] S13. Convert the enhanced image from RGB image format to YUV color space to obtain a preprocessed image.

[0045] S2. The present invention improves the existing YOLOv8 network structure, adds a small target detection branch and a dedicated small target detection head in the YOLOv8 network, and uses a T-C2F structure module to replace the C2F module in the YOLOv8 network, and uses the improved YOLOv8 network as a target detection model for training and learning.

[0046] Preferably, the present invention introduces a Transformer layer based on the C2F module of the YOLOv8 network, and further proposes a Transformer encoder deep semantic understanding unit (referred to as T-C2F structure module), such as Figure 2 As shown in the figure, the T-C2F structural module includes the cascaded first CBS layer, Split layer, Bottle module, Concat layer, Transformer layer and second CBS layer. The Bottle module includes N cascaded Bottleneck layers; the first CBS layer, Split layer and N Bottleneck layers are all residually connected to the Concat layer. The Transformer layer is used to build a deep semantic understanding unit, so that the feature points not only focus on local information, but also capture long-distance feature dependencies, thereby obtaining rich contextual information. Enhance the deep learning network's understanding of incomplete targets in complex scenes, and effectively improve the detection effect under target occlusion, stage, and blur.

[0047] Specifically, Figure 3 As shown in the figure, the Transformer layer includes two feature extraction modules and one Contraction layer; each feature extraction module includes a cascaded Contraction layer, MHSA layer and Expansion layer, and the input of the feature extraction module is concatenated with the output of the Expansion layer to form the output of the feature extraction module. The MHSA layer uses a multi-head attention mechanism as shown in Figure 4 As shown in (a), it consists of multiple self-attention layers, and the structure of each self-attention layer is as follows Figure 4 As shown in (b), it includes three parallel 1×1 convolutions. The first 1×1 convolution result is transposed and multiplied with the second 1×1 convolution result, and then processed by softmax to obtain the attention feature map. The third 1×1 convolution result is multiplied with the attention feature map to obtain the self-attention feature map.

[0048] Preferably, the present invention adopts the BiFPN network structure to replace the feature fusion strategy of the PANet network structure in the YOLOv8 network. In short, the present invention adds a jump connection based on the simplified PANet solution, that is, a jump connection is added between the input node and the output node of the two scales of 16x and 8x. In this way, each node in the top-down path fuses 2 features of the same scale, and each node in the bottom-up path fuses 3 features of the same scale. Figure 5 As shown in the figure, this feature fusion strategy cleverly integrates the input and output feature information of the same layer, retaining the fine detail features of the shallow layer and successfully integrating the rich semantic features of the deep layer, realizing the comprehensive utilization and efficient fusion of feature information. Although this improvement will slightly increase the amount of computation during the network training phase, they have shown significant effects in improving the performance of small target detection in images. In this way, YOLOv8 successfully provides better small target information for deep deep learning networks while maintaining computational efficiency, thereby improving the detection performance of small targets.

[0049] Specifically, general small-scale targets refer to targets below 32×32 pixels. From the perspective of a drone, dense small-scale targets are prone to appear, and most of them are below 15×15 pixels, which is very close to the limit of small target detection. In order to further improve the algorithm's ability to detect small-scale targets, the present invention adds a small target detection branch in the YOLOv8 network. Specifically, one more upsampling is performed in the top-down path, and one more downsampling is performed in the bottom-up path to construct a small target detector based on a 4x downsampled feature map. The newly added detector has more fine-grained information, is more sensitive to small-scale target features, and improves considerable performance without much reduction in model speed.

[0050] Preferably, the present invention adds a dedicated small target detection head after the small target detection branch, such as Figure 6 As shown in , the dedicated small target detection head includes a cascaded attention mechanism layer, an anchor box rotation layer, a deformable convolution layer, and a 1×1 convolution layer. Figure 7As shown, the attention mechanism layer includes a cascaded GAP layer, a first Conv2d layer, a first BN+Sigmoid layer, an upsample layer, a second Conv2d layer, and a second BN+Sigmoid layer; the input of the attention mechanism layer is concatenated and fused with the output of the second BN+Sigmoid layer to form the output of the attention mechanism layer. A dedicated small target detection head is added after the small target detection branch. First, the background information of the target area in the fusion feature map is suppressed by the attention mechanism; then the horizontal anchor frame is rotated by the anchor frame rotation network, and then a displacement sampling point prediction is calculated for each position on the feature map by rotating the anchor frame to guide the sampling position of the deformable convolution; finally, the position of the convolution sampling is adjusted by the deformable convolution, and the detection head is decoupled to further enhance the head feature information of the network, so as to more accurately describe the position information of the target in the drone aerial photography scene.

[0051] Specifically, the structure for decoupling the dedicated small target detection head is as follows: Fig. 9 As shown in the figure, two branches are separated from 1×1conv. The first branch includes 1 3×3conv and 1 1×1conv; the second branch includes 2 3×3conv and 2 1×1conv. After decoupling the dedicated small target detection head, three features are obtained, and different loss calculations are performed based on the three features.

[0052] Preferably, the overall structure of the target detection model obtained by improving the YOLOv8 network in the present invention is as follows: Figure 8 As shown, the target detection model includes a Backbone module, a Neck module and a Head module, wherein:

[0053] The Backbone module includes a first Conv layer, a second Conv layer, a first T-C2F structure module, a third Conv layer, a second T-C2F structure module, a fourth Conv layer, a third T-C2F structure module, a fifth Conv layer, a fourth T-C2F structure module and an SPPF layer which are connected layer by layer.

[0054] The Neck module adopts the BiFPN network, and adds a small target detection branch to the BiFPN network through a bidirectional cross-layer direct connection method to further introduce shallower features; specifically, the Neck module includes a first upsampling layer, a first Concat layer, a fifth T-C2F structure module, a second upsampling layer, a second Concat layer, a sixth T-C2F structure module, a third upsampling layer, a third Concat layer, a seventh T-C2F structure module, a sixth Conv layer, a fourth Concat layer, an eighth T-C2F structure module, a seventh Conv layer, a fifth Concat layer, a ninth T-C2F structure module, an eighth Conv layer, a sixth Concat layer, and a tenth T-C2F structure module connected layer by layer; among them, the fifth T-C2F structure module is connected to the fifth Concat layer, and the sixth T-C2F structure module is connected to the fourth Concat layer; the second T-C2F structure module is jump-connected to the fourth Concat layer, and the third T-C2F structure module is jump-connected to the fifth Concat layer.

[0055] The Head module includes detection head 1, detection head 2, detection head 3 and a special small target detection head.

[0056] S3. Input the preprocessed image into the target detection model to obtain a preliminary detection result, and post-process the preliminary detection result to obtain a detection result.

[0057] Specifically, filters are used to remove noise, errors are corrected to improve detection accuracy, and data is visualized to better understand the detection results. These post-processing steps can improve the accuracy and interpretability of the final detection results.

[0058] S4. Calculate the loss based on the detection results and back-propagate the training model parameters.

[0059] Specifically, the present invention uses mixed precision training technology to accelerate the training process and reduce the demand for computing resources, and also uses a gradient descent optimizer to update model parameters. In addition, the present invention uses a learning rate decay strategy to optimize model convergence. Through repeated iterative training and verification processes, the accuracy and performance of the target detection model are gradually improved.

[0060] S5. Use the target detection model obtained in S4 to perform target detection in UAV aerial images.

[0061] In the present invention, unless otherwise clearly stipulated and limited, the terms such as "installation", "setting", "connection", "fixation" and "rotation" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. Unless otherwise clearly defined, ordinary technicians in this field can understand the specific meanings of the above terms in the present invention according to the specific circumstances.

[0062] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting targets in drone aerial images based on improved YOLOv8, characterized in that: The following steps are involved: S1. Collect drone aerial images and preprocess them to obtain a preprocessed image set; S2. Introduce the BiFPN network into the YOLOv8 network, add a small target detection branch and a dedicated small target detection head, and use the T-C2F structure module to replace the C2F module in the YOLOv8 network to obtain the target detection model; S3. Input the preprocessed image into the target detection model to obtain a preliminary detection result, and post-process the preliminary detection result to obtain a detection result; S4. Calculate the loss based on the detection results and back-propagate the training model parameters; S5. Use the target detection model obtained in S4 to perform target detection in UAV aerial images.

2. The method for detecting targets in unmanned aerial images based on improved YOLOv8 according to claim 1, characterized in that: The T-C2F structural module includes a cascaded first CBS layer, a Split layer, a Bottle module, a Concat layer, a Transformer layer, and a second CBS layer. The Bottle module includes N cascaded Bottleneck layers. The first CBS layer, the Split layer, and the N Bottleneck layers are all residually connected to the Concat layer.

3. The method for detecting targets in unmanned aerial images based on improved YOLOv8 according to claim 2, characterized in that: The Transformer layer includes two feature extraction modules and one Contraction layer; each feature extraction module includes a cascaded Contraction layer, an MHSA layer, and an Expansion layer, and the input of the feature extraction module is concatenated and fused with the output of the Expansion layer to form the output of the feature extraction module.

4. The method for detecting targets in unmanned aerial images based on improved YOLOv8 according to claim 1, characterized in that: The target detection model includes a Backbone module, a Neck module and a Head module, wherein: The Backbone module includes a first Conv layer, a second Conv layer, a first T-C2F structure module, a third Conv layer, a second T-C2F structure module, a fourth Conv layer, a third T-C2F structure module, a fifth Conv layer, a fourth T-C2F structure module and an SPPF layer, which are connected layer by layer; The Neck module uses the BiFPN network and adds a small target detection branch to the BiFPN network through a bidirectional cross-layer direct connection to further introduce shallower features; The Head module includes detection head 1, detection head 2, detection head 3 and a special small target detection head.

5. The method for detecting targets in unmanned aerial images based on improved YOLOv8 according to claim 4, characterized in that: The Neck module includes a first upsampling layer, a first Concat layer, a fifth T-C2F structure module, a second upsampling layer, a second Concat layer, a sixth T-C2F structure module, a third upsampling layer, a third Concat layer, a seventh T-C2F structure module, a sixth Conv layer, a fourth Concat layer, an eighth T-C2F structure module, a seventh Conv layer, a fifth Concat layer, a ninth T-C2F structure module, an eighth Conv layer, a sixth Concat layer, and a tenth T-C2F structure module, which are connected layer by layer; Among them, the fifth T-C2F structure module is connected to the fifth Concat layer, and the sixth T-C2F structure module is connected to the fourth Concat layer; the second T-C2F structure module is jump-connected to the fourth Concat layer, and the third T-C2F structure module is jump-connected to the fifth Concat layer.

6. The method for detecting targets in unmanned aerial images based on improved YOLOv8 according to claim 1, characterized in that: The dedicated small object detection head consists of a cascaded attention mechanism layer, an anchor box rotation layer, a deformable convolution layer, and a 1×1 convolution layer.

7. The method for detecting targets in unmanned aerial images based on improved YOLOv8 according to claim 6, characterized in that: The attention mechanism layer includes a cascaded GAP layer, a first Conv2d layer, a first BN+Sigmoid layer, an upsample layer, a second Conv2d layer, and a second BN+Sigmoid layer; The input of the attention mechanism layer is concatenated with the output of the second BN+Sigmoid layer to form the output of the attention mechanism layer.

8. The method for detecting targets in unmanned aerial images based on improved YOLOv8 according to claim 1, characterized in that: The preprocessing process includes: S11. De-noising the drone aerial image by using a median filter to obtain a de-noised image; S12. performing contrast enhancement on the denoised image using histogram equalization to obtain an enhanced image; S13. Convert the enhanced image from RGB image format to YUV color space to obtain a preprocessed image.

Images