Unmanned aerial vehicle aerial photography small target detection method based on improved YOLOv8

Abstract

The invention provides an unmanned aerial vehicle aerial photography small target detection method based on improved YOLOv8, belongs to the field of image detection, and mainly comprises four steps of data preparation, model building, model training and model testing. The method comprises the following steps: firstly, collecting small target image data aerial-photographed by an unmanned aerial vehicle, and marking the data; then, an unmanned aerial vehicle aerial photography small target detection model based on the improved YOLOv8 is built, the model comprises a new aggregation and distribution mechanism and a Dynamic Head detection head, and the performance of the model is improved; thirdly, training set data are input into the model for training, and parameters are optimized; and finally, inputting test data of the verification set into the model to realize small target detection. The method combines an aggregation-distribution mechanism and a dynamic detection head, improves the precision of unmanned aerial vehicle small target aerial photography detection, and can be widely applied to the fields of environmental protection, agriculture, electric power and the like.

Description

Technical Field

[0001] The present invention belongs to the field of image detection, and more specifically relates to a method for detecting small targets in unmanned aerial vehicle aerial photography based on improved YOLOv8. Background Art

[0002] At present, drones are widely used in agriculture, forest environmental protection, power inspection and other fields. Especially in disaster relief, monitoring and reconnaissance, drones only need to carry corresponding equipment to obtain real-time information of a large area. However, ordinary image recognition technology, such as ObjectDetection, is mainly aimed at the recognition of large target objects, and the detection performance of small targets needs to be improved.

[0003] In the use of drones, especially in open areas and large-scale operations, the detection targets are usually small. Traditional target detection methods often produce high false detection rates and missed detection rates when identifying small targets. At present, several mainstream small target detection technologies, such as the YOLO series, FasterR-CNN series and SSD, have achieved good results in large target detection, but still have obvious deficiencies in the performance of small target detection. Summary of the invention

[0004] This paper proposes a method for detecting small targets in drone aerial photography based on improved YOLOv8. Through effective structural improvement and enhancement, the problem of small target detection is effectively solved. The introduction of new aggregation distribution mechanism and dynamic head technology on YOLOv8 can enable the model to achieve accurate detection when facing small targets, significantly improving the small target detection capability of drones.

[0005] In order to achieve the above object, the present invention is implemented by adopting the following technical scheme: the method comprises:

[0006] Data preparation: collect small target image data taken by drones, annotate the image data, and build training sets, validation sets, and test sets;

[0007] Model building: building a drone aerial photography small target detection model based on improved YOLOv8;

[0008] Model training: input the training set data into the model for training, optimize the model parameters, and improve the model accuracy;

[0009] Model testing: input the validation set test data into the model to detect small targets, analyze the model's detection results, and evaluate the model's detection effect.

[0010] In one solution, the data preparation includes: the improved YOLOv8 drone aerial photography small target detection model is improved as follows:

[0011] (1) A new aggregation and distribution mechanism GD is used to achieve efficient information exchange in YOLOv8 by globally fusing multi-level features and injecting global information into higher levels;

[0012] (2) The YOLOv8 head is replaced with the Dynamic Head detection head, and the performance of the model is improved by combining the three attention functions of scale, space, and task.

[0013] In one solution, the aggregation and distribution mechanism GD uses a feature alignment module FAM and an information fusion module IFM to aggregate features at different levels;

[0014] And distribute the fused information back to each level of the network through the information injection module Inject;

[0015] The information injection module Inject combines local features and global features through convolution and Sigmoid activation function operations to enhance the feature map with global context information.

[0016] In one embodiment, the Dynamic Head detection head is given a three-dimensional feature tensor F∈R on the detection layer L ×S×C The attention calculation formula is as follows:

[0017] V(F)=p c (p s (p L (F)×F)×F)×F

[0018] Among them, F represents an input three-dimensional tensor L×S×C, π L (.)πs(.) and πc(.) are the scale-aware attention module, space-aware attention module and task-aware attention module, respectively, and they only act on the L, S and C dimensions of the three-dimensional tensor F.

[0019] In one embodiment, the YOLOv8 also includes a lightweight neighboring layer fusion LAF module that uses average pooling and bilinear up / down sampling operations to align and merge feature maps.

[0020] In one embodiment, the YOLOv8 includes a backbone network Backbone, a neck Neck and a detection head Head;

[0021] The backbone network is responsible for feature extraction. Based on the Darknet-53 architecture, a new C2f module is introduced to perform residual learning.

[0022] In one solution, the neck is used for feature fusion, the C2f module is used to replace the C3 module, and the PAN-FPN strategy is used to implement top-to-bottom and bottom-to-top feature pyramid construction.

[0023] Beneficial effects of the present invention:

[0024] 1. Based on the YOLOv8 algorithm, this paper uses a new aggregation and distribution mechanism (GD) to achieve efficient information exchange in YOLOv8 by globally fusing multi-level features and injecting global information into higher levels. This significantly enhances the information fusion capability of the neck without significantly increasing latency, thereby improving the performance of the model at different object sizes.

[0025] 2. Replace the YOLOv8 head with the Dynamic Head detection head, and further improve the performance of the model under the above improvements by combining the three attention functions of scale, space and task. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flow chart of the method of the present invention;

[0027] Figure 2 This is the YOLOV8 framework structure diagram of the present invention;

[0028] Figure 3 This is a diagram of the traditional neck structure of the present invention;

[0029] Figure 4 For the present invention, a single π L (.)πs(.) and πc(.) attention module diagram

[0030] Figure 5 This is a structural diagram of the multi-linked DyHead of the present invention;

[0031] Figure 6 This is the overall framework diagram of the method of the present invention. DETAILED DESCRIPTION

[0032] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Typical embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in the present invention. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0033] Unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0034] like Figure 1 As shown, a method for detecting small targets in drone aerial photography based on improved YOLOv8 includes the following steps:

[0035] S1. Data preparation: collect small target image data taken by drones, annotate the image data, and construct training sets, validation sets, and test sets.

[0036] Collect image data of small targets taken by drones, and use drones with high-definition cameras to conduct a large amount of aerial photography in different environments, such as cities, farmlands, forests, etc.

[0037] The image data is labeled to form a corresponding data set. The labeling process includes two parts: target detection (object location) and target recognition (object classification). For target detection, in each image, the position of each target is clearly marked with a rectangular box, and each rectangular box needs to have corresponding coordinate data. For target recognition, the corresponding category label needs to be added to each target in the rectangular box.

[0038] The collected and labeled data are sorted to generate the data set required for training. According to the input requirements of YOLOv8, the data set is preprocessed accordingly. The preprocessed data set will be divided into training set, validation set and test set.

[0039] S2. Model construction: build a UAV aerial photography small target detection model based on improved YOLOv8.

[0040] like Figure 2 As shown in the figure, the YOLOv8 model is mainly composed of three parts: backbone network (Backbone), neck (Neck) and detection head (Head).

[0041] The backbone network is responsible for feature extraction. It is based on the Darknet-53 architecture and introduces a new C2f module for residual learning. This module combines the design ideas of CSP and ELAN, and incorporates gradient changes into the entire feature map by adding skip-layer connections and additional split operations.

[0042] The neck is used for feature fusion. The C2f module is used to replace the C3 module, and the PAN-FPN strategy is used to realize the top-to-bottom and bottom-to-top feature pyramid construction.

[0043] The detection head has undergone a major change, from the original anchor-based coupling head to an anchor-free decoupling head, removing the object branch and retaining only the decoupled classification branch and the regression branch using the DFL loss function. In order to accurately detect objects of different sizes, the detection head includes three detection parts that process feature maps of different scales.

[0044] like Figure 3 As shown in the figure, the NECK structure of the YOLO series adopts the traditional FPN structure, which contains multiple branches for multi-scale feature fusion. However, it can only completely fuse the features of adjacent layers, and the information of other layers can only be obtained indirectly "recursively". The information fusion structure of the traditional FPN: the existing level-1, 2, and 3 are arranged from top to bottom.

[0045] FPN is used for fusion between different levels. When level-1 obtains information from the other two levels, there are two different scenarios:

[0046] 1) If level-1 tries to utilize information from level-2, it can directly access and fuse that information.

[0047] 2) If level-1 wants to use level-3 information, level-1 should recursively call the information fusion module of the adjacent layer.

[0048] Specifically, the secondary and tertiary information must be fused first, and then the first level can indirectly obtain the tertiary information by combining the secondary information. This transmission mode may lead to a large amount of information loss during the calculation process. The information interaction between layers can only exchange the information selected by the intermediate layer, and the unselected information will be discarded during the transmission process. This leads to a situation where the information of a certain level can only fully help the adjacent layers, while weakening the help provided to other global layers. As a result, the overall effectiveness of information fusion may be limited.

[0049] In order to avoid information loss during transmission of the traditional FPN structure, we abandon the original recursive method and use a novel collection and distribution mechanism (GD). By using a unified module to collect and fuse information at each level and then distribute it to different levels, we not only avoid the inherent information loss in the traditional FPN structure, but also increase the partial information fusion capability of Neck without significantly increasing the latency. Therefore, our method can more effectively utilize the features extracted by the backbone.

[0050] The backbone network is used to extract features, and the detection head is used for classification and positioning. An efficient detection head can improve the performance of target detection, especially in dense scenes, and can locate targets more accurately. Most of the images taken by drones are taken from a bird's-eye view, which may cause complex background problems.

[0051] There are some limitations in the original detection head of YOLOv8, such as it uses a traditional single-scale prediction structure, which is not good at processing objects of different sizes. This is because it only predicts from feature maps of one scale, without using features of other scales, which limits the model's ability to detect multi-scale objects. In addition, the prediction head does not take into account contextual information, and the processing of each prediction position is independent of each other, lacking an overall perspective. At the same time, the prediction head has a small number of parameters, which limits its ability to mine spatial structural information in features.

[0052] To overcome these problems, the present invention introduces the Dynamic Head (DyHead) module in YOLOv8. This module adopts a dynamic routing mechanism, which can effectively integrate contextual information and identify multi-scale targets. The DyHead module optimizes the three dimensions of scale perception, space perception and task perception through the self-attention mechanism, thereby improving the performance of the model in processing vehicles of different scales, complex backgrounds and small targets.

[0053] The dynamic detection head unifies the attention mechanism with the target detection head, and regards the output of the backbone network as a three-dimensional tensor: level×space×channel, i.e., feature level, spatial position, and output channel. Scale-aware attention is performed on the feature level dimension, spatial-aware attention is performed on the spatial position dimension, and task-aware attention is performed on the output channel dimension. The three types of attention are combined to form the dynamic head structure. After unifying the scale of the multi-layer feature map output by the neck network, it is input into the dynamic detection head using the concat operation, and then passed through scale-aware attention, spatial-aware attention, and task-aware attention in turn. By enhancing and combining features in different dimensions, a target detection head with higher performance is obtained. The proposed method significantly improves the representation ability of the target detection head without any computational overhead.

[0054] The specific implementation process is as follows:

[0055] 1. Aggregation-distribution mechanism (GD): implemented through convolution and self-attention operations, this mechanism effectively fuses information from different layers of the network.

[0056] 2. Multi-scale feature fusion: The GD mechanism improves the fusion capability of multi-scale features, thereby improving the accuracy of target detection. The aggregation-distribution mechanism (GD) is one of the core features of this model, and its purpose is to solve the information fusion problem. In this mechanism, the feature alignment module (FAM) and the information fusion module (IFM) are used to aggregate features at different levels, and the fused information is distributed back to each level of the network through the information injection module (Inject). In this way, the model can more effectively utilize multi-scale features, thereby improving the accuracy of target detection while maintaining low latency.

[0057] (a) Information injection module (Inject): This module combines local features with global features through operations such as convolution and sigmoid activation function to enhance the feature map with global context information, which is crucial for accurate object detection.

[0058] (b) Lightweight Neighboring Fusion (LAF) module: This module is used to improve the fusion of feature maps of the phase layers. It uses operations such as average pooling and bilinear up / down sampling to align and merge feature maps, thereby ensuring that the local features of each layer are enriched with information from its direct neighboring layers.

[0059] Multi-scale feature fusion aims to improve the model's ability to detect objects of different sizes. By combining features from different layers of the network, the technology is able to capture information at multiple scales, from coarse to fine. Low-level features usually contain more details about small objects, while high-level features capture semantic information about large objects. Multi-scale feature fusion enhances the model's representation capabilities by aggregating features from these levels, allowing the model to more accurately identify and locate objects of various sizes in images.

[0060] Dynamic detection head, given a three-dimensional feature tensor F∈R on the detection layer L×S×C The attention calculation formula is as follows:

[0061] V(F)=p c (p s (p L (F)×F)×F)×F

[0062] Among them, F represents an input three-dimensional tensor L×S×C, π L (.)πs(.) and πc(.) are the scale-aware attention module, space-aware attention module and task-aware attention module, respectively, and they only act on the L, S and C dimensions of the three-dimensional tensor F.

[0063] π L (.)πs(.) and πc(.) attention modules and DyHead structure as shown Figure 4 and Figure 5 As shown. Among them, (a) represents π L The internal structure of the three attention modules (.), πs (.) and πc (.) and the single DyHead structure formed by the three modules in series. These three attention modules are applied to the detection head and can be stacked multiple times. (b) shows the process of stacking multiple DyHead modules.

[0064] S3, model training, input the training set data into the model, train it, optimize the model parameters, and improve the model accuracy.

[0065] Data input: Divide the preprocessed data set into training set, validation set and test set. The training set is used to train the model. Input these input data into the model.

[0066] Initialize the model: At the beginning of training, the model parameters (weights and biases) need to be initialized.

[0067] Forward propagation: The model will perform forward propagation, using the input data to calculate the output. In this process, the model will pass the input data layer by layer until it reaches the output layer and output the predicted result.

[0068] Calculate the loss: Compare the output obtained by forward propagation with the true label and calculate the loss value. In object detection, the commonly used loss function is the mean square error loss function.

[0069] Backpropagation: The gradient of the loss with respect to each parameter is calculated through backpropagation. The gradient points out the direction to reduce the loss.

[0070] Parameter update: Use optimizers such as gradient descent, Adam, RMSprop, etc. to update model parameters based on the calculated gradients.

[0071] The optimized model parameters include:

[0072] Hyperparameter tuning: including learning rate, batch size, number of training rounds, etc., all of which will affect the performance of the model. The optimal hyperparameters can be found through grid search, random search, etc.

[0073] Regularization: In order to prevent overfitting, a regularization term is introduced. And L1 regularization is adopted.

[0074] Use the validation set to adjust the model: adjust the model parameters based on the performance on the validation set to optimize the model's performance.

[0075] Continue training and tuning the model until the model converges, that is, the value of the loss function stops decreasing or the validation accuracy stops improving. At the same time, use the validation set regularly to verify the performance of the model to avoid overfitting.

[0076] S4. Model testing: input the validation set test data into the model to detect small targets, analyze the model's detection results, and evaluate the model's detection effect.

[0077] Test data input: First, the preprocessed test data needs to be input into the trained model. The test data is separated from the training data to ensure that the model test is fair and objective.

[0078] Predict the model: After all the test data is input into the model, the model will output the corresponding prediction results.

[0079] Analyze the prediction results of the model: This step is to compare the prediction results of the model with the actual labels to see the operation and accuracy of the model. For object detection tasks, commonly used evaluation indicators include mAP (meanAveragePrecision).

[0080] It mainly consists of two parts: Precision and Recall. Precision refers to the ratio of the number of correctly detected targets to the number of all detected targets. Recall refers to the ratio of the number of correctly detected targets to the number of all true targets.

[0081] The average precision (AP) calculated for each category is averaged over all categories to get mAP. The larger the value, the better the performance of the model.

[0082] During the calculation process, the threshold needs to be changed to obtain different Precision and Recall, and then the PR curve is obtained. AP is the area enclosed by the PR curve and the X-axis.

[0083] The main steps of mAP are: calculating the AP of each category, and then averaging the AP of all categories to get mAP. Its value can intuitively reflect the performance of the model in the entire dataset and is an important indicator for measuring the performance of the target detection model.

[0084] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0085] It should be understood that the detailed description of the technical solutions of the present invention with the help of the preferred embodiments is illustrative rather than restrictive. A person skilled in the art may modify the technical solutions described in the embodiments, or replace some of the technical features therein by equivalents, based on reading the specification of the present invention; and these modifications or replacements do not make the essence of the technical solutions of the corresponding specifications deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting small targets in drone aerial photography based on improved YOLOv8, characterized by: The method includes: Data preparation: collect small target image data taken by drones, annotate the image data, and build training sets, validation sets, and test sets; Model building: building a drone aerial photography small target detection model based on improved YOLOv8; Model training: input the training set data into the model for training, optimize the model parameters, and improve the model accuracy; Model testing: input the validation set test data into the model to detect small targets, analyze the model's detection results, and evaluate the model's detection effect.

2. The method for detecting small targets in drone aerial photography based on improved YOLOv8 according to claim 1, characterized in that: The data preparation includes: the improved YOLOv8 drone aerial photography small target detection model is improved as follows: (1) A new aggregation and distribution mechanism GD is used to achieve efficient information exchange in YOLOv8 by globally fusing multi-level features and injecting global information into higher levels; (2) The YOLOv8 head is replaced with the Dynamic Head detection head, and the performance of the model is improved by combining the three attention functions of scale, space, and task.

3. The method for detecting small targets in drone aerial photography based on improved YOLOv8 according to claim 2, characterized in that: The aggregation and distribution mechanism GD uses the feature alignment module FAM and the information fusion module IFM to aggregate features at different levels; And distribute the fused information back to each level of the network through the information injection module Inject; The information injection module Inject combines local features and global features through convolution and Sigmoid activation function operations to enhance the feature map with global context information.

4. The method for detecting small targets in drone aerial photography based on improved YOLOv8 according to claim 2, characterized in that: The Dynamic Head detection head is given a three-dimensional feature tensor F∈R on the detection layer L×S×C The attention calculation formula is as follows: V(F)=p c (p s (p L (F)×F)×F)×F Among them, F represents an input three-dimensional tensor L×S×C, π L (.)πs(.) and πc(.) are the scale-aware attention module, space-aware attention module and task-aware attention module, respectively, and they only act on the L, S and C dimensions of the three-dimensional tensor F.

5. The method for detecting small targets in drone aerial photography based on improved YOLOv8 according to claim 3, characterized in that: The YOLOv8 also includes a lightweight neighboring layer fusion LAF module that uses average pooling and bilinear up / down sampling operations to align and merge feature maps.

6. The method for detecting small targets in drone aerial photography based on improved YOLOv8 according to claim 1, characterized in that: The YOLOv8 includes a backbone network Backbone, a neck Neck and a detection head Head; The backbone network is responsible for feature extraction. Based on the Darknet-53 architecture, a new C2f module is introduced to perform residual learning.

7. The method for detecting small targets in drone aerial photography based on improved YOLOv8 according to claim 1, characterized in that: The neck is used for feature fusion, the C2f module is used to replace the C3 module, and the PAN-FPN strategy is used to realize the construction of feature pyramids from top to bottom and from bottom to top.

Images