Light-weight unmanned aerial vehicle target detection method based on DD-YOLO

Abstract

The invention discloses a DD-YOLO-based light-weight unmanned aerial vehicle target detection method, belongs to the field of unmanned aerial vehicle small target detection, and mainly optimizes the inspection capability of small targets in the view of an unmanned aerial vehicle to solve the problems of small targets, high density, serious shielding, complex models and the like in detection. According to the method, a deep convolutional lightweight network model is fused, a residual module is reconstructed, BottleNeck in C2f is replaced, and the feature extraction capability of a backbone network is enhanced while the network hierarchy is simplified; secondly, SE and MHSA attention mechanisms are introduced, and a spatial pyramid pooling layer is transformed, so that the model can more comprehensively aggregate local and global features; a quadruple down-sampling branch is added to a neck network to relieve the problem of insufficient receptive field of a small target, and the capacity of extracting key features of the small target is improved. The improved method is applied to unmanned aerial vehicle small target detection, and compared with an original YOLOv8s method, the improved method has the advantages of being high in precision, small in parameter quantity, low in calculation amount, high in calculation speed and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle image target recognition, and in particular to a lightweight unmanned aerial vehicle target detection method based on DD-YOLO. Background Art

[0002] As a flexible and convenient image acquisition tool, drones have been widely used in environmental monitoring, agriculture, urban planning and other fields. However, target detection in drone aerial images still faces many challenges. Since drones usually shoot at high altitudes, the targets in the images are often small and dense. In addition, occlusion between targets is also common. These factors may cause traditional target detection algorithms to have false detections and missed detections.

[0003] At present, the mainstream methods in the field of target detection are mainly based on deep learning, especially convolutional neural networks (CNNs). Among them, the YOLO (You Only Look Once) series of algorithms have gained wide attention in practical applications due to their advantages of fast detection speed and high accuracy. YOLOv8s is the latest version of the YOLO series, with good detection performance and high computational efficiency. However, the direct application of YOLOv8s for target detection in drone images may still be troubled by the following problems: 1. Drones usually shoot at high altitudes, resulting in smaller targets in the image, and traditional detection methods are difficult to capture the details of small targets. 2. In the scenes of drone aerial photography, target objects are often densely arranged, which increases the difficulty of target detection. 3. In dense scenes, mutual occlusion between objects makes it difficult to accurately extract target features, thereby affecting the detection effect. 4. Although YOLOv8s has improved detection performance, its high model complexity and computing resource requirements limit its application in resource-constrained environments.

[0004] In response to the above problems, existing technologies have attempted to improve model performance by improving network structure and other means. However, these improvements often increase the model complexity and computing resource requirements while improving detection accuracy. Therefore, how to simplify the model structure and reduce computing resource requirements while ensuring detection accuracy has become an important research direction in the current field of drone target detection.

[0005] Based on this, this paper proposes a lightweight UAV target detection method based on DD-YOLOv8. By introducing deep convolution, deformable convolution and various attention mechanisms, the feature extraction ability and detection accuracy of the model are effectively improved, and the detection performance of small targets is significantly enhanced while reducing the number of model parameters and computational complexity. Experimental results show that this method has achieved excellent detection results on multiple data sets, verifying its feasibility and effectiveness in practical applications. Summary of the invention

[0006] The purpose of the invention is to propose a lightweight UAV target detection method based on DD-YOLO to solve the problems of small targets, high density, severe occlusion and high model complexity faced by target detection in UAV aerial images. This method improves the detection accuracy and efficiency of the model by improving the YOLOv8s network structure, combining variable depth convolution and multiple attention mechanisms, and has broad application prospects.

[0007] In order to achieve the above purpose, the technical solution adopted by the present invention includes the following steps:

[0008] Step 1: Deploy the model environment to be trained and obtain the public drone image dataset at the same time;

[0009] Step 2: Design the network architecture of DD-YOLO and establish the DD-YOLO lightweight drone target detection model including the preprocessing part, backbone network, neck network and detection head part;

[0010] Step 3: training the DD-YOLO small target detection model based on the training set of the public data set to obtain the optimal small target detection model;

[0011] Step 4: Use the test set in the public dataset to perform performance evaluation on the optimal small target detection model to ensure that the model meets the predetermined accuracy and real-time processing requirements, and finally output the small target detection results to obtain the positioning and classification of targets in drone images.

[0012] The step 2 establishes the DD-YOLO detection model as follows:

[0013] Step S1: In the preprocessing part, the dataset images are translated, scaled, mixed up, and the Mosaic data augmentation method is used to stitch multiple images into one, thereby increasing the diversity of the training data.

[0014] Step S2.1: The backbone network (Backbone) adopts the CSP-Darknet53 network, and extracts the feature information of the input image through the Conv module convolution to form a feature map;

[0015] Step S2.2: Through the Dwconv module and the DdC2f module, the feature map of the base layer is first divided into two parts, and then they are merged through the cross-stage hierarchy;

[0016] Step S2.3: Through the attention spatial pyramid pooling layer (ASPPF) module, the features extracted by the backbone network (Backbone) are fused and passed to the neck network (Neck);

[0017] Step S3.1: The neck network (Neck) adopts FPN-PAN network,

[0018] Step S3.2: Improve the neck by adding a four-fold downsampling branch, which enhances the multi-scale feature fusion capability of the model and reduces the loss of detail features;

[0019] Step S3.3: extract the filtered information by using the Dwconv module and the DdC2f module and pass it to the head network (Head);

[0020] Step S4.1: The head network (Head) first uses CIoU as the bounding box regression method and adds a 160*160 high-resolution detection head to predict the information of the neck network (Neck);

[0021] Step S4.2: Using the non-maximum suppression (NMS) algorithm to quickly remove the excessive redundant bounding boxes obtained in the previous step, the detection result is finally obtained;

[0022] The performance evaluation of step 4 is as follows:

[0023] Step S1: First, verify the effectiveness of each improved module through ablation experiments on public datasets;

[0024] Step S2: Verify by comparative test;

[0025] Step S3: Visualize the performance of the network through tabular data and visualization results.

[0026] The present invention has the following beneficial effects and advantages:

[0027] (1) The model is lightweight first, ensuring that the accuracy is improved as much as possible without increasing the amount of model parameter calculations;

[0028] (2) Using multiple attention mechanisms to transform the SPPF network layer of the network, the transformed SPPF layer can dynamically adjust the importance of each channel, focus on different parts of the feature map, aggregate local and global features, and provide a more comprehensive feature representation;

[0029] (3) To address the loss of semantic information caused by different scales when detecting small targets, a four-fold downsampling branch is added to enhance the combination of deep semantic information and shallow semantic information. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a step diagram of a lightweight UAV target detection method based on DD-YOLO in the present invention.

[0031] Figure 2This is a flowchart for implementing the DD-YOLO target detection model of the present invention.

[0032] Figure 3 This is a schematic diagram of the Ddconv module claimed in the present invention.

[0033] Figure 4 This is a structural diagram of the DdC2f module described in the present invention.

[0034] Figure 5 This is a structural diagram of the ASPPF module described in the present invention.

[0035] Figure 6 This is a comparison chart of the PR curves of DD-YOLO and YOLOv8s described in the present invention.

[0036] Figure 7 This is the DD-YOLO visualization detection result described in the present invention. DETAILED DESCRIPTION

[0037] Example:

[0038] like Figure 1 As shown, the technical solution of the present invention includes five steps: environment deployment, data set acquisition, model construction, model training, and performance testing.

[0039] Step S1: Environment deployment: Ensure that the hardware platform meets the minimum configuration requirements. The system needs to install Python 3.8 or above, and install pytorch and cuda consistent with the graphics card version.

[0040] The step S2 obtains a data set: the data set VisDrone2019 used in the present invention is collected by the AISKYEYE team of the Machine Learning and Data Mining Laboratory of Tianjin University. The data set contains a total of 8599 static images taken by drones at high altitudes, including 6471 training sets, 1580 test sets, and 548 validation sets;

[0041] The step S3 constructs a model: the DD-YOLO target detection model of the present invention is constructed into four parts: data preprocessing, backbone network (backbone), neck network (neck), and detection head (head);

[0042] The step S4 trains the model: configure the training parameters as follows: the input image resolution is 640×640, the epochs is 200, the batchsize is 8, the workers is 8, the lr0 is 0.01, the lrf is 0.01, the momentum is 0.937, and the weight decay coefficient is 0.0005, and the optimal UAV small target detection model is obtained;

[0043] The step S5 performance test: the test data set is used to perform small target detection through the model to obtain the detection performance.

[0044] like Figure 2 As shown, the implementation flow chart of the DD-YOLO target detection model of the present invention is as follows: the present invention uses deep convolution to lighten the model, and applies the deformable convolution idea to form a new residual module to replace BottleNeck in C2f, thereby enhancing the feature extraction capability of the network backbone while lightening the network level; secondly, by introducing SE and MHSA attention ideas, the spatial pyramid pooling layer is transformed, so that the model can aggregate local and global features more comprehensively; and a four-fold downsampling branch is added to the neck network to alleviate the problem of insufficient receptive field for small targets, strengthen the focus on small targets, and effectively extract the key features of small targets;

[0045] The specific implementation process is as follows:

[0046] Step 1: Data preprocessing: Preprocess the images taken by the drone, including translation, scaling, mixup, and mosaic data enhancement operations on the images in the divided train to ensure that the input images meet the input requirements of the YOLOv8s model.

[0047] Step 2 Backbone network: extract features from the input image to provide basic data for subsequent detection and classification tasks. The backbone network of the present invention has the following improvements:

[0048] Step 2.1: Use deep convolution to replace the convolution in the network. The Dwconv module contains a deep convolution module. Deep convolution is a variant of the convolution operation that aims to significantly reduce the computational complexity and number of parameters. Assume that the input feature map The size is ,in is the height, is the width, is the number of channels. The mathematical expression is as follows: ,in Indicates that the output feature map is at position No. The value of the channel, Indicates that the convolution kernel is OK, Column and The weights at the channels, are the height and width of the convolution kernel.

[0049] Step 2.2: If Figure 3 As shown in the schematic diagram, the present invention borrows the idea of ​​deformable convolutional network to transform DwConv to obtain a new DdConv module. Assume that one of the input feature maps is The depth convolution of the position can be written as follows: ,in For point The bias term constant within the convolution kernel range introduces an offset at each position To achieve the effect of deforming the convolution kernel, the following formula The calculation process of deep convolution deformation is shown. In deep convolution, by introducing an offset, the position and shape of the convolution kernel are dynamically adjusted to adapt to the deformation of the target. This can not only keep the computational complexity and number of parameters low, but also capture the diverse characteristics of the target by dynamically adjusting the convolution kernel. The DdConv module structure is shown in the figure below. Figure 4 (d) shown.

[0050] Step 2.3: If Figure 4 As shown, Figure (a) is the residual module structure, Figure (b) is the residual structure (Bottleneck) in the original C2f module, and Figure (c) is the new DdC2f structure formed by replacing the conv module with the Ddconv module in Figure (d), which enhances the feature extraction capability of the network backbone while lightening the network layer.

[0051] Step 2.4: If Figure 5 As shown in the figure, a newly proposed ASPPF (Attention-SPPF) structure is used. The ASPPF module is composed of a feature fusion module (SPPF), a squeeze-excitation attention mechanism (SE), and a multi-head self-attention mechanism (MHSA). SPPF is a three-time parallel MaxPool2d; SE compresses the feature map through a global average pooling operation, extracts the global information of each channel, and calculates the attention weight of each channel through two fully connected layers, and finally weights the original features, so as to dynamically adjust the importance of each channel; while multi-head self-attention (MHSA) focuses on different parts of the feature map, so that the network can adaptively emphasize important areas according to the input context. Each attention head calculates an attention weight, and finally the outputs of multiple heads are linearly transformed and spliced ​​together. ASPPF enhances the model's ability to capture global context and long-range dependencies, strengthens the feature extraction capability, and can more comprehensively aggregate local and global features.

[0052] Step 3 Neck network: According to the characteristics of small targets in the drone's perspective image, the neck structure is modified. A four-fold downsampling branch is added to the FPN-PAN to improve the neck, enhance the model's multi-scale feature fusion capability, reduce the loss of detail features, and use four detection heads to assist in target detection tasks, thereby improving the accuracy of small target positioning and recognition.

[0053] Step 4: Detection head: A more flexible anchor-free mechanism is used. This mechanism can reduce the dependence on predefined anchors, directly regress the target box based on the points in the feature map, and apply the NMS algorithm to select the optimal box based on the confidence score and remove redundant boxes to improve detection accuracy.

[0054] like Figure 6 As shown in the figure, a comparison of the PR curves of DD-YOLO and YOLOv8s is shown.

[0055] like Figure 7 As shown, the visual detection effects of DD-YOLO and YOLOv8s are demonstrated.

[0056] In summary, compared with previous models, the lightweight UAV target detection method based on DD-YOLO in the present invention has the greatest advantage that it can effectively identify in complex environments. Aiming at the phenomenon of missed detection and false detection of small targets that may occur in the YOLOv8s model for high-altitude aerial images of UAVs, the improved model can effectively alleviate the problem of insufficient receptive field for small targets, strengthen the attention to small targets, better handle the relationship between multi-scale features and modeling space, better perform feature selection and enhance feature expression, thereby improving the accuracy and robustness of small target detection.

[0057] The above description is only a specific implementation mode of the present invention. Any feature disclosed in this specification, unless otherwise stated, can be replaced by other alternative features that are equivalent or have similar purposes; all the disclosed features, or all the steps in the methods or processes, except for mutually exclusive features and / or steps, can be combined in any way.

Claims

1. A lightweight UAV target detection method based on DD-YOLO, characterized in that The method comprises the following steps: Step S1: deploying a model environment to be trained; Step S2: obtaining a public UAV small target image dataset; S3: Design the network architecture of DD-YOLO and build a DD-YOLO lightweight UAV target detection model; Step S4: Train the DD-YOLO small target detection model based on the training set of the public data set to obtain the optimal small target detection model; Step S5: Use the test set in the public data set to perform performance evaluation on the optimal small target detection model to ensure that the model meets the predetermined accuracy and real-time processing requirements, and finally output the small target detection results to obtain the positioning and classification of targets in the UAV image.

2. A lightweight UAV target detection method based on DD-YOLO as described in claim 1, characterized in that: The network architecture of DD-YOLO includes a preprocessing part, a backbone network, a neck network and a detection head part.

3. A lightweight UAV target detection method based on DD-YOLO as described in claim 2, characterized in that: The preprocessing part is to translate, scale, mixup and use the Mosaic data enhancement method to stitch multiple images into one, thereby increasing the diversity of training data.

4. A lightweight UAV target detection method based on DD-YOLO as described in claim 2, characterized in that: The backbone network uses deep convolution to replace ordinary convolution and designs a new module DdC2f, and proposes a new ASPPF (Attention-SPPF) structure, which consists of SPPF, squeeze-incentive attention mechanism and multi-head self-attention mechanism.

5. A lightweight UAV target detection method based on DD-YOLO as described in claim 2, characterized in that: The neck network adds a four-fold downsampling branch in FPN-PAN, which improves the localization and recognition accuracy of small targets.

6. A lightweight UAV target detection method based on DD-YOLO as described in claim 2, characterized in that: The detection head part adopts a decoupled head structure and an anchor-free target detection method to directly predict the target center point, reduce the number of predictions, and improve efficiency.

7. A lightweight UAV target detection method based on DD-YOLO as described in claim 1, characterized in that: The performance evaluation usually uses accuracy (Precision), recall (Recall), and mean average precision (mAP) as the basis for evaluating the learning ability in the drone aerial image target detection task. The calculation formula is as follows: （1） （2） （3） （4） Where n represents the type of label, TP represents the number of correctly identified positive samples (true positive examples), FP represents the number of negative samples that are incorrectly identified as positive samples (false positive examples), and FN represents the number of positive samples that are incorrectly identified as negative samples (false negative examples). Secondly, when considering the effectiveness of model detection, the size and complexity of the model must also be considered. The model's parameters and GFLOPs (Floating Point Operations Per Second) are usually used for evaluation. The parameter quantity is obtained by summing the number of weight parameters of each layer; GFLOPs is the number of floating-point operations performed by the model in one second during the inference process.

Images