Method and system for detecting small target under aerial photography of unmanned aerial vehicle

Abstract

The invention discloses a method and a system for detecting a small target under aerial photography of an unmanned aerial vehicle, and belongs to the field of target detection. Comprising the steps of obtaining unmanned aerial vehicle aerial photography data to construct an unmanned aerial vehicle aerial photography data set; training the improved YOLOv8 model by using the unmanned aerial vehicle aerial photography data set to obtain an unmanned aerial vehicle aerial photography detection model; and inputting a to-be-detected unmanned aerial vehicle aerial image into the unmanned aerial vehicle aerial detection model to obtain an unmanned aerial vehicle aerial detection result. According to the method, C2f modules of P4 and P5 layers are replaced by CDR modules in a backbone network part, so that feature extraction is realized; biFPN is introduced to the neck, and cross-scale connection and weighted fusion are utilized to input features, so that multi-scale feature fusion is realized more efficiently; content guidance convolution and self-attention are introduced, an output feature map is adjusted through learned space weight, the gradient disappearance problem is relieved, and detail features are enhanced; a dynamic detection head is introduced into a detection head part, so that the detection head and attention are unified, and the detection capability is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and more particularly to a detection method and a detection system for small targets in drone aerial photography. Background Art

[0002] Small target detection is crucial to the popularization of drone technology and is the key to ensuring safety and efficiency. For example, in agricultural plant protection, using drones to monitor plants can detect pests and abnormalities in a timely manner; in power inspections, facing large-scale power equipment, quickly locating the location of defects can greatly improve the work efficiency of inspectors. In addition, during disaster relief, quickly locating the location of damaged facilities and trapped people is crucial to effectively carry out rescue work.

[0003] However, as the basis for realizing these intelligent tasks, the recognition of small targets in UAV remote sensing has been slow. The changes in the flight altitude and camera angle of the UAV cause the scale of the target to change dramatically, and the shadows, complex backgrounds, and different lighting conditions all interfere with the recognition of the target. Therefore, it is urgent to carry out research on the recognition of small targets in UAV remote sensing. Early methods were based on traditional methods of manually extracting features, and they were successful in target detection in complex scenes, but they could not fully capture the detailed information of the real target and could not guarantee robustness to data changes. Thanks to the progress of target detection methods, deep neural networks can automatically and efficiently learn appropriate and useful information from image features. As mentioned above, aerial images are full of small targets, and lower-resolution aerial images will prevent the network from learning features correctly, reducing detection accuracy. On the other hand, although modern drones can capture high-definition images to improve the detection accuracy of small targets, they will also increase the input size of the network and increase the complexity of the model.

[0004] Therefore, how to provide a method and system for detecting small targets under drone aerial photography that can solve the above problems is an issue that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a method and a system for detecting small targets under drone aerial photography, so as to solve the technical problems existing in the above-mentioned prior art.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A method for detecting small targets under drone aerial photography, comprising:

[0008] Obtain drone aerial photography data to build a drone aerial photography dataset;

[0009] Improve the YOLOv8 model, including: using the cascaded dilated residual CDR module to replace the CSP bottleneck C2f module with two convolutions in the P4 and P5 layers in the backbone network for feature extraction; introducing the weighted bidirectional feature pyramid network BiFPN structure in the neck, using cross-scale connections and weighted fusion input features to obtain multi-scale feature fusion features; introducing content-guided convolution and self-attention CGCA modules, adjusting the output feature map through the learned spatial weights to enhance detail features; in the detection head part, introducing a dynamic detection head;

[0010] The improved YOLOv8 model is trained using the drone aerial photography dataset to obtain a drone aerial photography detection model;

[0011] The drone aerial images to be detected are input into the drone aerial detection model to obtain the drone aerial detection results.

[0012] Preferably, the calculation formula of the cascade dilated residual CDR module is:

[0013]

[0014]

[0015] Among them, α1, α2, α3 represent the regional residual operations of different regions, β1, β2, β3 represent the semantic residual operations of different regions, Ω represents the batch normalization BN operation and then the ReLU activation function, F c It represents the output of regional residualization with the number of channels c. represents the dilated convolution with the dilation rate and convolution kernel being 1 and 3 respectively, Concat represents the channel concatenation operation, and BN represents the batch normalization operation.

[0016] Preferably, the node calculation formula of the weighted bidirectional feature pyramid network BiFPN structure is:

[0017]

[0018] in, It is the top-down input of the 4th layer of the backbone network. It is the top-down intermediate output of the 4th layer of the backbone network. is the output of the 4th layer of the backbone network from bottom to top, θ represents the resolution matching by upsampling or downsampling operation, and w i is a learnable weight, where i=1,2,3 represents the learned weights of different sizes, and ε is a minimum value.

[0019] Preferably, the calculation formula of the content-guided convolution and self-attention CGCA module is:

[0020]

[0021] in, and They represent the features processed by cross-space global average pooling, cross-channel global average pooling and cross-channel global maximum pooling operations respectively, ReLU represents the ReLU activation function, and Concat represents the channel cascade operation.

[0022] Preferably, in the local branches of the content-guided convolution and self-attention CGCA modules, a channel shuffle operation is used to further fuse channel information, and a 3×3×3 convolution is used to extract features. In the global branch, a transformer self-attention structure is used, and the calculation formula is:

[0023] F local =Conv 3×3×3 (CS(Conv 1×1 (X)));

[0024]

[0025] F′=Concat(F local +F global );

[0026] F out =Sigmoid(GConv 7×7 (CS(F′,X)));

[0027] Among them, F local represents the output of the local branch, CS represents the channel shuffle operation, GConv 7×7 represents a group convolution with a kernel size of 7, and ω is a learnable scaling factor used to control and The amount of matrix multiplication operations.

[0028] Preferably, the calculation formula for obtaining the multi-scale feature fusion feature is:

[0029] X=F low +F high ;

[0030] F fuse =Conv 1×1 (ρF low +(1-ρ)F high +X);

[0031] Among them, F low represents the low-level features, F high represents high-level features, F fuserepresents the output feature, and ρ represents the weight obtained by the CGCA module.

[0032] Preferably, the calculation formula of the dynamic detection head is:

[0033] Y=π C (π S (π L (X)));

[0034] π L =hard-sigmoid(f(X));

[0035]

[0036] π c =ρ(η 1 (X)X c +μ 1 (X),η 2 (X)X c +μ 2 (X));

[0037] Among them, π L , π S , π C are the attention functions applied to scale, space, and task, respectively. f represents the average pooling, 1×1 convolution, and ReLU activation function. p k +Δp k is the offset, Δm k is in p k The importance scalar of self-learning, X c represents the feature slice of the cth channel, and η and μ are hyperfunctions that learn to control the activation threshold.

[0038] A small target detection system under drone aerial photography, comprising:

[0039] Data acquisition module: obtains drone aerial photography data to build a drone aerial photography dataset;

[0040] Model building module: Improve the YOLOv8 model, including: using the cascaded dilated residual CDR module to replace the CSP bottleneck C2f module with two convolutions in the P4 and P5 layers in the backbone network for feature extraction; introducing the weighted bidirectional feature pyramid network BiFPN structure in the neck, using cross-scale connections and weighted fusion input features to obtain multi-scale feature fusion features; introducing content-guided convolution and self-attention CGCA modules, adjusting the output feature map through the learned spatial weights to enhance detail features; in the detection head part, introducing a dynamic detection head;

[0041] Model training module: Use the drone aerial photography dataset to train the improved YOLOv8 model to obtain the drone aerial photography detection model;

[0042] Detection module: Input the drone aerial photography image to be detected into the drone aerial photography detection model to obtain the drone aerial photography detection result.

[0043] It can be seen from the above technical solutions that compared with the prior art, the present invention discloses a detection method and a detection system for small targets under drone aerial photography. Aiming at the insufficient detection performance of YOLOv8n on small targets of drones, a CDR module based on dilated convolution is introduced to construct a more comprehensive feature representation in the backbone network; a BiFPN module is introduced to add an upsampling layer to the neck to increase the image resolution and improve the feature fusion of the neck; a CGCA fusion module is introduced to fuse low-level and corresponding high-level information respectively to enhance detail features; and DyHead is used to combine attention and detection head to improve the detection capability of the detection head. DCD-YOLO achieves a comprehensive detection accuracy of mAP@0.5 of 0.407 compared to the baseline model YOLOv8n (in all test samples, for each category, when using an IoU threshold of 0.5, the average accuracy of the DCD-YOLO model reached 0.407. In other words, on a given dataset, for those cases where the predictions overlap with the true bounding boxes by at least 50%, the DCD-YOLO model is able to achieve an average accuracy of about 40.7%), which is 21.9% higher than the baseline model, and it also has an advantage in lightweight. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0045] Figure 1 It is a schematic diagram of the detection method of the present invention;

[0046] Figure 2 A DCD-YOLO network model structure diagram provided by an embodiment of the present invention;

[0047] Figure 3 A structural diagram of a CDR module provided in an embodiment of the present invention;

[0048] Figure 4 An improved BiFPN structure diagram provided by an embodiment of the present invention;

[0049] Figure 5Channel and spatial attention structure diagram provided for embodiments of the present invention;

[0050] Figure 6 A structural diagram of the CGCA module provided in an embodiment of the present invention;

[0051] Figure 7 A structural diagram of DyHead provided in an embodiment of the present invention;

[0052] Figure 8 A structural diagram of a detection system provided by an embodiment of the present invention;

[0053] Fig. 9 A comparison chart of the detection effects of YOLOv8n provided in an embodiment of the present invention and DCD-YOLO of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] The purpose of the present invention is to provide a detection method and detection system for small targets under drone aerial photography, the method comprising: obtaining drone aerial photography data to construct a drone aerial photography data set; improving the YOLOv8 model; using the drone aerial photography data set to train the improved YOLOv8 model to obtain a drone aerial photography detection model; inputting the drone aerial photography image to be detected into the drone aerial photography detection model to obtain the drone aerial photography detection result. When the present invention improves the YOLOv8 model, the CDR module is used to replace the C2f module of the P4 and P5 layers in the backbone network part to achieve basic feature extraction; in the neck, BiFPN is introduced to use cross-scale connection and weighted fusion input features. Multi-scale feature fusion is achieved more quickly and efficiently; content-guided convolution and self-attention (CGCA) are introduced, and the output feature map is adjusted by the learned spatial weight, which alleviates the gradient vanishing problem and enhances the detail features; in the detection head part, a dynamic detection head is introduced to achieve the unification of the detection head and attention, and improve the detection capability.

[0056] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] Embodiment 1:

[0058] See also Figure 1 Embodiment 1 of the present invention discloses a method for detecting small targets in drone aerial photography, comprising:

[0059] Obtain drone aerial photography data to build a drone aerial photography dataset;

[0060] Improve the YOLOv8 model, including: using the cascaded dilated residual CDR module to replace the CSP bottleneck C2f module with two convolutions in the P4 and P5 layers in the backbone network for feature extraction; introducing the feature pyramid network BiFPN structure in the neck, using cross-scale connections and weighted fusion input features to obtain multi-scale feature fusion features; introducing content-guided convolution and self-attention CGCA modules, adjusting the output feature map through the learned spatial weights to enhance detail features; in the detection head part, introducing a dynamic detection head;

[0061] The improved YOLOv8 model is trained using the drone aerial photography dataset to obtain a drone aerial photography detection model;

[0062] The drone aerial images to be detected are input into the drone aerial detection model to obtain the drone aerial detection results.

[0063] In a specific embodiment, the improved model (DCD-YOLO network model) proposed based on the YOLOv8n algorithm is as follows Figure 2 As shown in the figure. In the backbone network, the CDR (cascade dilation-wise residual) module is used to replace the CSP bottleneck (C2f) module with two convolutions in the P4 and P5 layers to strengthen the basic feature extraction; the weighted bidirectional feature pyramid module BiFPN

[11] (bidirectional feature pyramid network) is introduced in the neck layer to strengthen the multi-scale feature extraction. The upsampling layer is added to increase the resolution of the feature map; the CGCA (content guided convolution and attention) module is introduced to combine the shallow feature map and the deep feature map to alleviate the gradient vanishing problem and enhance the detail features; in the detection head part, the DyHead (dynamic head) is used to replace the decoupling head of YOLOv8, and the attention is used to guide the detection head to improve the detection capability.

[0064] In a specific embodiment, the C2f feature extraction module CDR based on dilated residual includes:

[0065] The C2f module of YOLOv8 improves the nonlinear representation ability of the model and enhances the feature extraction ability of the model through the bottleneck design concept. However, it is still difficult to detect the small targets that are widely present in drone aerial photography. In view of the above problems, the present invention introduces a CDR module, which adopts an efficient two-step multi-scale context information acquisition method and has better detection ability for small-scale targets.

[0066] Specifically, Figure 3 As shown in the figure, the first step of the CDR module uses 3×3 convolution, BN and ReLU layers to achieve preliminary feature extraction, which is called regional residualization. In the second step, dilated convolutions with different expansion rates are used to capture multi-scale information, which is called semantic residualization.

[0067] Specifically, through the above two steps, multi-rate dilated convolution can simplify the acquisition of complex semantic information into filtering the feature map by channel, and after obtaining multi-scale information, multiple outputs are summarized. Specifically, all feature maps are connected for BN operation processing, and point-by-point convolution is used to fuse different channel information. Finally, the generated feature map is combined with the original feature map for output. The calculation formula of the CDR module is as follows:

[0068]

[0069]

[0070] Among them, α1, α2, α3 represent the regional residual operations of different regions, β1, β2, β3 represent the semantic residual operations of different regions, Ω represents the BN operation first and then the ReLU activation function, F c It represents the output of regional residualization with the number of channels c. represents the dilated convolution with the dilation rate and convolution kernel being 1 and 3 respectively, Concat represents the channel concatenation operation, and BN represents the batch normalization operation.

[0071] Specifically, feature pyramid FPN (feature pyramid network) is a commonly used multi-scale feature fusion method. However, FPN only fuses information from top to bottom. After YOLOv4, PANet has been widely used, adding bottom-up information fusion on the basis of FPN. For small target detection in drones, higher requirements are put forward for feature fusion. Therefore, an efficient weighted bidirectional feature pyramid module BiFPN (bidirectional feature pyramid network) structure is introduced to improve PANet's one-time bidirectional feature fusion, repeatedly stack the bidirectional feature fusion, and introduce learnable parameters to weight different feature maps according to their contribution.

[0072] In a specific embodiment, an efficient cross-scale bidirectional feature fusion module includes:

[0073] The introduced BiFPN structure is as follows Figure 4As shown in the figure, based on YOLOv8n, one upsampling layer is added to the neck to obtain a five-layer feature map. By detecting a larger feature map, small targets and their detailed information can be more finely distinguished, thereby improving the detection accuracy of small targets.

[0074] Specifically, the structural improvements are as follows:

[0075] First, inputs greater than two are considered nodes. Second, similar to the residual structure, the original input nodes of the same layer are jump-connected to the output nodes. Finally, the bidirectional feature fusion of PANet is improved by repeatedly stacking the bidirectional feature fusion to strengthen the feature fusion. The P1 layer is also used as an original input node to the P2 layer for feature fusion through 1×1 convolution, which increases the diversity of features while increasing less computational cost. The calculation formula of the node can be expressed as:

[0076]

[0077] in, It is the top-down input of the 4th layer of the backbone network. It refers to the top-down intermediate output of the 4th layer of the backbone network. refers to the output of the 4th layer of the backbone network from bottom to top, θ represents the matching resolution through upsampling or downsampling operations, and w i are learnable weights, where i=1, 2, 3 represent learned weights of different sizes, ε is a minimum value to avoid numerical instability, and the features of other layers are constructed in a similar way.

[0078] Specifically, low-level features (such as edges and contours) play an important role in feature fusion, but their role will gradually weaken after passing through many intermediate layers. To alleviate this problem, a fusion scheme based on content guided attention is further proposed, see Figure 5 shown.

[0079] In a specific embodiment, the content-guided convolution and self-attention fusion module CGCA includes: the content-guided attention is composed of channel attention and spatial attention, which adaptively represents the importance level of different regions, and the calculation formula can be expressed as:

[0080]

[0081] in, and They represent the features processed by cross-space global average pooling, cross-channel global average pooling and cross-channel global maximum pooling operations respectively, ReLU represents the ReLU activation function, and Concat represents the channel cascade operation.

[0082] Specifically, in order to further enhance the model's feature expression capability, the spatial and channel attention of content-guided attention is replaced with local and global attention to obtain the CGCA module. The structure of the CGCA module is as follows: Figure 6 As shown in the figure, in the local branch, the channel shuffle operation is used to further fuse the channel information, and the 3×3×3 convolution is used to extract features. In the global branch, the transformer self-attention structure is used:

[0083] F local =Conv 3×3×3 (CS(Conv 1×1 (X)));

[0084]

[0085] F′=Concat(F local +F global );

[0086] F out =Sigmoid(GConv 7×7 (CS(F′,X)));

[0087] Among them, F local represents the output of the local branch, CS represents the channel shuffle operation, GConv 7×7 represents a group convolution with a kernel size of 7, and ω is a learnable scaling factor used to control and The amount of matrix multiplication operations.

[0088] Specifically, the structure of the content-guided convolution and self-attention fusion module is as follows: Figure 2 As shown in the CGCA fusion module in Figure 2, the low-level features of the encoder and the corresponding high-level features are sent to the CGCA module to calculate the weights, and then combined by weighted summation, and the gradient vanishing problem is alleviated by skip connection. Finally, the fused features are integrated through 1×1 convolution to obtain the final feature F fuse :

[0089] X=F low +F high ;

[0090] F fuse =Conv 1×1 (ρF low +(1-ρ)F high +X);

[0091] Among them, F low represents the low-level features, F high represents high-level features, Ffuse represents the output feature, and ρ represents the weight obtained by the CGCA module.

[0092] Specifically, the detection head of YOLOv8 has good detection effect on objects of regular size, but its recognition ability for small targets is insufficient. Therefore, there is a higher demand for the performance of the detection head. This embodiment introduces a dynamic detection head (DyHead) and introduces an attention mechanism into the detection head.

[0093] In a specific embodiment, the efficient dynamic detection head includes:

[0094] like Figure 7 As shown in Figure 1, the dynamic detection head deploys an attention mechanism on each specific feature dimension, unifying scale perception, spatial perception, and task perception. The attention formula is as follows:

[0095] Y=π C (π S (π L (X)));

[0096] π L =hard-sigmoid(f(X));

[0097]

[0098] π c =ρ(η 1 (X)X c +μ 1 (X),η 2 (X)X c +μ 2 (X));

[0099] Among them, π L , π S , π C are the attention functions applied to scale, space, and task, respectively. f represents the average pooling, 1×1 convolution, and ReLU activation function. p k +Δp k is the offset, Δm k is in p k The importance scalar of self-learning, X c represents the feature slice of the cth channel, and η and μ are hyperfunctions that learn to control the activation threshold.

[0100] See also Figure 8 As shown, this embodiment 1 also discloses a small target detection system under drone aerial photography, including:

[0101] Data acquisition module: obtains drone aerial photography data to build a drone aerial photography dataset;

[0102] Model building module: Improve the YOLOv8 model, including: using the cascaded dilated residual CDR module to replace the CSP bottleneck C2f module with two convolutions in the P4 and P5 layers in the backbone network for feature extraction; introducing the weighted bidirectional feature pyramid network BiFPN structure in the neck, using cross-scale connections and weighted fusion input features to obtain multi-scale feature fusion features; introducing content-guided convolution and self-attention CGCA modules, adjusting the output feature map through the learned spatial weights to enhance detail features; in the detection head part, introducing a dynamic detection head;

[0103] Model training module: Use the drone aerial photography dataset to train the improved YOLOv8 model to obtain the drone aerial photography detection model;

[0104] Detection module: Input the drone aerial image to be detected into the drone aerial detection model to obtain the drone aerial detection result

[0105] Embodiment 2:

[0106] The method and system for detecting small targets under drone aerial photography provided by the present invention are used to conduct experiments and analyses.

[0107] Specifically, the experiment uses the VisDrone2019 dataset, which contains 10 types of object annotations and a total of 8629 images, including 6471 training set images, 548 validation set images, and 1610 test set images.

[0108] The data set contains a large number of small targets, and the category distribution is uneven. In order to further verify the generalization ability of the drone aerial photography detection model (DCD-YOLO) provided by the present invention, the Pascal VOC07+12 data set is selected for model training and verification. Among them, the training set uses the VOC07+12 training data set, which contains 16,551 pictures, and the verification set uses the VOC07 test data set, which contains 4,952 pictures and 20 types of object annotations.

[0109] Specifically, the experimental environment of this embodiment is the operating system Windows 10, the CPU is Inter (R) Core (TM) i5-10400F @ 2.9GHz, the memory is 32GiB, the graphics card is Nvidia GeForce RTX 2080Ti, and the video memory is 22GiB. The CUDA version is 11.7, and the cuDNN version v8.9.7 is used for GPU acceleration, the Pytorch version is 2.1.0, and the Python version is 3.9.19. All models are run under the same hyperparameters for training, verification and testing. The specific parameters of the experiment are shown in Table 1.

[0110] Table 1 Training parameter settings

[0111] Dataset VisDrone2019 VOC Optimizer SGD SGD Batch size 16 16 Cycle times 300 300 Input size 640 640 Learning Rate 0.01 0.01 Weight decay coefficient 0.0005 0.0005

[0112] In a specific embodiment, in order to verify the effectiveness of the proposed improvement strategy, YOLOv8n is used as the benchmark model, and an ablation experiment is performed on the VisDrone2019 dataset. The accuracy (Precision), recall rate (Recall), average precision (mAP), model parameter quantity (Params) and calculation amount (FLOPs) are used as evaluation indicators of the model. The specific calculation formula is as follows:

[0113]

[0114] Among them, TP represents the number of true positive examples; FN represents the number of false negative examples; FP represents the number of false negative examples, P A represents the average precision; P mA represents the mean of the average precision; P Ai represents the average precision of the i-th category.

[0115] Specifically, the results of the comparative experiments between DCD-YOLO and YOLO series algorithms are shown in Table 2. As an early target detection algorithm, YOLOv3Tiny performs poorly in the detection of small targets in drones. YOLOv5 and YOLOv8 are two versions of YOLO series networks launched by Ultralytics. YOLOv8n has reached 0.334 and 0.195 in mAP@0.5 and mAP@0.5-0.95, and YOLOv8s has reached 0.393 and 0.233 in mAP@0.5 and mAP@0.5-0.95. The indicators of YOLOv5 and YOLOv8 are relatively close. As target detection algorithms that have been widely used in recent years, they have achieved a good balance between accuracy and model parameter calculation. YOLOv6n is not superior to YOLOv8n, and YOLOv7Tiny is not superior to YOLOv8s. YOLOv9t, as the latest lightweight model version released in v9, achieved 0.338 and 0.197 in mAP@0.5 and mAP@0.5-0.95, which is better than YOLOv8n. YOLOv10s achieved 0.401 and 0.240 in mAP@0.5 and mAP@0.5-0.95, reflecting the advantages of YOLOv10 in network architecture. YOLOv11n and YOLOv11s, as the latest YOLO series detection algorithms, have similar small target detection performance to the YOLOv10 series. DCD-YOLO (the drone aerial photography detection model provided by the present invention) achieves precise measurement accuracy of 0.407mAP@0.5 and 0.245mAP@0.5-0.95 for the model comprehensive evaluation indicators with 2.9×106 model parameters and 13.7×109 model complexity. Comprehensively comparing all the above algorithms, DCD-YOLO (the drone aerial photography detection model provided by the present invention) has a huge advantage in detection accuracy while the number of parameters is 3.3% smaller than that of YOLOv8n, and it has a good balance between detection accuracy and model size.

[0116] Table 2 Comparison between DCD-YOLO (UAV aerial photography detection model provided by the present invention) and YOLO series algorithms

[0117] Model P R mAP@0.5 mAP@0.5-0.95 <![CDATA[Params / 10 6 ]]> <![CDATA[FLOPs / 10 9 ]]> YOLOv3Tiny 0.393 0.247 0.240 0.134 19.1 108 YOLOv5n 0.452 0.332 0.332 0.193 2.5 7.1 YOLOv5s 0.493 0.382 0.390 0.197 9.1 23.8 YOLOv6n 0.405 0.311 0.304 0.176 4.23 11.8 YOLOv7Tiny 0.470 0.399 0.368 0.193 6.04 13.3 YOLOv8n 0.448 0.328 0.334 0.195 3 8.1 YOLOv8s 0.505 0.384 0.393 0.233 11.1 28.5 YOLOv9t 0.451 0.337 0.338 0.197 2.0 7.7 YOLOv9s 0.530 0.384 0.407 0.242 7.2 21.4 YOLOv10n 0.433 0.329 0.330 0.192 2.7 8.2 YOLOv10s 0.516 0.395 0.401 0.240 7.17 26.7 YOLOv11n 0.439 0.330 0.332 0.191 2.58 6.3 YOLOv11s 0.503 0.379 0.392 0.235 9.42 21.3 DCD-YOLO 0.511 0.401 0.407 0.245 2.9 13.7

[0118] Specifically, DCD-YOLO (the drone aerial photography detection model provided by the present invention) is compared with other mainstream target detection algorithms as shown in Table 3. Faster-RCNN and Cascade-RCNN, as two-stage target detection algorithms, achieve detection accuracies of 0.321 and 0.318 at mAP@0.5, respectively. However, the number of parameters and the amount of calculation of the model are too large, which affects the reasoning speed and does not meet the real-time requirements of this detection task. DCD-YOLO (the drone aerial photography detection model provided by the present invention) improves mAP@0.5 by 21.5% compared to the one-stage algorithm TOOD. RetinaNet and RTMDet achieve detection accuracies of 0.271 and 0.308 at mAP@0.5, respectively. The detection accuracies are too low to be competent for the detection tasks of this study.

[0119] Specifically, the above experimental results show that DCD-YOLO (the drone aerial photography detection model provided by the present invention) has much lower parameters and calculation amount than the mainstream target detection model, has a huge advantage in detection accuracy, and is capable of detecting small drone targets.

[0120] Table 3 Comparison of DCD-YOLO (UAV aerial photography detection model provided by the present invention) and mainstream target detection algorithms

[0121]

[0122]

[0123] Specifically, in order to prove the effectiveness of adding the P1 layer as an input node in the improved BiFPN structure and replacing the two branch attentions in the CGCA fusion module proposed in the present invention, an internal module improvement ablation experiment was performed.

[0124] The experimental results are shown in Table 4. Adding the P1 layer as an input node improves the detection accuracy mAP@0.5 by 1.5% and mAP@0.5-0.95 by 1.7%. The number of parameters remains almost unchanged and the amount of calculation only increases by 3%. Replacing the attention layer also improves the detection accuracy mAP@0.5 by 0.7% and mAP@0.5-0.95 by 0.4%, while the number of parameters only increases by 3.6% and the amount of calculation only increases by 4.6%.

[0125] Table 4. Ablation experiment of module internal improvement

[0126] P1 layer input Replace attention P R mAP@0.5 mAP@0.5-0.95 <![CDATA[Params / 10 6 ]]> <![CDATA[FLOPs / 10 9 ]]> 0.521 0.382 0.401 0.241 2.9 13.3 √ 0.511 0.401 0.407 0.245 2.9 13.7 0.512 0.391 0.404 0.244 2.8 13.1 √ 0.511 0.401 0.407 0.245 2.9 13.7

[0127] The experimental results of the CGCA fusion module and other mainstream attention algorithms are compared in Table 5. Among them, YOLOv8n* indicates that the CDR module is used to replace the C2f module in the YOLOv8n model, the BiFPN structure is introduced and the upsampling layer is added, and the DyHead is used to replace the native detection head of YOLOv8n. From Table 5, it can be seen that the lightweight attention module simple parameter-free attention (SimAM), convolutional block attention module CBAM, and multi-scale attention module EMA cannot focus on the tiny features of the small target of the drone, resulting in a decrease in detection accuracy. The introduction of channel prior convolutional attention (CPCA), large separable kernel attention (LSKA), deformable attention (DAttention), and squeeze and excitation attention (SEAttention) modules did not significantly improve the model performance and did not make full use of context information, indicating that the mainstream attention module does not have a significant effect on the improvement of small target detection. The CGCA fusion module adds context information fusion on the basis of the attention module, and the detection accuracy mAP@0.5 is improved by 1.2%, and mAP@0.5-0.95 is improved by 1.2%.

[0128] Table 5 Comparison of attention module improvements

[0129] Methods P R mAP@0.5 mAP@0.5-0.95 <![CDATA[Params / 10 6 ]]> <![CDATA[FLOPs / 10 9 ]]> YOLOv8n 0.506 0.389 0.402 0.242 2.77 13.2 SimAM 0.513 0.383 0.395 0.237 2.96 14.8 CPCA 0.513 0.383 0.399 0.239 3.08 14.7 <![CDATA[CBAM [ ]]> 0.506 0.380 0.395 0.238 3.02 14.5 EMA 0.504 0.385 0.398 0.239 2.96 14.5 LSKA 0.499 0.391 0.401 0.239 3.02 14.5 DAttention 0.513 0.389 0.399 0.241 3.22 14.7 SEAttention 0.504 0.384 0.399 0.240 2.96 13.5 CGCAFM 0.511 0.401 0.407 0.245 2.9 13.7

[0130] In order to further verify the effectiveness of each improved module of DCD-YOLO, ablation experiments were conducted on each module. As shown in Table 6 below, the baseline model uses the YOLOv8n algorithm and achieves a detection accuracy of 0.334 on mAP@0.5. Using a single module improves the detection accuracy to varying degrees. On the basis of adding an upsampling layer, the BiFPN structure is introduced and the detection head is replaced with DyHead. The detection accuracy is increased to 0.384 and 0.389 in mAP@0.5 respectively. The upsampling layer is combined with the BiFPN structure and DyHead. The model accuracy is increased to 0.395 in mAP@0.5. On this basis, the CDR module is further introduced, and the mAP@0.5 is increased to 0.402. Finally, DCD-YOLO achieves a detection accuracy of 0.407mAP@0.5, which is 21.9% higher than the baseline model. The results of the ablation experiment prove the effectiveness of the improved modules in DCD-YOLO.

[0131] Table 6 Ablation experiment results

[0132] Baseline CDR BiFPN Upsample CGCAFM DyHead mAP@0.5 mAP@0.5-0.95 <![CDATA[Params / 10 6 ]]> <![CDATA[FLOPs / 10 9 ]]> √ 0.334 0.195 3 8.1 √ √ 0.336 0.196 2.95 8 √ √ 0.341 0.200 1.99 7.1 √ √ 0.382 0.227 3.97 14.8 √ √ 0.338 0.198 3.63 9.5 √ √ 0.366 0.216 3.49 9.6 √ √ √ 0.389 0.232 4.31 15.4 √ √ √ 0.384 0.230 2.08 11.1 √ √ √ √ 0.395 0.237 2.82 13 √ √ √ √ √ 0.402 0.242 2.77 13.2 √ √ √ √ √ √ 0.407 0.245 2.9 13.7

[0133] like Fig. 9, showing the actual detection performance comparison between YOLOv8n and DCD-YOLO. By comparing the two groups of pictures, it can be seen that YOLOv8n has poor detection performance in small targets, some occluded areas, distant areas and at night. DCD-YOLO solves these missed detection problems very well. For example, in the first group of pictures, it can be seen that DCD-YOLO can detect targets far away on the road and partially occluded targets. From the second group of pictures, it can be seen that DCD-YOLO can detect cars and vans that YOLOv8n misdetected. From the third group of pictures, it can be seen that DCD-YOLO can also accurately identify small targets in night scenes.

[0134] In order to further verify the generalization of DCD-YOLO proposed in this study, the baseline algorithm YOLOv8n and DCD-YOLO were experimented on the Pascal VOC07+12 dataset. From Table 7, it can be seen that DCD-YOLO is generally better than YOLOv8n, and the detection accuracy mAP@0.5 has increased from 0.746 to 0.768, an increase of 3%. Among the 20 classes in VOC, the detection accuracy of 18 classes has been significantly improved. This shows that DCD-YOLO also performs well on the Pascal VOC dataset with scattered small objects, and has a certain degree of generalization.

[0135] Specifically, through the above technical solution, the present invention proposes an improved algorithm DCD-YOLO based on the YOLOv8n model that balances performance and model size. In view of the insufficient detection performance of YOLOv8n on small targets of drones, a CDR module based on dilated convolution is introduced to construct a more comprehensive feature representation in the backbone network; a BiFPN module is introduced to add an upsampling layer to the neck to increase the image resolution and improve the feature fusion of the neck; a CGCA fusion module is introduced to fuse low-level and corresponding high-level information respectively to enhance detail features; DyHead is used to combine attention and detection head to improve the detection capability of the detection head. Compared with the baseline model YOLOv8n, the DCD-YOLO provided by the present invention achieves a mAP@0.5 comprehensive detection accuracy of 0.407, which is 21.9% higher than the baseline model, and also has advantages in lightweight.

[0136] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0137] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting small targets under drone aerial photography, characterized in that: include Obtain drone aerial photography data to build a drone aerial photography dataset; Improve the YOLOv8 model, including: using the cascaded dilated residual CDR module to replace the CSP bottleneck C2f module with two convolutions in the P4 and P5 layers in the backbone network for feature extraction; introducing the weighted bidirectional feature pyramid network BiFPN structure in the neck, using cross-scale connections and weighted fusion input features to obtain multi-scale feature fusion features; introducing content-guided convolution and self-attention CGCA modules, adjusting the output feature map through the learned spatial weights to enhance detail features; in the detection head part, introducing a dynamic detection head; The improved YOLOv8 model is trained using the drone aerial photography dataset to obtain a drone aerial photography detection model; The drone aerial images to be detected are input into the drone aerial detection model to obtain the drone aerial detection results.

2. The method for detecting small targets under drone aerial photography according to claim 1, characterized in that: The calculation formula of the cascade dilated residual CDR module is: Among them, α1, α2, α3 represent the regional residual operations of different regions, β 1, β 2, β3 represents the semantic residual operation of different regions, Ω represents the batch normalization BN operation followed by the ReLU activation function, and F c It represents the output of regional residualization with the number of channels c. represents the dilated convolution with the dilation rate and convolution kernel being 1 and 3 respectively, Concat represents the channel concatenation operation, and BN represents the batch normalization operation.

3. The method for detecting small targets under drone aerial photography according to claim 1, characterized in that: The node calculation formula of the weighted bidirectional feature pyramid network BiFPN structure is: in, It is the top-down input of the 4th layer of the backbone network. It is the top-down intermediate output of the 4th layer of the backbone network. is the output of the 4th layer of the backbone network from bottom to top, θ represents the resolution matching by upsampling or downsampling operation, and w i is a learnable weight, where i=1,2,3 represents the learned weights of different sizes, and ε is a minimum value.

4. The method for detecting small targets under drone aerial photography according to claim 1, characterized in that: The calculation formula of the content-guided convolution and self-attention CGCA module is: in, and They represent the features processed by cross-space global average pooling, cross-channel global average pooling and cross-channel global maximum pooling operations respectively, ReLU represents the ReLU activation function, and Concat represents the channel cascade operation.

5. The method for detecting small targets under drone aerial photography according to claim 4, characterized in that: In the local branch of the content-guided convolution and self-attention CGCA module, the channel shuffle operation is used to further fuse the channel information, and the 3×3×3 convolution is used to extract features. In the global branch, the transformer self-attention structure is used, and the calculation formula is: F local =Conv 3×3×3 (CS(Conv 1×1 (X))); F′=Concat(F local +F global ); F out =Sigmoid(GConv 7×7 (CS(F′,X))); Among them, F local represents the output of the local branch, CS represents the channel shuffle operation, GConv 7×7 represents a group convolution with a kernel size of 7, and ω is a learnable scaling factor used to control and The amount of matrix multiplication operations.

6. The method for detecting small targets under drone aerial photography according to claim 1, characterized in that: The calculation formula for obtaining the multi-scale feature fusion feature is: X=F low +F high ; F fuse =Conv 1×1 (ρF low +(1-p)F high +X); Among them, F low represents the low-level features, F high represents high-level features, F fuse represents the output feature, and ρ represents the weight obtained by the CGCA module.

7. The method for detecting small targets under drone aerial photography according to claim 1, characterized in that: The calculation formula of the dynamic detection head is: Y=π C (p S (p L (X))); π L =hard-sigmoid(f(X)); Among them, π L , π S , π C are the attention functions applied to scale, space, and task, respectively. f represents the average pooling, 1×1 convolution, and ReLU activation function. p k +Δp k is the offset, Δm k is in p k The importance scalar of self-learning, X c represents the feature slice of the cth channel, and η and μ are hyperfunctions that learn to control the activation threshold.

8. A small target detection system under drone aerial photography, characterized in that: include: Data acquisition module: obtains drone aerial photography data to build a drone aerial photography dataset; Model building module: Improve the YOLOv8 model, including: using the cascaded dilated residual CDR module to replace the CSP bottleneck C2f module with two convolutions in the P4 and P5 layers in the backbone network for feature extraction; introducing the weighted bidirectional feature pyramid network BiFPN structure in the neck, using cross-scale connections and weighted fusion input features to obtain multi-scale feature fusion features; introducing content-guided convolution and self-attention CGCA modules, adjusting the output feature map through the learned spatial weights to enhance detail features; in the detection head part, introducing a dynamic detection head; Model training module: Use the drone aerial photography dataset to train the improved YOLOv8 model to obtain the drone aerial photography detection model; Detection module: Input the drone aerial image to be detected into the drone aerial detection model to obtain the drone aerial detection result.

Images