YOLOv10-based unmanned aerial vehicle aerial image detection method for S-YOLO

Abstract

The invention provides an improved detection algorithm S-YOLO for unmanned aerial vehicle aerial images based on a YOLOv10 algorithm, and belongs to the technical field of target detection of unmanned aerial vehicle aerial remote sensing images. The algorithm mainly comprises the following steps: firstly, providing a C2fGCU module so as to enhance the small target feature extraction capability of backbone; secondly, an FFM module is provided, the multi-scale feature fusion capability of the neck is improved, and small target features are further enhanced; then, introducing a Dice loss function into the classification loss function to improve problems caused by label imbalance; thirdly, training parameters are set, and weight data are stored; finally, the optimal weight in the verification set is selected to be loaded into the network, and the test set is used for evaluation. The invention aims to improve the detection precision of the small target aerial photography of the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of drone image detection, and in particular to a drone aerial image detection method improved based on the YOLOv10 algorithm, which can be used for target detection tasks in drone aerial images. Background Art

[0002] UAV aerial photography technology has developed rapidly in recent years and is widely used in agricultural monitoring, urban planning, environmental protection, disaster assessment and other fields. Compared with traditional ground monitoring methods, UAVs can obtain high-resolution images of a larger range in a shorter time, greatly improving the efficiency and flexibility of data collection.

[0003] In drone aerial images, target objects usually have the following characteristics: small size, possible occlusion, low resolution, and easy to be confused in complex backgrounds. These characteristics make target detection in drone aerial images face many challenges, especially in the detection accuracy of small targets, which often leads to false detection and missed detection problems.

[0004] Traditional target detection algorithms often perform well when dealing with large-sized targets, but have limitations when dealing with small-sized targets. In order to improve detection accuracy, researchers have gradually proposed detection algorithms based on deep learning, such as the YOLO (You Only Look Once) series. These algorithms achieve real-time target detection through deep neural networks, but when processing drone aerial images, targeted improvements are still needed to adapt to the characteristics of small targets and complex environments.

[0005] Therefore, the research on target detection technology for drone aerial images will not only help improve the detection capability of small targets, but also help promote the development of related applications, such as precision agriculture, environmental monitoring, and public safety. Technological progress in this field will lay the foundation for the widespread popularization of drone applications. Summary of the invention

[0006] 1. Technical issues to be solved

[0007] The purpose of the present invention is to propose a new remote sensing image detection method in view of the above-mentioned shortcomings, which combines the attention mechanism and multi-scale fusion to better improve the detection accuracy of remote sensing images and solve the problems raised in the above-mentioned background technology.

[0008] (II) Technical solution

[0009] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0010] A network algorithm for detecting ground targets of unmanned aerial vehicles based on YOLOv10, comprising the following steps:

[0011] S1, based on the YOLOv10n framework, introduces a small target detection layer (P2);

[0012] S2, introduce the small target feature extraction module C2fGCU into the backbone network;

[0013] S3, introduce the feature fusion module FFM in the feature fusion part;

[0014] S4. In the detection part, the Dice loss function is introduced and combined with the BCE weighting as the classification loss function;

[0015] S5. Conduct ablation experiments to verify the effectiveness of each of the above steps and compare them with other algorithms to ensure that the improved model has improved performance.

[0016] A further improvement of the technical solution of the present invention is that in S1, the redundant P5 detection head (and related upsampling layer, Concat layer, Conv layer) for the small target detection task in the original network is deleted, and the P2 detection head (and related upsampling layer, Concat layer, Conv layer) for small targets is introduced.

[0017] The further improvement of the technical solution of the present invention is that: in S2, the original YOLOv10n backbone network is improved: a small target feature extraction module C2fGCU is added to the backbone network, and the small target features are enhanced through a gating mechanism.

[0018] The further improvement of the technical solution of the present invention is that in S3, the feature fusion part after the improvement of S1 is improved: an FFM feature fusion module is added to perform multi-scale feature fusion after strengthening the small target features.

[0019] The further improvement of the technical solution of the present invention is that: in S4, the original classification loss function BCE is improved by adding the Dice loss function and performing a weighted combination with the original BCE to improve the problem caused by label imbalance.

[0020] A further improvement of the technical solution of the present invention is that in S5, the model weight with the smallest loss function is selected and loaded into the network to test its detection accuracy and visualization result graph.

[0021] Due to the adoption of the above technical solution, the technical progress achieved by the present invention is:

[0022] The original YOLOv10 model mainly detects targets of general size, while the objects in drone aerial images are small, easily occluded, blurred, and numerous, so direct detection often results in false detection and missed detection. The improved S-YOLO model has made multiple optimizations to address this challenge: first, the P2 detection head is specially designed for small-sized targets and effectively detects tiny objects in images; second, the C2fGCU feature extraction module can efficiently extract features of small targets; the feature fusion module FFM solves the problem that the original model failed to effectively fuse multi-scale features by enhancing small target features and performing multi-scale fusion; finally, the combination of BCE and Dice weighted loss functions successfully addresses the label imbalance problem. These improvements make the S-YOLO model more suitable for target detection tasks in drone aerial images. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the process of the present invention;

[0024] Figure 2 It is a schematic diagram of the network model structure of S-YOLO of the present invention;

[0025] Figure 3 The schematic diagram of the network model structure after adding a small target detection layer to the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of the C2fGCU module proposed in the present invention;

[0027] Figure 5 This is a schematic diagram of the structure of the FFM module proposed in the present invention;

[0028] Figure 6 This is a detection effect diagram of the S-YOLO model proposed in the present invention in a complex environment;

[0029] Figure 7 This is a diagram showing the detection effect of the S-YOLO model proposed in the present invention on extremely small targets;

[0030] Figure 8 This is a detection effect diagram of the S-YOLO model proposed in the present invention in a night environment;

[0031] Fig. 9 This is a detection effect diagram of the S-YOLO model proposed in the present invention in a strong light environment;

[0032] Fig.10 This is a detection effect diagram of the S-YOLO model proposed in the present invention in a color light environment; DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] Example

[0035] like Figure 1 As shown, the present invention is an improved UAV aerial image detection method based on the YOLOv10 algorithm, comprising the following steps:

[0036] like Figure 2 As shown in the figure, S1 introduces a small target detection layer (P2) and deletes a large target detection layer (P5) to detect small-sized targets. S2 adds two C2fGCU modules to the backbone network for small target feature extraction. S3 adds two feature fusion modules FFM to the Neck part to fuse multi-scale features. Finally, S4 adds the Dice loss function to the detection head to solve the label imbalance problem.

[0037] like Figure 3 As shown, S1 introduces a detection layer specifically for small targets; the original network structure has a significant effect on targets of regular size, but the targets in drone aerial images are extremely small, resulting in incomplete detection of small target features. In response to the above problems, the present invention improves the original network architecture by adding two Concat layers, two C2f layers, an Upsample layer, a Conv layer, and a P2 detection head to the Neck part, and then deleting an SCDown layer, a Concat layer, a C2fCIB layer, and a P5 detection head. The optimized network architecture can better help the model capture global information and contextual small target information, and enhance the detection capability of small targets.

[0038] S2, add a small target feature extraction module C2fGCU. Since the target detection performed by YOLOv10 mainly focuses on targets of ordinary size, the effect of the feature extraction module G2f in the backbone may be reduced for small targets in drone images, and it cannot effectively extract the features of small targets. To address this problem, the present invention proposes a module G2fGCU (such as Figure 4The G2fGCU module mainly replaces the bottleneck of G2f with the GCU module. The GCU module contains three branches, of which the first two branches constitute a gating mechanism that can more accurately select and enhance the features of small targets while promoting nonlinear interactions between features. This interaction not only enhances useful features, but also suppresses irrelevant or noisy features, which is especially important for small targets because their features are often weak and easily disturbed by background noise. The remaining residual connections retain global information to ensure that the features of small targets are not diluted due to increasing depth.

[0039] The mathematical expression of the GCU module is:

[0040] W1=Conv3×3(x) (1)

[0041] W2=Sigmoid[Conv3×3(x)] (2)

[0042] W2=Sigmoid[Conv3×3(x)] (3)

[0043] W3=Conv3×3[W1⊙W2] (4)

[0044]

[0045] Among them, Conv3×3() represents the convolution operation with a convolution kernel of 3×3 on the feature, Sigmoid() represents the function mapping of the feature with the activation function of Sigmoid, and ⊙ represents the element-wise multiplication operation of the feature mapping. Represents the element-wise addition operation of the feature map. x is the input feature map, W1, W2, and W3 represent the feature maps after a series of operations, and Y represents the output feature map of the GCU module.

[0046] S3, add the feature fusion module FFM. The feature fusion part of YOLOv10 is located in the Neck in the model graph, and features of different depths (i.e., features of targets of different sizes) will be fused in the Neck. However, there are some problems in the Neck, which will lead to poor feature fusion effect of small targets. First, the feature fusion module (C2f) is easy to blur the low-level information without residual connection, and these low-level information are the key to the features of small targets. Therefore, the C2f module without residual connection will weaken the model's ability to detect small targets. Secondly, the feature fusion module needs to integrate shallow and deep features at the same time, where low-level features correspond to small targets and deep features correspond to large targets. However, the C2f module fails to fully consider the fusion requirements of multi-scale features, resulting in a significant decrease in the detection effect of the model on features of different scales. In response to the above problems, the present invention proposes a module FFM (such as Figure 5 As shown in the figure, FFM consists of two modules: Squeeze-and-Excitation Networks (SENet), a multi-scale feature extraction module (Multi-Scale Feature Extractor, MFE) and a hybrid convolution.

[0047] The mathematical expression of the FFM module is:

[0048] Y1=SENet(X) (6)

[0049] Y2=MFE(Y1) (7)

[0050] Y=MixGonv(Y2) (8)

[0051] Among them, X is the input feature map of the FFM module, Y1 is the output of the SE module, Y2 is the output of the MFE module, and Y is the output of the mixed convolutional layer MixGonv, which is also the output of the FFM module.

[0052] The SENet module enhances the model's ability to understand input features by learning the importance of each channel and applying these importance weights to the original feature map, making the model more focused on the key information in the input data. The residual connection ensures that the information is retained and will not be lost due to increased depth. The mathematical expression of the SENet module is:

[0053]

[0054] Y3=Y2.view(B,C,1,1)∈R B×C×1×1 (12)

[0055] Y=X⊙Y3,Y∈R B×C×H×W (13)

[0056] Among them, H and W are height and width respectively, R B×C is a real matrix, indicating that the shape of avg_pool is B×C. ⊙ represents the element-wise multiplication operation of the feature map. W represents the element addition operation of the feature map, and r is the scaling reduction. i and B i represents the weight and bias of the i-th fully connected layer. Y1 represents the output of avg_pool after being mapped by the first fully connected layer and then by the ReLU activation function. Y2 represents the output of Y1 after being mapped by the second fully connected layer and then by the Sigmoid activation function. Y3 reshapes Y2 from a matrix of shape B×C to a matrix of shape B×C×1×1. Y is the final output feature of the SENet module.

[0057] The MFE module extracts features of different scales through 1x1, 3x3 and 5x5 convolutions. 1x1 convolution can quickly obtain channel information, while 3x3 and 5x5 convolutions provide richer spatial information. These three convolutions can effectively capture the features of targets of different sizes, thereby improving detection accuracy. Among them, the depth-separable convolution with 3x3 and 5x5 as convolution kernels can further reduce model parameters and computational complexity while maintaining the effect of feature extraction. This convolution method is particularly effective in extracting features of small targets, and can reduce the burden of the model while maintaining spatial information. Then, feature maps of different scales are spliced ​​in the channel dimension, and the model can integrate diverse information from different convolutions. This fusion method not only improves the ability to comprehensively understand the target, but also enhances the capture of small target details. Finally, the fusion feature map and the input feature map are multiplied element by element, which enhances the model's attention to important features, improves the detection effect of small targets, and suppresses interference information, thereby effectively integrating multi-scale features and optimizing target detection performance.

[0058] The mathematical expression of the MFE module is:

[0059]

[0060] Among them, Concat() means concatenating the feature maps inside. represents a standard convolution with a convolution kernel of a×a. It represents a depth-wise separable convolution with a convolution kernel of a×a, and ⊙ represents the element-wise multiplication operation of the feature map.

[0061] After the input feature map passes through SENet and MFE, it will be input into the mixed convolution layer (Mix_conv). This layer uses group convolution, and groups is set to the maximum (the greatest common factor of the input and output channels of the mixed convolution). It can not only effectively reduce the number of parameters and computational complexity, but also enable the feature map to perform efficient and precise convolution operations, significantly enhancing the ability to capture small target features.

[0062] S4, introduces the weighted combination of Dice and BCE. In YOLOv10, the classification loss function uses BCE, but BCE is not very applicable when the labels are unbalanced. Therefore, this paper introduces the Dice loss function to effectively deal with the problem of label imbalance, and uses the weighted combination of BCE and Dice loss function as the classification loss function of the S-YOLO model, which can more comprehensively optimize the detection performance of the S-YOLO model.

[0063] The mathematical formula of the BCE loss function is as follows:

[0064]

[0065] Where: N is the total number of samples. i is the true label (0 or 1) of the i-th sample. is the probability that the model predicts that the i-th sample belongs to the foreground (a value between 0 and 1). log() is the natural logarithm.

[0066] The mathematical formula of the Dice loss function is:

[0067]

[0068] Where: p i is the predicted probability value (usually between 0 and 1) of the ith pixel (or sample). i is the true label value (0 or 1) of the i-th sample. N is the number of all pixels (or samples) in the image. ε is a small smoothing term used to avoid division by zero, which is set to 10 here. -6 The mathematical formula for the total classification loss is:

[0069] f loss =αf BCE +βf DiceLoss (18)

[0070] Among them, α and β are the weights of BCE and DiceLoss loss functions, which are set to 1 and 0.01 respectively in this paper.

[0071] S5. Conduct ablation experiments to verify the effectiveness of each of the above improvements, and compare with other algorithms to ensure that the improved model has improved performance. The data set of the present invention uses the VisDrone2019 data set, which is collected by the AISKYEYE team of the Machine Learning and Data Mining Laboratory of Tianjin University.

[0072] The experiment of the present invention uses precision (P), recall (R), mean average precision (mAP), model parameters (Params), and model complexity calculation (FLOPs) as the main evaluation indicators. Among them, multi-category average precision mAP is an important indicator for evaluating algorithms, which is used to evaluate the comprehensive performance of detection algorithms for target classification.

[0073] In order to verify the effectiveness of the algorithm S-YOLO proposed in the present invention, ablation experiments were conducted on the small target detection layer, C2fGCU module, FFM module, and Dice and BCE weighted combination proposed in the present invention. P, R, mAP50, mAP50:95, Params, and GFLOPs were used as evaluation indicators to detect the degree of influence of adding various methods to the original model. The experiment of the present invention was trained on the VisDrone2019 dataset for 500 epochs, and 10 groups of ablation experiments were performed under the same experimental conditions. The results after training are shown in Table 1.

[0074] Table 1 Ablation experiment on the VisDrone2019 test set

[0075]

[0076] Continuation Form - Form 1

[0077]

[0078] As can be seen from Table 1, each module is effective, and the final S-YOLO model improves mAP50:95 by 5.3% over the baseline (YOLOv10n).

[0079] In order to verify the effectiveness of the algorithm proposed in this invention, a comparative experiment was carried out on the VisDrone2019 dataset between the proposed algorithm and the current mainstream algorithm. The details are shown in Table 2.

[0080]

[0081] Experiments show that the S-YOLO proposed in this invention has the highest mAP50:95.

[0082] Finally, it should be noted that the above is only a preferred embodiment of the present invention and does not limit the scope of the present invention. Although the above embodiments describe the present invention in detail, those skilled in the art can still modify the technical solutions described in these embodiments or perform equivalent replacements on some technical features. Any modification, equivalent replacement or improvement made within the spirit and principle of the present invention shall be deemed to be included in the protection scope of the present invention.

Claims

1. A method for detecting small targets in drone aerial photography based on improved YOLOv10, characterized in that: Including Next steps: S1, based on the YOLOv10n framework, introduces a small target detection layer (P2); S2, introduce the small target feature extraction module C2fGCU into the backbone network; S3, introduce the feature fusion module FFM in the feature fusion part; S4. In the detection part, the Dice loss function is introduced and combined with the BCE weighting as the classification loss function; S5. Conduct ablation experiments to verify the effectiveness of each of the above steps and compare them with other algorithms to ensure that the improved model has improved performance.

2. According to claim 1, a method for detecting small targets in drone aerial photography based on improved YOLOv10, Features: In S1, a small object detection layer is introduced.

3. According to the improved UAV aerial photography small target detection method based on YOLOv10 according to claim 1, Features: In S2, under the conditions of S1, a small target feature extraction module C2fGCU is introduced into the backbone network.

4. According to claim 1, a method for detecting small targets in drone aerial photography based on improved YOLOv10, Features: In S3, under the conditions of S1, a feature fusion module FFM is introduced into the feature fusion part.

5. According to claim 1, a method for detecting small targets in drone aerial photography based on improved YOLOv10, Features: In S4, under the conditions of S1, the Dice loss function is introduced in the detection part and combined with the BCE weighted function as the classification loss function.

6. According to claim 1, a method for detecting small targets in drone aerial photography based on improved YOLOv10, Features: In S5, an ablation experiment is performed to verify the effectiveness of each of the above steps, and compared with other algorithms to ensure that the improved model has improved performance, and a detection effect diagram of the S-YOLO model proposed in the present invention is given.

Images