A method for detecting small targets under aerial photography of a drone

Abstract

The application discloses a kind of unmanned aerial vehicle aerial photography under small target detection method, applied to artificial intelligence technical field.The application comprises: obtaining unmanned aerial vehicle aerial photography data;The obtained unmanned aerial vehicle aerial photography data is preprocessed, and the data after preprocessing is obtained;The data after preprocessing is divided into training set, test set and validation set;YOLOv8n model is improved;Improved YOLOv8n model is iteratively trained using training set, and the test set is tested using the improved YOLOv8n model after training;Improved YOLOv8n model is verified using validation set, and the identification result of the small target detection of the improved YOLOv8n model to unmanned aerial vehicle aerial photography data is obtained.The application improves detection effect, improves the performance and generalization ability of model, speeds up network model convergence, reduces the missed detection and misdiagnosis of small target under the dense condition of unmanned aerial vehicle aerial photography.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for detecting small targets captured by drone aerial photography. Background Technology

[0002] With the continuous development of technology, object detection has been widely applied in many practical fields, such as intelligent vehicles, robot vision, and high-definition camera surveillance. Drone technology is a fundamental technology for realizing smart living and smart city construction. Its rapid development in recent years has also increased the presence of drones in people's lives, with increasing applications in logistics, transportation, photography, and other fields.

[0003] In existing technologies, Chen et al. added a 160×160 small target detection head to the YOLOv5 network structure and residually connected it to the high-level network. They also introduced a depthwise separable convolution algorithm in the Conv module, which effectively replaced ordinary convolution, thus significantly reducing the model size and improving its performance and accuracy. This improved the small target detection capability. Liu et al. proposed an improved YOLOv5 small target detection method, re-clustering anchor boxes, modifying the backbone network structure, adding convolution in the spatial pyramid pooling layer, and fusing decoupled detection heads to improve small target detection performance. Cui et al. proposed a target detection model based on an improved YOLOv7, introducing a lightweight RepVGG network into the backbone network and inserting the CBAM convolutional attention mechanism between the backbone network and the feature fusion network. Through real-time, high-precision technology, they were able to accurately locate small UAVs in complex environments. Hong et al. proposed a real-time drone tracking scheme, using a 5G network to transmit drone surveillance images to the cloud, and employing machine learning algorithms to detect and track multiple targets. They optimized the network structure of the target detector YOLO4, improved the target tracker DeepSORT, and adopted a detection-tracking mode. Wu et al. provided solutions for three research topics from the perspective of deep learning models: object detection from images, object detection from videos, and object tracking from videos. The performance of open datasets for drone-dominated target detection and tracking was evaluated using four benchmark datasets. Zhu et al. provided a powerful drone capture database, VisDrone, which can meet various needs. These data were captured in 14 different cities or suburbs in China from north to south. It includes four tracks: (1) image object detection, (2) video object detection, (3) single object tracking, and (4) multi-object tracking. A comprehensive review of target detection and tracking datasets and benchmarks was conducted, and the challenges of collecting large-scale drone-based target detection and tracking datasets were discussed. The VisDrone dataset was then described in response to the problems of low image contrast, small size, and large number of images in drone aerial photography environments. GERALDES et al. developed a situational awareness system based on unmanned aerial vehicles (UAVs) called Person-Moving Aligner (PAL). The PAL system addresses the first problem by analyzing video feeds from a UAV powered by a supercomputer on a module.

[0004] Currently, the core of the YOLO object detection algorithm lies in its small model size and high computation speed. YOLO's structure is very simple. It can directly output the position and category of bounding boxes through a neural network. YOLO is fast because it only needs to feed an image into the network to obtain the final detection result, so YOLO can also perform temporal detection in videos. YOLO directly uses the global image for detection, which can encode global information and reduce the error of detecting background as an object. YOLO has strong generalization ability because it can learn highly generalized features and apply them to other fields.

[0005] Therefore, proposing a method for detecting small targets under drone aerial photography, and improving the detection effect of small targets by improving the model, in order to solve the difficulties existing in the prior art, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of this, the present invention provides a method for detecting small targets under aerial photography by unmanned aerial vehicles, which is used to solve the technical problems existing in the prior art.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A method for detecting small targets captured by drone aerial photography includes the following steps:

[0009] S1. Acquire drone aerial photography data;

[0010] S2. Preprocess the acquired drone aerial photography data to obtain preprocessed data;

[0011] S3. Divide the preprocessed data into training set, test set and validation set;

[0012] S4. Improve the YOLOv8n model to obtain the improved YOLOv8n model;

[0013] S5. Iteratively train the improved YOLOv8n model using the training set, and then test the improved YOLOv8n model on the test set using the trained model.

[0014] S6. The effectiveness of the improved YOLOv8n model is verified using the validation set, and the recognition results of the improved YOLOv8n model for small target detection in UAV aerial data are obtained.

[0015] Optionally, in the above method, drone aerial data can be obtained from the Coklu deneme dataset in S1.

[0016] Optionally, in the above method, the synthetic fog enhancement algorithm is used in S2 to preprocess the acquired UAV aerial photography data.

[0017] Optionally, in the above method, the data in S3 can be divided into training set, validation set and test set according to the proportions of 87%, 10% and 3%.

[0018] The above method, optionally, includes the following improvements to the YOLOv8n model in S4:

[0019] The GCE-YOLO algorithm was introduced to improve the YOLOv8n structure; a P2 small object detection layer was added to improve the splicing and fusion of shallow and deep feature maps; a channel attention mechanism was introduced to recalibrate the weights of each channel in the SENet model; and the EfficiCIoU Loss function was introduced to measure the degree of overlap between target boxes.

[0020] Optionally, in S6, comparative experiments, ablation experiments, and YOLO series target detection algorithms were used to verify the effectiveness of the improved YOLOv8n model.

[0021] As can be seen from the above technical solution, compared with the prior art, the present invention has the following beneficial effects:

[0022] (1) Add a P2 small object detection layer to improve the small object detection effect. It is proposed to add a P2 small object detection layer after merging with the deep feature map to detect shallow feature maps. Adding a P2 small object detection layer can make the network pay more attention to the detection of small objects, thus improving the detection effect.

[0023] (2) SEAttention dynamically weights the importance of each channel, enabling the network to better focus on useful channels. Channel attention can be inserted into different layers of the neural network, thus helping the network to extract features more accurately and effectively at different levels. In this way, SEAttention helps the neural network to more adaptively focus on important channel features during the learning process, improving the model's performance and generalization ability.

[0024] (3) Using EfficiCIoU Loss as the bounding box loss function enhances the adjustment of predicted frames and accelerates the frame regression rate, speeds up the convergence of the network model, and reduces the number of missed and false detections of small targets in dense situations under UAV aerial photography. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0026] Figure 1 A flowchart of a method for detecting small targets under drone aerial photography provided by the present invention;

[0027] Figure 2 The image provided by this invention shows the preprocessed result, where a is the original data image and b is the preprocessed data image.

[0028] Figure 3 This is a diagram of the YOLOv8n architecture provided by the present invention;

[0029] Figure 4 An improved YOLOv8n architecture diagram provided for this invention;

[0030] Figure 5 This is a schematic diagram of a small target detection head structure provided in an embodiment of the present invention;

[0031] Figure 6 The above are comparison charts of detection results provided in the embodiments of the present invention, where a represents the detection result of YOLOv8n and b represents the detection result of GCE-YOLO. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Reference Figure 1 As shown, this invention discloses a method for detecting small targets under drone aerial photography, including the following steps:

[0034] S1. Acquire drone aerial photography data;

[0035] S2. Preprocess the acquired drone aerial photography data to obtain preprocessed data;

[0036] S3. Divide the preprocessed data into training set, test set and validation set;

[0037] S4. Improve the YOLOv8n model to obtain the improved YOLOv8n model;

[0038] S5. Iteratively train the improved YOLOv8n model using the training set, and then test the improved YOLOv8n model on the test set using the trained model.

[0039] S6. The effectiveness of the improved YOLOv8n model is verified using the validation set, and the recognition results of the improved YOLOv8n model for small target detection in UAV aerial data are obtained.

[0040] Furthermore, S1 obtains drone aerial photography data from the Coklu deneme dataset.

[0041] Furthermore, in S2, a synthetic fog enhancement algorithm is used to preprocess the acquired UAV aerial photography data.

[0042] Furthermore, in S3, the data is divided into training, validation, and test sets according to the proportions of 87%, 10%, and 3%.

[0043] Furthermore, the specific improvements to the YOLOv8n model in S4 are as follows:

[0044] The GCE-YOLO algorithm was introduced to improve the YOLOv8n structure; a P2 small object detection layer was added to improve the splicing and fusion of shallow and deep feature maps; a channel attention mechanism was introduced to recalibrate the weights of each channel in the SENet model; and the EfficiCIoU Loss function was introduced to measure the degree of overlap between target boxes.

[0045] Furthermore, in S6, comparative experiments, ablation experiments, and YOLO series target detection algorithms were used to verify the effectiveness of the improved YOLOv8n model.

[0046] In one specific embodiment, the following is included:

[0047] (I) Data Acquisition

[0048] Data was obtained using the Coklu Deneme dataset, which originates from the Roboflow dataset website, a platform focused on computer vision. This platform simplifies data annotation and model training, allowing developers to easily upload, download, annotate, and convert their datasets and models. It is also officially recommended by YOLOv8 and supports various data formats and model architectures. The platform provides users with a diverse library of models and free datasets, allowing them to easily obtain the necessary data formats for training models without manual conversion. The dataset is categorized as follows: UAP, UAI, insan (pedestrians), and tasit (vehicles). The pedestrians are small targets.

[0049] (II) Dataset Preprocessing and Data Partitioning

[0050] Because drones shoot from high altitudes, the actual aerial environment is far more complex and diverse than the environment depicted in the collected data. Sometimes, shooting is required in complex weather conditions such as thunderstorms and fog. Therefore, a fog enhancement process was applied to the collected data using a synthetic fog enhancement algorithm. Synthetic fog data enhancement is an image processing-based algorithm used to add fog effects produced by hazy weather, thereby improving image quality and usability. This algorithm simulates the formation of fog to process and synthesize images, making them appear more realistic and natural. Drones use aerial photography, which is greatly affected by weather conditions. For example, shooting in foggy weather will result in a certain degree of fogging in the data acquired by the drone. Simulating fog effects achieves natural scene fogging processing in images, thereby improving the visualization and usability of the images.

[0051] The dataset was augmented using a synthetic fog data augmentation algorithm, resulting in 10,440 enhanced images. Based on an 87%:10%:3% ratio, the dataset was divided into training, validation, and test sets, with 9,136 images in the training set, 976 images in the validation set, and 328 images in the test set. Preprocessed comparisons are shown below. Figure 2 As shown.

[0052] (III) Model Introduction

[0053] YOLOv8 was released in January 2023 by Ultralytics, the company that developed YOLOv5. YOLOv8 offers five scaling versions: YOLOv8n (nanometer), YOLOv8s (small), YOLOv8m (medium), YOLOv8l (large), and YOLOv8x (extra-large). YOLOv8 supports a variety of vision tasks, such as object detection, segmentation, pose estimation, tracking, and classification. It is a further improvement on YOLOv4 and YOLOv5, and compared to YOLOv5, YOLOv8 has made improvements in the following aspects:

[0054] 1. Model Architecture: A classic deep residual network, Darknet-53, is used as the backbone, consisting of 53 convolutional layers. These layers employ kernels of different scales to capture multi-scale features of the target. YOLOv5, on the other hand, uses CSPDarknet53. Darknet-53 is more complex and deeper than CSPDarknet53, with more parameters and greater potential representation capabilities. YOLOv8 uses a similar backbone to YOLOv5, but with some modifications to CSPlayer, now called the C2f module. The C2f module (with a cross-stage bottleneck of two convolutions) combines high-level features with contextual information to improve detection accuracy.

[0055] 2. Feature Pyramid: YOLOv8 introduces the FPN (Feature Pyramid Network) structure, using multiple feature maps of different scales for object detection. This enables YOLOv8 to detect targets of different sizes and scales, and improves the accuracy of small object detection.

[0056] 3. Detection Levels: YOLOv8 uses three different scale detection levels to detect targets, referred to as large, medium, and small. This multi-level detection allows the algorithm to better handle cross-scale features of targets, improving detection accuracy and stability. Simultaneously, YOLOv8 also utilizes a multi-scale feature fusion strategy, fusing features extracted from different scales to better predict targets of different sizes.

[0057] 4. Data Augmentation: YOLOv8 introduces more data augmentation techniques, such as CutMix and RandomErasing, which further improve the robustness and generalization ability of the model.

[0058] 5. Inference efficiency: YOLOv8 improves detection efficiency by performing dense sampling on the input image, inputting multiple small images into the model for detection, and then merging the detection results.

[0059] In YOLOv8, self-attention enhances the network's contextual understanding of targets; the CSPDarknet53 structure reduces computational complexity by dividing large feature maps into two smaller parts; and DropBlock regularization reduces redundant information in feature maps, improving model robustness. Simultaneously, YOLOv8 employs multi-scale training and data augmentation techniques during training. By training with input images at different scales, the model can better adapt to targets of varying sizes. Data augmentation generates transformed images, increasing data diversity and enhancing the model's generalization ability. In summary, YOLOv8 improves upon the original YOLO series algorithms by combining multi-scale feature fusion, self-attention, the CSPDarknet53 structure, DropBlock regularization, multi-scale training, and data augmentation, thereby increasing the accuracy and efficiency of target detection. The YOLOv8n network structure is shown below. Figure 3 As shown.

[0060] (iv) Model Improvement

[0061] For small target detection, YOLOv8n still suffers from issues such as false positives, missed detections, and relatively low detection accuracy. To address this, a new UAV target detection algorithm, GCE-YOLO, is proposed. GCE-YOLO significantly improves detection accuracy while reducing false positives and missed detections, without introducing unnecessary parameters, maintaining a balance between algorithm performance and lightweight design, and enhancing model robustness. GCE-YOLO optimizes the original YOLOv8n architecture by adding a P2 small target detection layer, incorporating SENet (a representative model of channel attention mechanisms), and integrating the EfficiCIoU Loss function. This combination of improvements enhances small target detection performance. The improved YOLOv8n architecture diagram is shown below. Figure 4 As shown.

[0062] 1. P2 Small Target Detection Layer

[0063] Small object detection has always been a challenging task in object detection algorithms. The fundamental reason for the unsatisfactory performance of small object detection lies in the lack of rich semantic and spatial information in the feature maps. Generally, shallow network feature maps possess rich spatial information, which is crucial for small object detection, while deep network feature maps possess rich semantic information, beneficial for object classification and large object detection. As the network feeds forward, the scale of the feature maps becomes smaller and smaller, resulting in more limited extracted information. Therefore, a new method is proposed: adding a small object detection layer to achieve the concatenation and fusion of shallow and deep feature maps. The original YOLOv8n output feature map sizes were 20×20, 40×40, and 80×80. Given that pedestrians are small targets in drone aerial photography, the three detection heads did not perform well in detecting pedestrians. Therefore, we added a 160×160 detection head, as shown below. Figure 5 As shown in the pink box, it can extract more specific and detailed information, and the added detection head is very suitable for detecting small targets. The four detection heads are responsible for detecting objects of different sizes, which more evenly optimizes this object size and provides richer fusion information for the detection of small targets, thereby improving the performance of small target detection.

[0064] 2. SE attention mechanism

[0065] Attention mechanisms were introduced into computer vision to mimic this aspect of the human visual system. This attention mechanism can be considered as a dynamic weight adjustment process based on the features of the input image. Attention mechanisms play a crucial role in many visual tasks. Classic network architectures such as Inception and ResNet demonstrate this; the Inception architecture shows that networks can achieve competitive accuracy by embedding multi-scale processes within their modules. In deep neural networks, different channels in different feature maps typically represent different objects. Channel attention adaptively recalibrates the weights of each channel; this process can be viewed as object selection, determining what to focus on. A key element of SENet is a squeeze and activation (SE) block, which collects global information, captures channel relationships, and improves representational power. Both squeeze and activation occur before summation using unit branches.

[0066] The SE module consists of two parts: a squeezing module and an activation module, which work together. The squeezing module collects global spatial information using global average pooling and then learns features along the channel dimension, representing the importance of each channel. The activation module captures channel relationships and outputs an attention vector, which is achieved by using fully connected layers and non-linear layers (ReLU and sigmoid). The network can recalibrate features, learning to selectively emphasize informative features and suppress less important ones using global information.

[0067] Squeezing is generally achieved by multiplying each channel of the feature as input with the corresponding element in the attention vector. Typically, the squeezing activation block Fse (with parameter θ) with X as input and Y as output can be represented as:

[0068] s=Fse(X,θ)=σ(W2δ(W1GAP(X))) (1)

[0069] Y = sX (2)

[0070] Channel attention is an attention mechanism used in computer vision tasks. It is used to enhance the expressive power of neural networks when processing images or feature maps. In computer vision, input images or feature maps contain many channels, each capturing different feature information. However, not all channels are useful for the task; some channels may contain noisy or redundant information, while others contain more important features.

[0071] By using Channel Attention, the importance of each channel can be dynamically weighted, allowing the network to better focus on useful channels. The network can automatically learn the importance of each channel, boosting useful features and suppressing useless ones. Channel attention can be inserted into different layers of the neural network, thus helping the network to extract features more accurately and effectively at different levels.

[0072] Typically, the calculation process for Channel Attention includes the following steps:

[0073] (1). Calculate the global average value for each channel to obtain a vector with channel dimensions.

[0074] (2). Transform the vector using a fully connected layer and perform a non-linear mapping on it using an activation function.

[0075] (3) The fully connected layer is used again to transform the mapped vector to obtain the final channel weights.

[0076] (4) The channel weights are summed with the original features to obtain the weighted features.

[0077] In this way, Channel Attention can help neural networks more adaptively focus on important channel features during the learning process, thereby improving the model's performance and generalization ability.

[0078] In the compression part, the input element feature map has dimensions of H x W x C, where H, W, and C represent height, width, and number of channels, respectively. The compression part compresses the dimension from H x W x C to 1 x 1 x C, that is, compresses H x W into 1 x 1 dimension. This process is achieved by global average pooling.

[0079] In the activation part, the 1x1xC dimension obtained from the compression part needs to be integrated into the fully connected layer to predict the importance of each channel, and then the corresponding channels are activated in the preceding feature map. A simple gating mechanism and sigmoid activation function are used. The SE block is a lightweight gating mechanism specifically designed to model the relationships between channels in a computationally efficient way, aiming to improve the representation of the basic modules of the entire network.

[0080] 3. Improved loss function

[0081] Directly improving the network part of object detection can lead to decreased detection accuracy and increased model complexity. Therefore, many researchers have considered improving the non-network parts of the detection algorithm without increasing network complexity. Object detection can generally be divided into two parts: localization and detection. The accuracy of localization is mainly dominated by the regression loss function. Therefore, many new regression loss functions have been proposed. YOLOv8n uses CIoU and DFL loss functions for bounding box loss and binary cross-entropy for classification loss. These losses improve object detection performance, especially when dealing with smaller objects.

[0082] By selecting appropriate positive and negative samples, IoU (Interference of the Uniform) becomes the most popular metric in bounding box regression, measuring the similarity between the predicted and ground truth bounding boxes. To further refine the IoU metric, IoU loss functions have been proposed. However, the IoU loss function becomes ineffective when the predicted bounding box and the IoU loss function do not overlap. To address these issues, various evaluation systems have been derived based on IoU, improving upon the shortcomings of the original IoU loss function from different perspectives and significantly enhancing its robustness. The most representative methods are generalized cross-joint, DIoU, and GIoU loss functions, which have played a fundamental role in the significant progress of object detection, but still have considerable room for optimization.

[0083] Among the methods described above, CloU is currently the best-performing boundary regression loss function, considering three important geometric factors: overlap region, center point distance, and aspect ratio. CloU uses loU, Euclidean distance, corresponding aspect ratio, and angle to measure the overlap region between the target and the ground truth bounding box.

[0084]

[0085]

[0086]

[0087] During regression, the width and height of the predicted bounding box cannot be increased or decreased simultaneously because it doesn't reflect the true width-to-height difference and its confidence level. Therefore, once it converges to the line-to-line ratio between the width and height of the predicted and ground truth frames, it can sometimes prevent the model from effectively optimizing similarity. The loss function EIoU Loss, based on CIoU Loss, decomposes the aspect ratio influence factor, calculates the width and height of the predicted and ground truth frames, and addresses the CIoU Loss problem.

[0088] When there are long edges, only EIoU Loss calculation slows down and fails to converge early. To better handle class imbalance and object localization problems, this invention employs a novel enhanced loss function, EfficiCIoU Loss, which can increase the adjustment of predicted frames and accelerate the frame regression rate. Combining CIoU and EIoU loss functions is fundamental to EfficiCIoU Loss. The aspect ratio of the predicted frame is first changed by CIoU until it converges to a suitable range. Then, each edge is carefully adjusted by EfficiCIoU Loss until it converges to the correct value. Experiments show that mAP@0.5 remains essentially unchanged, while mAP@0.5:0.95 shows a significant improvement.

[0089] EfficiCIoU Loss is calculated using the formula:

[0090]

[0091] EfficiCIoU Loss aims to address some of the problems of traditional object detection loss functions (such as IoU Loss and GIoU Loss), such as non-convergence or instability during optimization, and insufficient sensitivity to objects with large overlap. The core idea of ​​EfficiCIoU Loss is to more accurately measure the degree of overlap between bounding boxes by calculating the complete IoU (Intersection over Union) as part of the loss function. When calculating IoU, it introduces factors such as cell partitioning and center distance to comprehensively consider the position, size, and overlap of the bounding boxes.

[0092] Specifically, the calculation process of EfficiCIoU Loss includes the following steps:

[0093] 1. First, based on the predicted bounding boxes and the true bounding boxes, calculate their intersection and union regions to obtain the traditional IoU.

[0094] 2. Then, calculate the center point distance of the bounding box, that is, the Euclidean distance between the center point of the predicted box and the center point of the ground truth box.

[0095] 3. Next, calculate the diagonal distance between the two bounding boxes, which is the sum of the diagonal lengths of the predicted box and the ground truth box.

[0096] 4. Finally, the above three parts are combined using the EfficiCIoU formula to obtain the EfficiCIoU value. The EfficiCIoU Loss uses 1-EfficiCIoU as the loss and is used to optimize the object detection model.

[0097] (V) Experimental Environment and Configuration

[0098] The experimental platform was a 64-bit Windows 10 operating system. The experiment was based on the deep learning framework PyTorch-GPU 2.0.1. The GPU host was equipped with an NVIDIA GeForce RTX 2080Ti graphics card with 22GB of video memory. The development language was Python 3.9, and the CUDA version was 11.8. The specific environment configuration parameters for the experiment are shown in Table 1 below. Parameters not provided in this invention use the default parameters from the official YOLOv8n documentation.

[0099] Table 1 Experimental Parameter Configuration Table

[0100] Optimizer SGD Batch_Size 64 Epochs 300 Imges_Size 640 Learningrate 0.01 Amp true

[0101] The model was trained based on the original YOLOv8n model, with a batch size of 64, 300 epochs, and a learning rate of 0.01.

[0102] (vi) Evaluation Indicators

[0103] To verify the detection capability of the GCE-YOLO algorithm, this invention conducted ablation experiments and comparative experiments to explore the impact of the proposed method on the performance of the improved YOLOv8n model.

[0104] The experiments in this invention measure model performance using four evaluation criteria: mean accuracy (mAP), recall, model parameters, and computational power (GFLOPs).

[0105] Accuracy measures the proportion of correctly predicted samples out of the total number of samples. Accuracy evaluates the overall classification performance of the model. Recall measures the proportion of actual positive cases correctly identified by the model. Recall evaluates the model's recall rate for positive examples.

[0106]

[0107]

[0108] A true positive (TP) is a positive sample correctly predicted by the model; a true negative (TN) is a negative sample correctly predicted by the model; a false positive (FP) is a positive sample incorrectly predicted by the model; and a false negative (FN) is a negative sample incorrectly predicted by the model.

[0109] AP represents the average precision rate of a single class of labels, and its formula is as follows:

[0110]

[0111] mAP is the sum of the average precision of all labels divided by the total number of classes. A higher AP value indicates a higher average accuracy of the model. The formula is as follows:

[0112]

[0113] In the above formula, N represents the number of categories in the dataset.

[0114] The mAP metric solves this problem by calculating the mean accuracy (AP) for each class separately and then averaging these APs across all classes; hence, it is also called average mean precision. This approach ensures that the model's performance is evaluated for each class separately, thus providing a more comprehensive assessment of the model's overall performance.

[0115] (VII) Results and Analysis

[0116] The experimental results were compared with the YOLOv8n baseline. First, the backbone network was improved by replacing it with FasterNeXt. The experimental results are shown in Table 2. mAP@0.5% increased by 0.1, and mAP@0.5:0.95% decreased by 0.2. Then, based on the FasterNeXt backbone network, convolution PConv and PWConv were introduced respectively. The results showed that mAP@0.5% increased by 0.1 and 0.2 respectively, and mAP@0.5:0.95% decreased by 0.2. The number of parameters and floating-point operations decreased slightly. Subsequently, dynamic convolution ODConv was introduced. After experiments, mAP@0.5% increased by 0.2 and mAP@0.5:0.95% decreased by 0.2. Finally, ELAN, an aggregation network design, was introduced to improve the detection performance of the algorithm. It significantly improved the performance, with mAP@0.5 increasing by 0.9% and mAP@0.5:0.95 increasing by 1.9%. However, the number of parameters increased dramatically, becoming 2.27 times the original, and the number of floating-point operations also increased significantly, becoming 2.37 times the original. Considering the need for a lightweight model, this improvement method of the backbone network was not adopted.

[0117] Table 2 Comparison of Model Performance After Backbone Network Improvement

[0118] Experimental content backbone network mAP@0.5% mAP@0.5:0.95% Parameter quantity / M Number of floating-point operations / G YOLOv8n (baseline) C2f 96.2 82 3.0 8.2 YOLOv8n+FasterNeXt FasterNeXt 96.3 81.8 2.6 7.6 YOLOv8n+PConv FasterNeXt, PConv 96.4 81.8 2.6 7.5 YOLOv8n+PWConv FasterNeXt, PWConv 96.3 81.8 2.5 7.4 YOLOv8n+ODConv ODConv 96.4 81.8 3.0 7.6 YOLOv8n+ELANB ELANB 97.1 83.9 6.8 19.4

[0119] From the improvement of the backbone network, the model's performance was not significantly improved, and the better-performing improvement methods did not meet the model's requirement for lightweight design. To better capture features at multiple scales and semantic levels, this invention improved the neck network. Related experiments are shown in Table 3. An attention mechanism was added to the C2f module of the neck network. Adding the SimAm attention mechanism resulted in no change in mAP@0.5%, but a decrease of 0.3 in mAP@0.5:0.95%. Then, the SEAttention attention mechanism was added, resulting in an increase of 0.4 in both mAP@0.5% and mAP@0.5:0.95%, with minimal changes in the number of parameters and floating-point operations. Subsequently, dynamic convolution ODConv was introduced, resulting in no change in mAP@0.5%, but a decrease of 0.1 in mAP@0.5:0.95%. Finally, a new module design, ConXBv2, was introduced, combining MAE with Conv. Experiments showed no change in either mAP@0.5% or mAP@0.5:0.95%. In order to obtain more detailed features and information about smaller targets, experiments were conducted using YOLOv8n-P2. In the Head part, a small target detection head was added, which significantly improved the performance, with mAP@0.5 increasing by 1.4%.

[0120] Table 3. Performance Comparison of the Improved Model After Multi-Scale Fusion Stage

[0121] Experimental content Neck network mAP@0.5% mAP@0.5:0.95% Parameter quantity / M Number of floating-point operations / G YOLOv8n (baseline) C2f 96.2 82 3.0 8.2 YOLOv8n+C2f_SimAm C2f_SimAm 96.2 82 3.0 8.1 YOLOv8n+P2* C2f 97.6 83.9 2.9 12.7 YOLOv8n+ODConv ODConv 96.2 81.9 3.0 8.0 YOLOv8n+ConXBv2 ConXBv2 96.2 82 3.0 7.9 YOLOv8n+SEAttention* SEAttention 96.6 82.4 3.0 8.3

[0122] Note: "*" indicates the method used in this invention.

[0123] During the experiments, this invention was compared with several default loss functions of YOLOv8. Each experiment underwent 300 iterations, and the number of parameters remained unchanged at 3.0 / M, as shown in Table 4 below. The mAP@0.5% and mAP@0.5:0.95% of XIoU were the same as those of CIoU; the mAP@0.5% of DIoU decreased by 0.1, while mAP@0.5:0.95% remained unchanged; the mAP@0.5% of GIoU increased by 0.1, and mAP@0.5:0.95% increased by 0.8, showing a significant improvement and good results. Therefore, this loss function was integrated with two other improvements to the model. After the experiments, it was found that mAP@0.5% was 98 and mAP@0.5:0.95% was 84.2, both lower than the detection accuracy of the GCE-YOLO model; the mAP@0.5% of Wise-IoU was 0.5%. The accuracy improved significantly, with mAP@0.5 increasing by 0.6% and mAP@0.5:0.95% increasing by 1.2%. To verify the performance of Wise-IoU, this invention integrates this loss function with improvements from other modules. Experiments showed that mAP@0.5% was 98.2 and mAP@0.5:0.95% was 84.6, equal to or lower than the detection accuracy of the GCE-YOLO model. Although the introduction of the EfficiCIoU loss function did not show a significant performance improvement, its integration with other improvements yielded excellent results and a noticeable improvement. Therefore, after comprehensive consideration, the EfficiCIoU loss function was chosen. Compared to traditional IoU and GIoU losses, EfficiCIoU Loss performs better in handling target overlap and size variations, thus improving the performance and stability of the detection model.

[0124] Table 4 Comparison of different loss functions

[0125] Experimental content mAP@0.5% mAP@0.5:0.95% Number of floating-point operations / G CIoU 96.2 82 8.2 XIoU 96.2 82 8.1 DIoU 96.1 82 8.1 GIoU 96.3 82.8 8.1 Wise-IoU 96.7 83.1 8.1 EfficiCIoU* 96.5 82.1 8.1

[0126] ablation experiment

[0127] As shown in Table 5, the ablation comparison experiments reveal that adding the SEAttention mechanism increased mAP@0.5% and mAP@0.5:0.95% by 0.4, while the number of parameters and floating-point operations remained almost unchanged. Adding the P2 small object detection layer resulted in significant increases: mAP@0.5% increased by 1.4, and mAP@0.5:0.95% increased by 1.9, with the number of parameters decreasing instead of increasing. Incorporating the EfficiCIoU loss function improved mAP@0.5% by 0.5 and mAP@0.5:0.95% by 0.1. Further integration of SEAttention and the small object detection layer improved mAP@0.5% by 1.6 and mAP@0.5:0.95% by 2.2; simultaneously, adding both the EfficiCIoU loss function and the small object detection layer improved mAP@0.5% by 1.5 and mAP@0.5:0.95% by 1.9. After adding SEAttention and Efficacy loss function, mAP@0.5% improved by 0.8, and mAP@0.5:0.95% increased by 0.7. Experiments show that the improved model did not increase the number of parameters, but the number of floating-point operations increased by 4.5. In conclusion, this algorithm is suitable for detecting small targets in UAV aerial photography.

[0128] Table 5 Ablation Comparison Experiment

[0129]

[0130]

[0131] YOLO series detection algorithm experiment

[0132] As shown in Table 6, the GCE-YOLO model improves upon YOLOv5n, YOLOv5s, and YOLOv7-tiny by 4.0%, 1.2%, and 5.1% respectively in terms of mAP@0.5:0.95%. Although the improvement of the GCE-YOLO model compared to YOLOv5s is not significant, especially in terms of mAP@0.5%, the number of parameters in GCE-YOLO is only 3.0 (a decrease of 4 compared to YOLOv5s), and the number of floating-point operations is also reduced by 3.2, far lower than YOLOv5s. Compared to YOLOv7, the GCE-YOLO model shows a 0.1 decrease in mAP@0.5%, and the improvement in mAP@0.5:0.95% is also not significant. However, YOLOv7 has 10 times the number of parameters and approximately 9 times the number of floating-point operations of the GCE-YOLO model. To balance performance and lightweight design, YOLOv7 was not adopted. Compared to YOLOv8s, the GCE-YOLO model improved mAP@0.5% by 0.7 and decreased mAP@0.5:0.95% by 0.3. However, YOLOv8s has more than three times the number of parameters and more than twice the number of floating-point operations. Considering the balance between performance and lightweight design, YOLOv8s was not adopted.

[0133] Table 6 Comparison of Some Object Detection Algorithms

[0134]

[0135] As can be seen from the above, the GCE-YOLO model outperforms the seven models listed in the table, achieving more efficient drone target detection. Looking at the first column of Table 6, which shows the accuracy for small targets, using pedestrians as an example, we can see that the GCE-YOLO model significantly improves performance on small targets compared to other models, demonstrating superior performance. Compared to YOLOv5n, YOLOv8n, and YOLOv7-tiny, it improves mAP@0.5 by 11.3%, 10.1%, and 12.2% respectively on small targets. In conclusion, GCE-YOLO performs well in small target detection under drone aerial photography, meeting the higher accuracy requirements for small target scenarios.

[0136] Comparison chart of heat map and detection results

[0137] To further explore the improvement effect of GCE-YOLO, the detection performance of three aspects—the backbone network P4, SPPF, and the small target detection head—was visualized using heatmaps generated by the Grad-CAM method. In fast spatial pyramid pooling, GCE-YOLO's heatmaps show a more concentrated focus on feature regions and dense regions, thanks to its channel attention mechanism's ability to better extract useful features. Compared to YOLOv8n, where the small target features in the backbone network P4 and P4 detection head heatmaps are not specific and are rather vague, focusing only on certain partial features and failing to capture much of the small target's feature information, the GCE-YOLO model's backbone network P4 and P2 small target detection heads pay more specific attention to the details of small target features, solving the problems of missed detections and false detections, and exhibiting higher clarity, with feature regions almost comprehensively covering all small targets. Whether for medium-sized or small targets, GCE-YOLO outperforms YOLOv8n. The results clearly demonstrate the improvements of GCE-YOLO in three aspects: backbone network, fast spatial pyramid pooling, and small target detection head. They verify its significant advantages in urban road scenarios and provide an efficient and accurate solution for pedestrian detection in densely populated areas and for small targets.

[0138] Figure 6 The images show a comparison of detection results: a) is the prediction image of YOLOv8n, and b) is the prediction image of the improved model of this invention. It can be seen that the detection accuracy for pedestrians and vehicles is significantly improved, while the improvement in vehicle detection is not substantial, remaining essentially unchanged. However, for small targets such as pedestrians and humans, the detection accuracy is significantly improved, with no missed or false detections. From the above, it is clear that the model of this invention significantly improves the detection accuracy for small targets. Furthermore, since drone aerial photography often captures small objects, and the detection performance is significantly better than YOLOv8n even in low-light conditions, this improvement is attributed to the small target detection layer focusing on highlighting small targets, the channel attention mechanism extracting useful channels, suppressing some interference or useless information, and the loss function being adjusted by adding prediction frames and accelerating the frame regression rate. Through these measures, the GCE-YOLO model demonstrates excellent performance and advantages in handling complex scenes with dense data and small targets in drone aerial photography.

[0139] Conclusion: This invention proposes a small target detection algorithm for UAV aerial photography based on the improved YOLOv8n: GCE-YOLO. Addressing the issue of missed and false detections of small pedestrians and vehicles in densely populated scenes, such as the Coklu-Deneme dataset, traditional target detection algorithms suffer from this problem. Considering the balance between model lightweighting and performance, the following solutions are proposed: First, a P2 small target detection layer is added, allowing the network to focus more on small target detection and improving detection performance. Second, a channel attention mechanism, typical of models like SENet, is introduced. This first compresses the feature map space and then learns the importance of each channel along the channel dimension. Third, Efficiency Loss is used as the bounding box loss function, enhancing the adjustment capability of predicted frames, accelerating frame return rate, speeding up network model convergence, and reducing missed and false detections of small targets in densely populated UAV aerial photography scenarios.

[0140] Experimental results show that compared to YOLOv8n, GCE-YOLO, with the same number of parameters and a slight increase in the number of floating-point operations, improves mAP@.5 and mAP@0.5-0.95 by 2.0% and 2.7%, respectively. It demonstrates a significant improvement in detection performance for small targets. Taking human pedestrians as an example, compared to YOLOv8n, mAP@0.5 is improved by 7.6%, and mAP@0.5-0.95 by 11.3%. The single-stage detector GCE-YOLO improves the model's detection accuracy while maintaining a lightweight design. This makes it a promising algorithm for small target detection in UAV aerial photography.

[0141] This invention is an optimization based on the latest YOLOv8n model. YOLOv8 is the latest version in the YOLO (You Only Look Once) algorithm family. Compared to previous versions of YOLO, YOLOv8 has higher accuracy and faster inference speed, and it also follows the research ideas of previous generations. GCE-YOLO optimizes the original YOLOv8n structure and improves the detection performance for small targets by refining the model.

[0142] The above description of the disclosed embodiments is presented in a progressive manner to enable those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting small targets captured by drone aerial photography, characterized in that, Includes the following steps: S1. Acquire drone aerial photography data; S2. Preprocess the acquired drone aerial photography data to obtain preprocessed data; S3. Divide the preprocessed data into training set, test set and validation set; S4. Improve the YOLOv8n model to obtain the improved YOLOv8n model; S5. Iteratively train the improved YOLOv8n model using the training set, and then test the improved YOLOv8n model on the test set using the trained model. S6. The effectiveness of the improved YOLOv8n model is verified using the validation set, and the recognition results of the improved YOLOv8n model for small target detection in UAV aerial data are obtained.

2. The method for detecting small targets under drone aerial photography according to claim 1, characterized in that, S1 retrieves drone aerial data from the Coklu deneme dataset.

3. The method for detecting small targets under drone aerial photography according to claim 1, characterized in that, In S2, a synthetic fog enhancement algorithm is used to preprocess the acquired UAV aerial photography data.

4. The method for detecting small targets under drone aerial photography according to claim 1, characterized in that, In S3, the dataset is divided into training, validation, and test sets according to the proportions of 87%, 10%, and 3%.

5. The method for detecting small targets under drone aerial photography according to claim 1, characterized in that, The specific improvements to the YOLOv8n model in S4 are as follows: The GCE-YOLO algorithm was introduced to improve the YOLOv8n structure; a P2 small object detection layer was added to improve the splicing and fusion of shallow and deep feature maps; a channel attention mechanism was introduced to recalibrate the weights of each channel in the SENet model; and the EfficiCIoU Loss function was introduced to measure the degree of overlap between target boxes.

6. The method for detecting small targets under drone aerial photography according to claim 1, characterized in that, In S6, comparative experiments, ablation experiments, and YOLO series object detection algorithms were used to verify the effectiveness of the improved YOLOv8n model.

Images