Novel aerial image detection method

Abstract

The invention discloses a novel aerial image detection method, which covers three distinctive structure innovation modules, namely AKConv, C2f-DRAC and BAFPN, and provides a solution for many challenges in aerial image detection from three key dimensions of feature extraction, feature fusion and post-processing. Specifically, AKConv is introduced, a C2fDRAC structure is designed, and the problem of complex background interference is solved; a BAFPN structure is designed, the problems of complex background interference and small targets are solved, and the detection performance of the model is remarkably improved; the CIoU-Soft-NMS is introduced, and the advantages of Soft-NMS and CIoU are combined, so that the problems of loss of adjacent prediction frames and IoU calculation of overlapped bounding frames are effectively solved, and the problem of missing detection of dense targets is solved; an ADD-YOLO model is provided and is superior to other current mainstream detection models in aerial image detection.

Description

Technical Field

[0001] The present invention relates to the technical field of aerial image detection methods, and in particular to a novel aerial image detection method. Background Art

[0002] UAVs have certain limitations in their viewing angles during aerial photography. Since UAVs mainly take pictures of ground targets from a bird's-eye view, most of the images obtained appear as small targets. These small targets have low pixel values ​​and relatively simple features, and there is a lack of effective information that can be used, which undoubtedly adds many difficulties to target detection.

[0003] At present, deep learning-based detection technologies are mainly divided into two categories:

[0004] The first is a two-stage target detection algorithm based on regions. The two-stage model is a target detection algorithm that divides the detection task into two stages: selecting candidate regions and classifying the selected candidate regions, and finally outputs the target detection result. Therefore, target detection based on the two-stage method is also called the candidate region-based method.

[0005] In 2014, Girshick et al. proposed an R-CNN algorithm that combined RegionProposal and CNN, which greatly improved the detection performance compared to Overfeat. HeK et al. added a spatial pyramid pooling layer (SSP) between the convolutional layer and the fully connected layer of the convolutional neural network to solve the limitation of the fixed input size of the image in CNN. Compared with R-CNN, it avoids repeated operations when extracting features from images, and the detection performance is further improved.

[0006] In 2015, Girshick R et al. proposed Fast R-CNN based on R-CNN and combined the advantages of SPP-Net. Compared with R-CNN, the training and testing time was significantly shortened. After that, Faster R-CNN proposed by Ren S et al. adopted the Region Proposal Network (RPN), which greatly improved the detection speed of the algorithm. In 2017, He K et al. solved the problem of position deviation of feature maps by introducing the region of interest aggregation layer (RoIAlign) and proposed Mask R-CNN with more accurate detection effect.

[0007] The second is the one-stage target detection algorithm based on regression. The one-stage target detection method can output the detection result only through a single neural network after one operation, realizing the end-to-end conversion of the detection process. The detection speed is greatly improved compared with the two-stage target detection method, which can meet the requirements of real-time detection. Typical algorithms include SSD and YOLO series.

[0008] In 2016, Liu W et al. proposed the SSD (Single Shot Multi Box Detector) network model, which combines the advantages of YOLO and Faster R-CNN. Unlike YOLO, SSD uses a feature pyramid network structure, which can perform Softmax classification and position regression on multiple feature maps of different scales at the same time; in terms of pre-selection boxes, SSD refers to the Anchors mechanism of Faster R-CNN and proposes Priorbox, which can generate corresponding pre-selection boxes according to the size of feature maps.

[0009] The YOLO algorithm has been continuously developed since it was proposed in 2016. YOLOv1 uses a structure similar to GoogleNet, with 24 convolutional layers and 2 fully connected layers. Due to multiple downsampling, the feature map resolution is low and small target detection is weak. YOLOv2 uses the Darknet-19 backbone network, removes the fully connected layer, uses K-means clustering prior boxes and high-resolution image classifiers to improve speed and accuracy. YOLOv3 in 2018 uses the Darknet-53 feature extractor, replaces it with a logistic classifier, and uses 3 different scale feature maps to predict bounding boxes, highlighting the advantages of small target detection.

[0010] In 2020, YOLOv4 added practical skills to improve speed and accuracy, and YOLOv5 appeared in the form of a model family with multiple variants. It used Pytorch for the first time, which made it easier to deploy and improved reasoning and detection accuracy without losing accuracy.

[0011] In 2022, YOLOv7 has faster and more accurate performance than ever before, and its E-ELAN network and scaling strategy optimize the structure. YOLOv8 uses the C2f module and decoupling head structure, adopts new positive and negative sample matching and loss functions, and improves data enhancement. In 2024, YOLOv10 has multiple variants with accuracy scores between 38.5% and 54.4%; YOLOv11 adds C3K2, SPFF, and C2PSA blocks, which are used to optimize feature extraction, capture object attributes, and improve detection accuracy in complex scenes, respectively.

[0012] In conventional target detection algorithms, the single-layer feature map representation capability of convolutional neural networks is insufficient, making it difficult to properly handle complex small target conditions in images, ultimately resulting in unsatisfactory detection results. Given the current situation of poor small target detection performance in drone aerial images, it is necessary to re-plan and optimize the network structure to improve the network's feature extraction capabilities and enhance the algorithm's stability and reliability, in order to address challenges such as complex background interference, small-sized targets, and dense targets in aerial image detection.

[0013] To this end, the present invention proposes an innovative improved algorithm based on YOLOv8s

[0014] —ADD-YOLO, used in aerial image detection methods, solves many problems in aerial image detection. Summary of the invention

[0015] A new type of aerial image detection method proposed in the present invention aims to solve many problems in aerial image detection, and to develop an innovative improved algorithm based on YOLOv8s - ADD-YOLO, which is used in the aerial image detection method.

[0016] In order to achieve the above object, the present invention adopts the following technical solutions:

[0017] A novel aerial image detection method, including three innovative modules: AKConv, C2f-DRAC and BAFPN, and proposes three steps: feature extraction, feature fusion and post-processing;

[0018] S1. Feature extraction

[0019] The C2f_DRAC structure is designed to replace the backbone network C2f. This structure cleverly combines AKConv and CBAM and uses convolution operations with different dilation rates.

[0020] By replacing the original Conv and C2f in Backbone with AKConv and C2f_DRAC respectively, the feature extraction efficiency of the algorithm is significantly enhanced, thereby greatly improving the accuracy and stability of the algorithm in detecting targets under complex background conditions;

[0021] S2. Feature Fusion

[0022] The BAFPN structure is designed based on BiFPN, and an additional small target detection layer is added. The BAFPN structure successfully improves the feature fusion effect of the algorithm by using cross-layer links and feature weighted fusion technology;

[0023] S3, post-processing

[0024] Replacing the original NMS with CIoU-Soft-NMS ensures the reliability and accuracy of the algorithm in dense target detection scenarios, reduces the missed detection rate, and makes the detection results more complete and accurate.

[0025] As a further step in the method of the present invention, in the feature extraction stage, the ordinary Conv is replaced by AKConv, specifically, the variable kernel convolution AKConv is introduced. AKConv uses a new coordinate generation algorithm and cleverly introduces an offset to accurately adjust the sampling position of the irregular convolution kernel.

[0026] As a further step in the method of the present invention, the AKConv branch generates initial sampling coordinates by a new coordinate generation algorithm: first define a regular sampling grid P based on the central cell o (such as 3×3 or 5×5), and then construct an irregular sampling grid P with no center point and the upper left corner as the starting point n The two are spliced ​​into an initial sampling shape, which can flexibly adjust the position of the convolution kernel to adapt to local feature differences and deal with target occlusion and complex background problems in UAV target detection. In addition, the 5×5 sample network can improve the feature extraction accuracy and effectiveness by designing different initial sampling shapes.

[0027] As a further step in the method of the present invention, the main part of AKConv: first, a two-dimensional convolution is performed on the input feature map and the offset is obtained by back propagation and gradient descent, and an offset feature map with the same size as the input feature map and containing a 2d vector of offsets in the x and y directions is constructed; it is applied to the initial sampling shape to dynamically adjust it to form an adaptive sampling shape, and the shape is optimized according to the dynamic local features of the image to improve the efficiency and accuracy of feature extraction.

[0028] As a further step in the method of the present invention, a deformable sampling grid is used to resample the input feature map, and features are further extracted through shaping and convolution Conv. Then, a standardized Norm operation is performed for stable training, and finally, an activation function SiLU is used to add nonlinear factors to generate the final output Output, thereby providing a high-quality feature data foundation for subsequent image processing and promoting efficient and accurate operation of the image processing process.

[0029] As a further feature of the method of the present invention, in the feature fusion stage, BAFPN extracts features and adjusts the number of channels by performing AKConv convolution processing on feature maps of different scales output from Backbone to ensure that the number of channels is consistent during subsequent feature fusion.

[0030] As a further step in the method of the present invention, in the top-down path, upsampling is performed layer by layer and weighted fusion is performed to obtain a fused new feature map, and C2f-DRAC is used to integrate multi-scale information to improve the richness and accuracy of the feature map, thereby further improving the effect of feature fusion.

[0031] In DRAC, the first step is to use AKConv to perform a 3×3 convolution operation, which can halve the number of input channels. At the same time, by adjusting the sampling position of the irregular convolution kernel, it can adapt to the changes in local features at different positions, laying a solid foundation for subsequent feature extraction; the next step is to use 3×3 dilated convolution with dilation rates of 1, 3, and 5 to process the feature map after convolution.

[0032] As a further step in the method of the present invention, in the bottom-up path, information interaction of multiple paths is achieved through cross-layer jump connections, and after downsampling layer by layer and weighted fusion, a new fused feature map is obtained, and finally feature enhancement is completed through C2f to generate the final output.

[0033] As a further step in the method of the present invention, in the post-processing stage, the Soft-NMS algorithm formula is as follows:

[0034]

[0035] In formula (1), there are the following related definitions: represents the confidence of the i-th bounding box, is the threshold of IoU, M represents the current highest-scoring bounding box, is the i-th bounding box, and is the intersection-over-union ratio between M and ;

[0036] Soft-NMS uses a weight function to convert the degree of overlap into a real number between 0 and 1, calculates the weight based on IoU, and then performs a weighted summation on all bounding boxes.

[0037] As a further step in the method of the present invention, CIoU is used to replace IoU to enhance the performance of NMS, and it is named CIoU-Soft-NMS;

[0038] The CIoU calculation formula is as follows: In the above example, IoU represents the intersection over union of the predicted box and the true box; c represents the diagonal length of the minimum bounding rectangle that covers both the predicted box and the true box; b and b gt Represent the center points of the predicted box and the real box respectively; ρ(b,b gt ) represents b and b gt The Euclidean distance between

[0039] α represents the weight parameter that balances the impact of IoU and aspect ratio consistency, and its definition is shown in formula (3):

[0040] v represents a parameter that measures the difference in aspect ratio between the predicted box and the true box, and its definition is shown in formula (4):

[0041] w and h represent the width and height of the prediction box respectively; w gt and h gt Represent the width and height of the real box respectively;

[0042] The CIoU loss function is specifically expressed as formula (5):

[0043]

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The present invention introduces AKConv and designs the C2f_DRAC structure to solve the problem of complex background interference; designs the BAFPN structure to solve the problems of complex background interference and small targets, and significantly improves the detection performance of the model; introduces CIoU-Soft-NMS to combine the advantages of Soft-NMS and CIoU, effectively solves the IoU calculation problems of neighboring prediction box loss and overlapping bounding box, and solves the problem of missed detection of dense targets; proposes the ADD-YOLO model, which is superior to other current mainstream detection models in aerial image detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 AKConv structure diagram in the method of the present invention;

[0047] Figure 2 It is a structural diagram of C2f_DRAC in the method of the present invention;

[0048] Figure 3 It is a comparison diagram of the BAFPN structure and the BiFPN structure in the method of the present invention. DETAILED DESCRIPTION

[0049] The content of the present invention can be more easily understood by selecting the following detailed description of the preferred implementation method of the present invention and the embodiments included. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those of ordinary skill in the art to which the present invention belongs. When there is a conflict, the definition in this specification shall prevail.

[0050] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings and table data. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments.

[0051] Embodiment 1

[0052] A new aerial image detection method proposes an innovative and improved algorithm based on YOLOv8s, ADD-YOLO, for aerial image detection. It specifically covers three distinctive structural innovation modules, namely AKConv, C2f-DRAC and BAFPN. Starting from the three key dimensions of feature extraction, feature fusion and post-processing, it provides solutions to many challenges in aerial image detection.

[0053] In the feature extraction stage, AKConv is innovatively introduced. In the detection of aerial images, the shape and scale of the target are complex and varied. The introduction of AKConv enables the algorithm to adapt to various changes of the target in a more flexible way during the downsampling operation. At the same time, the C2f_DRAC structure is carefully designed. This structure cleverly combines AKConv and CBAM and uses convolution operations with different expansion rates.

[0054] In actual aerial photography scenes, the background conditions are often extremely complex. This design effectively enhances the algorithm's ability to capture multi-scale contextual information in complex background environments. By replacing the original Conv and C2f in Backbone with AKConv and C2f_DRAC respectively, the algorithm's feature extraction efficiency is significantly enhanced, thereby greatly improving the algorithm's detection accuracy and stability for targets under complex background conditions.

[0055] In the feature fusion stage, a BAFPN structure is specially designed, and an additional small target detection layer is added. In aerial images, small targets are not only tiny in size, but also often under complex background interference. The BAFPN structure uses cross-layer links and feature weighted fusion technology to successfully improve the feature fusion effect of the algorithm. In this way, the algorithm's detection performance for small targets under complex background interference has been greatly enhanced, and it can more accurately identify and locate small targets that are easily overlooked.

[0056] Finally, for the post-processing stage, the original NMS is replaced by CIoU-Soft-NMS. In aerial images, when facing dense target scenes, traditional NMS is prone to missed detection, while the use of CIoU-Soft-NMS effectively solves this problem, ensuring the reliability and accuracy of the algorithm in dense target detection scenarios, reducing the missed detection rate, and making the detection results more complete and accurate.

[0057] Embodiment 2

[0058] With respect to the three steps of feature extraction, feature fusion and post-processing in the first embodiment above, the following more specific content requirements are proposed;

[0059] 1. Replace ordinary Conv with AKConv

[0060] This invention innovatively introduces the variable kernel convolution (AKConv). AKConv uses a new coordinate generation algorithm and cleverly introduces an offset to accurately adjust the sampling position of the irregular convolution kernel, successfully overcoming the local limitation problem faced by traditional convolution. Not only that, AKConv also has extremely outstanding flexibility, and the number of convolution kernel parameters and the shape of the convolution kernel can be dynamically adjusted according to actual application requirements.

[0061] In actual application scenarios, especially when facing complex situations with variable targets, AKConv can flexibly adapt according to the specific features of the target, thereby significantly improving the accuracy and effectiveness of the model in feature extraction. Figure 1 .

[0062] The AKConv branch generates the initial sampling coordinates through a new coordinate generation algorithm: first define a regular sampling grid P based on the central cell o (such as 3×3 or 5×5), and then construct an irregular sampling grid P with no center point and the upper left corner as the starting point n The two are spliced ​​into an initial sampling shape, which can flexibly adjust the position of the convolution kernel to adapt to local feature differences and deal with target occlusion and complex background problems in UAV target detection. In addition, the 5×5 sample network can improve the feature extraction accuracy and effectiveness by designing different initial sampling shapes.

[0063] The main part of AKConv: First, perform a two-dimensional convolution on the input feature map and obtain the offset with the help of back propagation and gradient descent, and construct an offset feature map with the same size as the input feature map and containing the 2d vector of the x and y direction offset; apply it to the initial sampling shape to dynamically adjust it to form an adaptive sampling shape, optimize the shape according to the dynamic local features of the image, and improve the efficiency and accuracy of feature extraction;

[0064] The input feature map is resampled with a deformable sampling grid, and features are further extracted through shaping and convolution Conv. Then the standardized Norm operation is performed for stable training. Finally, the activation function SiLU is used to add nonlinear factors to generate the final output Output, providing a high-quality feature data foundation for subsequent image processing and promoting the efficient and accurate operation of the image processing process.

[0065] 2. Replace the backbone network C2f with C2f_DRAC

[0066] The C2f module of the backbone was optimized, and C2f_DRAC (Dilated Residual Fusion with AKConv and CBAM) was developed. C2f_DRAC combines AKConv and CBAM, and introduces convolution operations with different dilation rates. It can capture multi-scale context information, and combines the attention mechanism and adaptive convolution kernel to adjust the feature map, which enhances the accuracy of feature extraction and reduces false detection and missed detection. The structure of C2f_DRAC is as follows: Figure 2 shown.

[0067] In DRAC, the first step is to use AKConv to perform 3×3 convolution operations, which can halve the number of input channels. At the same time, by adjusting the sampling position of the irregular convolution kernel, it can adapt to the changes in local features at different positions, laying a solid foundation for subsequent feature extraction. The next step is to use 3×3 dilated convolutions with dilation rates of 1, 3, and 5 to process the feature map after convolution. With this dilated convolution with multiple dilation rates, contextual information of different scales can be effectively captured, so that the feature map is full of richer semantic information to calmly cope with complex and changing image content. After that, the output results of these dilated convolutions are spliced ​​in the channel dimension, and the spliced ​​results are input into the CBAM module. With the help of the unique attention mechanism of the CBAM module, the key features of the image are accurately highlighted, irrelevant or interference information is suppressed, and the effective information of the feature map is more prominent, thereby further enhancing the feature representation capability. Finally, 1×1 convolution is used to aggregate the previously acquired features to restore them to the original number of input channels, and the output is added to the input using residual connection. The design of residual connection can effectively enhance the efficiency of feature transfer, eliminate the gradient disappearance or gradient explosion in deep networks, ensure that the network can stably and efficiently learn image feature information, and thus improve the overall performance of DRAC structure in image processing and analysis tasks.

[0068] 3. Based on BIFPN combined with small target detection layer and proposed new neck structure

[0069] In this method, BAFPN (bidirectional adaptive feature pyramid network) is proposed based on BiFPN. Figure 3 As shown, Figure 3Comparison of the BAFPN structure (right) and the BiFPN structure (left). BAFPN extracts features and adjusts the number of channels by performing AKConv convolution processing on the feature maps P2, P3, P4, and P5 of different scales output from Backbone to ensure that the number of channels is consistent during subsequent feature fusion. In the top-down path, upsampling layer by layer and weighted fusion are performed to obtain a new fused feature map, and C2f-DRAC is used to integrate multi-scale information to improve the richness and accuracy of the feature map and further improve the effect of feature fusion; in the bottom-up path, cross-layer jump connections are used to realize information interaction of multiple paths, downsampling layer by layer and weighted fusion are performed to obtain a new fused feature map, and finally feature enhancement is completed through C2f to generate the final output.

[0070] First, BAFPN uses a bidirectional feature pyramid structure, with two paths, bottom-up and top-down, which retains more contextual information and enhances the efficiency of network feature information transmission; BAFPN uses cross-layer skip connections and deletes edge nodes with weak feature fusion capabilities, realizing cross-scale interaction of features at different levels, with stronger feature fusion capabilities and multi-level information interaction, effectively improving the model's ability to detect different targets. Secondly, BAFPN adds additional weights to each input feature, allowing the network to learn the importance of different input features and enhance the quality of feature fusion; in addition, after BAFPN introduces AKConv, the dynamic convolution kernel adjustment dynamically adjusts the shape and position of the convolution kernel according to the shape and size of the target, improving the accuracy and adaptability of feature extraction.

[0071] 4. Replace NMS with CIoU-Soft-NMS

[0072] Although traditional NMS is simple and effective, it may miss important targets, resulting in poor detection performance of the model for dense small targets. Soft-NMS can avoid this situation. The Soft-NMS algorithm formula is as follows:

[0073]

[0074] In formula (1), there are the following definitions: represents the confidence of the i-th bounding box, is the threshold of IoU, M represents the current highest-scoring bounding box, is the i-th bounding box, and is the intersection-over-union ratio between M and ; Soft-NMS uses a weight function to convert the degree of overlap into a real number between 0 and 1, and calculates the weight based on IoU, and then performs a weighted summation on all bounding boxes; in this way, Soft-NMS implements different degrees of suppression on bounding boxes according to the degree of overlap, rather than performing a complete binarization elimination operation as in the past; this method plays a positive role in dense target scenes, effectively solving the problem of easy loss of neighboring prediction boxes, and also reducing the amount of repeated calculations, thereby improving the operation speed.

[0075] Embodiment 3

[0076] Based on the content of the above second embodiment, the following content is also proposed:

[0077] Since Soft-NMS is implemented based on IoU, its inhibitory effect will be affected by the loss function. The original Soft-NMS uses the loss function based on the original Soft-NMS.

[0078] This loss function has many shortcomings: it only focuses on the overlapping area and cannot directly reflect the aspect ratio difference of the bounding box, which makes the detection effect unsatisfactory; when two bounding boxes do not intersect, the gradient disappears and the distance between the bounding boxes cannot be calculated; if there is an inclusion relationship between the bounding boxes, the improvement speed of IoU is relatively slow, which leads to a long time for model regression.

[0079] Compared with other IoU loss functions, CIoU has obvious advantages: it avoids the gradient vanishing problem that occurs when the IoU bounding boxes do not intersect; it optimizes the defect that GIoU does not consider the aspect ratio of the bounding box; on the basis of inheriting the advantages of DIoU, it also comprehensively considers the distance and aspect ratio to achieve comprehensive optimization; combined with the enhanced calculation function of EIoU, it further takes into account the aspect ratio of the bounding box, making the loss function more perfect; it improves the shape matching problem of SIoU, and considers the consistency of the center point distance and aspect ratio at the same time, which improves the accuracy and comprehensiveness of the regression.

[0080] In view of this, the method of the present invention also proposes an improved NMS method, which uses CIoU to replace IoU to enhance the performance of NMS, and named it CIoU-Soft-NMS;

[0081] The CIoU calculation formula is as follows: In the above example, IoU represents the intersection over union of the predicted box and the true box; c represents the diagonal length of the minimum bounding rectangle that covers both the predicted box and the true box; b and b gt Represent the center points of the predicted box and the real box respectively; ρ(b,b gt ) represents b and b gt The Euclidean distance between

[0082] α represents the weight parameter that balances the impact of IoU and aspect ratio consistency, and its definition is shown in formula (3):

[0083] v represents a parameter that measures the difference in aspect ratio between the predicted box and the true box, and its definition is shown in formula (4):

[0084] w and h represent the width and height of the prediction box respectively; w gt and h gt Represent the width and height of the real box respectively;

[0085] The CIoU loss function is specifically expressed as formula (5):

[0086]

[0087] Based on the above, it can be seen that the CIoU-Soft-NMS in the present invention combines the advantages of Soft-NMS and CIoU, and effectively solves the IoU calculation problem of neighboring prediction box loss and overlapping boundary boxes.

[0088] Embodiment 4

[0089] The present invention uses precision, recall, mAP@0.5, and mAP@0.5:0.95 as evaluation indicators to evaluate the performance of the model. mAP@0.5 means setting the IoU threshold to 0.5, calculating the AP of each category of images and averaging all categories. The higher the value, the stronger the comprehensive ability of the model; mAP@0.5:0.95 represents the average value of the mAP value obtained by setting the IoU threshold to 0.5 to 0.95 with a step size of 0.05. The higher the value, the more accurate the model is in locating the target.

[0090] 1. Comparative experiment of BAFPN:

[0091] The present invention uses the benchmark models YOLOv8n and YOLOv8s respectively, and conducts comparative tests on the Neck part using the original PANet structure, BiFPN structure, and BAFPN structure on the VisDrone2019 dataset to evaluate the performance improvement of BAFPN. The experimental results are shown in Table 1.

[0092]

[0093] The experimental data in Table 1 of the above experimental results clearly show that BAFPN, with its unique innovative feature fusion strategy and dynamic convolution adjustment mechanism, has greatly improved the detection ability of the model in the face of multi-target and multi-scale scenes; the positive impact of this improvement is multi-faceted, not only has the detection accuracy been significantly improved, but also greatly enhanced the adaptability and stability of the model in different environments, especially when dealing with complex background interference and small target detection tasks, it has demonstrated remarkable performance; the extremely significant performance improvement achieved by BAFPN on the YOLOv8n and YOLOv8s models provides a very convincing and powerful verification of the effectiveness of its structure.

[0094] 2. Comparative experiment of oft-NMS:

[0095] This paper uses the benchmark model YOLOv8s to conduct comparative tests on the original NMS, IoU-Soft-NMS, CIoU-Soft-NMS and other methods on the VisDrone2019 dataset to further evaluate the performance improvement of Soft-NMS and the impact of different loss functions on it. The experimental results are shown in Table 2.

[0096]

[0097] The data in Table 2 of the above experimental results show that the original NMS is not effective in dense target detection tasks; Soft-NMS uses a weight function to smooth adjacent boxes to avoid accidental deletion and improve detection performance; different loss functions affect the performance of Soft-NMS, and CIoU has a prominent advantage; it solves the problem of gradient vanishing when the IoU bounding boxes do not intersect, considers aspect ratio more than GIoU, inherits the advantages of DIoU and incorporates distance and aspect ratio, combines EIoU to enhance calculation and improve aspect ratio considerations, improves SIoU shape matching problems, and improves regression accuracy and comprehensiveness; experimental results confirm that when CIoU is used as the Soft-NMS loss function, the model performance is excellent.

[0098] 3. Ablation experiment:

[0099] This paper uses the YOLOv8s algorithm model as the benchmark model, VisDrone2019-DET-train as the training data set, and uses P, R, mAP@50, mAP@0.5:0.95, Parameters, and FPS as the main evaluation indicators. Under the same experimental conditions, ablation experiments are carried out. The experimental results are shown in the figure.

[0100] As shown in Table 3.

[0101]

[0102]

[0103] The experimental results in Table 3 show that compared with the baseline model, the improved model has improved mAP@0.5 by 15.7%, mAP@0.5:0.95 by 13.8%, and reduced Parameters by 27.22%. Although the FPS of the improved model has decreased, it still remains above 30FPS, which fully meets the requirements of real-time monitoring.

[0104] 4. Comparative experiment:

[0105] This paper compares YOLOv8s-ADB with the current mainstream single-stage and two-stage detection algorithms to verify the effectiveness of the proposed improved algorithm YOLOv8s-ADB. All comparison algorithms are tested and compared using the VisDrone2019 dataset, with a training cycle of 300 epochs, and mAP@0.5 and mAP@0.5:0.95 are used as evaluation indicators. The comparative experimental results are shown in Table 4.

[0106]

[0107]

[0108]

[0109] In order to further verify the versatility of YOLOv8s-ADB, this study also conducted comparative experiments on the UAVDT dataset. The UAVDT dataset is also an authoritative dataset widely used for target detection in drone aerial images. Its data characteristics are somewhat different from VisDrone2019, and it can more comprehensively evaluate the performance of the detection algorithm. Therefore, this paper conducts comparative experiments on YOLOv8s-ADB and other mainstream detection algorithms on the UAVDT dataset to verify the versatility and robustness of the improved model. All comparative algorithms are tested and compared using the UAVDT dataset, with a training cycle of 300 epochs. The evaluation indicators are still mAP@0.5 and mAP@0.5:0.95. The results of the comparative experiments are shown in Table 5.

[0110]

[0111]

[0112]

[0113] The experimental results in Table 5 of the above comparative experimental results clearly show that in terms of aerial image target detection tasks, the YOLOv8s-ADB model proposed in this paper shows the most outstanding performance. Taking the VsiDrone2019 dataset as an example, even compared with the latest and leading YOLOv9-C detection model, the YOLOv8s-ADB model is still 7.2% higher in mAP@0.5 and 0.95 higher in mAP@0.5. At the same time, compared with the baseline model YOLOv8s, YOLOv8s-ADB achieved a significant improvement of 15.7% in the mAP@0.5 indicator and a significant improvement of 13.8% in the mAP@0.5:0.95 indicator. In the UAVDT dataset, YOLOv8s-ADB improved its mAP@0.5 indicator by 7.3% and its mAP@0.5:0.95 indicator by 5.1% compared with YOLOv8s.

[0114] The examples referred to herein are merely illustrative and are used to explain some features of the method of the present invention. The appended claims are intended to claim the widest possible range that can be imagined, and the embodiments presented herein are only illustrations of selected implementations according to the combination of all possible embodiments. Therefore, it is the applicant's intention that the appended claims are not limited by the selection of examples that illustrate the features of the present invention. Some numerical ranges used in the claims also include sub-ranges within them, and changes in these ranges should also be interpreted as being covered by the appended claims where possible.

Claims

1. A novel aerial image detection method, characterized in that: It includes three innovative modules: AKConv, C2f-DRAC and BAFPN, and proposes three steps: feature extraction, feature fusion and post-processing; S1. Feature extraction The C2f_DRAC structure is designed to replace the backbone network C2f. This structure cleverly combines AKConv and CBAM and uses convolution operations with different dilation rates. By replacing the original Conv and C2f in Backbone with AKConv and C2f_DRAC respectively, the feature extraction efficiency of the algorithm is significantly enhanced, thereby greatly improving the accuracy and stability of the algorithm in detecting targets under complex background conditions; S2. Feature Fusion The BAFPN structure is designed based on BiFPN, and an additional small target detection layer is added. The BAFPN structure successfully improves the feature fusion effect of the algorithm by using cross-layer links and feature weighted fusion technology; S3, post-processing Replacing the original NMS with CIoU-Soft-NMS ensures the reliability and accuracy of the algorithm in dense target detection scenarios, reduces the missed detection rate, and makes the detection results more complete and accurate.

2. A novel aerial image detection method according to claim 1, characterized in that: In the feature extraction stage, ordinary Conv is replaced by AKConv. Specifically, variable kernel convolution AKConv is introduced. AKConv uses a new coordinate generation algorithm and cleverly introduces an offset to accurately adjust the sampling position of the irregular convolution kernel.

3. A novel aerial image detection method according to claim 2, characterized in that: The AKConv branch generates the initial sampling coordinates through a new coordinate generation algorithm: first define a regular sampling grid P based on the central cell o (such as 3×3 or 5×5), and then construct an irregular sampling grid P with no center point and the upper left corner as the starting point n The two are spliced ​​into an initial sampling shape, which can flexibly adjust the position of the convolution kernel to adapt to local feature differences and deal with target occlusion and complex background problems in UAV target detection. In addition, the 5×5 sample network can improve the feature extraction accuracy and effectiveness by designing different initial sampling shapes.

4. A novel aerial image detection method according to claim 3, characterized in that: The main part of AKConv is to first perform a two-dimensional convolution on the input feature map and use back propagation and gradient descent to obtain the offset, construct an offset feature map with the same size as the input feature map and containing a 2d vector of offsets in the x and y directions; apply it to the initial sampling shape to dynamically adjust it to form an adaptive sampling shape, optimize the shape according to the dynamic local features of the image, and improve the efficiency and accuracy of feature extraction.

5. A novel aerial image detection method according to claim 4, characterized in that: The input feature map is resampled with a deformable sampling grid, and features are further extracted through shaping and convolution Conv. Then the standardized Norm operation is performed for stable training. Finally, the activation function SiLU is used to add nonlinear factors to generate the final output Output, providing a high-quality feature data foundation for subsequent image processing and promoting the efficient and accurate operation of the image processing process.

6. The novel aerial image detection method according to claim 1, characterized in that: In the feature fusion stage, BAFPN extracts features and adjusts the number of channels by performing AKConv convolution processing on feature maps of different scales output from Backbone to ensure that the number of channels is consistent during subsequent feature fusion.

7. A novel aerial image detection method according to claim 6, characterized in that: In the top-down path, upsampling is performed layer by layer and weighted fusion is performed to obtain a new fused feature map. C2f-DRAC is used to integrate multi-scale information to improve the richness and accuracy of the feature map and further improve the effect of feature fusion.

8. A novel aerial image detection method according to claim 7, characterized in that: In the bottom-up path, information interaction of multiple paths is achieved through cross-layer skip connections. After downsampling layer by layer and weighted fusion, a new fused feature map is obtained, and finally feature enhancement is completed through C2f to generate the final output.

9. The novel aerial image detection method according to claim 1, characterized in that: In the post-processing stage, the Soft-NMS algorithm formula is as follows: In formula (1), there are the following related definitions: represents the confidence of the i-th bounding box, is the threshold of IoU, M represents the current highest-scoring bounding box, is the i-th bounding box, and is the intersection-over-union ratio between M and ; Soft-NMS uses a weight function to convert the degree of overlap into a real number between 0 and 1, calculates the weight based on IoU, and then performs a weighted summation on all bounding boxes.

10. A novel aerial image detection method according to claim 9, characterized in that: CIoU is used to replace IoU to enhance the performance of NMS, and it is named CIoU-Soft-NMS; The CIoU calculation formula is as follows: In formula (2), IoU represents the intersection over union ratio of the predicted box and the true box; c represents the diagonal length of the minimum bounding rectangle that covers both the predicted box and the true box; b and b gt Represent the center points of the predicted box and the real box respectively; ρ(b,b gt ) represents b and b gt The Euclidean distance between α represents the weight parameter that balances the impact of IoU and aspect ratio consistency, and its definition is shown in formula (3): v represents a parameter that measures the difference in aspect ratio between the predicted box and the true box, and its definition is shown in formula (4): w and h represent the width and height of the prediction box respectively; w gt and h gt Represent the width and height of the real box respectively; The CIoU loss function is specifically expressed as formula (5):

Images