Unmanned aerial vehicle aerial small target detection method based on fine-grained bidirectional multi-scale fusion

Abstract

The application discloses a UAV aerial small target detection method based on fine-grained bidirectional multi-scale fusion, and belongs to the technical field of UAV target detection. An AeroFineFusion-Det detection model is constructed based on a YOLOv8 as a basic network, a C2f_CMUNeXtB module is used to replace an original bottleneck structure in a backbone network, and the extraction capability of shallow fine-grained features is strengthened; a fine-grained bidirectional feature pyramid FS-BiFPN is built in a neck network, a high-resolution P2 feature layer is explicitly introduced, a bidirectional feature transmission path is retained, a double-attention weighted fusion DAWF module is embedded in each fusion node, a cross-layer local perception fusion head CLAFHead is used in a detection head, decoupling classification and regression prediction are completed based on three-scale features, and cross-layer feature compensation fusion is realized in a medium-scale branch. Through multi-module collaborative optimization, the application retains the small target detail information of aerial images, improves the feature fusion precision and detection stability, reduces the model calculation redundancy, and is suitable for real-time detection of dense small targets in complex aerial scenes.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) target detection technology, specifically to a method for detecting small targets in UAV aerial photography based on fine-grained bidirectional multi-scale fusion. Background Technology

[0002] The widespread application of drone technology in fields such as power line inspection, smart city monitoring, emergency rescue, and geological surveying has made aerial image target detection a core supporting technology for autonomous drone operations. Drone aerial images are characterized by high viewing angles, wide coverage, and complex and variable scenes. Targets are generally small in scale, densely distributed, frequently occluded, and lacking in texture information. They are also susceptible to factors such as changes in lighting, background interference, and viewing angle shifts, leading to frequent missed detections, false detections, and positioning errors in traditional detection models. Furthermore, drone-borne embedded platforms face objective limitations such as limited computing power, scarce storage resources, and strict power consumption constraints. High-precision detection models are often difficult to deploy in real-time on the edge due to their large number of parameters and high computational redundancy. The balance between detection accuracy and inference efficiency has become a key bottleneck restricting the engineering implementation of small target detection in drone aerial photography.

[0003] In recent years, the academic community has conducted a large number of deep learning methods for small target detection in UAV aerial photography. Among them, the one-stage detection method based on the YOLO series has become the mainstream direction due to its advantages such as fast inference speed and simple structure. In their paper "Small Target Detection in UAV Images by Fusion of Foreground Refinement and Multidimensional Inductive Bias Self-Attention" published in the *Journal of Image and Graphics*, Feng Qihan et al. proposed a fusion mechanism of foreground refinement and multidimensional inductive bias self-attention to address the problems of small target detail loss and background interference. This mechanism enhances the target region feature representation and improves detection stability in complex scenes. In their paper "FSTD-YOLO: A Target Detection Algorithm for Aerial Images Focusing on Small Targets" published in *Infrared Technology*, Ji Jiahao et al. designed a multi-receptive field feature extraction module and a dedicated small target detection head based on YOLOv8n. By enhancing shallow detail preservation and optimizing the attention mechanism, they alleviated the problem of small target feature attenuation. In their paper "DroneNet-Lite Lightweight Detector" published in the *Journal of South China Normal University (Natural Science Edition)*, Tang Xiaoyu et al. introduced a P2 high-resolution enhancement layer to improve the YOLO11 structure, achieving a balance between accuracy and lightweight design on the VisDrone2019 dataset. Although the aforementioned academic research has made some progress in terms of feature extraction, attention mechanisms, and lightweight design, it still generally suffers from defects such as insufficient utilization of high-resolution shallow features, severe information attenuation in cross-scale fusion, insufficient adaptive capability of multi-path feature aggregation, and strong coupling between detection head classification and localization. In drone aerial photography scenarios with dense occlusion and extreme scale changes, the detection accuracy and robustness of small targets are still difficult to meet the needs of practical applications.

[0004] To address the engineering challenges of small target detection in drone aerial photography, domestic patent technologies have made targeted improvements. Patent CN119625251A discloses a small target detection model for drone aerial photography scenarios. Using YOLOv8s as the base network, it reduces model parameters and improves small target detection performance by optimizing the backbone network feature extraction structure, improving the feature pyramid fusion method, and introducing a lightweight convolution module. Patent CN119495037A discloses a method and system for small target detection in drone aerial photography. Addressing the issues of large target scale differences and complex backgrounds in aerial images, it designs a multi-scale feature enhancement module and an adaptive non-maximum suppression strategy to strengthen small target feature responses and suppress duplicate prediction boxes. Patent CN120495641A discloses a method, system, medium, and device for small target detection in drone aerial photography based on RT-DETR. It introduces an efficient multi-scale attention mechanism based on the RT-DETR architecture and improves the accuracy and robustness of small target detection through cross-scale feature fusion and global context modeling. Although the aforementioned patented technologies have been optimized in terms of model structure, feature fusion, and post-processing strategies, they still have significant technical limitations: a fine-grained bidirectional multi-scale fusion mechanism is lacking, and high-resolution shallow details are significantly attenuated during cross-scale transmission; multi-path feature fusion relies on fixed weight allocation, resulting in weak adaptive aggregation capabilities; the feature extraction module lacks spatial modeling and channel interaction capabilities, leading to insufficient retention of shallow fine-grained features; the detection head has not achieved efficient decoupling and cross-layer local perception fusion, resulting in poor coordination between classification and localization in dense small target scenarios, making it difficult to simultaneously meet the requirements of high-precision, lightweight, and real-time UAV edge deployment.

[0005] Existing academic research and patented technologies have failed to systematically solve the core technical contradictions in small target detection in UAV aerial photography, such as shallow detail preservation, cross-scale information coupling, adaptive feature aggregation, and lightweight and efficient deployment. In complex scenarios, the problems of missed detection, false detection, and positioning drift of small targets are prominent, and high-precision models are difficult to adapt to UAV embedded platforms. Summary of the Invention

[0006] To address the aforementioned technical issues, this application discloses a method for detecting small targets in UAV aerial photography based on fine-grained bidirectional multi-scale fusion, comprising the following steps:

[0007] An AeroFineFusion-Det detection model based on YOLOv8 was constructed. The backbone network replaced the original C2f bottleneck structure with the C2f_CMUNeXtB module, and the neck network constructed a fine-grained bidirectional feature pyramid FS-BiFPN. Each fusion node of the FS-BiFPN was embedded with a dual attention weighted fusion DAWF module, and the detection head adopted a cross-layer local perception fusion head CLAFHead.

[0008] The FS-BiFPN explicitly incorporates a high-resolution P2 feature layer, retains the bidirectional feature transfer path from top to bottom and bottom to top, and the P5 feature layer only participates in semantic guidance and cross-scale fusion, and is not used as the input feature of the prediction branch.

[0009] The CLAFHead uses P2, P3, and P4 three-scale features to perform decoupled classification prediction and regression prediction.

[0010] Preferably, the dual-attention weighted fusion DAWF module performs channel attention recalibration, channel adjustment, and scale alignment operations on the K-way input features to obtain K-way aligned features. The formula for the alignment operation is:

[0011]

[0012] in, For the k-th input feature, For channel attention recalibration operation This is a 1×1 convolution operation. To scale features to the target scale The operation, , These represent the height and width of the target feature, respectively.

[0013] Preferably, the DAWF module calculates the fusion coefficients of each alignment feature based on learnable parameters, using the following formula:

[0014]

[0015] in, The number of input feature paths participating in the fusion. The learnable parameters corresponding to the k-th feature are... It is a non-linear activation function. To prevent the use of a pre-defined small constant with a denominator of zero, Let be the fusion coefficient of the k-th feature, satisfying and .

[0016] Preferably, the DAWF module performs weighted aggregation of the K-way alignment features based on the fusion coefficients to obtain fused features. The specific formula is as follows: ,in, The k-th input feature is the feature after channel attention recalibration and scale alignment. Let be the fusion coefficient of the k-th feature. The number of input feature paths participating in the fusion. This refers to the fusion feature after weighted aggregation.

[0017] Preferably, the DAWF module performs fusion on the fused features. Apply spatial attention modulation to output the final fused features. The specific formula is as follows:

[0018]

[0019] in, For spatial attention operations, The location weight map generated for spatial attention. This is an element-wise multiplication operation. This is a 1×1 convolution operation. This is the final fusion feature output by the DAWF module.

[0020] Preferably, the C2f_CMUNeXtB module internally uses CMUNEXtBlock as the bottleneck unit, and the CMUNEXtBlock adopts a residual connection structure for the input features. Perform residual mapping and output features The calculation formula is:

[0021]

[0022] in, For the input features of CMUneXtBlock, This is an incremental mapping operation that combines spatial updates and channel mapping. This is the output feature of CMUNEXtBlock.

[0023] Preferably, the incremental mapping It consists of a large kernel depthwise convolution space update, two 1×1 point convolution channel transformations, nonlinear activation, and normalized projection operations. The specific formula is as follows:

[0024]

[0025] in, For updating the space of large kernel depthwise convolutions. To increase the number of channels from Expansion to 1×1 point convolution operation, It is a non-linear activation function. To increase the number of channels from Compress back 1×1 point convolution operation, For normalization and projection operations, This represents the channel expansion ratio. This is the output of the incremental mapping operation.

[0026] Preferably, the CLAFHead employs a decoupled structure in the P2, P3, and P4 three-scale branches, introducing a cross-layer fusion unit CLLFusion only in the mid-scale P3 branch; the CLLFusion integrates high-resolution features With low-resolution features As input, after channel alignment and scale alignment, we obtain and Then, the initial fusion features are obtained by adding them element by element. The specific formula is as follows:

[0027]

[0028] in, High-resolution features Features after channel alignment and scale alignment Low-resolution features Features after channel alignment and scale alignment The initial fused feature is obtained by adding the two features element by element.

[0029] Preferably, the CLLFusion concatenates the aligned two features along the channel dimension to generate a channel-gated vector. The specific formula is as follows:

[0030]

[0031] in, This is the result of splicing the two aligned features along the channel dimension. This is a global average pooling operation. For lightweight multilayer sensor operation. It is the Sigmoid activation function. This is the generated channel gating vector.

[0032] Preferably, the CLLFusion modulates the initial fused features based on the channel gating vector and outputs the final fused features. The specific formula is as follows:

[0033]

[0034] in, The initial fused features are obtained by adding the two feature streams element by element. For the generated channel gating vector, This is a channel-by-channel multiplication operation. For depthwise convolution operations, For point convolution operations, For batch normalization operations, It is a non-linear activation function. The final fusion feature output by the CLFusion unit.

[0035] Compared with the prior art, the technical solution of this application has the following technical effects:

[0036] This invention constructs a fine-grained, bidirectional, multi-scale fusion detection model that can effectively preserve the detailed information of shallow high-resolution features in UAV aerial images, enhance the semantic and detail coupling ability between features at different levels, enable small targets to maintain complete feature expression during cross-scale transmission, improve the model's feature perception and information retention capabilities for small targets, and provide a stable and reliable feature foundation for subsequent accurate detection.

[0037] This invention employs a dual-attention weighted fusion mechanism, which can adaptively allocate weights and aggregate features from multiple input features. It combines channel and spatial dual attention to complete feature recalibration, autonomously strengthens target-related features and suppresses redundant background information, thereby improving the selectivity and stability of the feature fusion process and enabling the model to more accurately focus on the effective target region in complex backgrounds.

[0038] This invention uses an improved feature extraction module to enhance spatial modeling and channel interaction capabilities under lightweight constraints, efficiently extracting fine-grained texture and edge information from images, improving the expression quality of shallow features, and reducing the number of model parameters and computational redundancy. This makes the model more adaptable to the resource limitations of UAV embedded platforms and ensures efficient operation of edge inference.

[0039] This invention employs a decoupled cross-layer local perception fusion head, which improves classification stability and localization accuracy in dense small target scenes by independently modeling classification and regression branches and cross-layer compensation of mesoscale features, and optimizes the consistency and accuracy of target bounding box prediction, enabling the model to maintain reliable detection output in aerial photography scenes with dense targets and frequent occlusion.

[0040] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0041] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0043] Based on the description of the figures and their corresponding technical content in the document, the titles of the figures are as follows:

[0044] Figure 1 A schematic diagram of the overall architecture of the UAV aerial photography small target detection model based on fine-grained bidirectional multi-scale fusion in this invention;

[0045] Figure 2 A schematic diagram of the fine-grained bidirectional feature pyramid (FS-BiFPN) structure and the dual attention weighted fusion (DAWF) module of this invention;

[0046] Figure 3 A schematic diagram of the C2f_CMUNeXtB module and its internal CMUneXtBlock bottleneck unit structure of the present invention;

[0047] Figure 4 A schematic diagram of the CLFusion unit structure in the CLAFHead cross-layer local sensing fusion head of this invention;

[0048] Figure 5 A statistical diagram illustrating the category distribution and target bounding box scale distribution of the VisDrone dataset;

[0049] Figure 6 A schematic diagram of radar comparison of the AFFD model and comparison algorithm of this invention under multiple evaluation indicators;

[0050] Figure 7 The AP of the AFFD model and comparison algorithm of this invention on each category of the VisDrone dataset. 50 Heatmap comparison diagram;

[0051] Figure 8 A schematic diagram comparing the changes in detection accuracy and model complexity of ablation experiments for each module of the present invention;

[0052] Figure 9 A visual comparison diagram of the detection results of the AFFD model and the comparison algorithm of this invention on the VisDrone and UAVDT datasets;

[0053] Figure 10A schematic diagram showing a visual comparison of the feature activation heatmaps of the AFFD model and the YOLOv8s model of this invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0055] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0056] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0057] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0058] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0059] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0060] Example 1

[0061] This embodiment mainly describes a method for detecting small targets in UAV aerial photography based on fine-grained bidirectional multi-scale fusion, such as... Figure 1 As shown, it specifically includes:

[0062] An AeroFineFusion-Det detection model based on YOLOv8 was constructed. The backbone network replaced the original C2f bottleneck structure with the C2f_CMUNeXtB module, and the neck network constructed a fine-grained bidirectional feature pyramid FS-BiFPN. Each fusion node of the FS-BiFPN was embedded with a dual attention weighted fusion DAWF module, and the detection head adopted a cross-layer local perception fusion head CLAFHead.

[0063] The FS-BiFPN explicitly incorporates a high-resolution P2 feature layer, retains the bidirectional feature transfer path from top to bottom and bottom to top, and the P5 feature layer only participates in semantic guidance and cross-scale fusion, and is not used as the input feature of the prediction branch.

[0064] The CLAFHead uses P2, P3, and P4 three-scale features to perform decoupled classification prediction and regression prediction.

[0065] Furthermore, this application improves upon YOLOv8 as a baseline, proposing AeroFineFusion-Det, with the overall framework as follows: Figure 1 As shown, the backbone network introduces C2f_CMUNeXtB to enhance shallow feature representation; the neck network employs a fine-grained bidirectional fusion structure FS-BiFPN, and embeds a dual attention weighted fusion module (DAWF) in the fusion node to achieve adaptive feature aggregation; the prediction end uses a Cross-Level Local-Aware Fusion Head (CLAFHead) to output P2 / P3 / P4 scales, thereby strengthening the support of cross-layer local information for classification and localization. These improvements are optimized end-to-end under a unified training setting to enhance the stability and overall performance of small object detection.

[0066] Furthermore, the fine-grained bidirectional feature pyramid FS-BiFPN proposed in this application, such as Figure 2 As shown in (a); let the multi-scale features output by the backbone network be { ,in It boasts the highest spatial resolution. FS-BiFPN, while preserving bidirectional paths (top-down and bottom-up), will... Explicitly incorporate the fusion topology, and with As input to the detection head, it enables three-scale prediction; It only participates in top-down semantic guidance and cross-scale fusion, and is not used as input for the final prediction branch.

[0067] In the top-down path, high-level features are upsampled level by level to adjacent low-level scales and fused with the input features at the corresponding scale and the same-scale preserved branches to obtain intermediate features. Subsequently, the low-level fusion results are downsampled and fed back step by step from the bottom up, and then fused again with intermediate features from adjacent scales and branches preserved at the same scale to obtain the final output. .therefore, Figure 2 The topology shown in (a) can be viewed as introducing higher resolution onto the bidirectional fusion framework of BiFPN. This enhances the preservation and propagation of shallow details during cross-scale interactions, and supports subsequent predictions with three-scale outputs.

[0068] To improve the selectivity and stability of multi-path feature fusion, this application proposes a dual-attention weighted fusion module (DAWF) and implements it as a unified operator for each cross-layer fusion node in FS-BiFPN. Its structure is as follows: Figure 2 As shown in (b); a certain fusion node receives Road input features ;

[0069] For the Channel attention is applied to the input features of the path for recalibration, and then... Convolution completes channel mapping; then the spatial resolution of each branch is aligned to the target scale. Alignment features are obtained.

[0070]

[0071] In calculating branch weights, learnable parameters are introduced. After ReLU constraint to ensure non-negativity and normalization, the fusion coefficients are obtained. ,in This is a numerically stable term. Based on... The multi-path aligned features are weighted and aggregated to obtain the fused features: In terms of fusion features Spatial attention is introduced to reshape the response, enhancing the target region and suppressing redundant background; the spatial attention branch outputs a position weight map. The fusion features are then applied using an element-wise gating method to obtain the final output:

[0072]

[0073] in This indicates element-wise multiplication. The above process is similar to... Figure 2 (b) shows the same structure as the DAWF: channel recalibration and normalization weighting achieve cross-layer aggregation, and then spatial attention is used to enhance the fusion result at the location level, and the output is used for subsequent detection head prediction.

[0074] Furthermore, in small target detection in UAV aerial photography, feature extraction needs to preserve as much high-resolution texture and edge detail as possible in the shallow layers, and maintain information stability during multi-scale propagation and fusion to reduce the risk of missed detections and false detections in dense target scenes. YOLOv8's Backbone extensively uses C2f modules to enhance gradient flow and feature reuse, exhibiting good training stability and expression efficiency; however, its core still relies on local convolution operators. In aerial images with dense small targets and complex backgrounds, shallow details may still attenuate during cross-scale propagation, thus affecting the quality of subsequent fusion and prediction.

[0075] Based on the above analysis, this application proposes C2f_CMUNeXtB as a feature extraction unit to enhance shallow feature representation. Without altering the original YOLOv8 C2f main information flow organization, this application replaces the bottleneck transformation within C2f with CMUNEXtBlock, enabling the module to maintain its original branching, cascading, and splicing paradigms while introducing stronger spatial modeling and channel mixing capabilities. The key to this design is to enhance the shallow layer's response to fine-grained textures and edge cues with minimal structural perturbation, and to improve the preservation of shallow information during subsequent network propagation, providing more detailed support for multi-scale fusion and dense small target prediction.

[0076] The structure of CMUNeXtBlock is as follows: Figure 3 As shown, its core lies in enhancing spatial modeling with large-kernel depthwise separable convolutions and achieving efficient channel blending through inverse residual point convolutions, thereby enhancing the ability of shallow layers to express the texture and edge information of small targets with lower parameters and computational cost. Given input features CMUneXtBlock can be summarized as... Applying spatial updates and channel blending yields incremental mappings. And when the dimensions are consistent, residual join is used for output:

[0077]

[0078] in It consists of two parts: introducing a more comprehensive spatial context through depthwise convolution of a large kernel; followed by two... Point convolution performs channel expansion and back projection, enabling channel interaction. Its channel blending process can be written as:

[0079]

[0080] in Representation space updates (achieved by deep separable convolutions with large kernels). It is a non-linear activation. This represents lightweight transformations such as normalization and projection. This represents the channel expansion ratio. The design enhances the retention and propagation of shallow, fine-grained information through stronger spatial modeling and channel blending, providing a more stable feature base for subsequent multi-scale fusion and dense small-target prediction.

[0081] Furthermore, in the detection of dense small targets in UAV aerial photography, the detection head needs to maintain the stability of classification confidence and localization quality across multi-scale features, reducing confidence fluctuations and boundary drift caused by scale changes, occlusion, and complex backgrounds. Meanwhile, mesoscale features often represent a trade-off between detail and semantics, and are more susceptible to interference when targets are densely adjacent. Therefore, it is necessary to introduce appropriate cross-layer information compensation and local enhancement into this branch, and control additional computational overhead under edge deployment constraints.

[0082] The YOLOv8 detector head has good engineering versatility, but its multi-scale branches mostly use isomorphic mapping and lack targeted enhancement for the mesoscale branch. Moreover, in dense small target scenarios, the coupling between classification and regression targets is more prominent. Without clear decoupling design and cross-scale supplementation, the mesoscale representation is easily affected by nearby targets and background noise, thus restricting classification stability and positioning accuracy.

[0083] To this end, this application proposes a cross-layer local perception fusion head, CLAFHead, such as Figure 4 As shown, in Dense predictions are performed at three scales, and a decoupled structure is used to model classification and regression branches separately. To enhance the stability of localization and classification in dense small target scenes while controlling additional computational overhead, CLAFHead introduces a cross-layer fusion unit CLFusion only in the mesoscale branch: it uses fine-grained cues provided by the high-resolution branch to supplement the mesoscale semantic representation before feeding it into the subsequent prediction tower; the other scale branches maintain isomorphic mapping and do not introduce additional cross-layer coupling computation.

[0084] Let the two characteristics of the fusion be: (High-resolution side) and (Low resolution side). First, through... Convolution completes channel alignment and unifies the spatial scale to the low-resolution side (adaptive average pooling is used in the implementation), resulting in... The aligned features from the two paths are then added element-wise to form the initial fused features:

[0085]

[0086] To achieve cross-layer adaptive recalibration, CLFusion concatenates the two aligned features along the channel dimension and performs global average pooling to obtain the cross-layer channel description, which is then processed by a lightweight MLP (two-layer MLP). Convolution and SiLU generate gated vectors:

[0087]

[0088] This gating Channel-by-channel modulation is performed to suppress redundant responses and highlight more effective cross-layer complementary information during the fusion process.

[0089] To enhance the local structural representation of the fused features, CLFusion applies a lightweight local enhancement operator consisting of depthwise separable convolutions and pointwise convolutions (in conjunction with BN and SiLU) to the gated features to obtain the fused output:

[0090]

[0091] in As input to the mesoscale branch, it enters the subsequent decoupled prediction tower; by introducing cross-layer gating and lightweight local augmentation only at key scales, CLAFHead can improve classification stability and localization accuracy in dense small target scenes without significantly increasing additional computational overhead.

[0092] This embodiment, through a fine-grained bidirectional multi-scale fusion architecture, dual-attention weighted fusion, lightweight feature extraction, and the collaborative design of decoupled cross-layer perception detection heads, can fully preserve the shallow high-resolution detail information of aerial images, achieve adaptive aggregation of multi-path features, effectively enhance spatial modeling and channel interaction capabilities, reduce model parameters and computational redundancy, significantly improve the classification stability and positioning accuracy of dense small targets, enable the model to maintain efficient and accurate detection in complex aerial photography scenarios, and better adapt to the real-time deployment requirements of embedded UAVs.

[0093] Based on Example 1, this example conducts experiments on two UAV aerial target detection datasets: VisDrone2019-DET and UAVDT. VisDrone is used for the main comparative experiments and ablation analysis, while UAVDT is used for cross-dataset generalization validation. VisDrone scenarios are complex, with dense targets and widespread occlusion, covering 10 target categories including pedestrians and vehicles. Figure 5 As shown, Figure 5 (a) is the category distribution; Figure 5 (b) shows the width and height distribution of the bounding box (log density). Class imbalance and a high proportion of small targets can be observed, which places a stronger demand on multi-scale fusion for small target detection in UAV aerial photography.

[0094] The dataset exhibits a significant imbalance in class distribution (e.g., approximately 144,420 car annotations compared to approximately 3,241 awning-tricycle annotations). Most bounding boxes are smaller than 100×100 pixels, with the densest concentration of small targets within 50×50 pixels. This places higher demands on the model's fine-grained representation and multi-scale fusion capabilities.

[0095] UAVDT is a large-scale, challenging benchmark dataset built for target detection and tracking tasks on unmanned aerial vehicle (UAV) platforms. It contains approximately 10 hours of raw UAV video data, totaling about 80,000 frames. This paper conducts generalization experiments on UAVDT-M and divides the dataset based on locally prepared frame-sampling data: the training set contains 24,778 images, and the validation set contains 15,598 images. Other training parameters and evaluation metrics are kept consistent with the main dataset settings to ensure fairness and reproducibility of the comparison.

[0096] To ensure fairness and reproducibility in the experimental comparisons, all experiments were conducted under a unified training and testing configuration. Key hyperparameter settings for training and testing are shown in Table 1, and the hardware and software experimental environment configurations are shown in Table 2. Except for differences in network structure, all other training strategies and inference settings remained consistent.

[0097] Table 1 shows the training and testing hyperparameter settings. Training consisted of 250 epochs, with an input size of 640×640 and a batch size of 8. The optimizer used was SGD, and the initial learning rate was... and decay to The momentum coefficient is 0.937, and the weight decay is... The NMS IoU threshold was set to 0.7 during the inference phase. Table 2 shows the experimental hardware and software environment configuration. The experiment was conducted on Ubuntu 24.04, with an Intel(R) Core(TM) i7-14700KF CPU and an NVIDIA GeForce RTX3090 GPU (24 GB VRAM), and 31 GB of RAM. For the software environment, Python 3.9 was used, the deep learning framework was PyTorch 1.13.1, and CUDA version 11.6 was employed.

[0098] Table 1 Training and Testing Hyperparameter Settings

[0099] Table 2 Experimental hardware and software environment configuration

[0100] Evaluation Metrics. This application uses COCO-style mean accuracy (AP) as the primary evaluation metric, including... , as well as (IoU thresholds range from 0.50 to 0.95, with a step size of 0.05). Also reporting targets at different scales. , , This is used to measure the detection performance of small / medium / large targets. In terms of efficiency, the number of parameters, computational costs (MACs), and inference latency are given to comprehensively evaluate the accuracy-efficiency trade-off.

[0101] To verify the effectiveness of the proposed framework in UAV aerial photography of small targets, this application compares it with mainstream two-stage detectors (ATSS, VFNet, TOOD, etc.) and representative one-stage detectors (RTMDet-S, YOLOX-S, PPYOLOE, YOLOv7-tiny, YOLOv8s) on the VisDrone2019 validation set. Table 3 presents the quantitative results of overall accuracy and efficiency. It can be seen that AFFD-s achieves the best performance in key accuracy metrics: , and They reached 43.7% / 27.1% / 26.3% respectively, compared to the baseline YOLOv8s. (21.9%) increased by 4.4 percentage points. Meanwhile, AFFD-s improved in the small target indicator. The accuracy reached 17.7%, a 5.3 percentage point improvement over YOLOv8s (12.4%), indicating that the proposed fine-grained bidirectional fusion structure can more effectively enhance the expressive power of high-resolution hierarchical features, thereby improving the detection performance of small targets in aerial photography. In terms of efficiency, AFFD-s maintains high accuracy with only 7.36M parameters and 17.7 GMACs of computation, and an inference latency of 5.7ms, demonstrating a good balance between accuracy and efficiency. Furthermore, the lightweight version AFFD-n achieves 20.9% accuracy with only 1.99M parameters. Compared to 13.0% This indicates that the framework has good scalability and is suitable for the deployment needs of drones under different computing power constraints.

[0102] To more intuitively illustrate the trade-offs between multiple indicators, such as Figure 6 As shown, , , The parameters such as Params, MACs, and Latency are normalized and then plotted as radar charts. Overall, AFFD-s performs well in terms of accuracy (especially...). and The advantages of the proposed method are more prominent, while the efficiency remains within an acceptable range, further demonstrating the adaptability of the proposed method to small and dense targets in complex aerial photography scenarios.

[0103] In addition to the overall indicators, this application further provides category-level comparison results as shown in Tables 3 and 4, from each category As shown in Figure 7, AFD-s achieved consistent improvements across most categories, particularly in easily confused categories such as pedestrians and people, which often contain small-scale targets (reaching 51.1% and 40.7%, respectively). It also maintained stable gains in traffic target categories such as car, bus, and motor (83.2%, 60.1%, and 51.7%, respectively). The VisDrone validation set categories are illustrated in Figure 7. (%) Comparison heatmap, the horizontal axis represents the target category, the vertical axis represents different detection methods, and the color intensity indicates the corresponding category. The horizontal plane is used to visually represent category-level performance differences and overall improvement trends.

[0104] The results show that the proposed feature fusion mechanism not only improves the overall detection accuracy, but also enhances the ability to distinguish between dense small targets and similar categories.

[0105] Table 3 shows the overall comparison results of different detectors on the VisDrone2019 validation set.

[0106] Table 4. Categories on the VisDrone 2019-DET validation set (%)contrast

[0107] To further verify the generalization ability of the proposed framework under different data distributions, this application conducted supplementary comparative experiments on the UAVDT dataset, and the results are shown in Table 5. The experiments used the official UAVDT training / validation partition for training and evaluation, and maintained consistency with the VisDrone main experiment in terms of input resolution, training epochs, optimizer, and learning rate strategy, only changing the dataset and class configuration to ensure comparability. As shown in Table 5, AFFD-s achieves [performance / performance] on UAVDT. =30.6% / 18.7% / 17.7%, compared to the baseline YOLOv8s =16.4%, an increase of 1.3 percentage points. More notably, AFFD-s improved in the small target indicator... The percentage reached 18.2%, an increase of 6.4 percentage points compared to YOLOv8s (11.8%). The relatively limited variation indicates that the performance gain mainly comes from the enhancement of small-scale target representation and cross-scale information interaction. This result demonstrates that the proposed fine-grained bidirectional fusion and adaptive weighting mechanism can still stably improve the detection capability of small targets under cross-dataset settings, thereby enhancing the model's robustness and generalization performance under varying distribution conditions.

[0108] Table 5 Comparative experimental results on the UAVDT dataset

[0109] To verify that the proposed FS-BiFPN does not depend on a specific baseline structure for its gain, this application ported it to the YOLOv5, YOLOv8, and YOLO11 baselines respectively, and compared the results under the same training and evaluation settings. The results are shown in Table 6. It can be seen that FS-BiFPN brings consistent accuracy improvements and small target gains across all three baselines: on YOLOv5, YOLOv8, and YOLO11, They increased by 2.8, 3.7, and 3.5 percentage points respectively. These improvements were 3.9, 4.9, and 4.3 percentage points, respectively. Simultaneously, FS-BiFPN significantly reduced the parameter size while maintaining accuracy gains, with reductions of 34.3%, 34.1%, and 25.5% compared to their respective baselines. Furthermore, in the YOLOv8 baseline, FS-BiFPN further improved accuracy by 0.4 percentage points compared to conventional BiFPN. The gain was increased and the number of parameters was reduced by 24.6%, indicating that the fine-grained bidirectional fusion structure has better efficiency and transferability while enhancing cross-scale interaction.

[0110] Table 6. Comparison of the transferability of FS-BiFPN under different YOLO baselines

[0111] This application introduces four improved units—FS BiFPN, DAWF, C2f CMUNeXtB, and CLAFHead—on the original YOLOv8 framework to enhance the accuracy and robustness of small target detection in UAV aerial photography, while balancing model size and computational cost. To verify the effectiveness of each unit and the overall benefits of their combination, this application uses YOLOv8n and YOLOv8s as lightweight and standard-scale benchmark models, respectively, and sequentially integrates each module to conduct ablation experiments. Evaluation metrics and complexity measures are consistent with the Evaluation Metrics section. Ablation results are shown in Table 7. A–D are progressively introduced into YOLOv8s / YOLOv8n: A=FS-BiFPN, B=DAWF, C=C2f_CMUNeXtB, D=CLAFHead; Report , , , And Params(m), MACs(G);

[0112] As modules are gradually introduced along the chain, the overall accuracy of the model shows a continuous upward trend, with particularly stable improvement in small target metrics. This indicates that the improvement strategy in this application can effectively enhance the preservation of shallow details and cross-scale information interaction, thereby improving the recognizable representation capability of dense small targets. Taking YOLOv8s as an example, the baseline model... and They are 21.9 respectively. With 12.4 After introducing FS BiFPN, Increased to 25.6 Improved by 3.7 percentage points, Increased to 17.3 This represents a 4.9 percentage point improvement, indicating that fine-grained bidirectional feature fusion can explicitly enhance the role of high-resolution layers in multi-scale interactions, allowing for more complete preservation of small target details during cross-layer propagation. Further overlaying with DAWF... Upgraded to 26.0 This indicates that adaptive weighting and attention recalibration can improve the selectivity and stability of multi-path feature aggregation, leading to continuous improvement in fusion quality. Further addition of C2f_CMUNeXtB results in a more significant improvement in high IoU metrics, for example... Increased to 26.7 This indicates that the feature extraction unit is more inclined to improve representation quality and localization accuracy. Finally, after adding CLAFHead to obtain the complete model, Reached 26.3 , Reached 17.7 This indicates that decoupling prediction and cross-layer interaction provide supplementary gains to the localization stability and classification-localization synergy of dense small targets. Meanwhile, Table 7 also presents the comparison results when B, C, and D are introduced individually; their overall gains are smaller than those of the progressively stacked link starting from FS BiFPN, further illustrating that the main performance gain comes from the systematic enhancement of the multi-scale fusion path and high-resolution information flow, rather than the isolated replacement of a single module.

[0113] From a complexity perspective, compared to YOLOv8s, the proposed model reduces the number of parameters by approximately 33.9% while increasing computational complexity by approximately 23.8%. This result demonstrates that the structural modifications in this application, while introducing more comprehensive multi-scale fusion computation, significantly reduce the parameter scale through module replacement and structural compression, thereby achieving stable accuracy improvements with controllable computational increments, making it more suitable for edge deployment. A consistent trend is observed in YOLOv8n. From 18.2 Increased to 20.9 , From 9.6 Upgraded to 13.0 The number of parameters was reduced by 33.9%, which verifies the scalability and consistent effectiveness of the method for models at different scales.

[0114] like Figure 8 As shown, the overall trend more intuitively depicts the trade-off between performance and complexity. The matrix below shows the module combinations enabled for each configuration, where s and n represent YOLOv8s and YOLOv8n, respectively. Under different configurations, the AP index curve generally increases, while the parameter quantity curve generally decreases, indicating that the computational load is within a controllable growth range. The module enablement matrix also shows that the multi-unit combination proposed in this application is not a simple superposition, but rather forms a collaborative optimization link at the three levels of feature fusion, feature extraction, and detection head, thereby obtaining more stable comprehensive benefits.

[0115] Table 7 Ablation Results of Modules

[0116] Figure 8 Ablation experiment trend chart and module activation matrix. The left axis represents the AP index (%), and the right axis represents the complexity (Params, MACs); the matrix below shows the module combinations enabled for each configuration, where s and n represent YOLOv8s and YOLOv8n, respectively.

[0117] like Figure 9To evaluate the performance of AeroFineFusion-Det in small object detection, this application selected challenging images from the VisDrone2019 and UAVDT datasets and compared them with existing methods such as YOLOv8, RTMDet, and TOOD. Key detection regions are marked in the figures. These comparisons provide a more intuitive demonstration of the differences between AeroFineFusion-Det and other methods in terms of detection accuracy and small object localization.

[0118] Qualitative results show that AeroFineFusion-Det exhibits more stable small target detection capabilities in complex backgrounds and dense target scenes. Compared with baseline methods, this method demonstrates higher detection completeness for small-scale instances such as distant vehicles / pedestrians, significantly reduces missed detections in magnified local areas, and provides more compact bounding boxes for adjacent dense targets, suppressing duplicate boxes and false detections. Particularly in the far-end areas of road scenes and densely parked areas, this method maintains a more consistent confidence distribution and more coherent target coverage, indicating that the improved multi-scale representation is more conducive to the continuous identification and localization of small targets.

[0119] To further verify the model's response characteristics to the effective target region, Figure 10 Visualizations of the feature responses of AeroFineFusion-Det and baseline methods are presented. It can be observed that the baseline method exhibits strong but dispersed activations in large background areas (such as building outlines, vegetation, and areas of strong light / reflection), easily introducing responses irrelevant to the target. In contrast, AeroFineFusion-Det's activation distribution is more concentrated in roads and target clusters, with a significantly reduced high response to the background. The red-marked areas show that this method can reduce unnecessary attention to distant buildings and large-area textures, and form more continuous target responses in crowded areas, thus providing more discriminative feature support for classification and localization.

[0120] This embodiment demonstrates that by introducing the C2f_CMUNeXtB feature extraction module, the modeling capability of shallow features is enhanced; and by employing FS-BiFPN and DAWF modules for multi-scale feature fusion, the model's robustness to background noise is improved. The detection head uses CLAFHead, achieving high-precision small target localization through cross-layer local perceptual fusion and decoupled regression classification mechanisms. Experimental results on the VisDrone2019 and UAVDT datasets show that AeroFineFusion-Det outperforms existing methods in small target detection, particularly demonstrating significant advantages in localization accuracy and reduction of missed detections.

[0121] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter changes made to these embodiments within the spirit and principles of the present invention, without departing from the principles and spirit of the present invention, through conventional substitutions or to achieve the same function, fall within the scope of protection of the present invention.

Claims

1. A method for detecting small targets in UAV aerial photography based on fine-grained bidirectional multi-scale fusion, characterized in that, Includes the following steps: An AeroFineFusion-Det detection model based on YOLOv8 was constructed. The backbone network replaced the original C2f bottleneck structure with the C2f_CMUNeXtB module, and the neck network constructed a fine-grained bidirectional feature pyramid FS-BiFPN. Each fusion node of the FS-BiFPN was embedded with a dual attention weighted fusion DAWF module, and the detection head adopted a cross-layer local perception fusion head CLAFHead. The FS-BiFPN explicitly incorporates a high-resolution P2 feature layer, retains the bidirectional feature transfer path from top to bottom and bottom to top, and the P5 feature layer only participates in semantic guidance and cross-scale fusion, and is not used as the input feature of the prediction branch. The CLAFHead uses P2, P3, and P4 three-scale features to perform decoupled classification prediction and regression prediction.

2. The method for detecting small targets in UAV aerial photography based on fine-grained bidirectional multi-scale fusion according to claim 1, characterized in that, The dual-attention weighted fusion DAWF module performs channel attention recalibration, channel adjustment, and scale alignment operations on the K-way input features to obtain K-way aligned features. The formula for the alignment operation is: in, For the k-th input feature, For channel attention recalibration operation This is a 1×1 convolution operation. To scale features to the target scale The operation, , These represent the height and width of the target feature, respectively.

3. The method for detecting small targets in UAV aerial photography based on fine-grained bidirectional multi-scale fusion according to claim 2, characterized in that, The DAWF module calculates the fusion coefficients of each aligned feature based on learnable parameters, using the following formula: in, The number of input feature paths participating in the fusion. The learnable parameters corresponding to the k-th feature are... It is a non-linear activation function. To prevent the use of a pre-defined small constant with a denominator of zero, Let be the fusion coefficient of the k-th feature, satisfying and .

4. The method for detecting small targets in UAV aerial photography based on fine-grained bidirectional multi-scale fusion according to claim 3, characterized in that, The DAWF module performs weighted aggregation of the K-way alignment features based on the fusion coefficients to obtain the fused features. The specific formula is as follows: ,in, The k-th input feature is the feature after channel attention recalibration and scale alignment. Let be the fusion coefficient of the k-th feature. The number of input feature paths participating in the fusion. This refers to the fusion feature after weighted aggregation.

5. The method for detecting small targets in UAV aerial photography based on fine-grained bidirectional multi-scale fusion according to claim 4, characterized in that, The DAWF module performs fusion on the features. Apply spatial attention modulation to output the final fused features. The specific formula is as follows: in, For spatial attention operations, The location weight map generated for spatial attention. This is an element-wise multiplication operation. This is a 1×1 convolution operation. This is the final fusion feature output by the DAWF module.

6. The method for detecting small targets in UAV aerial photography based on fine-grained bidirectional multi-scale fusion according to claim 1, characterized in that, The C2f_CMUNeXtB module internally uses CMUNEXtBlock as the bottleneck unit. CMUNEXtBlock employs a residual connection structure for the input features. Perform residual mapping and output features The calculation formula is: in, For the input features of CMUneXtBlock, This is an incremental mapping operation that combines spatial updates and channel mapping. This is the output feature of CMUNEXtBlock.

7. The method for detecting small targets in UAV aerial photography based on fine-grained bidirectional multi-scale fusion according to claim 6, characterized in that, The incremental mapping It consists of a large kernel depthwise convolution space update, two 1×1 point convolution channel transformations, nonlinear activation, and normalized projection operations. The specific formula is as follows: in, For updating the space of large kernel depthwise convolutions. To increase the number of channels from Expansion to 1×1 point convolution operation, It is a non-linear activation function. To increase the number of channels from Compress back 1×1 point convolution operation, For normalization and projection operations, This represents the channel expansion ratio. This is the output of the incremental mapping operation.

8. The method for detecting small targets in UAV aerial photography based on fine-grained bidirectional multi-scale fusion according to claim 1, characterized in that, The CLAFHead employs a decoupled structure in its three-scale branches (P2, P3, and P4), introducing a cross-layer fusion unit CLLFusion only in the mid-scale P3 branch; the CLLFusion integrates high-resolution features... With low-resolution features As input, after channel alignment and scale alignment, we obtain and Then, the initial fusion features are obtained by adding them element by element. The specific formula is as follows: in, High-resolution features Features after channel alignment and scale alignment Low-resolution features Features after channel alignment and scale alignment The initial fused feature is obtained by adding the two features element by element.

9. The method for detecting small targets in UAV aerial photography based on fine-grained bidirectional multi-scale fusion according to claim 8, characterized in that, The CLLFusion concatenates the aligned two features along the channel dimension to generate a channel-gated vector. The specific formula is as follows: in, This is the result of splicing the two aligned features along the channel dimension. This is a global average pooling operation. For lightweight multilayer sensor operation. It is the Sigmoid activation function. This is the generated channel gating vector.

10. The method for detecting small targets in UAV aerial photography based on fine-grained bidirectional multi-scale fusion according to claim 9, characterized in that, The CLLFusion modulates the initial fused features based on the channel gating vector and outputs the final fused features. The specific formula is as follows: in, The initial fused features are obtained by adding the two feature streams element by element. For the generated channel gating vector, This is a channel-by-channel multiplication operation. For depthwise convolution operations, For point convolution operations, For batch normalization operations, It is a non-linear activation function. The final fusion feature output by the CLFusion unit.

Images