An unmanned aerial vehicle aerial small target detection method and system based on scale-crossing feature fusion

Abstract

The application discloses a kind of UAV aerial small target detection method and system based on scale cross feature fusion, it is related to computer vision and UAV target detection technical field.The application includes: obtaining aerial image and preprocessing constructs training set;Extract multi-scale feature, realize high-dimensional mapping under low computing cost;SCFM module is designed, combined with space-channel collaborative attention and multi-scale random pooling attention, adjacent scale features are aligned and residual enhancement fusion;P2-P5 four-scale collaborative detection structure is constructed, P2 enhanced feature is generated by shallow high-resolution feature and deep semantic feature through SCFM cross fusion;The MBConv detection head of embedded SE attention is used to predict;Introduce QFL to realize classification and positioning joint optimization with IoU as quality soft label.The application reduces the parameter quantity and computing amount, and improves the detection precision and robustness of small, dense and occluded targets in complex aerial scene.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and UAV target detection technology, and in particular to a method and system for detecting small targets in UAV aerial photography based on scale cross-feature fusion. Background Technology

[0002] Unmanned aerial vehicle (UAV) visual target detection has played a crucial role in recent years in fields such as battlefield reconnaissance, intelligent transportation, precision agriculture, environmental monitoring, and emergency rescue, thanks to its advantages of high mobility, wide field of view, and flexible deployment. However, UAV aerial images are typically captured by airborne cameras during dynamic flight, facing severe challenges due to drastic changes in flight altitude and perspective, including extremely low target resolution (very small scale), drastic scale variations, dense target distribution, and susceptibility to interference from complex backgrounds. Traditional target detectors are primarily designed for natural images of medium to large-sized targets with sufficient computing power. When directly transferred to UAV platforms with limited computing resources, they inevitably encounter severe performance bottlenecks. Specifically, large models struggle to meet the constraints of limited airborne computing resources for real-time inference, while overly lightweight models often suffer severe performance degradation when facing small, dense, or partially occluded targets, leading to frequent false positives and false negatives.

[0003] In recent years, with the rapid development of deep learning technology, more and more researchers have begun to focus on solving these challenges in UAV vision, and have made significant progress in lightweight network design and feature enhancement. Regarding network architecture optimization, Liu et al. proposed ConvNeXt, which significantly enhanced feature representation capabilities while maintaining computational efficiency by rethinking the design paradigm of convolutional networks. Li et al. introduced EfficientViT, combining a lightweight self-attention mechanism with convolutional operations, achieving a better balance between accuracy and efficiency for high-resolution dense prediction tasks. To address the problem of insufficient global modeling capabilities, Carion et al. proposed DETR, transforming object detection into an ensemble prediction problem. Subsequently, Zhang et al. and Liu et al. successively introduced DINO-DETR and DN-DETR, significantly improving the training stability and localization accuracy of end-to-end detectors in small target scenarios by improving denoising training and matching strategies. Furthermore, for practical deployment and real-time requirements, Lyu et al. proposed RT-DETR, which significantly improved inference efficiency while retaining the advantages of Transformer; Wu et al. and Koonce et al. introduced RepViT and MobileViTv3, respectively, demonstrating good performance on mobile and embedded platforms. Meanwhile, the latest lightweight YOLO variants (such as MSUD-YOLO and CS-YOLO) are also widely used in the field of UAV vision in order to achieve a balance between detection accuracy and computational efficiency.

[0004] While existing deep learning and lightweight detection techniques have achieved some performance improvements on UAV platforms, they still face significant limitations in key target feature extraction, cross-scale feature fusion, and detection result reliability in increasingly complex application scenarios. First, some methods still rely on relatively complex network structures, resulting in high parameter counts and computational costs, which are unfavorable for real-time deployment on resource-constrained UAV edge platforms. Second, the fusion of shallow spatial detail features and deep semantic features often employs direct concatenation or simple weighting, easily introducing redundant background information. Furthermore, the semantic differences between features at different scales lack sufficient modeling, making it difficult to effectively enhance the features of small targets in densely distributed and occluded regions. Third, even when high-resolution shallow features are introduced, existing methods often lack scale-cross-attention constraints with deep semantic features, and the edge structure and fine-grained texture information of small targets may still degrade during continuous downsampling and feature transfer. Finally, traditional detection loss functions typically optimize classification confidence and bounding box localization quality in a relatively independent manner, making it difficult for predicted classification scores to accurately reflect the actual localization accuracy of candidate boxes, thus affecting the reliability of candidate box ranking during non-maximum suppression in dense scenes.

[0005] To address the aforementioned issues, proposing a method and system for small target detection in UAV aerial photography based on scale-cross feature fusion, while simultaneously considering lightweight feature extraction, scale-cross feature fusion, collaborative enhancement of shallow details and deep semantics, and optimization of classification and localization quality consistency under strict airborne computing power constraints, remains a key technical problem that urgently needs to be solved in the field of complex UAV aerial target detection. Summary of the Invention

[0006] In view of this, the present invention provides a method and system for detecting small targets in UAV aerial photography based on scale cross-feature fusion, which solves the problems of missed detection and false alarm caused by extremely small target scale, dense distribution, severe occlusion and complex background interference in UAV aerial images in the prior art. At the same time, it overcomes the real-time deployment bottleneck caused by the limited computing power of airborne edge devices. Through lightweight star-shaped feature extraction, scale cross-attention fusion, shallow detail and deep semantic collaborative enhancement and quality perception loss optimization, the detection accuracy and robustness of small targets, dense targets and occluded targets in complex aerial photography scenarios are improved.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A method for detecting small targets in UAV aerial photography based on scale-cross feature fusion includes the following steps: S1. Acquire drone aerial image data and preprocess it to build a target detection model training set; S2. The StarNet lightweight backbone network is used to extract multi-scale features from the preprocessed UAV aerial images and output multi-level original feature maps from shallow to deep layers. The StarNet lightweight backbone network has a built-in Star Block, which realizes the implicit mapping of features to high-dimensional space through star-shaped element-wise multiplication operations, so as to enhance the non-linear interaction capability and semantic expression capability of features with low computational cost. S3. Construct multi-scale feature paths based on multi-level original feature maps, and set up scale cross-feature fusion module SCFM in adjacent scale feature fusion nodes; after the scale cross-feature fusion module SCFM performs spatial resolution alignment on adjacent scale features, it generates cross-scale feature enhancement fusion attention weights, and applies them to the scale-aligned fusion features in a residual enhancement manner to obtain multi-scale enhanced fusion features. S4. Based on retaining the P5 deep semantic detection layer, the scale cross-feature fusion module SCFM is used to perform scale cross-attention fusion of shallow high-resolution features and upsampled deep semantic features to generate P2 enhanced detection features. The P2 enhanced detection features, together with the P3, P4 and P5 detection features, constitute the P2-P5 four-scale collaborative detection structure. S5. Input the detection features of the four scales P2, P3, P4 and P5 into the MBConv lightweight detection head for target prediction; S6. During the model training phase, Quality Focus Loss (QFL) is introduced, with the Intersection over Union (IoU) between the predicted bounding box and the ground truth bounding box used as a soft label for classification quality, and the classification confidence and bounding box localization quality are optimized simultaneously and jointly. S7. Input the drone aerial image to be detected into the trained target detection model, and output the category information and bounding box position of the small target captured by the drone aerial image.

[0008] Optionally, in S2, the StarNet lightweight backbone network adopts a four-stage hierarchical structure. In this structure, the Star Block sequentially performs 7×7 depthwise separable convolution, 1×1 point convolution for channel dimension expansion, star-shaped element-wise multiplication, and 1×1 point convolution for channel compression on the input feature map. Finally, the compressed output features are residually connected with the original input features to generate second-order enhanced features containing nonlinear interaction relationships.

[0009] Optionally, in the above method, the Scale Cross-Feature Fusion (SCFM) module in S3 includes a spatial-channel collaborative attention unit and a multi-scale random pooling attention unit. The spatial-channel collaborative attention unit extracts local spatial structure information through multi-scale depthwise separable convolution and generates spatial attention weights and channel attention weights respectively. The multi-scale random pooling attention unit performs random pooling response calculations at different pooling scales on the scale-aligned fused features and performs normalized weighted fusion of multiple pooling responses to obtain a scale-invariant attention map. The spatial attention weights, channel attention weights, and scale-invariant attention map are then fused to obtain the fused attention weights.

[0010] Optionally, in the above method, the generation process of P2 enhanced detection features in S4 includes: The C2 shallow high-resolution features are used as spatial detail inputs, and the deep semantic features obtained by upsampling C3, C4 or C5 features are used as contextual semantic inputs. After spatial resolution alignment of the spatial detail inputs and contextual semantic inputs, they are input into the scale cross-feature fusion module SCFM, and P2 enhanced detection features are obtained through scale cross-attention fusion.

[0011] Optionally, in the above method, in S5, the MBConv lightweight detection head includes a 1×1 channel extended convolutional layer, a 3×3 depthwise separable convolutional layer, an SE channel attention layer, and a 1×1 channel compressed convolutional layer connected in sequence; wherein, the SE channel attention layer obtains channel description vectors through global average pooling, and generates channel weights through fully connected layers and activation functions, and applies the channel weights to the feature map output by the depthwise separable convolutional layer.

[0012] In the above method, optionally, the detection features at the four scales P2, P3, P4 and P5 are respectively set with MBConv lightweight detection heads. The convolution parameters of each scale detection head are independent of each other, and are used to adapt to the detection feature distribution under different spatial resolutions and different target scales.

[0013] Optionally, in S6 of the above method, the Quality Focus Loss (QFL) uses the Intersection over Union (IoU) between the predicted bounding box and the ground truth bounding box as the localization quality score within the continuous interval [0,1] during positive sample supervision, and replaces the positive sample label value in the discrete category label with the localization quality score; during negative sample supervision, the quality label of the corresponding category is set to 0; a modulation factor is constructed by the difference between the classification prediction score and the quality label, so that the model can simultaneously optimize the category discrimination ability and the bounding box localization quality during training.

[0014] A small target detection system for UAV aerial photography based on scale cross-feature fusion, used to implement the small target detection method for UAV aerial photography based on scale cross-feature fusion described above, includes a data acquisition and preprocessing module, a lightweight multi-scale feature extraction module, a scale cross-feature fusion module, a P2-P5 four-scale collaborative detection structure, an MBConv lightweight detection head, a quality focus loss (QFL) optimization module, and a detection result output module connected in sequence. The data acquisition and preprocessing module is used to acquire drone aerial image data and perform preprocessing to build a training set for the target detection model. The lightweight multi-scale feature extraction module is used to extract multi-scale features from preprocessed UAV aerial images using the StarNet lightweight backbone network, and output multi-level original feature maps from shallow to deep layers. The StarNet lightweight backbone network has a built-in Star Block, which realizes the implicit mapping of features to high-dimensional space through star-shaped element-wise multiplication operations, so as to enhance the non-linear interaction capability and semantic expression capability of features with low computational cost. The Scale Cross-Feature Fusion (SCFM) module is used to construct multi-scale feature paths based on multi-level original feature maps and set up SCFM modules in adjacent scale feature fusion nodes. After spatial resolution alignment of adjacent scale features, the SCFM module generates fusion attention weights for cross-scale feature enhancement and applies them to the scale-aligned fusion features in a residual enhancement manner to obtain multi-scale enhanced fusion features. The P2-P5 four-scale collaborative detection structure is used to generate P2 enhanced detection features by using the scale cross-feature fusion module SCFM to fuse shallow high-resolution features with upsampled deep semantic features, while retaining the P5 deep semantic detection layer. The P2 enhanced detection features, together with the P3, P4 and P5 detection features, constitute the P2-P5 four-scale collaborative detection structure. The MBConv lightweight detection head is used to predict targets based on detection features at four scales: P2, P3, P4, and P5, achieving channel feature compression, key channel enhancement, and background redundancy suppression. The Quality Focus Loss (QFL) optimization module is used to introduce Quality Focus Loss (QFL) during the model training phase. It uses the Intersection over Union (IoU) between the predicted bounding box and the ground truth bounding box as a soft label for classification quality, and simultaneously optimizes the classification confidence and bounding box localization quality. The detection result output module is used to input the drone aerial image to be detected into the trained target detection model and output the category information and bounding box position of the small target captured by the drone aerial image.

[0015] In the aforementioned system, optionally, the Scale Cross-Feature Fusion (SCFM) module includes a spatial-channel collaborative attention unit and a multi-scale random pooling attention unit. The spatial-channel collaborative attention unit extracts local spatial structure information through multi-scale depthwise separable convolution and generates spatial attention weights and channel attention weights respectively. The multi-scale random pooling attention unit performs random pooling response calculations at different pooling scales on the scale-aligned fused features and performs normalized weighted fusion of multiple pooling responses to obtain a scale-invariant attention map. The spatial attention weights, channel attention weights, and scale-invariant attention map are then fused to obtain the fused attention weights.

[0016] In the aforementioned system, optionally, the MBConv lightweight detection head includes a 1×1 channel extended convolutional layer, a 3×3 depthwise separable convolutional layer, an SE channel attention layer, and a 1×1 channel compressed convolutional layer connected in sequence. The SE channel attention layer obtains channel description vectors through global average pooling and generates channel weights through a fully connected layer and an activation function, and applies the channel weights to the feature map output by the depthwise separable convolutional layer.

[0017] As can be seen from the above technical solution, compared with the prior art, the present invention provides a method and system for detecting small targets in UAV aerial photography based on scale cross-feature fusion, which has the following beneficial effects: (1) This invention uses StarNet as a lightweight backbone network. Through the star-shaped element-wise multiplication operation in Star Block, the implicit mapping of features to high-dimensional space is realized at a low computational cost, which enhances the nonlinear interaction capability and semantic expression capability between features. At the same time, compared with traditional heavy backbone networks, it can effectively reduce the number of model parameters and computational complexity, and is more suitable for the real-time deployment needs of resource-constrained edge devices such as drones.

[0018] (2) The Scale Cross-Feature Fusion (SCFM) module designed in this invention combines the spatial-channel collaborative attention mechanism with the multi-scale random pooling attention mechanism to perform resolution alignment, attention weight generation, and residual enhancement fusion on adjacent scale features. This module can strengthen the interaction between shallow spatial details and deep semantic information, effectively suppress redundant information in complex backgrounds, and improve the model's feature perception ability and detection robustness for densely distributed, heavily occluded, and small-scale targets.

[0019] (3) This invention constructs a four-scale collaborative detection structure from P2 to P5 while retaining the P5 deep semantic detection layer. The P2 detection features are not formed by directly inputting shallow high-resolution features into the detection head, but rather by fusing shallow high-resolution features output by StarNet with upsampled deep semantic features through scale-based cross-attention fusion via the SCFM module. The resulting enhanced P2 detection features simultaneously contain the edge structure, fine-grained texture, and contextual semantic information of small targets, effectively mitigating the problem of detail loss in small targets caused by continuous downsampling and improving the model's sensitivity and detection accuracy for extremely small-scale targets.

[0020] (4) This invention constructs a lightweight MBConv detection head with an embedded SE channel attention mechanism. Through point convolution channel expansion, depthwise separable convolution spatial feature extraction, SE channel recalibration, and point convolution compression dimensionality reduction, it achieves lightweight representation enhancement of detection features. This detection head can highlight key target channels and suppress background noise channels with low computational overhead, thereby improving the expression efficiency of multi-scale detection features.

[0021] (5) This invention introduces Quality Focus Loss (QFL) during the model training phase, using the Intersection over Union (IoU) between the predicted and ground truth bounding boxes as a soft label for classification quality. This allows the classification score output by the detection head to simultaneously represent the target class probability and the bounding box localization quality. This design alleviates the problem of inconsistency between traditional classification confidence and localization accuracy, improves the reliability of non-maximum suppression (NMS) candidate box ranking in dense UAV aerial photography scenarios, and further reduces false detections and false negatives. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0023] Figure 1 This is a flowchart of a method for detecting small targets in UAV aerial photography based on scale cross-feature fusion disclosed in this invention. Figure 2 This is a schematic diagram of the overall structure of the UAV aerial photography small target detection network disclosed in this invention; Figure 3 This is a schematic diagram of the StarNet lightweight backbone network and Star Block core unit structure disclosed in this invention; Figure 4 Schematic diagram of the Scale Cross-Feature Fusion Module (SCFM) and its internal components; Figure 5 This is a schematic diagram of the P2-P5 four-scale collaborative detection network architecture that generates P2 enhanced detection features via SCFM, as disclosed in this invention. Figure 6 This is a schematic diagram comparing the traditional target detection representation disclosed in this invention with the detection representation based on mass focus loss (QFL); Figure 7 This is a schematic diagram of the lightweight detection head structure of MBConv with embedded channel attention mechanism disclosed in this invention; Figure 8 This is a comparison chart of the precision and recall evaluation index curves of the model disclosed in this invention during the training process, wherein... Figure 8 (a) is a schematic diagram comparing the precision evaluation metric curve of the model during the training process with that of the mainstream baseline model. Figure 8 (b) A schematic diagram comparing the recall evaluation metric curve of the model during training with that of mainstream baseline models; Figure 9 This is a feature attention heatmap generated by the model disclosed in this invention in a complex UAV aerial photography scenario, wherein... Figure 9 (a) Feature attention heatmap generated for the YOLOv11n baseline model. Figure 9 (b) is the feature attention heatmap generated by the model of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] In this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0026] See Figure 1 and Figure 2 As shown, this invention discloses a method for detecting small targets in UAV aerial photography based on scale-cross feature fusion, comprising the following steps: S1. Acquire drone aerial image data and preprocess it, converting the ground truth bounding box labels corresponding to the drone aerial images into the annotation format required for target detection model training, and constructing a target detection model training set. S2. The StarNet lightweight backbone network is used to extract multi-scale features from the preprocessed UAV aerial images and output multi-level original feature maps from shallow to deep layers. The StarNet lightweight backbone network has a built-in Star Block, which realizes the implicit mapping of features to high-dimensional space through star-shaped element-wise multiplication operations, so as to enhance the non-linear interaction capability and semantic expression capability of features with low computational cost. S3. Construct a multi-scale feature path based on the multi-level original feature map, and set up a scale cross-feature fusion module (SCFM) in the adjacent scale feature fusion node; after the scale cross-feature fusion module (SCFM) aligns the spatial resolution of adjacent scale features, it combines the spatial-channel collaborative attention mechanism with the multi-scale random pooling attention mechanism to generate cross-scale feature enhancement fusion attention weights, and applies them to the scale-aligned fusion features in a residual enhancement manner to obtain multi-scale enhanced fusion features; S4. Based on retaining the P5 deep semantic detection layer, the scale-cross feature fusion module SCFM is used to perform scale-cross attention fusion between the shallow high-resolution features output by the StarNet lightweight backbone network and the upsampled deep semantic features to generate P2 enhanced detection features. The P2 enhanced detection features, together with the P3, P4 and P5 detection features, constitute the P2-P5 four-scale collaborative detection structure. Among them, the P2 enhanced detection features are not formed by directly accessing the detection head from the shallow features, but after fusing fine-grained texture, edge structure and contextual semantic information, they are combined with the P3, P4 and P5 detection features to form the P2-P5 four-scale collaborative detection structure. S5. Input the detection features of the four scales P2, P3, P4 and P5 into the MBConv lightweight detection head for target prediction; S6. During the model training phase, Quality Focus Loss (QFL) is introduced, with the Intersection over Union (IoU) between the predicted bounding box and the ground truth bounding box used as a soft label for classification quality, and the classification confidence and bounding box localization quality are optimized simultaneously and jointly. S7. Input the drone aerial image to be detected into the trained target detection model, and output the category information and bounding box position of the small target captured by the drone aerial image.

[0027] Furthermore, preprocessing in S1 includes image size unification, pixel normalization, and label coordinate format conversion; The multi-level original feature maps include C2, C3, C4 and C5 feature maps; the multi-scale feature paths include four-scale feature paths: P2, P3, P4 and P5.

[0028] Further, see Figure 3 In S2, the StarNet lightweight backbone network adopts a four-stage hierarchical structure, with each stage outputting feature maps whose spatial resolution gradually decreases and the number of channels gradually increases. Among them, Star Block performs 7×7 depthwise separable convolution, 1×1 point convolution for channel dimension expansion, star-shaped element-wise multiplication, and 1×1 point convolution for channel compression on the input feature map in sequence. Finally, the compressed output features are residually connected with the original input features to generate second-order enhanced features containing nonlinear interaction relationships.

[0029] Further, see Figure 4 In S3, the Scale Cross-Feature Fusion (SCFM) module includes a spatial-channel collaborative attention unit and a multi-scale random pooling attention unit. The spatial-channel collaborative attention unit extracts local spatial structure information through multi-scale depthwise separable convolution and generates spatial attention weights and channel attention weights respectively. The multi-scale random pooling attention unit performs random pooling response calculations at different pooling scales on the scale-aligned fused features and performs normalized weighted fusion of multiple pooling responses to obtain a scale-invariant attention map. The spatial attention weights, channel attention weights, and scale-invariant attention map are then fused to obtain the fused attention weights.

[0030] Further, see Figure 5 In S4, the generation process of P2 enhanced detection features includes: The C2 shallow high-resolution features are used as spatial detail input, while the deep semantic features obtained by upsampling C3, C4, or C5 features are used as contextual semantic input. After spatial resolution alignment, the spatial detail input and contextual semantic input are input into the Scale Cross-Feature Fusion (SCFM) module. P2 enhanced detection features are obtained through scale cross-attention fusion. The P2 enhanced detection features, together with the P3, P4, and P5 detection features, participate in detection prediction to form a four-scale collaborative detection structure that preserves both shallow details and deep semantics.

[0031] Further, see Figure 7 In S5, the MBConv lightweight detection head includes a 1×1 channel extended convolutional layer, a 3×3 depth-separable convolutional layer, an SE channel attention layer, and a 1×1 channel compressed convolutional layer connected in sequence. The SE channel attention layer obtains the channel description vector through global average pooling and generates channel weights through a fully connected layer and activation function. The channel weights are applied to the feature map output by the depth-separable convolutional layer to enhance the effective target channels and suppress background noise channels.

[0032] Furthermore, the detection features at the four scales of P2, P3, P4 and P5 are respectively set with MBConv lightweight detection heads. The convolution parameters of each scale detection head are independent of each other, which are used to adapt to the detection feature distribution under different spatial resolutions and different target scales.

[0033] Furthermore, the specific content of S6 is as follows: During the model training phase, the Quality Focus Loss (QFL) is combined to jointly optimize the classification confidence and localization quality. The Intersection over Union (IoU) between the predicted bounding box and the ground truth bounding box is introduced into the classification loss function as a soft label of the corresponding category, so that the classification score output by the detector head can simultaneously represent the target category probability and the bounding box localization quality.

[0034] Further, see Figure 6 In S6, the Quality Focus Loss (QFL) uses the Intersection over Union (IoU) between the predicted bounding box and the ground truth bounding box as the localization quality score within the continuous interval [0,1] during positive sample supervision, and replaces the positive sample label value in the discrete category label with the localization quality score; during negative sample supervision, the quality label of the corresponding category is set to 0; a modulation factor is constructed by the difference between the classification prediction score and the quality label, so that the model can simultaneously optimize the category discrimination ability and the bounding box localization quality during training, thereby improving the reliability of non-maximum suppression (NMS) candidate box ranking in dense aerial photography scenarios.

[0035] and Figure 1 Corresponding to the method described above, this invention also discloses a UAV aerial small target detection system based on scale cross-feature fusion, used to implement the UAV aerial small target detection method based on scale cross-feature fusion described above. The system includes, in sequence, a data acquisition and preprocessing module, a lightweight multi-scale feature extraction module, a scale cross-feature fusion module, a P2-P5 four-scale collaborative detection structure, an MBConv lightweight detection head, a quality focus loss (QFL) optimization module, and a detection result output module. The data acquisition and preprocessing module is used to acquire drone aerial image data and perform preprocessing to build a training set for the target detection model. The lightweight multi-scale feature extraction module is used to extract multi-scale features from preprocessed UAV aerial images using the StarNet lightweight backbone network, and output multi-level original feature maps from shallow to deep layers. The StarNet lightweight backbone network has a built-in Star Block, which realizes the implicit mapping of features to high-dimensional space through star-shaped element-wise multiplication operations, so as to enhance the non-linear interaction capability and semantic expression capability of features with low computational cost. The Scale Cross-Feature Fusion (SCFM) module is used to construct multi-scale feature paths based on multi-level original feature maps and set up SCFM modules in adjacent scale feature fusion nodes. After spatial resolution alignment of adjacent scale features, the SCFM module generates fusion attention weights for cross-scale feature enhancement and applies them to the scale-aligned fusion features in a residual enhancement manner to obtain multi-scale enhanced fusion features. The P2-P5 four-scale collaborative detection structure is used to generate P2 enhanced detection features by using the scale cross-feature fusion module SCFM to fuse shallow high-resolution features with upsampled deep semantic features, while retaining the P5 deep semantic detection layer. The P2 enhanced detection features, together with the P3, P4 and P5 detection features, constitute the P2-P5 four-scale collaborative detection structure. The MBConv lightweight detection head is used to predict targets based on detection features at four scales: P2, P3, P4, and P5, achieving channel feature compression, key channel enhancement, and background redundancy suppression. The Quality Focus Loss (QFL) optimization module is used to introduce Quality Focus Loss (QFL) during the model training phase. It uses the Intersection over Union (IoU) between the predicted bounding box and the ground truth bounding box as a soft label for classification quality, and simultaneously optimizes the classification confidence and bounding box localization quality. The detection result output module is used to input the drone aerial image to be detected into the trained target detection model and output the category information and bounding box position of the small target captured by the drone aerial image.

[0036] Furthermore, the Scale Cross-Feature Fusion (SCFM) module includes a spatial-channel collaborative attention unit and a multi-scale random pooling attention unit. The spatial-channel collaborative attention unit extracts local spatial structure information through multi-scale depthwise separable convolution and generates spatial attention weights and channel attention weights respectively. The multi-scale random pooling attention unit performs random pooling response calculations at different pooling scales on the scale-aligned fused features and performs normalized weighted fusion of multiple pooling responses to obtain a scale-invariant attention map. The spatial attention weights, channel attention weights, and scale-invariant attention map are then fused to obtain the fused attention weights.

[0037] Furthermore, the MBConv lightweight detection head includes a 1×1 channel extended convolutional layer, a 3×3 depthwise separable convolutional layer, an SE channel attention layer, and a 1×1 channel compressed convolutional layer connected in sequence. Among them, the SE channel attention layer obtains the channel description vector through global average pooling, and generates channel weights through fully connected layers and activation functions, and applies the channel weights to the feature map output by the depthwise separable convolutional layer.

[0038] See Figures 1 to 8This invention presents a method for small target detection in UAV aerial photography based on scale-cross feature fusion. The method comprises seven key steps: data acquisition and preprocessing, lightweight multi-scale feature extraction based on StarNet, scale-cross attention fusion based on SCFM, generation of P2-enhanced detection features through scale-cross fusion, lightweight detection prediction based on MBConv-SE, joint optimization of classification confidence and localization quality based on QFL, and output of detection results. Details are as follows: Step 1: Obtain the target data and construct the training set.

[0039] Aerial image data from drones at different flight altitudes, shooting angles, and in complex environments is collected. This data can originate from publicly available drone aerial small target datasets, such as the VisDrone2019 dataset and the TinyPerson dataset, or from self-built datasets collected by the drone platform. To ensure the object detection model can correctly interpret the supervision signals, the preprocessing stage converts the bounding box annotations provided in the original dataset into the annotation format required for the object detection model training. Image resizing, pixel normalization, and necessary data augmentation are then performed to construct the object detection model training set.

[0040] Step 2: Lightweight multi-scale feature extraction is performed using the StarNet backbone network.

[0041] An aerial image with an input resolution of 640×640 is fed into a four-stage hierarchical StarNet lightweight backbone network to extract multi-scale feature maps (C2, C3, C4, and C5) from shallow to deep layers. The shallow feature maps have high spatial resolution and contain rich edge structures and fine-grained texture information; the deep feature maps have strong semantic expressive power, providing target category and contextual semantic information. For an input dimension of... The feature map is processed by the core module StarBlock, which first performs a depthwise separable convolution with a kernel size of 7, and then expands the channels to 4 using a 1×1 point convolution. d This provides sufficient redundancy. Next, element-wise multiplication is performed using star schema operations. The formula for calculating a single-layer star schema is as follows:

[0042] Finally, the channel dimension is restored using a fully connected layer. d And add residual connections. This design implicitly achieves this without increasing the computational overhead of a single layer. Nonlinear interactions and feature mappings in dimensions.

[0043] Step 3: Scale cross-feature fusion and attention enhancement are performed using the SCFM module.

[0044] Based on the C2, C3, C4, and C5 multi-scale features output by StarNet, a four-scale feature path (P2, P3, P4, and P5) is constructed, and a scale-cross feature fusion module (SCFM) is set in the adjacent scale feature fusion node. The SCFM module is used to perform spatial resolution alignment, attention weight generation, and residual enhancement fusion on features of different scales. MCATTN employs a multi-scale random pooling strategy to generate a scaling-invariant attention map. Given input features... Its attention map The calculation formula is:

[0045] In the formula, Indices representing attention map scales. The average pooling function for the corresponding scale ( Set to 3). These are the normalized weighting coefficients. This mechanism significantly enhances the feature responses in heavily occluded and small regions.

[0046] Step 4: Generate P2 enhanced detection features using SCFM and construct a P2-P5 four-scale collaborative detection structure.

[0047] Building upon traditional multi-scale detection structures, this invention does not simply directly input shallow P2 features into the detection head. Instead, while retaining the P5 deep semantic detection layer, it uses the SCFM module to perform scale-cross attention fusion of shallow high-resolution features and deep semantic features to generate P2 enhanced detection features. Specifically, the shallow high-resolution features output by StarNet are used as spatial detail input, and the semantic features obtained by upsampling deep features are used as contextual semantic input. In the SCFM module, the two are spatially aligned, and fused attention weights are generated through spatial-channel collaborative attention and multi-scale random pooling attention. These fused attention weights are then applied to the fused features using residual enhancement, resulting in P2 enhanced detection features that combine edge structure, fine-grained texture, and deep contextual semantic information. Thus, the P2 enhanced detection features, together with the P3, P4, and P5 detection features, constitute the P2-P5 four-scale collaborative detection structure. This structure can maintain the advantages of shallow high resolution while introducing deep semantic constraints, effectively mitigating the detail degradation problem that occurs during continuous downsampling of small targets, and improving the model's sensitivity in locating extremely small-scale targets, dense targets, and occluded targets.

[0048] Step 5: Use the MBConv-SE lightweight detector head to perform channel feature compression and representation enhancement.

[0049] Input features into the detection head Spatial features are extracted sequentially using 1×1 expanded-dimensional convolutions and 3×3 depthwise separable convolutions. Then, SE channel attention is introduced, and the feature vector is obtained through global average pooling (GAP). Then, the weighted score is calculated. The calculation process is as follows: ; ; ; In the formula, For the input feature map, This is the channel description vector obtained from global average pooling. and These are the weight matrices for the channel compression layer and the channel recovery layer, respectively. For the first Channel attention weights For the weighted number of Each outputs a feature channel. Target-related features are enhanced and background noise is suppressed through channel recalibration. Finally, 1×1 point convolution is used to complete channel compression and feature fusion.

[0050] Step 6: Combine the Quality Focus Loss (QFL) to jointly optimize classification confidence and localization quality.

[0051] During training, the positive sample classification supervision signal is transformed from discrete values ​​into soft labels representing the intersection-over-union (IoU) ratio between the ground truth bounding boxes and the predicted bounding boxes. For classification confidence scores Its loss function is defined as: ; ; This mechanism effectively solves the problem of the disconnect between classification scores and localization quality, and improves the reliability of candidate box selection in the NMS stage.

[0052] Step 7: Results Output and Evaluation.

[0053] After network training is complete, the drone aerial images of the complex scene to be detected are input into the trained target detection model. The model sequentially undergoes StarNet lightweight multi-scale feature extraction, SCFM scale cross-attention fusion, P2-P5 four-scale collaborative detection, MBConv-SE lightweight detection head prediction, and QFL quality-aware optimization, ultimately outputting the category information and bounding box location of the small target. Simultaneously, the detection accuracy and lightweight deployment performance of the target detection model can be evaluated using metrics such as precision, recall, mean precision, model parameter count, computational cost, and inference speed.

[0054] In one specific embodiment, the experimental environment and dataset are as follows: The main parameters of the computing platform used in this invention are as follows: operating system: Ubuntu 22.04; processor: Intel(R) Xeon(R) Platinum 8255C @ 2.50 GHz; paired with an NVIDIA RTX 3090 graphics card with 24 GB of video memory. The programming language used is Python 3.12, and the deep learning framework is PyTorch 2.1.2 with CUDA 11.8 acceleration. Information on the public datasets used is shown in Table 1. Table 1 Experimental Dataset Information

[0055] Analysis of experimental results: To quantitatively analyze the performance differences of various detection methods in complex aerial photography scenarios, this embodiment adopts the generally accepted evaluation metrics in the field of object detection, including mean accuracy (mAP@0.5 and mAP@0.5:0.95) to evaluate detection accuracy, and parameter count (Params / M) and computational complexity (GFLOPs) to evaluate the lightweight nature of the model. The experiment compared this invention with classic two-stage methods such as Faster R-CNN, mainstream single-stage methods based on deep learning such as YOLOv5n, YOLOv8n, YOLOv10n, and YOLOv11n, and the latest improved methods for UAV scenarios such as EUAVDet-n, YOLOFLY, and Mamba-YOLO, on the VisDrone2019 dataset. The quantitative results are shown in Table 2 (higher accuracy scores are better, and lower parameter counts and computational complexity are better).

[0056] Table 2. Comparative experiments of various detection methods on the VisDrone2019 dataset.

[0057] Table 2 shows that while Faster R-CNN performs well on certain metrics in the classic two-stage approach, its high parameter count and computational complexity make it unsuitable for real-time deployment of UAV edge devices. Mainstream lightweight baseline models (such as YOLOv11n) have low computational cost, but their limited feature extraction capabilities result in a measurable mAP@0.5 of only 27.8% when dealing with densely packed small targets. Latest improved methods like Mamba-YOLO increase mAP@0.5 to 34.3%, but at the cost of a significant increase in parameter count and computational cost. In conclusion, this invention achieves the best balance between model accuracy and lightweight design, with mAP@0.5 and mAP@0.5:0.95 reaching maximum values ​​of 38.6% and 24.3% respectively. Simultaneously, the parameter count is drastically compressed to 1.9M, and the computational cost is only 5.5 GFLOPs, perfectly breaking the trade-off between "high accuracy" and "lightweight design."

[0058] To further verify the generalization ability and robustness of the present invention in extremely small targets and high-frequency dense scenes, this embodiment conducted cross-dataset generalization tests on the TinyPerson dataset, which is specifically designed for small pedestrian detection. The quantitative comparison results are shown in Table 3.

[0059] Table 3. Generalization comparison experiments of various detection methods on the TinyPerson dataset.

[0060] Table 3 shows that due to the extremely low target resolution and strong visual interference from ocean and land in the TinyPerson dataset, the mAP@0.5 of conventional lightweight models (such as YOLOv8n, YOLOv10n, and YOLOv11n) generally hovers between 11% and 12%, indicating a very high false negative rate. While some improved algorithms, such as LMF-UAV, improve accuracy to 15.8%, their high parameter count of 14.7M and computational cost of 61.8 GFLOPs negate their value for edge deployment. In contrast, the prediction results generated by this invention significantly outperform other methods in both overall accuracy and localization precision, with mAP@0.5 jumping to 20.7% and mAP@0.5-0.95 reaching 9.85%. This demonstrates that the proposed SCFM-generated P2 enhanced detection feature mechanism and scale cross-fusion structure can improve the recognition ability of small targets in complex backgrounds.

[0061] Evaluation of model training convergence and stability. To visually demonstrate the learning dynamics of this invention during the training phase, such as... Figure 8As shown, this embodiment extracts the precision and recall evaluation metric curves of the model during training and compares them with mainstream baseline models. Observation Figure 8 The curve trend shows that in the early stage of training, the performance of the model of this invention shows a rapid upward trend, and gradually stabilizes in the middle and late stages, demonstrating excellent optimization stability and convergence characteristics.

[0062] Visual analysis of target perception and attention in complex scenarios. To further qualitatively verify the feature extraction advantages of the model in actual UAV deployment, such as... Figure 8 As shown, this embodiment extracts and compares the feature attention heatmaps and detection results generated by the baseline model and the model of this invention in complex UAV aerial photography scenarios. The experimental verification scenarios cover extreme visual challenge environments such as multi-scale dense target distribution, severe building occlusion, and low light at night.

[0063] Depend on Figure 9 The heatmap response distribution shows that traditional baseline models are easily affected by complex aerial perspectives when extracting features, resulting in a large number of scattered and detached background activation responses, and it is difficult to form continuous high-response hot zones in small target areas. In contrast, the attention response generated by the model of this invention is highly convergent, and can accurately focus on the edges of multi-scale target entities (such as dense groups of vehicles and extremely small-scale pedestrians), while irrelevant background noise is significantly suppressed.

[0064] In summary, the proposed method and architecture for small target detection in UAV aerial photography based on scale cross-feature fusion significantly outperforms existing methods in terms of completeness and overall detection quality, generating the most accurate and robust target perception model. In particular, it demonstrates high engineering application value and technological advancement when dealing with extremely small and heavily occluded areas that are generally considered the most difficult to recover in aerial vision.

[0065] The present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement, for example... Figure 1 The steps of a UAV aerial photography small target detection method based on scale cross-feature fusion are shown.

[0066] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting small targets in UAV aerial photography based on scale-cross feature fusion, characterized in that, Includes the following steps: S1. Acquire drone aerial image data and preprocess it to build a target detection model training set; S2. The StarNet lightweight backbone network is used to extract multi-scale features from the preprocessed UAV aerial images and output multi-level original feature maps from shallow to deep layers. The StarNet lightweight backbone network has a built-in Star Block, which realizes the implicit mapping of features to high-dimensional space through star-shaped element-wise multiplication operations, so as to enhance the non-linear interaction capability and semantic expression capability of features with low computational cost. S3. Construct multi-scale feature paths based on multi-level original feature maps, and set up scale cross-feature fusion module SCFM in adjacent scale feature fusion nodes; scale After the cross-scale feature fusion module aligns the spatial resolution of adjacent scale features, it generates fusion attention weights for cross-scale feature enhancement and applies them to the scale-aligned fusion features in a residual enhancement manner to obtain multi-scale enhanced fusion features. S4. Based on retaining the P5 deep semantic detection layer, the scale cross-feature fusion module SCFM is used to perform scale cross-attention fusion of shallow high-resolution features and upsampled deep semantic features to generate P2 enhanced detection features. The P2 enhanced detection features, together with the P3, P4 and P5 detection features, constitute the P2-P5 four-scale collaborative detection structure. S5. Input the detection features of the four scales P2, P3, P4 and P5 into the MBConv lightweight detection head for target prediction; S6. During the model training phase, Quality Focus Loss (QFL) is introduced, with the Intersection over Union (IoU) between the predicted bounding box and the ground truth bounding box used as a soft label for classification quality, and the classification confidence and bounding box localization quality are optimized simultaneously and jointly. S7. Input the drone aerial image to be detected into the trained target detection model, and output the category information and bounding box position of the small target captured by the drone aerial image.

2. The method for detecting small targets in UAV aerial photography based on scale-cross feature fusion according to claim 1, characterized in that, In S2, the StarNet lightweight backbone network adopts a four-stage hierarchical structure. The Star Block sequentially performs 7×7 depthwise separable convolution, 1×1 point convolution for channel dimension expansion, star-shaped element-wise multiplication, and 1×1 point convolution for channel compression on the input feature map. Finally, the compressed output features are residually connected with the original input features to generate second-order enhanced features containing non-linear interaction relationships.

3. The method for detecting small targets in UAV aerial photography based on scale cross-feature fusion according to claim 1, characterized in that, In S3, the scale cross-feature fusion module SCFM includes a spatial-channel collaborative attention unit and a multi-scale random pooling attention unit; The spatial-channel collaborative attention unit extracts local spatial structure information through multi-scale depthwise separable convolution and generates spatial attention weights and channel attention weights respectively. The multi-scale random pooling attention unit performs random pooling response calculations at different pooling scales on the scale-aligned fused features, and performs normalized weighted fusion of multiple pooling responses to obtain a scale-invariant attention map; the spatial attention weights, channel attention weights and scale-invariant attention map are fused to obtain fused attention weights.

4. The method for detecting small targets in UAV aerial photography based on scale cross-feature fusion according to claim 1, characterized in that, In S4, the generation process of P2 enhanced detection features includes: The C2 shallow high-resolution features are used as spatial detail inputs, and the deep semantic features obtained by upsampling C3, C4 or C5 features are used as contextual semantic inputs. After spatial resolution alignment of the spatial detail inputs and contextual semantic inputs, they are input into the scale cross-feature fusion module SCFM, and P2 enhanced detection features are obtained through scale cross-attention fusion.

5. The method for detecting small targets in UAV aerial photography based on scale-cross feature fusion according to claim 1, characterized in that, In S5, the MBConv lightweight detection head includes a 1×1 channel extended convolutional layer, a 3×3 depthwise separable convolutional layer, an SE channel attention layer, and a 1×1 channel compressed convolutional layer connected in sequence. The SE channel attention layer obtains channel description vectors through global average pooling and generates channel weights through fully connected layers and activation functions. The channel weights are then applied to the feature map output by the depthwise separable convolutional layer.

6. The method for detecting small targets in UAV aerial photography based on scale cross-feature fusion according to claim 5, characterized in that, The detection features at the four scales P2, P3, P4 and P5 are respectively set with MBConv lightweight detection heads. The convolution parameters of each scale detection head are independent of each other, and are used to adapt to the detection feature distribution under different spatial resolutions and different target scales.

7. The method for detecting small targets in UAV aerial photography based on scale cross-feature fusion according to claim 1, characterized in that, In S6, the Quality Focus Loss (QFL) uses the Intersection over Union (IoU) between the predicted bounding box and the ground truth bounding box as the localization quality score within the continuous interval [0,1] during positive sample supervision, and replaces the positive sample label value in the discrete category label with the localization quality score; during negative sample supervision, the quality label of the corresponding category is set to 0; a modulation factor is constructed by the difference between the classification prediction score and the quality label, so that the model can simultaneously optimize the category discrimination ability and the bounding box localization quality during training.

8. A small target detection system for UAV aerial photography based on scale-cross feature fusion, characterized in that, To implement the UAV aerial photography small target detection method based on scale cross-feature fusion as described in any one of claims 1-7, the method comprises, in sequence, a data acquisition and preprocessing module, a lightweight multi-scale feature extraction module, a scale cross-feature fusion module, a P2-P5 four-scale collaborative detection structure, an MBConv lightweight detection head, a mass focus loss QFL optimization module, and a detection result output module. The data acquisition and preprocessing module is used to acquire drone aerial image data and perform preprocessing to build a training set for the target detection model. The lightweight multi-scale feature extraction module is used to extract multi-scale features from preprocessed UAV aerial images using the StarNet lightweight backbone network, and output multi-level original feature maps from shallow to deep layers. The StarNet lightweight backbone network has a built-in Star Block, which realizes the implicit mapping of features to high-dimensional space through star-shaped element-wise multiplication operations, so as to enhance the non-linear interaction capability and semantic expression capability of features with low computational cost. The Scale Cross-Feature Fusion (SCFM) module is used to construct multi-scale feature paths based on multi-level original feature maps, and sets the SCFM module in adjacent scale feature fusion nodes; scale After the cross-scale feature fusion module aligns the spatial resolution of adjacent scale features, it generates fusion attention weights for cross-scale feature enhancement and applies them to the scale-aligned fusion features in a residual enhancement manner to obtain multi-scale enhanced fusion features. The P2-P5 four-scale collaborative detection structure is used to generate P2 enhanced detection features by using the scale cross-feature fusion module SCFM to fuse shallow high-resolution features with upsampled deep semantic features, while retaining the P5 deep semantic detection layer. The P2 enhanced detection features, together with the P3, P4 and P5 detection features, constitute the P2-P5 four-scale collaborative detection structure. The MBConv lightweight detection head is used to predict targets based on detection features at four scales: P2, P3, P4, and P5, achieving channel feature compression, key channel enhancement, and background redundancy suppression. The Quality Focus Loss (QFL) optimization module is used to introduce Quality Focus Loss (QFL) during the model training phase. It uses the Intersection over Union (IoU) between the predicted bounding box and the ground truth bounding box as a soft label for classification quality, and simultaneously optimizes the classification confidence and bounding box localization quality. The detection result output module is used to input the drone aerial image to be detected into the trained target detection model and output the category information and bounding box position of the small target captured by the drone aerial image.

9. A small target detection system for UAV aerial photography based on scale cross-feature fusion according to claim 8, characterized in that, The Scale-Cross Feature Fusion (SCFM) module includes a spatial-channel collaborative attention unit and a multi-scale random pooling attention unit. The spatial-channel collaborative attention unit extracts local spatial structure information through multi-scale depthwise separable convolution and generates spatial attention weights and channel attention weights respectively. The multi-scale random pooling attention unit performs random pooling response calculations at different pooling scales on the scale-aligned fused features, and performs normalized weighted fusion of multiple pooling responses to obtain a scale-invariant attention map; the spatial attention weights, channel attention weights and scale-invariant attention map are fused to obtain fused attention weights.

10. A small target detection system for UAV aerial photography based on scale cross-feature fusion according to claim 8, characterized in that, The MBConv lightweight detection head consists of a 1×1 channel extended convolutional layer, a 3×3 depthwise separable convolutional layer, an SE channel attention layer, and a 1×1 channel compressed convolutional layer connected in sequence. The SE channel attention layer obtains channel description vectors through global average pooling and generates channel weights through a fully connected layer and activation function. The channel weights are then applied to the feature map output by the depthwise separable convolutional layer.

Images