An aerial image small target detection method based on stage-wise adaptive feature transformation

Abstract

The application relates to a kind of aerial image small target detection methods based on stage adaptive feature transformation, for aerial image using small target detection network based on improved single-stage target detection framework to carry out detection;Through backbone network extraction multilevel image features, using progressive multi-scale feature aggregation module carries out cross-stage incremental enhancement, and using adaptive Inception hybrid module carries out geometric perception feature transformation to enhanced multi-scale features;Through light neck network to generate fusion detection features, light neck network only retains P3 detection branch and P4 detection branch, and uses multi-level attention fusion module to replace splicing operation, and uses progressive multi-scale feature aggregation module to replace convolution fusion layer;Fusion detection features are input into detection head, and the detection head output is post-processed to generate the final target detection result.Compared with the prior art, the application improves the missed detection problem of aerial image small target, and improves the robustness of target detection under complex aerial view.

Description

Technical Field

[0001] This invention relates to the fields of image processing and target detection technology, and in particular to a method for small target detection in aerial images based on staged adaptive feature transformation. Background Technology

[0002] Traditional target detection primarily relies on manually designed features and machine learning algorithms, such as sliding window methods and template matching techniques. These methods typically employ handcrafted features like Histogram of Oriented Gradients (HOG) and Scale-Invariant Feature Transform (SIFT) to describe the appearance characteristics of targets. However, when faced with complex backgrounds and multi-scale targets in aerial images, traditional algorithms suffer from limitations such as limited feature representation capabilities, high computational complexity, and sensitivity to environmental changes, making it difficult to meet the real-time and accuracy requirements of modern UAV applications.

[0003] Convolutional neural network-based detection methods, through end-to-end learning, can automatically extract rich hierarchical feature representations, significantly improving detection accuracy and robustness. Deep learning methods are mainly divided into two-stage detectors (such as Faster R-CNN) and single-stage detectors (such as YOLO and SSD series). Among them, single-stage detectors have been widely used in aerial photography due to their advantages of fast inference speed and high memory efficiency.

[0004] Although object detection and YOLOv11n are relatively mature in aerial small object detection, they still face many technical challenges and problems in practical applications. First, small objects in aerial images occupy a very small percentage of pixels, resulting in sparse feature information. This makes them prone to losing crucial details during multiple downsampling processes in deep networks, leading to high false negative rates. Second, traditional static convolutional kernels and feature fusion strategies lack adaptability to targets of different shapes, orientations, and scales, making it difficult to effectively handle the diverse target features in aerial scenes. Furthermore, semantic differences and information mismatches exist between feature layers of different scales, and simple feature concatenation or addition operations cannot fully utilize the complementarity of multi-scale information. Finally, while the P5 detection layer can handle large-scale targets, its computational resource consumption is disproportionate to its actual contribution in aerial scenes with dense small targets, affecting the overall efficiency of the algorithm.

[0005] In summary, while there is existing research on small target detection in aerial photography, there is still a lack of an effective method that can simultaneously alleviate the problems of easy feature loss, semantic mismatch between multi-scale features, interference from complex backgrounds, and the contradiction between lightweight models and detection accuracy. Existing methods struggle to balance real-time performance and computational efficiency while maintaining detection accuracy. In particular, there is a lack of an efficient target detection method specifically designed for UAV aerial photography scenarios that can adaptively enhance the representation of small target features, optimize multi-scale feature fusion, and reduce the computational overhead of redundant detection layers. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing UAV aerial small target detection technologies in terms of computational resource allocation, multi-scale feature representation, geometric shape adaptation, and multi-level feature fusion, and to provide an aerial image small target detection method based on staged adaptive feature transformation.

[0007] The objective of this invention can be achieved through the following technical solutions: A method for small target detection in aerial images based on staged adaptive feature transformation, comprising the following steps: Acquire the aerial image to be detected and preprocess the aerial image; The preprocessed aerial images are input into a small target detection network based on an improved single-stage target detection framework. The small target detection network extracts multi-level image features through a backbone network, which uses a progressive multi-scale feature aggregation module for cross-stage incremental enhancement; and an adaptive Inception hybrid module is used to perform geometric perception feature transformation on the enhanced multi-scale features. The output features of the backbone network are input into the lightweight neck network. The lightweight neck network replaces the splicing operation with a multi-level attention fusion module and replaces the traditional convolutional fusion layer with a progressive multi-scale feature aggregation module. Only the P3 detection branch and P4 detection branch for small-scale and medium-scale targets are retained. The outputs of the P3 detection branch and P4 detection branch are fused by the multi-level attention fusion module in combination with global prompts to obtain the final fused detection features. The fused detection features are input into the detection head, and the output of the detection head is post-processed to generate the final target bounding box, target category information, and target confidence score.

[0008] As a preferred technical solution, the progressive multi-scale feature aggregation module is specifically implemented as follows: The input feature map is segmented into two branches via a cross-stage local connection structure; The first sub-feature map is processed by multiple cascaded multi-scale feature aggregation units to obtain the first branch enhanced feature; the second sub-feature map retains its original features. The first branch enhanced feature is concatenated with the second sub-feature map through channels, and then channel fusion is performed through convolution. The fusion result is residually connected to the original input feature map of the progressive multi-scale feature aggregation module to obtain the output features of the progressive multi-scale feature aggregation module.

[0009] As a preferred technical solution, the multi-scale feature aggregation unit adopts a progressive feature partitioning strategy, specifically implemented as follows: For the input features of the multi-scale feature aggregation unit, a 3×3 convolution is performed to obtain local structural response features. The local structural response features are then segmented according to the channel dimension to obtain two sub-features. The first local structural response sub-feature enters the branch for subsequent medium-scale feature extraction, and the second local structural response sub-feature is used for the small-scale feature branch of the final aggregation. For the first local structural response sub-feature, perform 5×5 grouped convolution to obtain the medium-scale response feature. The medium-scale response feature is then divided into two sub-features according to the channel dimension. The first medium-scale response sub-feature enters the branch for subsequent large-scale feature extraction, and the second medium-scale response sub-feature is used for the final aggregated medium-scale feature branch. For the first medium-scale response sub-feature, perform 7×7 grouped convolution to obtain the large-scale contextual response feature; The second local structural response sub-feature, the second medium-scale response sub-feature, and the large-scale context response feature are concatenated along the channel dimension, and the number of channels is compressed by convolution to obtain the aggregated feature; The aggregated features are residually concatenated with the original input features of the multi-scale feature aggregation unit to obtain the final output of the multi-scale feature aggregation unit.

[0010] As a preferred technical solution, the adaptive Inception hybrid module adopts a two-level serial processing structure, including an adaptive Inception hybrid unit and a gated feature processing unit.

[0011] As a preferred technical solution, the adaptive Inception hybrid unit is specifically implemented as follows: The input features of the adaptive Inception mixing unit are divided into multiple channel groups along the channel dimension, and geometric perceptual kernel modulation is performed independently on each channel group, as follows: Three complementary depthwise convolution operations are performed on the sub-features of the channel group input: square depthwise convolution to capture compact local features; horizontal bar depthwise convolution to model horizontal structures; and vertical bar depthwise convolution to model vertical structures. Global average pooling is performed on the sub-features of the channel group input to obtain a global statistical vector; convolution is performed on the global statistical vector and it is reshaped into a 3C×1×1 shape, and the weights of three depth convolutions are generated through the Softmax operation; where C is the number of channels of the input feature of the adaptive Inception hybrid unit; The generated weights are used to weight and fuse the outputs of the three depthwise convolutions, and the feature distribution is stabilized by batch normalization. The outputs of the geometric perception kernels of each channel group are spliced ​​together along the channel dimension, and cross-channel information is fused through convolution to obtain the output of the adaptive Inception hybrid unit.

[0012] As a preferred technical solution, the kernel size of the square depth convolution is k×k; the kernel size of the horizontal strip depth convolution is 1×(3k+2); and the kernel size of the vertical strip depth convolution is (3k+2)×1; where k is a set kernel size parameter.

[0013] As a preferred technical solution, the gated feature processing unit uses the output of the adaptive Inception hybrid unit as the input feature, expands the input feature to a higher dimension through convolution, and divides the expanded feature into a spatial extraction branch and a gated branch along the channel dimension. The spatial extraction branch extracts the spatial response through a 3×3 depthwise convolution and generates gating coefficients for feature selection through an activation function. The gating coefficients are multiplied element-wise with the gating branches, then projected back to the target dimension via convolution, and residually connected with the original input features of the gating feature processing unit to obtain the final output of the gating feature processing unit.

[0014] As a preferred technical solution, the multi-level attention fusion module is specifically implemented as follows: Align the number of channels with the resolution of the two input features received by the multi-level attention fusion module to obtain the preprocessed features; The two preprocessed features are fed into two regional block context modeling branches for further processing. The output features of the four regional block context modeling branches are concatenated along the channel dimension, and the final output of the multi-level attention fusion module is obtained through multiple layers of continuous convolution operations.

[0015] As a preferred technical solution, the region block context modeling branch is specifically implemented as follows: The input features are rearranged and divided into multiple regions of a set size. For each region block, mean pooling is performed along the channel dimension to obtain a compact representation of the spatial features; The first layer of MLP maps the compact representation of the region block to an intermediate representation with a dimension half that of the output dimension; layer normalization is performed on the intermediate representation; the normalized features are extended to the final output dimension through the second layer of MLP to obtain the region block embedding feature matrix. Calculate the self-attention matrix of the region block features, and use the self-attention matrix to weight the region block embedding feature matrix to obtain the region block self-attention enhanced features; Calculate the cosine similarity between the features of each region block after self-attention enhancement and the global cue; perform element-wise weighting on the features after self-attention enhancement based on the cosine similarity, and then optimize the feature representation through a learnable linear transformation matrix to obtain the final output of the region block context modeling branch.

[0016] As a preferred technical solution, the multi-level attention fusion module adopts a reparameterized convolution mechanism and a progressive feature refinement strategy: for the splicing result of the output features of the four RPCM branches, the spliced ​​high-dimensional features are compressed back to the target dimension through 1×1 convolution, structured feature learning is performed through reparameterized 3×3 convolution, and feature refinement and output adjustment are achieved through the final 1×1 convolution.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1) This invention provides a small target detection method based on staged adaptive feature transformation for UAV aerial images. Through a continuous processing flow of lightweight decoding, cross-stage multi-scale enhancement, geometric perception feature transformation and multi-level attention fusion, it can perform staged optimization for problems such as small targets, dense distribution, large scale changes and complex backgrounds in aerial photography scenes.

[0018] 2) This invention proposes PMFA and MSFA, which enhance the expression of small target boundary, texture and context information by progressively aggregating features of different receptive fields such as 3×3, 5×5 and 7×7, thereby improving the problem of missed detection caused by the sparsity of small target features.

[0019] 3) This invention proposes AIMBlock and GAKM, which use dynamic weight fusion of square, horizontal bar and vertical bar convolution kernels to adaptively capture features of targets with different directions, different geometric shapes and different aspect ratios, thereby improving the detection robustness under complex aerial photography perspectives.

[0020] 4) This invention employs a lightweight neck network. By removing the P5 detection branch, computational resources are concentrated on the P3 and P4 detection layers, which are more suitable for small-scale and medium-scale targets. This reduces the number of parameters and computational load caused by redundant large target branches and reduces the loss of small target features caused by deep downsampling.

[0021] 5) This invention proposes MLAF and RPCM, which combine local region block context modeling with global prompting guidance fusion, to alleviate semantic mismatch between features at different levels and feature confusion between dense targets, and improve the discriminativeness of multi-scale feature fusion. Attached Figure Description

[0022] Figure 1 This is a flowchart of the aerial image small target detection method based on staged adaptive feature transformation according to the present invention.

[0023] Figure 2 This is a schematic diagram of the overall structure of the SAFT-DET small target detection network proposed in this invention, showing the lightweight structure with the P5 detection layer removed and the collaborative workflow of the PMFA, AIMBlock, and MLAF modules.

[0024] Figure 3 This is a schematic diagram of the PMFA module structure of the present invention.

[0025] Figure 4 This is a schematic diagram of the AIMBlock module structure of the present invention.

[0026] Figure 5 This is a schematic diagram of the GAKM mechanism structure of the present invention.

[0027] Figure 6 This is a schematic diagram of the MLAF module structure of the present invention.

[0028] Figure 7 This is a schematic diagram of the workflow of the RPCM mechanism of the present invention.

[0029] Figure 8 This diagram illustrates a comparison of the detection results of the method of the present invention with those of YOLOv8 and YOLOv11 in different drone aerial photography traffic scenarios.

[0030] Figure 9 This is a schematic diagram comparing the Grad-CAM feature activation heatmap of the method of the present invention with YOLOv8 and YOLOv11 in UAV aerial photography scenarios. Detailed Implementation

[0031] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0032] Example 1 This embodiment provides a small target detection method for aerial images based on staged adaptive feature transformation. Based on the YOLO series of single-stage target detection frameworks, and considering the characteristics of UAV aerial images such as a high proportion of small-scale targets, dense target distribution, strong background interference, and significant changes in target geometry and orientation, the method improves the decoding structure, feature enhancement structure, geometric perception feature transformation structure, and multi-level feature fusion structure of the detection network to form the SAFT-DET small target detection network. This method can improve the detection accuracy of small targets, dense targets, and targets with changing geometric shapes in UAV aerial scenes while reducing the number of model parameters and computational resources. It is suitable for scenarios such as UAV inspection, traffic monitoring, security monitoring, disaster relief, and remote sensing target recognition.

[0033] Specifically, this invention divides the small target detection process in UAV aerial photography into several interconnected feature transformation stages: First, by removing or eliminating the P5 detection branch for large-scale targets, a lightweight decoding structure for small- and medium-scale targets is formed; second, a Progressive Multi-Scale Feature Aggregation (PMFA) module is used to enhance image features under different receptive fields; then, an Adaptive Inception Mixer Block (AIMBlock) and Geometry-Aware Kernel Modulation (GAKM) mechanism are used to perform adaptive feature transformation on small targets of different shapes, orientations, and scales; finally, a Multi-Level Attention Fusion (MLAF) module and Regional Patch Context Modeling (RPCM) are used to fuse features at different levels, and the detection head is used to generate candidate target bounding boxes, class scores, and confidence scores; non-maximum suppression (NMS) post-processing is performed on the candidate detection results to output the final target bounding boxes, target class information, and target confidence scores.

[0034] like Figure 1 As shown, the method for small target detection in aerial images based on staged adaptive feature transformation has the following specific steps.

[0035] Step 1: Acquire aerial images of the drone to be detected and preprocess the drone aerial images.

[0036] Specifically, during the inference detection phase, the preprocessing includes image resizing, pixel value normalization, channel order adjustment, and batch processing format encapsulation to ensure that the input image meets the input format requirements of the small object detection network. During the training phase, the preprocessing may further include one or more methods such as random horizontal flipping, color perturbation, Mosaic enhancement, and MixUp enhancement to enhance the model's adaptability to different aerial photography altitudes, lighting conditions, target densities, and background complexity.

[0037] Step 2: Input the preprocessed UAV aerial images into the small target detection network, and extract multi-level image features through the backbone feature extraction part.

[0038] like Figure 2As shown, the SAFT-DET small target detection network includes a backbone feature extraction stage, a lightweight decoding stage, PMFA, AIMBlock, MLAF, and a detection output stage. These stages are not simply stacked, but rather address different technical bottlenecks in the UAV aerial small target detection process sequentially: the lightweight decoding stage reduces redundant computation caused by large-scale detection branches; the PMFA stage enhances the feature representation of small targets in different receptive fields; the AIMBlock stage improves the network's adaptive perception capability for targets with different geometric shapes and orientations; and the MLAF stage alleviates semantic mismatches between features at different levels and feature confusion between dense targets.

[0039] Specifically, the small target detection network can use a single-stage target detection network as its basic framework, extracting image features of different spatial resolutions and semantic levels from the input image through a backbone feature extraction part. The multi-level image features include P3 layer features suitable for small-scale target detection and P4 layer features suitable for medium-scale target detection; in the original three-scale detection structure, there may also be a P5 layer detection branch for large-scale targets, but this invention does not set or removes the P5 layer detection branch in the decoding and detection output structure.

[0040] Step 3: Construct a lightweight decoding structure.

[0041] Traditional three-scale detection structures typically have three detection branches: P3, P4, and P5, with the P5 branch primarily used for large-scale target detection. Considering the dominance of small- and medium-scale targets in UAV aerial images, this invention removes or eliminates the P5 branch (for large-scale targets) during the network structure design phase, while retaining the P3 branch (for small-scale targets) and the P4 branch (for medium-scale targets), thus forming a lightweight decoding structure.

[0042] In the lightweight decoding structure of this invention, the detection head performs target classification and bounding box regression based solely on features from layers P3 and P4, instead of setting up a separate large-scale detection output branch based on features from layer P5. This reduces the backbone connections, decoding processing, and detection head computation associated with the P5 detection branch, allowing computational resources to be more focused on feature representation and detection output for small and medium-scale aerial targets.

[0043] The improved network computational cost can be expressed as: In the formula, This indicates the computational cost of the lightweighted network. This indicates the computational cost of extracting the core features retained after lightweighting. This indicates the computational cost associated with the P3 detection branch; This indicates the computational cost associated with the P4 detection branch. This represents the total computational cost of the original three-scale detection network; and These represent the computational costs of the backbone and the head related to the P5 detection branch in the original three-scale detection network, respectively.

[0044] Through the aforementioned lightweight decoding structure, the network avoids allocating computational resources to low-resolution, large-scale detection branches, reduces the impact of redundant detection layers on small target detection tasks, and improves adaptability to drone aerial photography of dense small target scenes.

[0045] Step 4: Perform cross-stage multi-scale feature enhancement through the Progressive Multi-Scale Feature Aggregation (PMFA) module to obtain enhanced multi-scale features.

[0046] Traditional fixed-scale convolutional structures struggle to simultaneously capture local texture, edge contours, and broader contextual information of small targets. To address this, this invention introduces a PMFA module at key feature extraction locations to enhance feature representation at different scales while controlling computational overhead.

[0047] Specifically, such as Figure 3 As shown, the PMFA module will input the feature map. The structure is divided into Cross Stage Partial (CSP) segments. and Two branches, of which go through The first branch of enhanced features is obtained by processing a series of cascaded multi-scale feature aggregation (MSFA) units. Maintain the original features; combine the enhanced features of the first branch with... Perform channel splicing, and through Convolution is used to fuse channels, and then the fused result is combined with the input features. The residuals are summed to obtain the output features of the PMFA module.

[0048] 4.1) The output of the PMFA module is represented as follows: In the formula, Represents input features; and These represent the features of the first branch and the features of the second branch, respectively. Indicates the first branch passed through Features after processing by each MSFA unit; This indicates a channel splicing operation; This represents a 1×1 convolution operation.

[0049] 4.2) The Multi-Scale Feature Aggregation Unit (MSFA) employs a progressive feature partitioning strategy. Input features are first processed through a 3×3 convolution to obtain local structural responses, and then through 5×5 and 7×7 grouped convolutions to obtain medium- and large-scale contextual responses. Through the combination of 3×3, 5×5, and 7×7 convolution kernels, the MSFA unit output is represented as: in, This represents the large-scale features extracted through 7×7 grouped convolution. and These represent feature segmentation operations at different processing stages.

[0050] MSFA units can expand the receptive field coverage with lower computational overhead, enabling more complete expression of small target boundaries, local textures, and surrounding contextual information.

[0051] Step 5: Perform geometrically perceptual feature transformation on the enhanced multi-scale features using the adaptive Inception hybrid module AIMBlock.

[0052] like Figure 4 As shown, AIMBlock employs a two-stage serial processing structure, including an Adaptive Inception Mixer (AIMIxer) and a Gated Feature Processing Unit (GFPU).

[0053] 5.1) The first stage is the Adaptive Inception Hybrid Unit (AIMixer). This stage will input features... Divided along the channel dimension Channel group And perform Geometric Sensing Kernel Modulation (GAKM) processing on each channel group. The first-stage output can be represented as: In the formula, Indicates the first Group input sub-features; Indicates the first Group GAKM operations; This indicates that the outputs of each channel group are spliced ​​together; This represents a 1×1 convolution operation.

[0054] like Figure 5As shown, Geometric Aware Kernel Modulation (GAKM) sets three complementary convolutional kernel shapes, including a square depthwise convolutional kernel, a horizontal stripe depthwise convolutional kernel, and a vertical stripe depthwise convolutional kernel. Specifically, the three convolutional kernels can be set to... , and Among them, square depthwise convolutional kernels are used to capture compact local features, horizontal bar depthwise convolutional kernels are used to model horizontal structures, and vertical bar depthwise convolutional kernels are used to model vertical structures.

[0055] Let the input features be GAKM extracts global statistics from input features using global average pooling and generates normalized weight coefficients corresponding to three types of convolution kernels using 1×1 convolution. In the formula, This means reshaping the features into a 3C×1×1 shape; the Softmax operation ensures the reshaping of the three weights. The sum of these values ​​is 1, thus achieving weight allocation in a probabilistic sense.

[0056] Based on dynamically generated weight coefficients, GAKM performs weighted fusion of three different shapes of depthwise convolution operations: In the formula, Representing squares respectively Horizontal bar and vertical bars Depth convolution operation, This indicates batch normalization operation.

[0057] 5.2) The second stage is gated feature processing (GFPU). GFPU expands the input features to a higher dimension through 1×1 convolutions and divides the expanded features into a spatial extraction branch and a gated branch. The spatial extraction branch extracts the spatial response through 3×3 depthwise convolutions and activation functions, while the gated branch controls the selective passage of feature information. The extracted features are then projected back to the target dimension through 1×1 convolutions. GFPU can be represented as: In the formula, Indicates to use first Convolution expands the channels. This means that the expanded feature is divided equally along the channel: Extract branches for the space; For gated branches, Indicates the extraction of branches from the space. Do Depthwise separable convolution, express Convolutional projection.

[0058] Therefore, AIMBlock performs geometrically perceptual kernel modulation on the input features through the AIM stage, and then inputs the obtained intermediate features into GFPU for gating feature processing to form geometrically perceptual adaptive transformation features, thereby improving the network's ability to adapt to small targets of different shapes, directions and scales.

[0059] Step 6: Utilize the Multi-Level Attention Fusion (MLAF) module to perform region block context modeling on features at different levels through RPCM, and combine global prompts to guide fusion to obtain fused detection features.

[0060] This invention utilizes the MLAF module to achieve intelligent fusion and optimization of features at different scales by introducing a local-global attention mechanism and a multi-level feature interaction strategy.

[0061] 6.1) As Figure 6 As shown, MLAF receives input features and The input features are then refined using a multi-branch RPCM. The multi-level feature fusion process of MLAF can be represented as: In the formula, and These represent the block context modeling (RPCM) operations with patch sizes of 2 and 4, respectively, with the superscript indicating the corresponding patch size parameter.

[0062] 6.2) As Figure 7 As shown, RPCM will input feature maps Rearranged as The feature map is divided into a form of size [formula missing]. , where This is a configurable patch size parameter. For each patch, the module first performs mean pooling along the channel dimension to obtain a compact representation of the spatial features, and then performs nonlinear transformation and dimension adjustment through a two-layer MLP network, where the first MLP layer will... The patch representation of a dimension is mapped to The intermediate representation of the dimension is further extended by the second-layer MLP. The final representation of dimensional: In the formula, Indicates the first The spatial location of each channel The eigenvalue at that location.

[0063] 6.3) After obtaining the local patch feature representations, the RPCM module introduces a self-attention mechanism to enhance the intrinsic correlation and discriminative ability of the features. By calculating the similarity matrix between patch features, the module can identify and strengthen patch regions with similar semantic content, while suppressing irrelevant or conflicting feature information. The self-attention matrix can be represented as: In the formula, This represents the feature matrix of region block embedding. This represents the number of hidden layer channels. Through this self-attention matrix, RPCM can enhance the association between regions with similar semantic content and suppress interference from irrelevant background regions.

[0064] The self-attention matrix is ​​used to weight the region block embedding feature matrix to obtain the self-attention enhanced features: .

[0065] 6.4) RPCM can set a learnable global prompt, and evaluate the matching degree between each patch feature and the global prompt through cosine similarity calculation. The feature modulation process guided by the global prompt includes three key steps: similarity calculation, mask generation, and feature transformation. in, The global matching score is calculated using cosine similarity. The clamp function restricts the similarity value to the range [0,1] to ensure the effectiveness of the mask. It is a learnable linear transformation matrix used to further optimize feature representations under the guidance of a global prompt.

[0066] 6.5) To further optimize feature representation and improve the network's generalization ability, MLAF introduces a reparameterized convolution (RepConv) mechanism and a progressive feature refinement strategy. Reparameterized convolution enhances feature learning capabilities by using a multi-branch structure during training and merges it into a single convolution during inference to maintain efficiency. The final output of MLAF achieves feature dimensionality compression, structured processing, and final refinement through three consecutive convolution operations: In the formula, Responsible for compressing the concatenated high-dimensional features back to the target dimension. , Structured feature learning is achieved through reparameterized 3×3 convolutions. The final 1×1 convolution is then used to refine features and adjust the output.

[0067] Step 7: Input the fused detection features into the detection head, and use the detection head to generate candidate target bounding boxes, category scores, and confidence scores; perform non-maximum suppression (NMS) post-processing on the candidate detection results, and output the final target bounding boxes, target category information, and target confidence scores.

[0068] Specifically, such as Figure 2 As shown, after processing by the lightweight decoding structure, PMFA module, AIMBlock, and MLAF module, the network obtains fused detection features on the retained P3 and P4 detection branches, and inputs these fused detection features into the detection head. The detection head generates candidate target bounding boxes, class scores, and confidence scores; it performs non-maximum suppression (NMS) post-processing on the candidate detection results, and outputs the final target bounding box, target class information, and target confidence score.

[0069] This invention proposes a small target detection method based on staged adaptive feature transformation for UAV aerial images. By introducing a lightweight decoding structure, a PMFA cross-stage multi-scale feature enhancement module, an AIMBlock geometric perception feature transformation module, and a MLAF multi-level attention fusion module, it achieves efficient detection of small-scale targets, dense targets, and targets with geometrically changing shapes in UAV aerial images. This method can enhance the multi-scale feature representation capability of small targets while reducing the computational overhead of redundant detection branches, thereby improving the target detection accuracy in complex aerial photography scenarios.

[0070] To verify the effectiveness of the proposed method for small target detection in UAV aerial images based on staged adaptive feature transformation, this embodiment conducts experimental verification on several publicly available benchmark datasets for aerial target detection.

[0071] In terms of datasets, this embodiment uses three widely used aerial target detection benchmark datasets: VisDrone2019, UAVDT, and DOTA1.0.

[0072] In terms of training configuration, the model training epochs were set to 300, the batch size to 16, and the input image size was uniformly adjusted to 640×640. During training, the SGD optimizer was used, with an initial learning rate of 0.01, momentum of 0.937, and weight decay of 0.0005. Cosine annealing was employed to adjust the learning rate. For data augmentation, random horizontal flipping, color perturbation, Mosaic augmentation, and MixUp augmentation were used during the training phase. The probability of Mosaic augmentation was set to 1.0, and the probability of MixUp augmentation was set to 0.15.

[0073] Regarding evaluation metrics, this embodiment adopts standard evaluation metrics in the field of object detection, using Precision (P), Recall (R), mAP@50, and mAP@50-95 as the evaluation metrics for detection accuracy. Model complexity is measured by the number of parameters and GFLOPs. The test conditions are a batch size of 1, FP32 accuracy, and a 640×640 input size; all models are evaluated under the same conditions to ensure fair comparison.

[0074] Table 1 shows the ablation experiment results for each module of this invention. As can be seen from Table 1, after removing the P5 detection layer, the mAP50 of the model improved after adding PMFA, AIMBlock, and MLAF respectively. When PMFA, AIMBlock, and MLAF were added simultaneously, the mAP50 of the model reached 37.5, the number of parameters was 1.24M, and the FPS was 235.1. These results indicate that PMFA, AIMBlock, and MLAF can synergistically improve the detection accuracy of small targets in UAV aerial photography, while maintaining a low number of parameters and good inference efficiency.

[0075] Table 1. Module ablation experiment results after removing the P5 detection layer. This invention provides a comprehensive comparison with mainstream detectors, including the YOLO series (YOLOv5, YOLOv8, YOLOv9, YOLOv10, YOLOv12, YOLOv13) and advanced lightweight detectors (Hyper-YOLO, Mamba-YOLO, TA-YOLO, FFCA-YOLO, PP-PicoDet, URS-YOLO, and TPH-YOLO). Specific experimental comparison results can be found in Table 2.

[0076] Table 2 Comparison Experiments with Different Mainstream Models Table 2 shows that the SAFT-DET-n model of this invention achieves a 4.1 percentage point improvement in mAP50 and a 3.2 percentage point improvement in mAP50-95 compared to the YOLOv11n baseline model, while reducing the number of parameters by approximately 52%.

[0077] Figure 8 The results show that, based on the visualization results of traffic scenarios with four different perspectives and target densities, the model of this invention consistently outperforms the two baseline models in all scenarios, significantly reduces missed detections, achieves more accurate bounding box localization for multi-scale vehicles, and maintains robust performance under different perspectives and traffic densities.

[0078] To verify the cross-dataset generalization ability, this embodiment will further test the model trained on the VisDrone dataset on the UAVDT and DOTA datasets. Table 3 shows that compared with the baseline model, the mAP50 is improved by 1.52 percentage points and the mAP50-95 is improved by 2.82 percentage points on UAVDT; and the mAP50 is improved by 1.9 percentage points and the mAP50-95 is improved by 2.4 percentage points on DOTA.

[0079] Table 3 Generalization experiments on UAVDT and DOTA datasets Figure 9 The Grad-CAM feature activation heatmaps are presented in parking lot and nighttime urban scenes. The baseline model exhibits scattered and weak activation patterns, with insufficient response to individual vehicle features. The improved YOLOv11 baseline model shows improved focus but suffers from uneven intensity distribution and weak response to distant vehicles. The proposed method consistently generates strong and concentrated activation patterns, with prominent red-yellow regions; it generates independent and robust responses for each vehicle in dense scenes and maintains balanced activation across all targets under low-light conditions, thus validating the effectiveness of the staged adaptive feature transformation mechanism.

[0080] Example 2 As one embodiment of the present invention, the present invention also provides a small target detection system for UAV aerial images based on staged adaptive feature transformation. The system performs the method described in Embodiment 1 above, specifically including: The image input and preprocessing module is used to acquire the drone aerial image to be detected and to preprocess the drone aerial image. The backbone feature extraction module is used to input the preprocessed UAV aerial images into the small target detection network and extract multi-level image features through the backbone network; The lightweight decoding module is used to construct a lightweight decoding structure, removing the P5 detection branch for large-scale targets and retaining the P3 and P4 detection branches for small-scale and medium-scale targets. The PMFA module is used to perform cross-stage incremental enhancement on the multi-level image features to obtain enhanced multi-scale features. The AIMBlock module is used to perform geometrically perceptual feature transformation on the enhanced multi-scale features. The MLAF module is used to perform region block context modeling and global cue-guided fusion of features at different levels to obtain fused detection features; The detection output module is used to input the fused detection features into the detection head, which generates candidate target bounding boxes, category scores, and confidence scores. Non-maximum suppression (NMS) post-processing is performed on the candidate detection results to output the final target bounding boxes, target category information, and target confidence scores.

[0081] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for small target detection in aerial images based on staged adaptive feature transformation, characterized in that the steps include... include: Acquire the aerial image to be detected and preprocess the aerial image; The preprocessed aerial images are input into a small target detection network based on an improved single-stage target detection framework. The small target detection network extracts multi-level image features through a backbone network, which uses a progressive multi-scale feature aggregation module for cross-stage incremental enhancement; and an adaptive Inception hybrid module is used to perform geometric perception feature transformation on the enhanced multi-scale features. The output features of the backbone network are input into the lightweight neck network. The lightweight neck network replaces the splicing operation with a multi-level attention fusion module and replaces the traditional convolutional fusion layer with a progressive multi-scale feature aggregation module. Only the P3 detection branch and P4 detection branch for small-scale and medium-scale targets are retained. The outputs of the P3 detection branch and P4 detection branch are fused by the multi-level attention fusion module in combination with global prompts to obtain the final fused detection features. The fused detection features are input into the detection head, and the output of the detection head is post-processed to generate the final target bounding box, target category information, and target confidence score.

2. The method for small target detection in aerial images based on staged adaptive feature transformation according to claim 1, characterized in that, The progressive multi-scale feature aggregation module is implemented as follows: The input feature map is segmented into two branches via a cross-stage local connection structure; The first sub-feature map is processed by multiple cascaded multi-scale feature aggregation units to obtain the first branch enhanced feature; the second sub-feature map retains its original features. The first branch enhanced feature is concatenated with the second sub-feature map through channels, and then channel fusion is performed through convolution. The fusion result is residually connected to the original input feature map of the progressive multi-scale feature aggregation module to obtain the output features of the progressive multi-scale feature aggregation module.

3. The method for small target detection in aerial images based on staged adaptive feature transformation according to claim 2, characterized in that, The multi-scale feature aggregation unit adopts a progressive feature partitioning strategy, which is implemented as follows: For the input features of the multi-scale feature aggregation unit, a 3×3 convolution is performed to obtain local structural response features. The local structural response features are then segmented according to the channel dimension to obtain two sub-features. The first local structural response sub-feature enters the branch for subsequent medium-scale feature extraction, and the second local structural response sub-feature is used for the small-scale feature branch of the final aggregation. For the first local structural response sub-feature, perform 5×5 grouped convolution to obtain the medium-scale response feature. The medium-scale response feature is then divided into two sub-features according to the channel dimension. The first medium-scale response sub-feature enters the branch for subsequent large-scale feature extraction, and the second medium-scale response sub-feature is used for the final aggregated medium-scale feature branch. For the first medium-scale response sub-feature, perform 7×7 grouped convolution to obtain the large-scale contextual response feature; The second local structural response sub-feature, the second medium-scale response sub-feature, and the large-scale context response feature are concatenated along the channel dimension, and the number of channels is compressed by convolution to obtain the aggregated feature; The aggregated features are residually concatenated with the original input features of the multi-scale feature aggregation unit to obtain the final output of the multi-scale feature aggregation unit.

4. The method for small target detection in aerial images based on staged adaptive feature transformation according to claim 1, characterized in that, The adaptive Inception hybrid module adopts a two-level serial processing structure, including an adaptive Inception hybrid unit and a gated feature processing unit.

5. The method for small target detection in aerial images based on staged adaptive feature transformation according to claim 4, characterized in that, The adaptive Inception hybrid unit The input features of the adaptive Inception mixing unit are divided into multiple channel groups along the channel dimension, and geometric perceptual kernel modulation is performed independently on each channel group, as follows: Three complementary depthwise convolution operations are performed on the sub-features of the channel group input: square depthwise convolution to capture compact local features; horizontal bar depthwise convolution to model horizontal structures; and vertical bar depthwise convolution to model vertical structures. Global average pooling is performed on the input sub-features of the channel group to obtain a global statistical vector; The global statistical vector is convolved and reshaped into a 3C×1×1 shape, and the weights of three depthwise convolutions are generated through a Softmax operation; where C is the number of channels of the input features of the adaptive Inception hybrid unit. The generated weights are used to weight and fuse the outputs of the three depthwise convolutions, and the feature distribution is stabilized by batch normalization. The outputs of the geometric perception kernels of each channel group are concatenated along the channel dimension, and cross-channel information is fused through convolution to obtain the output of the adaptive Inception hybrid unit.

6. The method for small target detection in aerial images based on staged adaptive feature transformation according to claim 5, characterized in that, The kernel size of the square depthwise convolution is k×k; the kernel size of the horizontal bar depthwise convolution is 1×(3k+2); the kernel size of the vertical bar depthwise convolution is (3k+2)×1; where k is a set kernel size parameter.

7. The method for small target detection in aerial images based on staged adaptive feature transformation according to claim 4, characterized in that, The gated feature processing unit uses the output of the adaptive Inception hybrid unit as the input feature, expands the input feature to a higher dimension through convolution, and divides the expanded feature into a spatial extraction branch and a gated branch along the channel dimension. The spatial extraction branch extracts the spatial response through a 3×3 depthwise convolution and generates gating coefficients for feature selection through an activation function. The gating coefficients are multiplied element-wise with the gating branches, then projected back to the target dimension via convolution, and residually connected with the original input features of the gating feature processing unit to obtain the final output of the gating feature processing unit.

8. The method for small target detection in aerial images based on staged adaptive feature transformation according to claim 1, characterized in that, The multi-level attention fusion module is implemented as follows: Align the number of channels with the resolution of the two input features received by the multi-level attention fusion module to obtain the preprocessed features; The two preprocessed features are fed into two regional block context modeling branches for further processing. The output features of the four regional block context modeling branches are concatenated along the channel dimension, and the final output of the multi-level attention fusion module is obtained through multiple layers of continuous convolution operations.

9. The method for small target detection in aerial images based on staged adaptive feature transformation according to claim 8, characterized in that, The specific implementation of the region block context modeling branch is as follows: The input features are rearranged and divided into multiple regions of a set size. For each region block, mean pooling is performed along the channel dimension to obtain a compact representation of the spatial features; The first layer of MLP maps the compact representation of the region block to an intermediate representation with a dimension half that of the output dimension; layer normalization is performed on the intermediate representation; the normalized features are extended to the final output dimension through the second layer of MLP to obtain the region block embedding feature matrix. Calculate the self-attention matrix of the region block features, and use the self-attention matrix to weight the region block embedding feature matrix to obtain the region block self-attention enhanced features; Calculate the cosine similarity between the features of each region block after self-attention enhancement and the global cue; perform element-wise weighting on the features after self-attention enhancement based on the cosine similarity, and then optimize the feature representation through a learnable linear transformation matrix to obtain the final output of the region block context modeling branch.

10. The method for small target detection in aerial images based on staged adaptive feature transformation according to claim 8, characterized in that, The multi-level attention fusion module adopts a reparameterized convolution mechanism and a progressive feature refinement strategy: for the splicing result of the output features of the four RPCM branches, the spliced ​​high-dimensional features are compressed back to the target dimension through 1×1 convolution, structured feature learning is performed through reparameterized 3×3 convolution, and feature refinement and output adjustment are achieved through the final 1×1 convolution.

Images