A lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention

Abstract

This invention discloses a lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention. First, multi-scale feature extraction based on stacked phantom-guided reparameterization is performed on the remote sensing image. Frequency-domain guided gated edge enhancement is employed, and global context modeling is performed on the enhanced feature map based on a multi-head self-attention mechanism. After residual connection and layer normalization, the enhanced feature map is fed into a feedforward network for nonlinear transformation, outputting globally encoded high-dimensional features. A target query based on self-attention information interaction is then performed, interacting the target query with the high-dimensional features based on cross-attention. Lightweight multidimensional attention is then used to perform channel and spatial weighting on the interacted high-dimensional features, mapping them to target category and bounding box coordinates via a prediction head, outputting the final detection result. This method addresses problems such as complex backgrounds, large target scale differences, low pixel ratio of small targets, and inherent speckle noise in synthetic aperture radar images.

Description

Technical Field

[0001] This invention relates to the field of small target detection technology in remote sensing images, and in particular to a lightweight method for small target detection in remote sensing based on edge enhancement and multidimensional attention. Background Technology

[0002] With the rapid development of satellite and drone technologies, the acquisition of high-resolution remote sensing images has become increasingly convenient, playing an irreplaceable role in fields such as military reconnaissance, disaster assessment, smart cities, and traffic monitoring. As one of the core tasks of remote sensing image interpretation, target detection aims to accurately locate and identify targets of interest from complex image scenes. Remote sensing target detection has evolved from traditional methods to deep learning methods. Early methods, such as CFAR-based detection, relied on assumptions about the statistical distribution of background clutter, resulting in poor adaptability to complex scenes. Template matching-based methods were sensitive to changes in target pose and scale. The rise of deep learning has brought revolutionary changes to this field. Two-stage methods, such as Faster R-CNN and Cascade R-CNN based on candidate regions, generate candidate boxes through a region proposal network, followed by secondary classification and regression, achieving high accuracy but slow speed. Single-stage methods, represented by the YOLO series, treat detection as a regression problem, directly predicting bounding boxes and categories, achieving fast end-to-end detection, and have become the most widely used paradigm.

[0003] However, compared to natural scene images, existing detection methods are more difficult for target detection in remote sensing images. Complex backgrounds and speckle noise cause target edges to become blurred and textures to be lost. Target scales vary drastically, from large targets with thousands of pixels to small targets with only tens of pixels, requiring models to have strong multi-scale feature representation capabilities. Small targets are easily lost during downsampling, and weak targets are blurred from the background due to low contrast. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the purpose of this invention is to provide a lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention, which aims to solve the problems of existing small target detection methods being unable to integrate redundant background noise and struggling to represent multi-scale features when the target scale changes drastically, resulting in low small target detection accuracy.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention, the detection method includes: Step 1: Acquire remote sensing images, perform multi-scale feature extraction based on stacked phantom-guided reparameterization on the remote sensing images to obtain multi-level feature maps, use frequency domain-guided gated edge enhancement to enhance details of the multi-level feature maps, and obtain enhanced feature maps. Step 2: Based on the multi-head self-attention mechanism, perform global context modeling on the enhanced feature map, and then feed it into the feedforward network for nonlinear transformation after residual connection and layer normalization, outputting high-dimensional features after global encoding. Step 3: Based on self-attention, acquire target query information interaction, interact the target query with high-dimensional features based on cross-attention, and then perform channel and spatial weighting processing on the high-dimensional features after interaction through lightweight multi-dimensional attention to obtain lightweight target query features. These features are then mapped to target category and bounding box coordinates by the prediction head, and the final detection result is output to realize small target detection in remote sensing images.

[0006] Furthermore, in step 1, multi-scale feature extraction based on stacked phantom-guided reparameterization is performed on the remote sensing image. The phantom convolution concept is used to generate compact intrinsic features, which are then input into the remote sensing image. m intrinsic feature maps are generated through 1×1 convolution, denoted as m. This reduces the computational cost of standard convolution to only a fraction of its original value. .

[0007] Furthermore, regarding intrinsic characteristics Multi-scale reparameterization enhancement is performed using three parallel reparameterized convolutional branches, corresponding to dilation rates r=1, 3, and 6, respectively. Each branch consists of two parallel sub-branches: a 3×3 set of convolutions and a 1×1 point convolution. The outputs are element-wise summed to obtain multi-receptive field features. Represented as: ; Where * denotes convolution. For a 3×3 convolution kernel with an inflation rate of r, The kernel is 1×1, and BN is batch normalization. During the inference phase, the two branches are merged into a single 3×3 dilated convolution through structural reparameterization to maintain efficient inference. A linear transformation is used to absorb the two BN layers into the convolutional kernel, merging them into a single convolution. ; The outputs of the three branches are added element-wise to obtain the enhanced multi-receptor field feature. , is represented as: .

[0008] Furthermore, an adaptive channel weighting mechanism is introduced, which first considers multi-receptive field features. Spatial information is aggregated through global average pooling to generate channel descriptors. : ; Subsequently, inter-channel dependencies are captured through two fully connected layers, and channel weights are generated by Sigmoid activation. : ; Where δ is ReLU and σ is Sigmoid; finally, the weights s are... Channel-by-channel multiplication yields the recalibrated enhanced multi-receptive field features: ; To restore the target number of channels, Phantom expansion is performed: a 3×3 depthwise convolution is applied to each channel to generate s-1 phantom features, which are then concatenated with the original channels to obtain a total of m×s feature maps; finally, a 1×1 convolution is used to adjust the channels to... The final output Y is obtained, where the phantom expansion factor s=2 is set, and a phantom feature is generated for each intrinsic channel, resulting in a multi-level feature map.

[0009] Furthermore, in the frequency-domain guided gated edge enhancement in step 1, the input multi-level feature map is mapped to the frequency domain using a fast Fourier transform to obtain the spectrum. ; through a set of predefined binary masks The spectrum is divided to separate different frequency components, as shown below: ; After inverse transformation, spatial feature groups corresponding to different frequency bands are obtained. To perform adaptive frequency fusion for different spatial locations, the module learns a spatial selection feature map (SM), which generates spatial weights corresponding to the feature maps of each frequency band. Selectively enhance high-frequency components in edge regions of an image; weighted fusion of features Represented as: .

[0010] Furthermore, in the frequency-domain guided gating edge enhancement of step 1, a lightweight gating attention unit is used. First, the weighted fused features are... Channel attention is applied, and channel weights are generated through global average pooling and a multilayer perceptron. And perform channel reweighting. Subsequently, a 3×3 depthwise separable convolution is used to extract the spatial edge response, and a spatial gate mask G is generated using the Sigmoid function, expressed as: ; Ultimately, gated enhancement features Depend on Element-wise multiplication with G yields, i.e. This enables precise focusing on key edge areas; To capture multi-scale contextual information related to edges, lightweight multi-scale context aggregation and output employ a set of parallel, depth-separable convolutional processes with dilation. The void ratio is set to The outputs of each branch are spliced ​​along the channel dimension to form multi-scale features. ; through a 1×1 standard convolution pair Perform fusion and dimensionality reduction, and combine with the original input. Perform residual connections to output enhanced feature maps: .

[0011] Furthermore, in step 3, lightweight multidimensional attention is used to perform channel and spatial weighting on the high-dimensional features after interaction. The lightweight multidimensional attention first adopts spatial channel co-calibration and uses grouped convolution and dual pooling strategies to achieve a lightweight design. For the input high-dimensional features, channel descriptors are first generated through global average pooling, then summed after passing through two shared fully connected layers, and finally the channel attention weights are obtained through the Sigmoid function. Simultaneously, the maximum and average values ​​of the high-dimensional input features are taken along the channel dimension, concatenated, and then processed through a... Grouped convolution generates spatial attention weights , is represented as: ; in This indicates element-wise multiplication.

[0012] Furthermore, the lightweight multidimensional attention in step 3 also employs a local-global interaction mechanism, introducing deformable sampling and a simplified state-space model to achieve dynamic interaction between local details and the global context; firstly, for Each spatial location in the image is predicted using a lightweight subnetwork. Each sampling offset and their corresponding attention weights Subsequently, bilinear interpolation was used to extract features from the sampling points, and weighted summation was used to obtain the local focusing features. , is represented as: ; At the same time, in order to capture global dependencies, Unfolding along the spatial dimension into a sequence, and then performing linear recursion through depthwise separable convolutions to obtain global context features. The complexity of this process is only And it is accelerated through parallel scanning; Finally, the local focused features and global features are added and fused to obtain the fused lightweight target query features: .

[0013] The technical solution adopted in this invention has the following beneficial effects: This invention addresses the challenges of complex backgrounds, noise interference, large target scale differences, and limited computational resources in small target detection of remote sensing images. It designs a Frequency Domain Guided Gated Edge Enhancement Module (FGEM), which uses Fast Fourier Transform to suppress noise and enhance texture in the frequency domain, combined with a gated attention mechanism to sharpen target contours in the spatial domain, improving the model's robustness to noise and its ability to perceive edge details. A lightweight feature extraction network, GhostPG-Net, is constructed, with a Phantom Guided Reparameterization Module (PGM) at its core, integrating phantom convolution, multi-branch reparameterization, and adaptive channel weighting to significantly reduce computational complexity while enhancing multi-scale feature representation. Finally, a lightweight Multidimensional Attention Module (LMDA) is introduced, which achieves detail perception and contextual modeling of small targets through spatial channel collaborative calibration and dynamic local-global interaction. Attached Figure Description

[0014] Figure 1 A schematic diagram of the model architecture for a lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention provided by the present invention; Figure 2 A schematic diagram of the phantom-guided reparameterization architecture for a lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention provided by the present invention; Figure 3 A schematic diagram of a frequency domain guided gating edge enhancement architecture for a lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention provided by the present invention; Figure 4 A schematic diagram of a lightweight multidimensional attention architecture for a lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention provided by the present invention; Figure 5 This is a schematic diagram comparing the detection results of an embodiment of the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0016] A lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention, such as Figure 1 As shown, the detection method of the present invention employs the following... Figure 1 The architecture shown is tested, and the testing methods include: Step 1: Acquire remote sensing images, perform multi-scale feature extraction based on stacked phantom-guided reparameterization on the remote sensing images to obtain multi-level feature maps, use frequency domain-guided gated edge enhancement to enhance details of the multi-level feature maps, and obtain enhanced feature maps. For object detection tasks based on convolutional neural networks, achieving lightweight design and reducing computational resource consumption while maintaining model accuracy is one of the important research challenges. To address this issue, this section proposes GhostPG-Net, a lightweight feature extraction network based on PGM (Phantom-Guided Reparameterization Module) reconstruction. Building upon GhostNet, this network significantly enhances feature representation capabilities while maintaining lightweight design by integrating a multi-branch reparameterization structure and an adaptive channel weighting mechanism.

[0017] GhostNet, by introducing the Ghost module and the channel shuffle mechanism, maximizes the preservation of the network's feature extraction capabilities while reducing computational complexity and the number of parameters. In traditional convolutional neural networks, the standard convolution operation is represented as:

[0018] ; Where Y is the output feature map, X is the input feature map, W is the convolution kernel, and b is the bias term. The Ghost module decomposes the standard convolution into two parts: first, it generates the basic feature map Y1 through lightweight convolution, and then it generates the redundant feature map Y2 through inexpensive operations. The final output is... This design significantly reduces the number of parameters and computational cost, but has the following limitations: (1) The redundant features generated by the linear transformation have limited expressive power and are difficult to fully capture multi-scale contextual information; (2) There is a lack of an adaptive recalibration mechanism for feature channels, making it difficult to distinguish between important features and redundant information; (3) The design with a fixed receptive field is difficult to adapt to drastic changes in the target scale in remote sensing images.

[0019] To address the above requirements, this invention proposes a Phantom-Guided Reparameterization Module (PGM). This module integrates phantom convolution, reparameterized multi-branch structure, and adaptive channel weighting mechanism, significantly enhancing the model's ability to perceive small targets at multiple scales without introducing additional inference overhead, and greatly reducing computational complexity. As a supplement to existing edge enhancement modules, PGM focuses on multi-scale feature aggregation and lightweight representation, forming a synergistic optimization with edge enhancement modules. The innovation of PGM lies in its organic integration of the above three technologies, fully leveraging their respective advantages while eliminating redundant computation through integrated design, providing an efficient feature enhancement solution for remote sensing small target detection. The overall architecture of PGM is as follows: Figure 2 As shown.

[0020] To reduce the computational cost of standard convolution, we first employ the concept of phantom convolution to generate compact intrinsic features. Input m intrinsic feature maps are generated through 1×1 convolution, denoted as m. This step compresses the computational cost of standard convolution to only a fraction of its original value. This provides a lightweight foundation for subsequent enhancements.

[0021] To compensate for the insufficient expressive power of phantom convolution linear transformation, intrinsic features are... Multi-scale reparameterization enhancement is performed by designing three parallel reparameterized convolutional branches with dilation rates r=1, 3, and 6, respectively, to capture multi-receptive field information from local details to global context. During training, each branch consists of two parallel sub-branches: a 3×3 set of convolutions and a 1×1 point convolution, with the outputs summed element-wise. During inference, structural reparameterization merges the two branches into a single 3×3 dilated convolution, maintaining efficient inference. The outputs of the three branches are summed element-wise to obtain the enhanced multi-receptive field features. .

[0022] Taking the branch of expansion rate r as an example, let the input be... The output is The above process can be represented as: ; Where * denotes convolution. For a 3×3 convolution kernel with an inflation rate of r, The kernel is 1×1, and batch normalization (BN) is used. During inference, two BN layers are absorbed into the kernel through a linear transformation and merged into a single convolution. ; The outputs of the three branches are added element-wise to obtain the enhanced multi-receptor field feature. : ; To enhance the response to key channels of small targets and suppress background noise, an adaptive channel weighting mechanism is introduced. Firstly, the... Spatial information is aggregated through global average pooling to generate channel descriptors. : ; Subsequently, inter-channel dependencies are captured through two fully connected layers, and channel weights are generated by Sigmoid activation. : ; Where δ is ReLU and σ is Sigmoid. Finally, the weights s are... Channel-by-channel multiplication yields the recalibrated features: ; To restore the target number of channels, Phantom expansion is performed: a 3×3 depthwise convolution is applied to each channel to generate s-1 phantom features, which are then concatenated with the original channels to obtain a total of m×s feature maps. Finally, a 1×1 convolution is used to adjust the channels to... This yields the final output Y. Here, the phantom expansion factor s=2, meaning that one phantom feature is generated for each intrinsic channel.

[0023] In remote sensing image target detection, speckle noise, complex backgrounds, and the diversity of target scales severely interfere with the accurate extraction of target edge and texture features. Existing methods typically perform feature enhancement in the spatial domain, which has limited capabilities for noise suppression and edge refinement. To address these challenges, a novel Frequency-Guided Gated Edge Enhancement Module (FGEM) is proposed. The core innovation of this module lies in its combination of the physical interpretability of frequency domain analysis with the adaptive capability of spatial domain gated attention. The FGEM module effectively separates noise from target structure and specifically enhances multi-scale edge features. Simultaneously, the module employs lightweight designs such as depthwise separable convolutions, ensuring high performance while maintaining low parameter count and computational complexity, making it easy to embed into existing detection networks and deploy in practice. The FGEM architecture diagram is shown below. Figure 3 As shown.

[0024] The frequency domain feature decoupling and spatial guided weighting part utilizes Fast Fourier Transform to transform the input feature map. Mapping to the frequency domain yields the spectrum. Through a set of predefined binary masks The spectrum is divided to separate different frequency components, as shown below: ; After inverse transformation, spatial feature groups corresponding to different frequency bands are obtained. To perform adaptive frequency fusion for different spatial locations, the module learns a spatial selection feature map (SM), which generates spatial weights corresponding to the feature maps of each frequency band. This process selectively enhances high-frequency components in edge regions of an image. The weighted fused features... Represented as: ; To further refine edges and suppress noise, a lightweight gated attention unit was designed. First, for... Channel attention is applied, and channel weights are generated through global average pooling and a multilayer perceptron. And perform channel reweighting. Subsequently, spatial edge responses are extracted using a 3×3 depthwise separable convolution, and a spatial gated mask G is generated using the Sigmoid function: ; Ultimately, gated enhancement features Depend on Element-wise multiplication with G yields, i.e. This allows for precise focusing on key edge areas.

[0025] To capture multi-scale contextual information related to edges, lightweight multi-scale context aggregation and output employ a set of parallel, depth-separable convolutional processes with dilation. The void ratio is set to The outputs of each branch are concatenated along the channel dimension to form multi-scale features. This design significantly reduces the number of parameters in traditional multi-branch structures by using depthwise separable convolutions. Finally, it is achieved through a 1×1 standard convolution pair. Perform fusion and dimensionality reduction, and combine with the original input. Perform residual connections to obtain the final output of the module: ; .

[0026] Step 2: Based on the multi-head self-attention mechanism, perform global context modeling on the enhanced feature map, and then feed it into the feedforward network for nonlinear transformation after residual connection and layer normalization, outputting high-dimensional features after global encoding.

[0027] Step 3: Based on self-attention, acquire target query information interaction, interact the target query with high-dimensional features based on cross-attention, and then perform channel and spatial weighting processing on the high-dimensional features after interaction through lightweight multi-dimensional attention to obtain lightweight target query features. These features are then mapped to target category and bounding box coordinates by the prediction head, and the final detection result is output to realize small target detection in remote sensing images.

[0028] While multi-receptive-field aggregation and channel attention significantly enhance the feature representation capabilities of small targets, their fixed dilation rate convolution and static channel weight allocation mechanisms still have two limitations: first, the receptive field expansion of dilated convolution is limited by a preset dilation rate, making it difficult to adapt to drastic changes in target scale in complex remote sensing scenes; second, the channel attention module only performs global recalibration of features, lacking the ability to dynamically model spatial details and long-range dependencies. To overcome these limitations while controlling model computational overhead to meet practical deployment requirements, this invention further introduces a lightweight multi-dimensional attention module (LMDA). This module integrates spatial-channel collaborative calibration (SCC) and dynamic local-global interaction (DLG) mechanisms, significantly improving the model's ability to perceive details and model context of small targets while maintaining high efficiency. The LMDA structure diagram is shown below. Figure 4 As shown.

[0029] The SCC submodule aims to synergistically enhance feature discriminative power from both spatial and channel dimensions. Unlike channel attention based on fully connected layers, SCC employs a lightweight design using grouped convolutions and dual pooling strategies. For input features... First, channel descriptors are generated through global average pooling, then summed after passing through two shared fully connected layers, and finally the channel attention weights are obtained through a sigmoid function. Simultaneously, the maximum and average values ​​of the input features along the channel dimension are taken, concatenated, and then processed through a... Grouped convolution generates spatial attention weights , is represented as: ; in This indicates element-wise multiplication. This design significantly reduces the number of parameters through grouped convolutions, while dual pooling preserves richer spatial and channel statistics.

[0030] To overcome the limitations of a fixed receptive field, the DLG submodule introduces deformable sampling and a simplified state-space model to achieve dynamic interaction between local details and the global context. Firstly, for Each spatial location in the image is predicted using a lightweight subnetwork. Each sampling offset and their corresponding attention weights Subsequently, bilinear interpolation was used to extract features from the sampling points, and weighted summation was performed to obtain the local focusing features. , is represented as: ; At the same time, in order to capture global dependencies, Unfolding along the spatial dimension into a sequence, and then performing linear recursion through depthwise separable convolutions to obtain global context features. The complexity of this process is only... Furthermore, this can be accelerated through parallel scanning. Finally, the local and global features are added and fused: .

[0031] To further verify the detection effect of the present invention, the following experiments were conducted: VisDrone2019 is an authoritative benchmark dataset in the field of UAV vision, released in 2019 by the AISKYEYE team at Tianjin University. It aims to address the challenges of small target detection and tracking based on UAV platforms in complex airspace environments. This dataset was collected using multi-rotor UAVs in 14 typical urban scenes in China, covering various environments such as transportation hubs, densely built-up areas, and suburban roads, and includes image data under different lighting and weather conditions. The dataset contains a total of 10,209 high-resolution images, divided into training, validation, and test sets in a ratio of 6471:548:1610, and includes 288 aerial video clips. Data annotation includes 10 typical target categories such as pedestrians and vehicles, with over 2.6 million manually annotated bounding boxes. Each instance provides attribute information such as bounding box coordinates, occlusion level, and category.

[0032] This invention uses multiple metrics to evaluate the model, with the primary accuracy metric being mAP (mean precision) at two IoU thresholds (0.5 and 0.5:0.95). The formula for calculating mAP is shown below.

[0033] ; ; in N For the number of categories, AP i Indicates the first i AP values ​​for each category P For accuracy, R This refers to the recall rate. mAP The value ranges from [0,1], and the higher the value, the better the overall detection performance of the model.

[0034] As shown in Table 1, under similar parameter and computational requirements, the proposed GhostPG-Net achieves the best detection performance. Compared to C3Ghost, GhostPG-Net improves mAP@0.5 by 2.1 percentage points while maintaining a similar parameter count and higher inference speed. Compared to GhostConv, the proposed method achieves 2.6 percentage points higher mAP@0.5 with a lower parameter count. Although ShuffleNetv2 and MobileNetv3 have certain advantages in parameter and computational requirements, their detection accuracy is significantly lower than GhostPG-Net, differing by 13.0 and 8.0 percentage points, respectively. These results demonstrate that GhostPG-Net achieves stronger feature representation capabilities with extremely low computational overhead through the synergistic design of phantom convolution and reparameterized multi-branch structures.

[0035] Table 1. Comparison with experiments using different lightweight networks (%)

[0036] Among various mainstream attention modules, the lightweight multidimensional attention module LMDA proposed in this invention exhibits the best overall performance, as shown in Table 2. LMDA achieves mAP@0.5:0.95 of 20.4% and mAP@0.5 of 34.9% with only 3.24M parameters and 9.2G of computation, both higher than GAM, CBAM, EMA, ECA, and SE. Specifically, LMDA's mAP@0.5 is 0.6 percentage points higher than the second-best performing EMA, while having fewer parameters and comparable inference speed. Compared to channel-based attention SE, LMDA improves mAP@0.5 by 0.8 percentage points with only a 0.15M increase in parameters by introducing spatial channel collaborative calibration and a dynamic local-global interaction mechanism, validating the effectiveness of the multidimensional attention mechanism.

[0037] Table 2. Results of comparative experiments on attention mechanisms (%)

[0038] To verify the overall performance of FAR-DETR in complex scenarios, a comprehensive comparative experiment was conducted on the VisDrone2019 dataset. As shown in Table 3, the proposed FAR-DETR achieves 50.2% mAP@0.5 and 29.4% mAP@0.5:0.95, both significantly outperforming the baseline model Salience-DETR and other mainstream methods. In terms of model efficiency, FAR-DETR has 30.2M parameters and a computational cost of 133.0 GFLOPs, far lower than the Base model with the same backbone, while significantly outperforming lightweight detectors such as YOLOv8L and YOLOv7M in accuracy. Compared to YOLOv7M, FAR-DETR improves mAP@0.5 by 1.9 percentage points with a lower parameter count; compared to Deformable DETR, it reduces the parameter count by 21%, the computational cost by 35%, and the accuracy by more than 5 percentage points, verifying the effectiveness of the lightweight design and module collaboration of this invention.

[0039] Table 3. Comparative Experiment Results of VisDrone 2019

[0040] Ablation experiment: To verify the effectiveness of the improved modules proposed in this invention, ablation experiments were conducted on the VisDrone2019 dataset using Salience-DETR as the baseline model. The experiments progressively introduced three core modules: the lightweight feature extraction network GhostPG-Net, the frequency-domain guided gating edge enhancement module FGEM, and the lightweight multidimensional attention module LMDA. The impact of each module and its combination on detection performance was quantitatively analyzed. The experimental results are shown in Table 4. Introducing GhostPG-Net alone improved mAP@0.5 by 1.3 percentage points to 49.2%, while significantly reducing the number of parameters from 55.9M to 28.5M and the computational cost from 201.0G to 124.5G, verifying the efficiency of the lightweight backbone network. Introducing FGEM and LMDA alone brought improvements of 0.7 and 0.4 percentage points, respectively, indicating the effectiveness of the frequency-domain enhancement and multidimensional attention mechanisms, but with a slight increase in the number of parameters and computational cost. The performance was further improved after combining the modules. Combining GhostPG-Net with FGEM resulted in an mAP@0.5 of 49.8%, a 1.9 percentage point improvement over the baseline; combining GhostPG-Net with LMDA achieved 49.6%. While the combination of FGEM and LMDA offered some improvement, the parameter count and computational cost remained high, indicating that a lightweight backbone network is key to achieving overall efficiency. The final complete model, integrating all three modules, achieved an mAP@0.5 of 50.2%, a 2.3 percentage point improvement over the baseline, and an mAP@0.5:0.95 improvement of 1.5 percentage points. The parameter count was 30.2M, and the computational cost was 133.0G, significantly lower than the baseline. Experimental results demonstrate that the three modules exhibit a good synergistic enhancement effect, significantly improving detection accuracy while maintaining the model's lightweight characteristics.

[0041] Table 4. Results of ablation experiments on different parts of Visdrone 2019

[0042] To verify the performance of the proposed model in detecting complex backgrounds and small objects, we selected... Figure 5 The five typical scenarios (1), (2), (3), (4), and (5) are respectively a top-down shot, a view of dense small targets in the distance at eye level, a night scene, a bright light scene, and a scene with object occlusion. A visual comparison is then made, such as... Figure 5 As shown. In the overhead shot, Figure 5 In (c), there are multiple missed detections of small targets at the shadow edges, indicating insufficient bounding box fit. Figure 5 (d) contains multiple false detections, such as misidentifying a metal box as a car; in scenes with dense small targets at a distance viewed from eye level, Figure 5 (c) and (d) have high false negative rates, with overlapping bounding boxes in dense areas; in nighttime scenes, Figure 5 (c) Among them, false positives are more frequent due to the influence of light. Figure 5 There is one missed detection in both (d) and (e); in strong light scenes, Figure 5 In example (d), a positioning error occurs in the highlight area, such as misdetecting an obstacle as a pedestrian. Figure 5 (c) also has multiple missed detections; in object occlusion scenarios, Figure 5 In cases (c) and (d), occluded targets are prone to being missed or falsely merged. In a comprehensive comparison, FAR-DETR demonstrates the best performance in terms of false detection and missed detection under challenges such as complex lighting, dense small targets, and occlusion, with the fewest identification errors, thus validating the effective improvement of the proposed module for remote sensing small target detection.

[0043] This invention addresses the challenges of small target detection in remote sensing images, such as complex backgrounds, noise interference, large target scale differences, and limited computational resources, by proposing an enhanced detection framework, FAR-DETR. Based on Salience-DETR, this framework designs a Frequency-Guided Gated Edge Enhancement Module (FGEM), which uses Fast Fourier Transform to suppress noise and enhance texture in the frequency domain, combined with a gated attention mechanism to sharpen target contours in the spatial domain, improving the model's robustness to noise and its ability to perceive edge details. A lightweight feature extraction network, GhostPG-Net, is constructed, with a Phantom-Guided Reparameterization Module (PGM) at its core, integrating phantom convolution, multi-branch reparameterization, and adaptive channel weighting to significantly reduce computational complexity while enhancing multi-scale feature representation. A lightweight Multidimensional Attention Module (LMDA) is introduced, which achieves detail perception and contextual modeling of small targets through spatial channel collaborative calibration and dynamic local-global interaction. Experimental results on the VisDrone2019 dataset show that FAR-DETR achieves an mAP@0.5 of 50.2%, a 2.3 percentage point improvement over the baseline, while maintaining low parameter count and computational complexity, validating its advantage in balancing detection accuracy and model efficiency. Future work will further optimize the model structure, extend it to related vision tasks, and explore rotated bounding box representation methods to improve its applicability in more complex scenarios.

Claims

1. A lightweight remote sensing small target detection method based on edge enhancement and multi-dimensional attention, characterized in that, The detection methods include: Step 1: Acquire remote sensing images, perform multi-scale feature extraction based on stacked phantom-guided reparameterization on the remote sensing images to obtain multi-level feature maps, use frequency domain-guided gated edge enhancement to enhance details of the multi-level feature maps, and obtain enhanced feature maps. Step 2: Based on the multi-head self-attention mechanism, perform global context modeling on the enhanced feature map, and then feed it into the feedforward network for nonlinear transformation after residual connection and layer normalization, outputting high-dimensional features after global encoding. Step 3: Based on self-attention, acquire target query information interaction, interact the target query with high-dimensional features based on cross-attention, and then perform channel and spatial weighting processing on the high-dimensional features after interaction through lightweight multi-dimensional attention to obtain lightweight target query features. These features are then mapped to target category and bounding box coordinates by the prediction head, and the final detection result is output to realize small target detection in remote sensing images.

2. The lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention according to claim 1, characterized in that, In step 1, the remote sensing image is subjected to multi-scale feature extraction based on stacked ghost guided reparameterization, a ghost convolution idea is adopted to generate compact intrinsic features, and the remote sensing image is input m intrinsic feature maps are generated through 1x1 convolution, denoted as The calculation amount of the standard convolution is compressed to .

3. The lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention according to claim 2, characterized in that, Regarding intrinsic characteristics Multi-scale reparameterization enhancement is performed using three parallel reparameterized convolutional branches, corresponding to dilation rates r=1, 3, and 6, respectively. Each branch consists of two parallel sub-branches: a 3×3 set of convolutions and a 1×1 point convolution. The outputs are element-wise summed to obtain multi-receptive field features. Represented as: ； Where * denotes convolution. For a 3×3 convolution kernel with an inflation rate of r, The kernel is 1×1, and BN is batch normalization. During the inference phase, the two branches are merged into a single 3×3 dilated convolution through structural reparameterization to maintain efficient inference. A linear transformation is used to absorb the two BN layers into the convolutional kernel, merging them into a single convolution. ； The outputs of the three branches are added element-wise to obtain the enhanced multi-receptor field feature. , represented as: 。 4. The lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention according to claim 3, characterized in that, An adaptive channel weighting mechanism is introduced, firstly, multi-receptive field features are... Spatial information is aggregated through global average pooling to generate channel descriptors. : ； Subsequently, inter-channel dependencies are captured through two fully connected layers, and channel weights are generated by Sigmoid activation. : ； Where δ is ReLU and σ is Sigmoid; finally, the weights s are... Channel-by-channel multiplication yields the recalibrated enhanced multi-receptive field features: ； To restore the target number of channels, Perform phantom expansion: Apply a 3×3 depthwise convolution to each channel to generate Each phantom feature is concatenated with the original channels to obtain m×s feature maps; finally, the channels are adjusted using a 1×1 convolution. The final output Y is obtained, where the phantom expansion factor s=2 is set, and a phantom feature is generated for each intrinsic channel, resulting in a multi-level feature map.

5. The lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention according to claim 1, characterized in that, In step 1, frequency-domain guided gated edge enhancement, the input multi-level feature map is mapped to the frequency domain using Fast Fourier Transform to obtain the spectrum. ; through a set of predefined binary masks The spectrum is divided to separate different frequency components, as shown below: ； After inverse transformation, spatial feature groups corresponding to different frequency bands are obtained. To perform adaptive frequency fusion for different spatial locations, the module learns a spatial selection feature map (SM), which generates spatial weights corresponding to the feature maps of each frequency band. To selectively enhance high-frequency components in edge regions of an image; Features after weighted fusion Represented as: 。 6. The lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention according to claim 1, characterized in that, In step 1, frequency-domain guided gating edge enhancement employs a lightweight gating attention unit. First, the weighted fused features are... Channel attention is applied, and channel weights are generated through global average pooling and a multilayer perceptron. And perform channel reweighting. Subsequently, a 3×3 depthwise separable convolution is used to extract the spatial edge response, and a spatial gate mask G is generated using the Sigmoid function, expressed as: ； Ultimately, gating enhancement features Depend on Element-wise multiplication with G yields, i.e. This enables precise focusing on key edge areas; To capture multi-scale contextual information related to edges, lightweight multi-scale context aggregation and output employ a set of parallel, depth-separable convolutional processes with dilation. The void ratio is set to The outputs of each branch are spliced ​​along the channel dimension to form multi-scale features. ; Through a 1×1 standard convolution pair Perform fusion and dimensionality reduction, and combine with the original input. Perform residual connections to output enhanced feature maps: 。 7. The lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention according to claim 1, characterized in that, In step 3, lightweight multidimensional attention is used to perform channel and spatial weighting on the high-dimensional features after interaction. The lightweight multidimensional attention first adopts spatial channel co-calibration and uses grouped convolution and dual pooling strategies to achieve a lightweight design. For the input high-dimensional features, channel descriptors are first generated through global average pooling, then summed after passing through two shared fully connected layers, and finally the channel attention weights are obtained through the Sigmoid function. Simultaneously, the maximum and average values ​​of the high-dimensional input features are taken along the channel dimension, concatenated, and then processed through a... Grouped convolution generates spatial attention weights , represented as: ； in This indicates element-wise multiplication.

8. The lightweight remote sensing small target detection method based on edge enhancement and multidimensional attention according to claim 7, characterized in that, Step 3's lightweight multidimensional attention also employs a local-global interaction mechanism, introducing deformable sampling and a simplified state-space model to achieve dynamic interaction between local details and the global context; firstly, for Each spatial location in the image is predicted using a lightweight subnetwork. Each sampling offset and their corresponding attention weights Subsequently, bilinear interpolation was used to extract features from the sampling points, and weighted summation was used to obtain the local focusing features. , represented as: ； At the same time, in order to capture global dependencies, Unfolding along the spatial dimension into a sequence, and then performing linear recursion through depthwise separable convolutions to obtain global context features. ; The complexity of this process is only And it is accelerated through parallel scanning; Finally, the local focused features and global features are added and fused to obtain the fused lightweight target query features: 。

Images