Remote sensing small target detection method based on multi-scale detail context space attention

Abstract

The invention provides a remote sensing small target detection method based on multi-scale detail context space attention (MDCSA). According to the method, an RT-DETR framework is used as a backbone, and a basic module of a backbone network Backbone stage is replaced by an MDCSA module, so that multi-scale feature depth fusion and small target edge weak feature extraction are realized. The MDCSA module is formed by connecting a multi-scale dynamic attention sub-module, a local and global fusion sub-module, a scale routing and coordinate attention sub-module and a geometric gating attention sub-module in series, and all the sub-modules achieve the functions of weak edge clue highlighting, local and global feature balance, cross-scale adaptive distribution and background redundancy suppression. And finally, the output is added with the input features through residual connection to obtain enhanced features. According to the method, the feature representation capability of dense small targets, sheltered targets and complex backgrounds in the remote sensing image can be enhanced, and a higher mAP value and a more stable recall rate can be realized in a remote sensing small target detection task.

Description

Technical Field

[0001] This invention relates to deep learning and computer vision technologies, and more particularly to a remote sensing small target detection method that utilizes multi-scale detail contextual spatial attention (MDCSA) to enhance feature representation. Background Technology

[0002] Remote sensing imagery is more complex than conventional natural scenes in terms of acquisition methods, imaging height, and transmission links. Small targets typically occupy only a dozen pixels or even less, and are affected by clouds, ground cover occlusion, and sensor noise, resulting in blurred edges, weak texture, and low contrast. Common downsampling and convolution operations can erase these valuable details early in the backbone network. Even with subsequent multi-scale fusion using structures like FPN and PANet, it often only roughly adds or concatenates shallow high-resolution features with deep semantic features, applying fixed weights to feature maps at each scale by default. Because deep, small-scale features carry little information and are noisy, this "equal-weight fusion" further amplifies background interference, making it difficult to distinguish dense small targets from ground texture.

[0003] Furthermore, remote sensing targets are characterized by their density, large scale span, and variable orientation. Convolutions with a single scale or a single dilation rate cannot simultaneously capture details and context. Traditional attention models typically focus only on a single channel or spatial dimension, lacking collaborative modeling of edges, location, and local context, leading to instability in the localization and classification of small targets. Especially in dense remote sensing datasets like SIMD, where targets are extremely close together, without explicit emphasis on fine-grained edges and local regions, the model is prone to mistaking small targets for background or confusing them with each other. Therefore, constructing a small target detection module that can adaptively schedule multi-scale weights, enhance edge and location awareness, and preserve details during downsampling has become a key requirement for remote sensing scenarios.

[0004] To address the aforementioned issues, this invention designs a remote sensing small target detection framework based on Multi-Scale Detail Context Space Attention (MDCSA). The overall approach is to replace the original structure in the RT-DETR backbone with MDCSA, allowing features to undergo a complete expression from the backbone stage: edge perception → multi-scale refinement → detail / context collaboration → coordinate / scale routing → geometric gating. MDCSA internally connects multi-scale dynamic attention, local-global fusion modules, scale routing and coordinate attention, and geometric gating attention in sequence, and adds them to the backbone input via residuals to achieve end-to-end multi-scale adaptive fusion. MSDAM is responsible for extracting shared basic features and guiding three different receptive field branches to combine between dilation=1 and dilation=2, while simultaneously using Sobel edge gating and a local context weight generator to highlight fine-grained edges. LGFM maintains texture and semantic balance using a dual detail / context path. ScaleRouter explicitly encodes positions and dynamically assigns scale weights through coordinate attention and multi-scale routing. GCGA combines geometric and channel gating at the output to suppress background noise and highlight key regions. The four sub-modules work in a progressive manner, enabling the network to retain details and enhance positioning capabilities in high-noise, dense, and small-sized remote sensing scenarios, ultimately significantly improving detection accuracy and recall. Summary of the Invention

[0005] The purpose of this invention is to design a deep learning network that can detect small target objects in remote sensing images with high detection accuracy.

[0006] To address the issues of small remote sensing targets being easily obscured and losing edge and location signals during multi-scale fusion, this invention provides a remote sensing small target detection network based on multi-scale detail context space attention. The network mainly includes a backbone network for extracting features from the input image for training subsequent networks; an MDCSA module for multi-scale feature enhancement of the features extracted by the backbone network, enabling the network to learn more detailed and location information about the small targets; an RT-DETR decoder for matching the enhanced features with the target and the predicted bounding box; and a target detection module for outputting the target detection results. The detection method includes the following main steps:

[0007] 1. Obtain the labeled dataset of relevant small target images and convert the dataset label format to YOLO format;

[0008] 2. The RT-DETR backbone is used as the backbone network. Initial convolution and downsampling operations are performed on the input image to obtain multi-scale feature maps. The original basic module is replaced with the MDCSA module to make the backbone output have a stronger ability to express small targets.

[0009] 3. Construct the MDCSA module. MDCSA is composed of the multi-scale dynamic attention module MSDAM, the local-global fusion module LGFM, the scale routing and coordinate attention module ScaleRouter, and the geometric gated attention module GCGA, which are connected in series and added to the main input through residual method to form the final output.

[0010] 4. Input the multi-scale features output by MDCSA into the decoder and matching network of RT-DETR, use Hungarian matching to constrain the predicted bounding boxes and ground truth labels, and then perform classification and regression through the detection head to obtain the remote sensing small target detection results;

[0011] 5. Based on the training set, perform multiple rounds of training and parameter tuning on the constructed network, and use the obtained optimal weights for deployment and inference.

[0012] The core of the entire network lies in embedding MDCSA into the RT-DETR backbone, forming a feature enhancement path of "multi-module serialization + residual closed loop". The MDCSA module sequentially includes Multi-Scale Dynamic Attention (MSDAM), Local-Global Fusion (LGFM), Scale Routing and Coordinate Attention (ScaleRouter), and Geometric Gated Attention (GCGA). These four sub-modules are explicitly connected, and the output of each level serves both as input to the next level and as addition to the original features of the backbone through a residual loop. In this way, each layer of the backbone possesses detailed multi-scale / multi-semantic modeling capabilities, eliminating the need to rely on backend FPN or extended necks to compensate for information loss.

[0013] The construction method of the MSDAM submodule in the above steps includes the following:

[0014] In MSDAM, the input features are denoted as... First, preprocessed features are generated through 1×1 1×1 pointwise convolution and GELU activation, while retaining the original features as residual branches. To highlight the weak edge features of small remote sensing targets, MSDAM embeds Sobel edge enhancement gating. Specifically, the average along the x-channel is taken to obtain a single-channel image m, which is then convolved with Sobel_X and Sobel_Y to obtain the horizontal and vertical gradients g. x g y And calculate the amplitude Here, ε is a small constant to prevent numerical instability. Then, mag is fed into a 1×1 1×1 convolution and a sigmoid function to obtain the gate, which is then controlled by x. edge The form = x×(1+0.1×gate) emphasizes the edges without destroying the original semantics.

[0015] After edge enhancement, MSDAM constructs shared base features x using the method DWConv3×3→BN→ReLU. base The channel was then proportionally divided into three paths (x1, x2, x3), corresponding to a small receptive field (3×3), a medium receptive field (5×5), and a large receptive field (7×7), respectively. Each path used depthwise separable convolution to generate the underlying response. To avoid the inability of fixed convolutional kernels to adapt to different target sizes, MSDAM introduces a local context weight generator: for x edge Apply DWConv 5×5 and DWConv 3×3 Adaptive average pooling is then applied, followed by MLP output of ctx_logits∈R. B ×3×max_dilation After Softmax, weights w3, w5, and w7 are obtained. These weights guide the corresponding branches to adaptively fuse between depthwise convolutional branches with dilation=1 and dilation=2, ensuring that local details are preserved while capturing extended context. Finally, the weighted results are fused using PWConv1×1 and concatenated in the channel dimension, merging with the residual signal to provide multi-scale enhancement features for the next module.

[0016] The construction method of LGFM in the above steps includes the following:

[0017] This module consists of detail branches and context branches to balance local textures and wide-area semantics. The detail branches use DWConv 3×3 +PWConv 1×1 The structure focuses on capturing high-frequency information and edge textures; the context branches are connected to DWConv. 5×5 With DWConv 5×5,d＝2 This is equivalent to obtaining a 9×9 receptive field, used to aggregate environmental information over a larger area. The outputs of the two branches are first unified in terms of channel count, and then passed through learnable weights (α). d ,α c =Softmax([w d ,w d Adaptive fusion is performed while ensuring the main path remains dominant. The fusion result is linearly combined with the MSDAM output and then fed into the next module.

[0018] The ScaleRouter construction method, as described above, includes the following:

[0019] The module first performs coordinate attention encoding, averaging the input features along both the height and width, to obtain... and The two components are concatenated, dimensionality reduced using PWConv, activated by GELU, and then split into x. h_att ,x w_att The channel is restored using a separate PWConv and a is obtained using Sigmoid. h ,a w Finally, press x. pos =x×a h ×a w Explicit position information is injected. The scale adaptation part then uses x... pos The channel is divided into three segments, and 3×3 average pooling and max pooling are performed on each segment. The segments are then stacked and fed into an MLP to obtain three scale weights (α3, α5, α7). These weights are applied to the corresponding channel blocks, and then the blocks are concatenated again for output.

[0020] The GCGA construction method in the above steps includes the following:

[0021] This part is implemented through a combination of depthwise convolution and channel attention. First, DWConv 7×7 Obtain spatial gate sp =σ(DWConv 7×7 (x) Highlight regions with geometric structures; secondly, use a SE-style two-layer MLP to generate channel gates. ch This is used to emphasize important channel dimensions. Finally, the output is calculated according to x. gcga =x×gate sp ×gate ch Combined, and then processed by PWConv 1×1 The original number of channels is projected back and added to the MDCSA input through residual connections to ensure the stability of feature enhancement and gradient propagation.

[0022] By adopting the above technical solution, the present invention has the following advantages:

[0023] 1. By replacing the basic module in the backbone with the MDCSA module, the feature extraction capability for small target edges and context is enhanced, thereby improving detection recall and accuracy;

[0024] 2. The cascaded structure of MSDAM, LGFM, ScaleRouter, and GCGA enables dynamic multi-scale feature extraction and scale-aware routing, effectively suppressing background interference and highlighting the target area, providing stable and efficient feature representation for remote sensing small target detection. Attached Figure Description

[0025] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration:

[0026] Figure 1This is a schematic diagram of the remote sensing small target detection method with multi-scale detail context space attention according to the present invention.

[0027] Figure 2 This is a schematic diagram of the remote sensing small target detection method based on the multi-scale detail context space attention (MDCSA) module of the present invention.

[0028] Figure 3 This is a schematic diagram of the overall structure of the multi-scale dynamic attention MSDAM module of the present invention.

[0029] Figure 4 This is a schematic diagram of the LGFM submodule structure of the local-to-global fusion module of the present invention.

[0030] Figure 5 This is a schematic diagram of the ScaleRouter module structure of the present invention.

[0031] Figure 6 This is a schematic diagram of the geometrically gated attention (GCGA) structure of the present invention.

[0032] Figure 7 This is a schematic diagram of the overall architecture of the RT-DETR remote sensing small target detection network based on the improved MDCSA module of the present invention. Detailed Implementation Plan

[0033] The present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the invention. The present invention proposes a remote sensing small target detection method based on multi-scale detail context space attention, the specific implementation of which includes the following:

[0034] 1. Select a labeled dataset of small target images relevant to a specific scenario (e.g., the SIMD dataset of small targets in urban traffic monitoring scenarios).

[0035] 2. The RT-DETR backbone is selected as the main network, and each basic module in the backbone is replaced with the MDCSA module proposed in this invention to enhance the multi-scale feature extraction capability. The network as a whole still outputs three resolution features (corresponding to P3, P4, and P5), but the output of each stage has already undergone multi-stage attention enhancement of MDCSA before entering the RT-DETR decoder.

[0036] 3. Build the MDCSA module and insert it into Backbone.

[0037] In the appendix Figure 2In the MDCSA module, there are four concatenated sub-modules: Multi-Scale Dynamic Attention (MSDAM), Local-Global Fusion (LGFM), Scale Router and Coordinate Attention (ScaleRouter), and Geometric Gated Attention (GCGA). The input of MDCSA comes from the features of the previous layer of the Backbone, and the output is concatenated with the residual and returned to the main path of the Backbone before being fed into the decoder.

[0038] Combined with appendix Figure 3 This explains the specific method for constructing the MSDAM submodule: The MSDAM submodule first processes the input features... It performs 1×1 pointwise convolution and GELU preprocessing, retaining the original input as a residual. Then, it applies Sobel edge enhancement to the preprocessed features: the channel-averaged result m is convolved with Sobel_X and Sobel_Y respectively to obtain the gradient g. x g y Calculate the amplitude Then, a 1×1 convolution and a sigmoid function are applied to obtain the gate, forming x. edge = x×(1+0.1×gate) to enhance weak edges commonly found on small remote sensing targets. After shared basis extraction via DWConv3×3→BN→ReLU, the features are divided into three paths x1, x2, and x3, and then processed by DWConv... 3×3 DWConv 5×5 DWConv 7×7 Get the basic response To achieve adaptive multi-scale fusion, MSDAM will also adjust x edge Apply DWConv 5×5 3×3 adaptive average pooling and MLP are used to output Softmax-normalized weights w3, w5, w7. Guided by this, the three feature paths are weighted and combined with convolutional branches of dilation=1 and dilation=2, respectively. For example, the small-scale branch satisfies... The same applies to the other branches, and through PWConv 1×1 After fusion, the data is spliced ​​back to the main path in the channel dimension to form the MSDAM output.

[0039] Combined with appendix Figure 4 This section explains the specific method for constructing the LGFM submodule: The LGFM submodule receives the output of MSDAM and is internally divided into detail branches and context branches: the former uses DWConv 3×3 +PWConv 1×1 To maintain spatial resolution and highlight local textures; the latter is concatenated with DWConv 5×5 With DWConv 5×5,d＝2This is equivalent to expanding the receptive field to 9×9 to capture region semantics. The outputs of the two branches are adaptively fused through learnable weights and linearly combined with the MSDAM result in a ratio of 0.7:0.3 to achieve a balance between detail and context.

[0040] Combined with appendix Figure 5 This section explains the specific method for constructing the ScaleRouter submodule: The ScaleRouter submodule is responsible for injecting coordinate attention and scale adaptation mechanisms. It first performs global average pooling on the input features along both the height and width to obtain x. h With x w After processing with PWConv+GELU, it is split into x h_att x w_att Further mapping to attention a h a w Thus forming x pos =x×a h ×a w The location encoding result. Then, ScaleRouter will use x pos The channel is divided into three segments: small, medium, and large. Local statistics are obtained by combining 3×3 average pooling and max pooling. The normalized weights are output by MLP, and the features of the three segments are weighted separately and then concatenated, so that information at different scales is enhanced in a customized manner before entering the next stage.

[0041] Combined with appendix Figure 6 This section explains the specific method for constructing the GCGA submodule: the GCGA submodule simultaneously models spatial geometry and channel importance. It utilizes DWConv... 7×7 Obtain spatial gate sp Highlighting areas with geometric structures; and generating channel gating gates through a two-layer SE-type MLP. ch Emphasizing the critical path. The final output is x. gcga =x×gate sp ×gate ch via PWConv 1×1 The original number of channels is projected back and added to the residual of the MDCSA input to return to the Backbone main path, providing enhanced multi-scale features for the decoder.

Claims

1. A remote sensing small target detection method based on a multi-scale detail context spatial attention (MDCSA) module, characterized in that, Using the RT-DETR network model as the base network, the network is optimized according to the characteristics of remote sensing small target images. The basic module in the RT-DETR backbone network is replaced with the MDCSA module. The constructed remote sensing small target detection network mainly includes: a backbone network with the MDCSA module as the basic module, an RT-DETR decoder, and a target detection module. The data flow of the entire network is as follows: First, the input image is processed through the first two stages of convolution and downsampling operations of the backbone network to obtain a multi-scale feature map. Then, the feature map is input into the MDCSA module for multi-scale feature enhancement. The MDCSA module includes a multi-scale dynamic attention module (MSDAM), a local-global fusion module (LGFM), a scale routing and coordinate attention submodule (ScaleRouter), and a geometrically gated attention submodule (GCGA) connected in sequence. The output of the MSDAM submodule is connected to the input of the LGFM submodule, the output of the LGFM submodule is connected to the input of the ScaleRouter submodule, the output of the ScaleRouter submodule is connected to the input of the GCGA submodule, and the output of the GCGA submodule is added to the input features of the MDCSA module through a residual connection to obtain the enhanced feature map. Then, the feature map output by the MDCSA module is input into the RT-DETR decoder, and the target and the predicted bounding box are matched using the Hungarian matching algorithm. Finally, the matching result is input into the target detection module for target classification and bounding box regression prediction, which is used for the detection of small targets in remote sensing.

2. The remote sensing small target detection method according to claim 1, characterized in that, The MSDAM submodule in the MDCSA module performs multi-scale edge context attention processing on the input features. The construction of the MSDAM submodule includes the following steps: Step 1: The input features are first processed by 1×1 pointwise convolution and GELU activation to form preprocessed features, and the original features are saved as residual branches; Step 2: Perform edge enhancement gating on the preprocessed features. The input features are averaged to obtain a single-channel image. The horizontal and vertical gradients are extracted by Sobel X / Y convolution. The gradient magnitude is calculated. The edge gating signal is generated by 1×1 pointwise convolution and Sigmoid activation. The features are enhanced in a lightweight way. Step 3: Apply 3×3 depthwise separable convolution, BatchNorm, and ReLU operations to the enhanced features to form a unified basic feature map; Step 4: Divide the shared basic features into three paths according to the channel dimension. The number of channels in the first path, split1, is dim / 3; the number of channels in the second path, split2, is dim / 3; and the number of channels in the third path is dim-split1-split2. Step 5: The three branches are processed by separable convolutions of depths of 3×3, 5×5, and 7×7 respectively to obtain the basic responses under different receptive fields; Step 6: Extract context features from the input features using 5×5 depthwise convolution, then obtain spatial partitioning using 3×3 adaptive average pooling, and output weights with a dimension of 3×max_dilation through pointwise convolutional MLP. After Softmax normalization, we obtain three weight vectors w3, w5, and w7. Step 7: Configure two types of deep convolution kernels, dilation=1 and dilation=2, for each branch. Use the weight vector generated in Step 6 to perform a weighted sum of the convolution outputs with different dilation rates to obtain the refined features. Step 8: Perform channel fusion on the three refined features using 1×1 pointwise convolution, and then concatenate them along the channel dimension to obtain output features with the same number of input channels.

3. The remote sensing small target detection method according to claim 1, characterized in that, The MDCSA module also includes the LGFM submodule, the ScaleRouter submodule, and the GCGA submodule, specifically including: (1) LGFM submodule: includes detail branch and context branch. The detail branch consists of 3×3 depthwise convolution and 1×1 pointwise convolution. The context branch consists of two 5×5 depthwise convolution layers, where the latter layer has a dilation of 2. The outputs of the two branches are weighted and fused through learnable weights and linearly mixed with the output of the MSDAM submodule in a ratio of 0.7:0.

3. (2) ScaleRouter submodule: First, the coordinate attention mechanism is used to extract horizontal / vertical statistics, generate a_h and a_w and modulate the input features element by element. Then, it is divided into three sub-blocks according to the channel dimension. The statistics are extracted by the context extractor composed of 3×3 average pooling and max pooling. The normalized weights α3, α5 and α7 of the three scales are generated by MLP and applied to the three channel blocks respectively to achieve cross-scale adaptive allocation. (3) GCGA submodule: captures geometric contours through depth convolution, combines channel gating generated by pointwise convolution to perform secondary filtering on the fused features, and projects back to the original dimension through 1×1 pointwise convolution, and adds it to the input residual of the MDCSA module to obtain the final output.

Images