A remote sensing small target detection method and device based on density-oriented attention enhancement

Abstract

A remote sensing small target detection method and device based on density-oriented attention enhancement, the method comprising: acquiring a preprocessed remote sensing image dataset; building a remote sensing small target detection model based on density-oriented attention enhancement, the structure of the remote sensing small target detection model being: sequentially connecting a backbone network, a deformable encoder, a feature alignment module, a density map generation module, a feature-guided enhancement module, a deformable decoder and an output layer from an input layer; inputting a training set into the built remote sensing small target detection model for training to obtain a trained remote sensing small target detection model; verifying the trained remote sensing small target detection model by using a verification set to obtain a final remote sensing small target detection model; and inputting a test set into the final remote sensing small target detection model to complete the identification of different targets in the test set. The present application improves the accuracy and reliability of small targets while maintaining the advantages of an end-to-end detection framework.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of remote sensing technology, computer vision, and target detection, and in particular to a method and apparatus for detecting small remote targets based on density-guided attention enhancement. Background Technology

[0002] In complex and diverse remote sensing applications, small target detection technology is a core requirement in fields such as environmental monitoring, disaster early warning, and land planning. Remote sensing images are characterized by wide coverage, high background complexity, and significant differences in target scale. Especially in scenarios such as marine monitoring and farmland identification, targets such as ships and small buildings often exhibit low pixel ratios and sparse feature information. The DETR model based on the Transformer architecture effectively avoids the anchor box design bias problem through an end-to-end detection mechanism. Its global attention mechanism can capture the correlation features between targets and complex backgrounds in remote sensing images, but it still has bottlenecks in the detection of small targets in remote sensing. First, there is a spatial misalignment problem during multi-scale feature fusion. Existing methods directly fuse features from different levels of the backbone network, ignoring the spatial misalignment problem of feature maps caused by downsampling operations. This geometric distortion makes it impossible to accurately match shallow details with deep semantics, severely limiting the localization accuracy of small targets. Second, the attention mechanisms used often lack effective guidance. DETR has high computational cost for global attention and, under guided conditions, it is difficult to automatically focus on targets with extremely small pixel proportions in complex backgrounds, resulting in the submergence of small target features and a high false negative rate. Summary of the Invention

[0003] To address the shortcomings of the existing technologies, this invention takes small remote sensing targets in complex backgrounds as the detection targets. Based on the collaborative design of feature alignment, density-oriented and attention-enhanced features, it proposes a method and device for detecting small remote sensing targets based on density-oriented attention enhancement. This method effectively improves the detection accuracy and recall rate of small targets while ensuring detection efficiency.

[0004] The remote sensing small target detection method based on density-guided attention enhancement proposed in this invention includes the following steps:

[0005] Step 1: Obtain the preprocessed remote sensing image dataset and divide it into training, validation, and test sets according to the proportions.

[0006] Step 2: Construct a remote sensing small target detection model based on density-guided attention enhancement. The structure of the remote sensing small target detection model is as follows: starting from the input layer, the backbone network, deformable encoder, feature alignment module, density map generation module, feature-guided enhancement module, deformable decoder, and output layer are connected in sequence.

[0007] Step 3: Input the training set into the built remote sensing small target detection model for training to obtain the trained remote sensing small target detection model; and use the validation set to validate the trained remote sensing small target detection model to obtain the final remote sensing small target detection model.

[0008] Step 4: Input the test set into the final remote sensing small target detection model to complete the identification of different targets in the test set.

[0009] Furthermore, step 1 includes:

[0010] Step 1.1: Obtain remote sensing images containing small targets, and label the category labels and bounding box coordinates of the small targets to construct a dataset;

[0011] Step 1.2: Preprocess the images in the dataset, including data size normalization, data augmentation, and pixel value normalization;

[0012] Step 1.3: Divide the preprocessed dataset into training set, validation set and test set according to the set ratio.

[0013] Furthermore, the backbone network described in step 2 is used to extract multi-scale features; the deformable encoder is used to initially enhance the multi-scale features; the feature alignment module is used to achieve spatial alignment and fusion of multi-scale features; the density map generation module is used to generate a density map reflecting the probability distribution of small targets from the fused features; the feature-guided enhancement module is used to use the density map to guide attention to further enhance the multi-scale features; the deformable decoder is used to receive the finally enhanced multi-scale features and output the prediction result through decoding operations; and the output layer is used to output the final target category and bounding box coordinates.

[0014] Furthermore, the structure of the remote sensing small target detection model includes: the backbone network adopts a ResNet50 architecture, containing five convolutional stages, which sequentially output multi-scale feature maps P2 (1 / 4 resolution), P3 (1 / 8 resolution), P4 (1 / 16 resolution), and P5 (1 / 32 resolution), where P2 retains small target detail information and P5 contains deep semantic information; the deformable encoder consists of multiple deformable attention layers, each of which contains a multi-head deformable self-attention module and a feedforward network, enhancing the P2-P5 features output by the backbone network to generate optimized multi-scale features S2-S5; the feature alignment module sequentially includes the following components: a first convolutional layer, a bilinear interpolation upsampling layer, and a feature concatenation layer. The second convolutional layer, deformable convolutional layer, and residual connection layer; the density map generation module sequentially includes: a third convolutional layer, three parallel dilated convolutional layers, a spatial attention submodule, and a fourth convolutional layer for output transformation; the feature guidance enhancement module includes: a multi-scale pooling layer, a feature concatenation layer, a spatial attention submodule, a channel attention submodule, and a feature fusion layer; the deformable decoder consists of multiple cascaded deformable attention layers, each layer containing a cross-attention module and a feedforward network; the output layer contains two independent components: the classification head consists of two fully connected layers and a Softmax activation function, responsible for outputting the class probability distribution of the target; the regression head also consists of two fully connected layers, specifically used to predict the coordinate offset of the bounding box.

[0015] Preferably, the first, third, and fourth convolutional layers are all 1×1 convolutional layers, and the second convolutional layer is a 3×3 convolutional layer; the deformable encoder consists of 6 deformable attention layers; and the deformable decoder consists of 6 cascaded deformable attention layers.

[0016] Furthermore, the process of building the network also includes: setting the number of attention heads and feature dimensions of the deformable encoder and deformable decoder; designing the kernel size, stride and number of kernels of each convolutional layer of each module according to the input and functional requirements of each module of the network; and setting the number of target query vectors of the deformable decoder according to the distribution density of the target in the dataset.

[0017] Furthermore, step 3 includes:

[0018] Step 3.1: Input the training set into the model in batches for forward propagation. The input images are processed by the backbone network, deformable encoder, feature alignment module, density map generation module, feature guidance enhancement module, and deformable decoder, and the output target class probability and bounding box prediction are given.

[0019] Step 3.2: Calculate the loss function, including: Focal Loss for classification loss, GIoULoss for regression loss, and the total loss is the weighted sum of classification loss and regression loss;

[0020] Step 3.3: Use the AdamW optimizer to iteratively update the model parameters by setting the initial learning rate, weight decay coefficient, and learning rate strategy;

[0021] Step 3.4: During training, periodically evaluate model performance using the validation set and save the weights of the best-performing model; terminate training early when the model's performance on the validation set no longer improves.

[0022] Step 3.5: Use the model with the best performance on the validation set as the final remote sensing small target detection model.

[0023] A second aspect of the present invention relates to a remote sensing small target detection device based on density-guided attention enhancement, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the remote sensing small target detection method based on density-guided attention enhancement of the present invention.

[0024] A third aspect of the invention is a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the density-guided attention-enhanced remote sensing small target detection method of the present invention.

[0025] In summary, this invention provides a remote sensing small target detection method based on density-guided attention enhancement. The core process includes: first, acquiring a small target dataset from remote sensing images; preprocessing the dataset and then dividing it proportionally into training, validation, and test sets to provide standardized input for subsequent network training and validation; extracting multi-scale features P2-P5 and their optimized versions S2-S5 from the input images using a ResNet50 convolutional neural network backbone and a deformable encoder; and inputting P2 and S5 into a feature alignment module for cross-scale alignment and fusion to mitigate cross-scale errors. While addressing the location issue, the module fuses shallow detail features and deep semantic features. The fused features are input into the density map generation module, which captures the spatial distribution model of small targets through operations such as dilated convolution, generating a probability density map of the small target distribution. This provides prior knowledge of the small target location for subsequent feature enhancement. Based on the prior knowledge of the density map, S2-S5 are enhanced with channel and spatial dual-dimensional features to further improve feature discrimination. Finally, the target query vector and the enhanced features are input into the deformable decoder for decoding and prediction of classification results and bounding boxes, completing the detection and localization of remote sensing small targets.

[0026] The innovation of this invention is:

[0027] 1. This invention proposes a feature alignment module based on deformable convolution. Through the process of "1×1 convolution dimensionality reduction → bilinear upsampling to match size → 3×3 convolution to learn spatial offset → deformable convolution dynamic calibration", it actively compensates for the spatial structure deviation of shallow features, so that deep and shallow features are accurately matched in geometric position and semantic information, significantly improving the accuracy of small target bounding box regression.

[0028] 2. This invention innovatively introduces a density map generation module to display prior knowledge of the distribution of small targets. By designing a density estimation network that fuses multi-scale dilated convolution with spatial attention, it accurately captures clustered regions of small targets. Furthermore, it proposes using this density map to modulate spatial attention, achieving a shift from global attention to density-guided focusing, effectively reducing the false negative rate of small targets.

[0029] 3. This invention constructs a "spatial-channel" serialization enhancement pipeline, which first strengthens positional information through density-modulated spatial attention, and then filters semantic features through channel attention, thereby enhancing small target features in two dimensions, suppressing background interference, and improving feature discrimination power.

[0030] 4. This invention deeply integrates the above three innovative modules with the ResNet50 backbone network and deformable encoder / decoder, forming a closed-loop synergy of "input → feature optimization → guidance → enhancement → output". While maintaining the advantages of DETR end-to-end detection, it breaks through the bottleneck of accuracy and recall rate in small target detection.

[0031] The beneficial effects of adopting the above technical solution are as follows:

[0032] This invention achieves accurate multi-scale feature extraction and attention enhancement for small targets in remote sensing images through the synergistic effect of a feature alignment module, a density map generation module, and a feature-guided enhancement module. The key features of this detection method are: utilizing deformable convolution to achieve spatial alignment of shallow and deep features, effectively solving the problem of decreased accuracy in small target localization caused by network downsampling, and significantly improving the accuracy of bounding box regression; explicitly modeling the prior distribution of small targets through the density map generation module, guiding the model's attention from the complex global background to target-rich regions, improving the recall rate for small targets, and effectively mitigating the problem of missed detections; and employing a density-guided dual attention enhancement mechanism to refine features from both spatial and channel dimensions, strengthening the spatial location information of small targets and improving the semantic discriminative ability of features, enabling the model to maintain excellent detection performance in complex backgrounds and dense target scenes. Ultimately, this invention improves the accuracy and reliability of detecting small targets such as ships and vehicles in remote sensing images while maintaining the advantages of an end-to-end detection framework. Attached Figure Description

[0033] Figure 1This is a flowchart of the method of the present invention.

[0034] Figure 2 This is a structural diagram of the remote sensing small target detection model based on density-guided attention enhancement in the method of this invention.

[0035] Figure 3 This is a structural diagram of the feature alignment module in the method of the present invention.

[0036] Figure 4 This is a structural diagram of the density map generation module in the method of the present invention.

[0037] Figure 5 This is a structural diagram of the feature guidance enhancement module in the method of the present invention. Detailed Implementation

[0038] To facilitate understanding of this application, specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. The following embodiments are illustrative of the invention but are not intended to limit its scope. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application.

[0039] Example 1

[0040] This embodiment proposes a remote sensing small target detection method based on density-guided attention enhancement, such as... Figure 1 As shown, the method includes the following steps:

[0041] Step 1: Obtain the preprocessed remote sensing image dataset and divide it into training, validation, and test sets according to the proportions.

[0042] Step 1.1: Obtain remote sensing images containing small targets, and label the category labels and bounding box coordinates of the small targets to construct a dataset;

[0043] Step 1.2: Preprocess the images in the dataset, including data size normalization, data augmentation, and pixel value normalization;

[0044] Step 1.3: Divide the preprocessed dataset into training set, validation set and test set according to the set ratio.

[0045] In this embodiment, since the density-guided attention-enhanced remote sensing small target detection method is proposed for the application scenario of small target detection in complex backgrounds, to verify its effectiveness, typical small targets such as aircraft, ships, vehicles, and small buildings from the AI-TOD-v2 remote sensing dataset are used as detection objects to construct a sample dataset. The sample dataset is divided into training set, validation set, and test set in a 7:2:1 ratio. The batch size for each training iteration is set to 8 to adapt to GPU memory limitations and ensure training stability. The RGB three-channel features of the sample images are determined, and the three-channel feature tensor of each batch of 8 sample images is denoted as... , where H and W represent the height and width dimensions of the image, respectively.

[0046] Step 2: Build a remote sensing small target detection model based on density-guided attention enhancement:

[0047] like Figure 2 As shown, the structure of the model is as follows: starting from the input layer, the backbone network, deformable encoder, feature alignment module, density map generation module, feature guidance enhancement module, deformable decoder and output layer are connected in sequence.

[0048] Step 2.1: The input remote sensing image is used to extract features through the ResNet50 backbone network, generating multi-scale feature maps P2-P5. Among them, the shallow feature P2 (1 / 4 resolution) retains more detailed information and is crucial for the localization of small targets; the deep feature P5 (1 / 32 resolution) contains richer semantic information. Subsequently, these features are input into the deformable Transformer encoder to further extract and enhance the optimized multi-scale features S2-S5.

[0049] Step 2.2: Furthermore, in the feature pyramid, due to the continuous pooling and convolution stride operations of the backbone network, there is a significant spatial misalignment between deep feature maps (such as S5) and shallow feature maps (such as P2), which severely restricts the localization accuracy of small targets. Figure 3 As shown, the steps of the feature alignment module are as follows:

[0050] Step 2.2.1: Input multi-scale features P2 and S5, and perform 1×1 convolution operations on both to compress the number of channels and filter redundant information;

[0051] Step 2.2.2: Perform bilinear upsampling on the compressed deep feature S5 to match its spatial size with P2. Then, fuse the upsampled deep feature with the shallow feature using a concatenation operation to obtain a preliminary fused feature that integrates shallow details and deep semantics.

[0052] Step 2.2.3: The fused features are processed by a 3×3 standard convolution. This convolutional layer is used to learn the spatial offset field. For each position on the input feature map, this convolutional layer will output 2N values ​​(N equals the kernel size), representing the offset (Δx, Δy) of the N sampling points in the x and y directions, respectively.

[0053] Step 2.2.4: Using the learned offset, apply deformable convolution to the original shallow feature P2 without 1×1 convolution compression. This operation performs irregular and adaptive sampling on the P2 feature map according to the learned offset, dynamically adjusting its spatial structure to align its spatial structure with the semantic content of the deep feature S5, thereby compensating for the spatial misalignment caused by network downsampling;

[0054] Step 2.2.5: The aligned features and the upsampled S5 features are added element-wise to further fuse details and semantic information, and the aligned features are output.

[0055] Step 2.3: Further, to provide the model with prior knowledge of the target distribution, multiple deep separable dilated convolutions are used to generate a predicted density map from the fused feature map. This density map reflects the probability of small targets existing at various locations in the image, such as... Figure 4 As shown, the steps of the density map generation module are as follows:

[0056] Step 2.3.1: Perform a 1×1 convolution on the fused features to further refine the features;

[0057] Step 2.3.2: By using three cascaded depthwise separable dilated convolutions with dilation rates of 1, 3, and 5, multi-scale contextual information is captured. At the same time, a spatial attention mechanism is embedded to automatically calculate the importance weight of each spatial location, enhance the response of the target potential region, and suppress background interference.

[0058] Step 2.3.3: Optimize the features again using dilated convolution. A convolutional layer maps the features to a single-channel density map, which is then activated by a sigmoid activation function to generate a density map with the same spatial size as the input feature map. Each pixel value in this map represents the probability density of the presence of a small object at that location.

[0059] Step 2.4: Further, to achieve feature optimization for region awareness, the feature-guided enhancement module is used. This module combines the location prior knowledge provided by the density map with spatial attention and channel attention to enhance the features. For example... Figure 5 As shown, the steps of the feature-guided enhancement module are as follows:

[0060] Step 2.4.1: Downsample the density map generated in step 2.3.3 by different factors to obtain a multi-scale density weight map that matches the size of the multi-scale features (S2-S5) output by the encoder.

[0061] Step 2.4.2: For each scale feature map Si, calculate its initial spatial attention weights Ms. Specifically, Si is subjected to max pooling and average pooling along the channel dimension to obtain two single-channel feature maps with spatial dimensions H×W. These are then concatenated and processed through 1×1 convolution and sigmoid activation to obtain the initial spatial attention weights Ms. Subsequently, the density weight map at the corresponding scale is multiplied element-wise with Ms to obtain the density-modulated spatial attention map, thereby accurately locating and enhancing the spatial region of small targets. Finally, the multi-scale density-guided enhanced spatial attention map Si′ is output.

[0062] Step 2.4.3: Further apply channel attention by adjusting the weights of the channel dimensions to enhance feature representation capabilities, especially for the semantic features of small targets. Specifically, perform global max pooling and global average pooling on the spatial attention map Si′ obtained in Step 2.4.2 to obtain two 1×1×C feature maps. Compress the feature maps in the spatial dimension to reduce the image size, facilitating subsequent learning of channel features. Then, feed the two results into a shared multilayer perceptron (MLP) for learning, again using two 1×1×C feature maps. The MLP activation function is ReLU. Finally, add the two feature maps element-wise and then map them using a Sigmoid activation function to obtain the channel attention weight matrix Mc. Multiplying the spatial feature map Si′ and the channel attention weight matrix Mc yields the final enhanced feature Ei. This feature retains multi-scale information and has higher resolution and discriminative power in densely detected regions of small targets.

[0063] Step 2.5: Input the enhanced multi-scale features and learnable object queries into the decoder to complete the localization and classification of small targets: Input the enhanced features Ei into the deformable decoder, and combine them with the learnable object queries. The encoder can use the deformable attention mechanism to focus on the high-probability existence area of ​​small targets indicated by the dense map and decode the features. The target category is predicted by the classification head (fully connected layer + softmax), and the target bounding box coordinates are predicted by the regression head.

[0064] Furthermore, the number of attention heads and feature dimensions of the deformable encoder and deformable decoder are set; the kernel size, stride and number of kernels of each convolutional layer of each module are designed according to the input and functional requirements of each module of the network; and the number of target query vectors of the deformable decoder is set according to the distribution density of the target in the dataset.

[0065] The density-guided attention enhancement-based remote sensing small target detection method proposed in this embodiment achieves accurate detection of small targets in complex backgrounds through a synergistic mechanism of feature alignment, density guidance, and attention enhancement. Therefore, for remote sensing images of different resolutions, the structural parameters of each module are designed according to the scale distribution and feature representation requirements of small targets to effectively extract detailed features of small targets and suppress background interference.

[0066] In this embodiment, deep neural networks possess powerful feature learning capabilities, enabling direct end-to-end detection of remote sensing images. Feature alignment, density-guided detection, and attention enhancement modules are the core of the detection network, primarily used to enhance the feature representation of small targets. The quality of feature extraction by the network is closely related to the module structure and parameter design; therefore, the overall network architecture is determined based on the characteristics and detection challenges of small remote sensing targets. This embodiment leverages the complementarity between the detailed information of shallow features and the semantic information of deep features for small targets, achieving precise localization through feature alignment and enhancing the representational ability of small target regions using a density-guided attention mechanism.

[0067] The model network comprises: a ResNet50 backbone network, a deformable encoder, a feature alignment module, a density map generation module, a feature-guided enhancement module, a deformable decoder, and classification and regression output heads. The backbone network extracts multi-scale features; the deformable encoder enhances features through a deformable attention mechanism; the feature alignment module achieves spatial calibration of shallow and deep features through deformable convolution; the density map generation module generates target distribution priors through dilated convolution and spatial attention; the feature-guided enhancement module utilizes a density map-guided dual attention mechanism; and the deformable decoder generates prediction results through the interaction between the target query and the enhanced features.

[0068] Based on the size distribution of the input image and the scale range of small targets, the specific parameter configuration of the network in this embodiment is as follows: the number of attention heads in the deformable encoder and decoder is set to 8, and the feature dimension of each head is 64; the number of output channels of the 3×3 convolutional layer in the feature alignment module is set to 256; the dilation rate of the dilated convolution in the density map generation module is set to 1, 3, and 5 respectively; and the number of target query vectors in the deformable decoder is set to 300 to adapt to the distribution density of targets in the dataset. Through the above parameter configuration, the network can effectively balance detection accuracy and computational efficiency, achieving accurate detection of small targets in remote sensing images.

[0069] Step 3: Input the training set into the built remote sensing small target detection model for training to obtain the trained remote sensing small target detection model; and use the validation set to validate the trained remote sensing small target detection model to obtain the final remote sensing small target detection model.

[0070] Step 3.1: Input the training set into the model in batches for forward propagation. The input images are processed by the backbone network, deformable encoder, feature alignment module, density map generation module, feature guidance enhancement module, and deformable decoder, and the output target class probability and bounding box prediction are given.

[0071] In this implementation, the essence of the training process is to teach the model how to map input images to the correct target set. Let a batch of input images be... ,in The batch size. The forward propagation process of the model is the function... The process of action:

[0072] (1)

[0073] in The model is for the first The prediction results for the images include the probability distribution of the target category and the bounding box coordinates. This represents all the parameters of the model that need to be optimized.

[0074] The core challenge of training lies in matching the predictions (object queries) output by 300 models with a variable number of unordered real targets. This invention employs the Hungarian algorithm for optimal bipartite graph matching, assigning a unique prediction to each real target. Matching cost function. It consists of classification cost and bounding box cost:

[0075] (2)

[0076] in, It is the cross-entropy loss between the predicted class and the true class. and It is the L1 distance loss and GIoU loss between the predicted box and the ground truth box. , , These are hyperparameters used to balance the importance of different loss terms. By solving this matching problem, a unique "prediction-true" pairing is determined for each batch, thus transforming a set prediction problem into a supervised learning problem.

[0077] Step 3.2: Calculate the loss function, including: Focal Loss for classification loss, GIoULoss for regression loss, and the total loss is the weighted sum of classification loss and regression loss;

[0078] After completing the Hungarian matching, calculate the total loss used for backpropagation. Similar to the matching cost, the total loss also consists of multiple components, but the classification loss has been optimized by employing Focal Loss to address the class imbalance problem between small targets and background. The classification loss for a prediction matched as a positive sample is calculated as follows:

[0079] (3)

[0080] in: The model predicts that the target belongs to the true category of the match. The probability, It is a balance factor, which is set to 0.25 in this embodiment. This is the modulation factor, which is set to 2 in this embodiment. When Very small (difficult-to-classify samples) When the value approaches 1, the loss remains essentially unchanged; when When the value is large (for easily classified samples), this term approaches 0, thus reducing its loss weight. This forces the model to focus on learning small, difficult-to-classify targets.

[0081] For bounding box regression, a combination of L1 loss and GIoU loss is used:

[0082] (4)

[0083] in and These are the coordinate vectors of the ground truth bounding box and the predicted bounding box, respectively. The GIoU loss is calculated as follows:

[0084] (5)

[0085] in and These are the areas of the predicted bounding box and the ground truth bounding box, respectively. It is the area of ​​the smallest closed convex surface (smallest bounding rectangle) that contains them. GIoU not only focuses on overlapping regions, but also penalizes the separation of boxes when they do not overlap, which can provide more effective gradient guidance for bounding box regression, especially beneficial for improving the localization accuracy of small targets.

[0086] Step 3.3: Use the AdamW optimizer to iteratively update the model parameters by setting the initial learning rate, weight decay coefficient, and learning rate strategy;

[0087] In this embodiment, the AdamW optimizer is selected for parameter updates, and its formula is as follows:

[0088] (6)

[0089] in: It is the first Model parameters at the next iteration This is the initial learning rate, set to... , and These are the first and second moment estimates after bias correction. It is the weight decay coefficient, set to This helps prevent overfitting. The learning rate is scheduled using a cosine annealing strategy, where it smoothly decays from its initial value to 0 throughout the training process.

[0090] Step 3.4: During training, periodically evaluate model performance using the validation set and save the weights of the best-performing model; terminate training early when the model's performance on the validation set no longer improves.

[0091] Step 3.5: Use the model with the best performance on the validation set as the final remote sensing small target detection model.

[0092] Step 4: Input the test set into the final remote sensing small target detection model to complete the identification of different targets in the test set.

[0093] In this implementation, the AI-TOD-V2 dataset is used for model training and validation. The specific training process is as follows: the training set is input into the network in batches of 8, and Focal Loss is selected as the classification loss function (parameters...). =0.25, =2), GIoU loss is used as the regression loss function, and the total loss is the weighted sum of the two (weight ratio 1:1). The AdamW optimizer is used for parameter updates, with an initial learning rate of 1×1. The weight decay coefficient is 1×1 The learning rate is adjusted using a cosine annealing strategy.

[0094] During training, after each training epoch, the performance of the current model is evaluated using the validation set, and the mean accuracy (AP) and accuracy (A) are calculated. A Indicators, with particular attention to A for micro-targets. and A Indicators. When the validation set loss function does not decrease for 10 consecutive epochs, the learning rate is reduced to 0.5 of its original value. The convergence threshold M=20 is set. When the validation set performance (with mAP as the main reference indicator) no longer improves for 20 consecutive epochs, the model is considered to have converged, and training is terminated early. At this point, the trained remote sensing small target detection model is obtained.

[0095] To further optimize model performance and ensure its generalization ability, K-fold cross-validation was used to perform final validation on the trained model. The validation set was divided into 5 parts, with 4 parts used as the training validation set and 1 part as the test validation set in turn. This process was repeated 5 times, and the average of the 5 validation results was used as the final performance evaluation metric. The model parameters with the best overall performance during cross-validation were selected to complete the model network construction, resulting in the final remote sensing small target detection model. Using the above training strategy, the final model obtained in this implementation achieved a mAP of 26.4% on the AI-TOD-V2 test set, with a high accuracy for detecting small targets. The indicator reached 13.8%, effectively improving the accuracy of remote sensing small target detection.

[0096] Example 2

[0097] This embodiment proposes a remote sensing small target detection method based on density-guided attention enhancement. By introducing feature alignment, density map guidance, and attention enhancement modules, it achieves the detection of remote sensing small targets. The model flow is shown in Figure 1. The core steps include multi-scale feature extraction, feature alignment, density map generation, feature-guided enhancement module, and detection result output.

[0098] The specific steps are as follows:

[0099] Step 1: Multi-scale feature extraction;

[0100] Model structure as follows Figure 2 As shown, in order to obtain multi-scale feature information of targets in remote sensing images, the remote sensing image is first input, and features at different scales are extracted using the backbone network ResNet50 and the deformable Transformer encoder. Among them, shallow features retain detailed information, while deep features contain richer semantic information.

[0101] Step 1-1: Obtain the training dataset for small target detection in remote sensing images and perform preprocessing operations on the dataset;

[0102] Steps 1-2: Feature extraction is performed using ResNet50 convolutional neural images, where shallow features (such as P2) retain more detailed information;

[0103] Steps 1-3: Input the features extracted from the backbone network into the deformable encoder to further extract multi-scale features, among which deep features (such as S5) contain richer semantic information.

[0104] Step 2: Feature alignment processing;

[0105] In the feature pyramid, due to the continuous pooling and convolution stride operations of the backbone network, there is a significant spatial misalignment between deep feature maps (such as S5) and shallow feature maps (such as P2). This misalignment leads to a mismatch between the precise localization information of shallow features and the rich semantic information of deep features during direct fusion, severely limiting the localization accuracy of small targets. To address this, this invention introduces a feature alignment module to actively compensate for this spatial bias, laying the foundation for subsequent accurate feature fusion and enhancement.

[0106] Step 2-1: Initial state, input multi-scale features P2 and S5, and perform 1×1 convolution operation on P2 and S5 respectively to compress the number of channels, filter redundant information, and reduce computational complexity;

[0107] Step 2-2: Perform bilinear upsampling on the compressed deep feature S5 to match its spatial size with P2. Then, fuse the upsampled deep feature with the shallow feature using a concatenation operation to obtain a preliminary fused feature that integrates shallow details and deep semantics.

[0108] Steps 2-3: The fused features are then passed through a standard 3×3 convolution. The key role of this convolutional layer is not to output features, but to learn the spatial offset field for subsequent deformable convolutions. For each location on the input feature map, this convolutional layer outputs 2N values ​​(N equals the kernel size), which represent the offsets (Δx, Δy) of the N sampling points in the x and y directions, respectively.

[0109] Steps 2-4: Using the offsets learned in Step 2-3, we apply deformable convolution to the original shallow features P2 that have not been compressed by 1×1 convolution. The deformable convolution samples at irregular, adaptive positions on the P2 feature map based on the learned offsets. This process dynamically "stretches" the P2 feature map, aligning its spatial structure with the semantic content of the deep features S5, thereby accurately compensating for spatial misalignment caused by network downsampling.

[0110] Steps 2-5: The aligned features and the upsampled S5 features are further fused together through element-wise addition to obtain detailed and semantic information, resulting in aligned features that maintain high resolution and are rich in semantics.

[0111] Step 3: Build the density map generation module;

[0112] This invention utilizes multi-layer deep separable dilated convolution to generate a predicted density map from the fused feature map. This density map reflects the probability of small targets existing at various locations in the image, providing the model with prior knowledge of target distribution.

[0113] Step 3-1: Perform a 1×1 convolution on the fused features to further refine the features;

[0114] Step 3-2: By using three cascaded depthwise separable dilated convolutions with dilation rates of 1, 3, and 5, multi-scale contextual information is captured. At the same time, a spatial attention mechanism is embedded to automatically calculate the importance weight of each spatial location, enhance the response of the target potential region, and suppress background interference.

[0115] Step 3-3: Optimize the features again using dilated convolution. A convolutional layer maps the features to a single-channel density map, which is then activated by a sigmoid function to generate a density map with the same spatial dimensions as the input feature map. Each pixel value in this map represents the probability density of the presence of a small object at that location.

[0116] Step 4: Build the feature-guided enhancement module;

[0117] This invention utilizes a feature-guided enhancement module, which combines the location prior knowledge provided by the density map with spatial attention and channel attention to enhance features, highlight the features of small remote sensing targets, and achieve feature optimization for region perception.

[0118] like Figure 5 As shown, the steps to build the feature-guided enhancement module are as follows:

[0119] Step 4-1: Downsample the density map generated in Step 3-3 by different factors to obtain a multi-scale density weight map that matches the size of the multi-scale features (S2-S5) output by the encoder.

[0120] Step 4-2: For each scale feature map Si, use its corresponding density weight map as a guide to calculate the spatial attention weights. Specifically, perform max pooling and average pooling on Si along the channel dimension to obtain two single-channel feature maps with spatial dimensions H×W. Concatenate these two feature maps along the channel dimension, perform a 1×1 convolution operation on the result to reduce the number of channels to 1, and then map them through the Sigmoid activation function to obtain the initial spatial attention weights M. s Then, the multi-scale density weight map obtained in step 4-1 is compared with the initial spatial attention weights M at the corresponding scales. sElement-wise multiplication yields a density-modulated spatial attention map, which helps the model more accurately locate and enhance the spatial regions of small targets. All these operations are performed within a single-scale feature map, ultimately outputting a multi-scale density-guided enhanced spatial attention map Si′.

[0121] Step 4-3: Next, channel attention is further applied. By adjusting the weights of the channel dimensions, the feature representation capability is further enhanced, especially the semantic features of small targets. Specifically, global max pooling and global average pooling are performed on the spatial attention map Si′ obtained in Step 4-2 to obtain two 1×1×C feature maps. The feature maps are compressed in the spatial dimension to reduce the image size, which facilitates the subsequent learning of channel features. Then, the two results are fed into a shared multilayer perceptron (MLP) for learning. Again, two 1×1×C feature maps are used. The activation function of the MLP is ReLU. Finally, the two feature maps are added element-wise and then mapped through a Sigmoid activation function to obtain the channel attention weight matrix M. c Then the spatial feature map Si′ and the channel attention weight matrix M are... c Multiplying these results yields the final enhanced feature Ei. This feature retains multi-scale information and offers higher resolution and discriminative power in densely populated areas of small targets.

[0122] Step 5: Target detection and result output;

[0123] The enhanced multi-scale features and learnable object queries are fed into the decoder for decoding, thus completing the localization and classification of small targets.

[0124] Step 5-1: Input the enhanced feature Ei into the deformable decoder. Combined with the learnable object query, the encoder can focus on the high-probability existence area of ​​small targets indicated by the dense map through the deformable attention mechanism to decode the features.

[0125] Step 5-2: Predict the target category using the classification head (fully connected layer + softmax) and predict the target bounding box coordinates using the regression head;

[0126] Step 5-3: The decoder iterates through 6 layers to gradually optimize the category score and bounding box coordinates of each query. After each iteration, the semantic and location accuracy of the query continues to improve.

[0127] Step 5-4: The last layer of the decoder outputs the feature vector for each query. The classification branch predicts the class label for each query and uses Focal Loss to handle the class imbalance problem where small objects are easily classified as background. The regression branch predicts the bounding box coordinates for each query and uses GIou Loss to optimize the overlap between the bounding box and the ground truth box.

[0128] Step 5-5: To further optimize the output results, standard non-maximum suppression (NMS) post-processing can be used to sort all predicted boxes by category score, suppress duplicate boxes with the highest overlap, and finally output the category label and bounding box coordinates of each target.

[0129] First, a dataset of small targets in remote sensing images is acquired. After preprocessing, the dataset is divided into training, validation, and test sets according to a set ratio, providing standardized input for subsequent network training and validation. The input images are processed by a ResNet50 convolutional neural network backbone and a deformable encoder to extract multi-scale features P2-P5 and their optimized counterparts S2-S5. P2 and S5 are then input into a feature alignment module for cross-scale alignment and fusion, alleviating cross-scale misalignment while fusing shallow detail features with deep semantic features. The fused features are then input into a density map generation module, which captures the spatial distribution model of small targets through operations such as dilated convolution, generating a probability density map of small target distribution, providing prior knowledge of small target location for subsequent feature enhancement. Based on the prior knowledge of the density map, S2-S5 is enhanced with channel and spatial dual-dimensional features to further improve feature discrimination. Finally, the target query vector and the enhanced features are input into the deformable decoder for decoding and prediction of classification results and bounding boxes, completing the detection and localization of remote sensing small targets.

[0130] Example 3

[0131] This embodiment relates to a remote sensing small target detection device based on density-guided attention enhancement, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the remote sensing small target detection method based on density-guided attention enhancement of Embodiment 1.

[0132] Example 4

[0133] This embodiment relates to a computer-readable storage medium storing a program that, when executed by a processor, implements the remote sensing small target detection method based on density-guided attention enhancement as described in Embodiment 1.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it.

Claims

1. A remote sensing small target detection method based on density-guided attention enhancement, characterized in that, Includes the following steps: Step 1: Obtain the preprocessed remote sensing image dataset and divide it proportionally into training, validation, and test sets, including: Step 1.1: Obtain remote sensing images containing small targets, and label the category labels and bounding box coordinates of the small targets to construct a dataset; Step 1.2: Preprocess the images in the dataset, including data size normalization, data augmentation, and pixel value normalization; Step 1.3: Divide the preprocessed dataset into training set, validation set and test set according to the set ratio; Step 2: Construct a remote sensing small target detection model based on density-guided attention enhancement. The structure of the remote sensing small target detection model is as follows: starting from the input layer, it sequentially connects to a backbone network, a deformable encoder, a feature alignment module, a density map generation module, a feature-guided enhancement module, a deformable decoder, and an output layer. The backbone network is used to extract multi-scale features; the deformable encoder is used to initially enhance multi-scale features; the feature alignment module is used to achieve spatial alignment and fusion of multi-scale features; the density map generation module is used to generate a density map reflecting the probability distribution of small targets from the fused features; the feature-guided enhancement module is used to further enhance multi-scale features using the density map-guided attention; the deformable decoder is used to receive the finally enhanced multi-scale features and output the prediction result through decoding operations; the output layer is used to output the final target category and bounding box coordinates. Step 3: Input the training set into the built remote sensing small target detection model for training to obtain the trained remote sensing small target detection model; and use the validation set to validate the trained remote sensing small target detection model to obtain the final remote sensing small target detection model. Step 4: Input the test set into the final remote sensing small target detection model to complete the identification of different targets in the test set.

2. The remote sensing small target detection method based on density-guided attention enhancement as described in claim 1, characterized in that, The structure of the remote sensing small target detection model further includes: the backbone network adopts a ResNet50 architecture, containing five convolutional stages, which sequentially output multi-scale feature maps P2, P3, P4, and P5, where P2 retains the detailed information of small targets and P5 contains deep semantic information; the deformable encoder consists of multiple deformable attention layers, each of which contains a multi-head deformable self-attention module and a feedforward network, which enhances the P2-P5 features output by the backbone network to generate optimized multi-scale features S2-S5; the feature alignment module sequentially includes the following components: a first convolutional layer, a bilinear interpolation upsampling layer, a feature concatenation layer, a second convolutional layer, a deformable convolutional layer, and a residual connection. The density map generation module sequentially includes: a third convolutional layer, three parallel dilated convolutional layers, a spatial attention submodule, and a fourth convolutional layer for output transformation; the feature-guided enhancement module includes: a multi-scale pooling layer, a feature concatenation layer, a spatial attention submodule, a channel attention submodule, and a feature fusion layer; the deformable decoder consists of multiple cascaded deformable attention layers, each containing a cross-attention module and a feedforward network; the output layer contains two independent components: a classification head consisting of two fully connected layers and a Softmax activation function, responsible for outputting the class probability distribution of the target; and a regression head also consisting of two fully connected layers, used to predict the coordinate offset of the bounding box.

3. The remote sensing small target detection method based on density-guided attention enhancement as described in claim 2, characterized in that, The process of building a remote sensing small target detection model also includes: setting the number of attention heads and feature dimensions of the deformable encoder and deformable decoder; designing the kernel size, stride and number of kernels of each convolutional layer of each module according to the input and functional requirements of each module of the network; and setting the number of target query vectors of the deformable decoder according to the distribution density of targets in the dataset.

4. The remote sensing small target detection method based on density-guided attention enhancement as described in claim 3, characterized in that, Step 3 includes: Step 3.1: Input the training set into the model in batches for forward propagation. The input images are processed by the backbone network, deformable encoder, feature alignment module, density map generation module, feature guidance enhancement module, and deformable decoder, and the output target class probability and bounding box prediction are given. Step 3.2: Calculate the loss function, including: Focal Loss for classification loss, GIoU Loss for regression loss, and the total loss is the weighted sum of classification loss and regression loss; Step 3.3: Use the AdamW optimizer to iteratively update the model parameters by setting the initial learning rate, weight decay coefficient, and learning rate strategy; Step 3.4: During training, periodically evaluate model performance using the validation set and save the weights of the best-performing model; terminate training early when the model's performance on the validation set no longer improves. Step 3.5: Use the model with the best performance on the validation set as the final remote sensing small target detection model.

5. A remote sensing small target detection device based on density-guided attention enhancement, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the remote sensing small target detection method based on density-guided attention enhancement as described in any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the remote sensing small target detection method based on density-guided attention enhancement as described in any one of claims 1-4.

Images