Infrared image small target detection system based on spatial multi-scale adaptive gating and attention aggregation
By combining multi-scale contextual attention aggregation and gated spatial attention fusion modules with a deep supervised loss function, the problem of insufficient feature modeling in infrared image small target detection is solved, and more efficient infrared small target detection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
- Filing Date
- 2025-12-13
- Publication Date
- 2026-05-01
AI Technical Summary
Existing infrared image small target detection methods fail to effectively consider the characteristics of infrared small targets, resulting in poor detection performance. In particular, it is difficult to distinguish targets from background noise in complex backgrounds with low signal-to-noise ratio and lack of texture information.
We employ a multi-scale contextual attention aggregation module, a gated spatial attention fusion module, and a deep supervised loss function. By fusing feature extraction, spatial and channel attention enhancement, we dynamically select meaningful spatial features and optimize gradient propagation through a deep supervised loss function to improve feature representation capabilities.
It improves the performance of small target detection in infrared images, enhances the ability to identify small targets, and improves the robustness and inference speed of the detection model.
Smart Images
Figure CN121962572A_ABST
Abstract
Description
A Small Target Detection System for Infrared Images Based on Spatial Multi-Scale Adaptive Gating and Attention Convergence Technical Field
[0001] This invention relates to the fields of infrared image processing and computer vision technology, and in particular to an infrared image small target detection system based on spatial multi-scale adaptive gating and attention aggregation. Background Technology
[0002] Compared to visible light, infrared light has stronger penetrating power and can adapt to complex environments, especially in low-light conditions, where infrared imaging can effectively acquire target information. Infrared small target detection, as a common and powerful technique, is widely used in fields such as wilderness rescue, military early warning, and security surveillance. Although deep learning technology has made good progress in ordinary natural optical images, compared with ordinary optical images, infrared images, due to their imaging mechanism, exhibit characteristics such as small and dark targets, low signal-to-noise ratio, and lack of texture information. The limited information content and complex backgrounds of infrared small targets often lead to poor detection results in infrared images. Therefore, further improving the detection performance of small targets in infrared images is an urgent need to promote the automation, intelligence, and efficiency of infrared image detection technology.
[0003] Current popular methods for small target detection in infrared images involve feature modeling of the infrared image during the feature extraction process of the network. This method is generally achieved by enhancing local information, contextual information, or fusion enhancement of the feature map. However, most of these methods only consider a few aspects and are too simple to delve into the characteristics of the infrared small target itself. As a result, the detection network cannot accurately model the infrared small target and distinguish the target from background noise. Summary of the Invention
[0004] The technical problem to be solved by this invention is to provide an infrared image small target detection system based on spatial multi-scale adaptive gating and attention aggregation, which can solve the problem of low infrared small target detection performance caused by insufficient feature modeling.
[0005] To address the aforementioned technical problems, this invention provides an infrared image small target detection system based on spatial multi-scale adaptive gating and attention aggregation, comprising: a multi-scale contextual attention aggregation module, a gated spatial attention fusion module, and a deep supervised loss function; the multi-scale contextual attention aggregation module includes a fusion feature extraction module, a spatial attention aggregation module, and a channel attention aggregation module. The fusion feature extraction module extracts fusion features from infrared images at different scales, the spatial attention aggregation module assigns different spatial weights to the image to enhance spatial information in the infrared image, and the channel attention aggregation module assigns different weights to different channels of the image to enhance channel information in the infrared image; the gated spatial attention fusion module includes a multi-scale feature fusion module, a spatial selection module, and a spatial cross-modulation module. The multi-scale feature fusion module performs semantic aggregation of features at different resolutions, the spatial selection module dynamically selects meaningful spatial features from the infrared image to enhance the spatial information expression of the infrared image, and the spatial cross-modulation module uses local and global contextual information to enrich the feature map, suppress noise, and enhance the local feature expression of the target region; the deep supervised loss function is a combination of the multi-scale prediction map and the ground truth label cross-entropy loss.
[0006] Furthermore, the fusion feature extraction module achieves feature alignment by upsampling or downsampling operations to address the multi-scale characteristics of infrared small targets. These represent four feature maps at different resolutions extracted by the backbone network. The dimensions of the input image. For the corresponding number of channels, the aligned feature map is as follows: :
[0007]
[0008]
[0009]
[0010] .
[0011] in In order to Perform contextual information aggregation for We selected dimensionality reduction parameters, using 1×1 convolution to reduce the dimensionality by a factor of 60, and further extracted image features.
[0012]
[0013] Next Multi-scale features are extracted using convolutional kernels of different sizes and then summed to generate multi-scale fused features. ;
[0014] .
[0015] Furthermore, the spatial attention aggregation module focuses on features. To perform spatial attention calculations, taking into account the low contrast between the target and background in infrared images, the spatial attention module calculates spatial weights using a 7×7 convolutional kernel. Strengthen the salience of small target areas;
[0016]
[0017]
[0018] in It is a feature Spatial weights, It is the result of features being spatially attention-weighted, where For the Hadamard operator.
[0019] Furthermore, the channel attention aggregation module will Channel-weighted calculations are performed, and channel weights are learned through global pooling and fully connected layers. Suppress redundant noise channels;
[0020]
[0021]
[0022] The obtained spatial attention and channel attention are interacted through a pointwise product, a residual connection is added to ensure stability, and the input channels are recovered through a 1×1 convolutional layer. The aggregated multi-scale context information is obtained.
[0023]
[0024] Output of feature extraction module and Perform residual connection;
[0025]
[0026] Finally, the aggregated infrared image feature map is obtained. Together with Input gating attention fusion module.
[0027] Furthermore, the multi-scale feature fusion module integrates local and global contextual information, fusing high-resolution spatial details with low-resolution semantic information; the gated spatial attention fusion module receives two inputs and is computed by the deep and shallow layers of the network, in the following order: Get output , Get output , Get output , Get output First, high-resolution details With low resolution details The data is fed into a multi-scale fusion module to extract both local contextual information and global information. Performing a 2x upsampling makes the size and same
[0028]
[0029]
[0030] in It is a depthwise separable convolution. Diffuse convolution with an inflation factor of 2. For average pooling, For max pooling, Perform channel adjustments, then obtain the local context features. and global features Add
[0031]
[0032] It includes features at multiple scales, and then... Send to the space selection module.
[0033] Furthermore, the spatial selection module is designed with a gating mechanism to obtain two spatial weights. and The two weights are respectively with Pixel-by-pixel multiplication to emphasize Identify key areas in the space and perform residual connections;
[0034]
[0035]
[0036]
[0037] in To normalize, the spatially refined result is then... It is fed into the spatial cross-modulation module.
[0038] Furthermore, the spatial cross-modulation module achieves complementarity between local and global features through sigmoid gating, enriching local details with global context details, and adjusting global features with rich local details, thereby improving robustness to small targets;
[0039]
[0040]
[0041] Finally, pixel-wise multiplication is used to stabilize the fusion features and obtain the fused features of high resolution and low resolution. ;
[0042]
[0043] As a new , As a new The input is sent to the entry-level spatial attention fusion module for calculation, and finally obtained
[0044] .
[0045] Furthermore, the deep supervised loss function is composed of a combination of cross-entropy loss calculations of multiple resolution prediction maps and ground truth labels, which... Using 1×1 convolution and Function to obtain multi-resolution prediction map ,in ;
[0046]
[0047] Secondly, Perform bilinear interpolation splicing Obtain the overall forecast map ;
[0048]
[0049] in For bilinear interpolation, the cross-entropy loss between the overall predicted map and the true value GT is then calculated;
[0050]
[0051] Next, the cross-entropy loss between each multi-resolution prediction map and the true value GT is calculated;
[0052]
[0053]
[0054] in The predicted graph needs to be upsampled to the same size as the ground plane before calculating the cross-entropy loss. Therefore, the overall loss function of the network is:
[0055]
[0056] in Set to 1.
[0057] The beneficial effects of the present invention are as follows: (1) A multi-scale contextual attention aggregation module is designed, which can aggregate multi-level features of infrared images in the network and use spatial attention and channel attention to enhance the global contextual information and local information of the input image, so that the network can pay attention to more useful spatial information and channel information and generate more effective features; (2) A gated spatial attention fusion module is designed. This module is located after the multi-scale contextual attention aggregation module. This module aggregates features of different resolutions with semantic similarity. The gating is designed so that the network can dynamically select meaningful spatial features and use spatial interaction modulation so that the features can pay attention to both local information and global contextual information; (3) A deep-supervised multi-scale prediction map combination loss function is designed. This structure sets loss functions in multiple intermediate layers to calculate weighted loss, which can improve gradient propagation efficiency; guide the detection network to generate meaningful feature maps, improve the expressive power of features, and thus improve the detection performance of small targets. In the testing phase, ordinary supervision is used to replace this module to achieve lightweight detection model and improve inference speed. Attached Figure Description
[0058] Figure 1 is a schematic diagram of the system structure of the present invention.
[0059] Figure 2 is a system architecture diagram of the present invention.
[0060] Figure 3 is a diagram of the multi-scale contextual attention aggregation module of the present invention.
[0061] Figure 4 is a diagram of the gated spatial attention fusion module of the present invention.
[0062] Figure 5 shows the detection results of this invention on the test dataset. Detailed Implementation
[0063] As shown in Figure 1, an infrared image small target detection system based on spatial multi-scale adaptive gating and attention aggregation includes: a multi-scale contextual attention aggregation module, a gated spatial attention fusion module, and a deep supervised loss function. The multi-scale contextual attention aggregation module includes a fusion feature extraction module, a spatial attention aggregation module, and a channel attention aggregation module. The fusion feature extraction module is used to extract fusion features from infrared images at different scales. The spatial attention aggregation module is used to assign different spatial weights to the image to enhance the spatial information in the infrared image. The channel attention aggregation module is used to assign different weights to different channels of the image to enhance the channel information in the infrared image. The gated spatial attention fusion module includes a multi-scale feature fusion module, a spatial selection module, and a spatial cross-modulation module. The multi-scale feature fusion module is used to semantically aggregate features at different resolutions. The spatial selection module dynamically selects meaningful spatial features from the infrared image to enhance the spatial information expression of the infrared image. The spatial cross-modulation module uses local and global contextual information to enrich the feature map, suppress noise, and enhance the local feature expression of the target region. The deep supervised loss function is the cross-entropy loss of the multi-scale prediction map and the ground truth label.
[0064] During the training phase, the Adam optimizer is used to optimize the network weight parameters, the batch size is set to 16, and the initial learning rate is... The learning rate is set to 0.001, and a cosine annealing strategy is used to gradually reduce the learning rate to 10. -5 The network training iterates until the loss function stabilizes, at which point training stops. During training, the total training epochs are set to 1000. Performance verification is performed every 10 iterations, and the model is saved. During detection, test images are input to verify the method's performance, and the experimental results are saved.
[0065] As shown in Figure 2, the image X is input into the U-Net residual block structure to extract feature maps of different sizes. :
[0066]
[0067] Will Four feature maps of different sizes are input into the multi-scale context aggregation attention module. The multi-scale context aggregation attention module is shown in Figure 3. First, the feature maps are input into the multi-scale context aggregation attention module. Input fusion feature extraction module to obtain ;
[0068]
[0069]
[0070]
[0071] .
[0072]
[0073]
[0074] Next, the spatial attention aggregation module processes the features. Perform spatial attention calculations:
[0075]
[0076]
[0077] in It is a feature Spatial weights, It is the result of features being spatially attention-weighted, where For the Hadamard operator. The channel attention aggregation module will... Perform channel-weighted calculations to calculate the channel weights of features and learn the correlation between channels:
[0078]
[0079]
[0080] The obtained spatial attention and channel attention are interacted through a pointwise product, a residual connection is added to ensure stability, and the input channels are recovered through a 1×1 convolutional layer. ;
[0081]
[0082] Will and Perform residual connection;
[0083]
[0084] Finally, the aggregated infrared image feature map is obtained. Together with Input gating attention fusion module.
[0085] The next step is for the gating spatial attention fusion module to analyze features. Semantic similarity fusion is performed. The gated spatial attention fusion module is shown in Figure 4. To unify the symbols... Named The multi-scale feature fusion module integrates local and global contextual information, fusing high-resolution spatial details with low-resolution semantic information. The gated spatial attention module receives two inputs and is computed by both deep and shallow layers of the network. Therefore, the computation order is as follows: Get output , Get output , Get output , Get output First, The data is fed into a multi-scale fusion module for simultaneous extraction of local contextual information and global information. Performing a 2x upsampling makes the size and same;
[0086]
[0087]
[0088] in It is a depthwise separable convolution. Diffuse convolution with an inflation factor of 2. For average pooling, For max pooling, Perform channel adjustments. Then, use the obtained local context features... and global features Add
[0089]
[0090] It includes features at multiple scales, and then... The data is fed into the space selection module. The space selection module employs a gating mechanism to obtain two space weights. These two weights are respectively correlated with… Pixel-by-pixel multiplication is performed to emphasize important areas in space, and residual connections are made.
[0091]
[0092]
[0093]
[0094] in To normalize, the spatially refined result is then... The data is fed into the spatial cross-modulation module. The spatial cross-modulation module enriches local details with global context details, and adjusts global features with the enriched local details.
[0095]
[0096]
[0097] Finally, pixel-wise multiplication is used to stabilize the fused features:
[0098]
[0099] As a new , As a new The input is sent to the entry-level spatial attention fusion module for calculation, and the final result is:
[0100]
[0101] The next step is to use a deep supervision strategy to calculate the cross-entropy loss between prediction maps at multiple resolutions and the ground truth labels. This will be done to unify the notation. Named First of all, Using 1×1 convolution and The function obtains multi-resolution prediction maps:
[0102]
[0103] Secondly, Perform bilinear interpolation splicing Obtain the overall forecast map:
[0104]
[0105] in For bilinear interpolation, the cross-entropy loss between the overall predicted plot and the true value GT is then calculated:
[0106]
[0107] Next, the cross-entropy loss between each multi-resolution predicted map and the true value GT is calculated:
[0108]
[0109]
[0110] in The predicted graph needs to be upsampled to the same size as the ground plane before calculating the cross-entropy loss; therefore, the overall network loss function is:
[0111]
[0112] in Set to 1 based on experience. Use a combined loss function and The parameters of the detection network are optimized using the total loss function of the detection network. After multiple iterations of optimization, the final optimal network model is obtained.
[0113] During the testing phase, test samples are input into the network to obtain the final detection results.
[0114] Figure 5 shows the detection effect of this embodiment, where a and c are ground detection label images, and b and d are detection result images after using the present invention. Based on the detection results in Figure 5, the detection network based on spatial multi-scale adaptive gating and attention aggregation proposed in this invention can effectively detect small targets in infrared images.
Claims
1. A small target detection system for infrared images based on spatial multi-scale adaptive gating and attention aggregation, characterized in that, include: Multi-scale contextual attention aggregation module, gated spatial attention fusion module, and deep supervised loss function; The multi-scale contextual attention aggregation module includes a fusion feature extraction module, a spatial attention aggregation module, and a channel attention aggregation module. The fusion feature extraction module is used to extract fusion features from infrared images at different scales. The spatial attention aggregation module is used to assign different spatial weights to the image to enhance the spatial information in the infrared image. The channel attention aggregation module is used to assign different weights to different channels of the image to enhance the channel information in the infrared image. The gated spatial attention fusion module includes a multi-scale feature fusion module, a spatial selection module, and a spatial cross-modulation module. The multi-scale feature fusion module is used to semantically aggregate features of different resolutions. The spatial selection module dynamically selects meaningful spatial features from the infrared image to enhance the spatial information expression of the infrared image. The spatial cross-modulation module uses local and global contextual information to enrich the feature map, suppress noise, and enhance the local feature expression of the target region. The deep supervision loss function is the cross-entropy loss of the multi-scale prediction map and the real label.
2. The infrared image small target detection system based on spatial multi-scale adaptive gating and attention aggregation as described in claim 1, characterized in that, The fusion feature extraction module achieves feature alignment for the multi-scale characteristics of small infrared targets through upsampling or downsampling operations. These represent four feature maps at different resolutions extracted by the backbone network. The dimensions of the input image. For the corresponding number of channels, the aligned feature map is as follows: ： in In order to Perform contextual information aggregation for We selected dimensionality reduction parameters, using 1×1 convolution to reduce the dimensionality by a factor of 60, and further extracted image features. Next Multi-scale features are extracted using convolutional kernels of different sizes and then summed to generate multi-scale fused features. ； 。 3. The infrared image small target detection system based on spatial multi-scale adaptive gating and attention convergence as described in claim 1, characterized in that, Spatial attention aggregation module for features Spatial attention is calculated, and the spatial attention module uses 7×7 convolutional kernels to compute spatial weights. Strengthen the salience of small target areas; in It is a feature Spatial weights, It is the result of features being spatially attention-weighted, where For the Hadamard operator.
4. The infrared image small target detection system based on spatial multi-scale adaptive gating and attention aggregation as described in claim 1, characterized in that, The channel attention aggregation module will Channel-weighted calculations are performed, and channel weights are learned through global pooling and fully connected layers. Suppress redundant noise channels; The obtained spatial attention and channel attention are interacted through a pointwise product, a residual connection is added to ensure stability, and the input channels are recovered through a 1×1 convolutional layer. The aggregated multi-scale context information is obtained. Output of feature extraction module and Perform residual connection; Finally, the aggregated infrared image feature map is obtained. Together with Input gating attention fusion module.
5. The infrared image small target detection system based on spatial multi-scale adaptive gating and attention aggregation as described in claim 1, characterized in that, The multi-scale feature fusion module integrates local and global contextual information, fusing high-resolution spatial details with low-resolution semantic information; the gated spatial attention fusion module receives two inputs and is computed by the deep and shallow layers of the network, in the following order: Get output ， Get output ， Get output ， Get output ； First, high-resolution details With low resolution details The data is fed into a multi-scale fusion module to extract both local contextual information and global information. Performing a 2x upsampling makes the size and same in It is a depthwise separable convolution. Diffuse convolution with an inflation factor of 2. For average pooling, For max pooling, Perform channel adjustments, then obtain the local context features. and global features Add It includes features at multiple scales, and then... Send to the space selection module.
6. The infrared image small target detection system based on spatial multi-scale adaptive gating and attention aggregation as described in claim 1, characterized in that, The spatial selection module is designed with a gating mechanism to obtain two spatial weights. and The two weights are respectively with Pixel-by-pixel multiplication to emphasize Identify key areas in the space and perform residual connections; in To normalize, the spatially refined result is then... It is fed into the spatial cross-modulation module.
7. The infrared image small target detection system based on spatial multi-scale adaptive gating and attention convergence as described in claim 1, characterized in that, The spatial cross-modulation module achieves complementarity between local and global features through sigmoid gating; Finally, pixel-wise multiplication is used to stabilize the fusion features and obtain the fused features of high resolution and low resolution. ； As a new ， As a new The input is sent to the entry-level spatial attention fusion module for calculation, and finally obtained 。 8. The infrared image small target detection system based on spatial multi-scale adaptive gating and attention aggregation as described in claim 1, characterized in that, The deep supervised loss function is composed of a combination of cross-entropy loss calculations between multiple resolution prediction maps and the ground truth labels. Using 1×1 convolution and Function to obtain multi-resolution prediction map ,in ； Secondly, Perform bilinear interpolation splicing Obtain the overall forecast map ； in For bilinear interpolation, the cross-entropy loss between the overall predicted map and the true value GT is then calculated; Next, the cross-entropy loss between each multi-resolution prediction map and the true value GT is calculated; in The predicted graph needs to be upsampled to the same size as the ground plane before calculating the cross-entropy loss. Therefore, the overall loss function of the network is: in Set to 1.