Infrared small target detection method based on multi-scale nested attention network

Abstract

This invention discloses an infrared small target detection method based on a multi-scale nested attention network, belonging to the field of image processing technology. The method includes the following steps: constructing a multi-scale densely nested U-shaped encoder-decoder backbone network; embedding parallel contextual image block attention modules into each scale layer of the encoder in the backbone network to extract and fuse multi-dimensional features of the feature map, generating enhanced features for each scale layer of the encoder; embedding a hole-frequency multi-scale channel enhancement module into the decoder layer of the backbone network to extract and fuse multi-receptive field spatial features and frequency domain features of the feature map; constructing a multi-scale skip connection structure in the backbone network to fuse the low-level spatial detail features of the encoder with the high-level semantic features of the decoder layer by layer; applying an activation function to the final output feature map to generate a normalized saliency map; and using a pixel-level segmentation algorithm to extract the target region from the saliency map, thereby achieving the detection and localization of infrared small targets.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an infrared small target detection method based on a multi-scale nested attention network. Background Technology

[0002] Infrared small target detection is a crucial technology in image processing and target detection, possessing core application value in both military and security fields. It is widely used in scenarios such as early battlefield warning, air and sea target monitoring, missile guidance, border patrol, and autonomous perception for unmanned systems. The core technical challenge lies in the fact that distant targets occupy only a small number of pixels in the image. After atmospheric attenuation, sensor noise, and complex background clutter, the image signal-to-noise ratio and local contrast are significantly low. Targets exhibit weak texture, irregular shapes, and discontinuous edges, making them highly susceptible to confusion with background interference such as cloud edges and sea clutter. This places stringent demands on the target perception and background suppression capabilities of the detection method.

[0003] Existing infrared small target detection methods are mainly divided into two categories: model-driven and data-driven. Model-driven methods rely on prior assumptions about the features of the image background and target, and achieve target enhancement by manually designing filters or constructing local contrast measures. However, they have poor robustness in complex environments and are prone to false alarms and missed detections. Data-driven methods learn discriminative features by training neural networks, which has better performance, but multiple downsampling can easily lead to severe loss of fine-grained structural information of small targets. Furthermore, the design of multi-scale feature fusion is insufficient, and it cannot fully model the contrast characteristics of targets and backgrounds at different scales, ultimately resulting in poor target perception and background suppression in complex backgrounds. Therefore, there is an urgent need to design a high-performance detection model with high-resolution feature extraction capabilities, strong background interference suppression capabilities, and adaptability to multi-scale target modeling requirements to meet the engineering and practical requirements of infrared small target detection in complex application environments. Summary of the Invention

[0004] The purpose of this invention is to provide an infrared small target detection method based on a multi-scale nested attention network to solve the problems raised in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an infrared small target detection method based on a multi-scale nested attention network, the method comprising the following steps:

[0006] Construct a multi-scale densely nested U-shaped encoding-decoding backbone network; Parallel context image block attention modules are embedded in each scale layer of the encoder in the backbone network to extract and fuse multi-dimensional features of the feature maps, generating enhanced features for each scale layer of the encoder. A hole-frequency multi-scale channel enhancement module is embedded in the decoder layer of the backbone network to extract and fuse multi-receptive field spatial features and frequency domain features of the feature map. A multi-scale jump connection structure is constructed in the backbone network to fuse the low-level spatial detail features of the encoder with the high-level semantic features of the decoder layer by layer. Apply the Sigmoid activation function to the feature map of the final output of the backbone network to generate a normalized saliency map; A pixel-level segmentation algorithm is used to extract the target region from the saliency map, thereby enabling the detection and localization of small infrared targets.

[0007] Specifically, it can be divided into the following steps: Step S1: Construct a multi-scale densely nested attention network based on an improved Transformer; The multi-scale dense nested attention network based on the improved Transformer specifically refers to the following: under a unified U-shaped encoder-decoder network framework, by sequentially executing the parallel context image block attention module construction described in step S2, the hole-frequency multi-scale channel enhancement module construction described in step S3, and the multi-scale skip-connection feature fusion operation described in step S4, corresponding attention mechanisms and multi-scale feature modeling units are gradually introduced at different levels of the network, ultimately forming a complete multi-scale dense nested attention network structure.

[0008] Step S2: Establish a parallel contextual image patch attention module; The module includes multiple parallel attention branches. Each branch constructs attention weights for different ranges of image regions and performs block-level weighting, feature selection, and channel recalibration on the branch output to form a fusion feature representation that takes into account both local salient information and global semantic information. Step S3: Establish the hole-frequency multi-scale channel enhancement module; The module contains multiple dilated convolution branches with different dilation rates and frequency branches, which extract target edge and shape feature information under different receptive fields respectively; Step S4: A multi-scale jump connection structure is used to establish connections between each layer of the encoder and the corresponding layer of the decoder, so that the spatial detail features contained in the lower layers of the encoder can be fully integrated with the higher semantic features in the decoding and reconstruction stage. Step S5: The final output layer introduces the Sigmoid activation function to normalize the output results and generate a saliency map with values ​​between 0 and 1. Step S6: Use a pixel-level segmentation algorithm to extract regions from the saliency map and separate small infrared targets from the complex background.

[0009] Furthermore, in step S2, the specific processing procedure is as follows: S21: First, convolution is performed on the input feature map of the current scale layer to obtain the feature map; at the same time, three parallel attention branches are constructed. S22: Establish local convolution branches to obtain local enhanced features F local ; S23: Establish a global convolutional branch to obtain the global enhanced feature F. global ; S24: Establish a sequential convolution branch to obtain the structural enhancement feature F. conv ; S25: Output the local branch F local Global branch output F global and serial convolution branch output F conv By performing element-wise addition, the fusion feature is obtained: F =F local +F global +F conv ∈R C'×H'×W' ; Among them, F Represents the fused feature map; + indicates element-wise addition operation, R C'×H'×W' In the fusion feature map, R indicates that the element values ​​belong to the real number field; C' indicates the number of channels in the fusion feature map; H' indicates the spatial resolution of the fusion feature map in the height direction; and W' indicates the spatial resolution of the fusion feature map in the width direction.

[0010] S26: Regarding fusion feature F Channel attention M is applied sequentially. c Spatial attention M s Generate weighted feature F s ; S27: Weighted feature F s First, HiLo attention is introduced to perform cross-branch feature interaction within a frequency decoupling framework: high-frequency detail responses are enhanced through local window self-attention in the Hi-Fi branch, and the global dependency of the low-frequency signal after average pooling within the window is modeled using the Lo-Fi branch; then, the outputs of the two branches are concatenated and linearly projected to obtain feature F. out .

[0011] S28: To F out Execute Dropout and batch normalization sequentially. ReLU activation function ( After processing, the final output feature is obtained: F''=δ(B(dropout(F out ))),F''∈R C'×H'×W' ; Where dropout represents random deactivation operation; F'' represents the final output feature map of step S2.

[0012] This output is the final result of the PCP-HF module.

[0013] Furthermore, in step S22, the specific processing procedure is as follows: S221: Input feature map F'∈R C'×H'×W' C' represents the number of channels in the feature map; H' represents the spatial resolution of the feature map in the height direction; W' represents the spatial resolution of the feature map in the width direction; non-overlapping partitioning is performed along the spatial dimensions at local small scales p×p (e.g., 2×2), forming a four-dimensional tensor with dimensions p×p, C', H' / p, W' / p; the first dimension: p×p represents the spatial size within a single local image patch; the second dimension: C' represents the number of channels contained in each local image patch, compared to the original input feature map. Figure 1 The third dimension, H' / p, represents the number of image blocks obtained by dividing the image in the height direction; the fourth dimension, W' / p, represents the number of image blocks obtained by dividing the image in the width direction.

[0014] S222: Perform average pooling along the channel dimension on each local "image patch" to achieve channel compression. This process only compresses the number of channels C' of the "image patch" and does not change the other dimensions of the "image patch", thus obtaining a dimension of p×p,(H'×W') / p. 2 The three-dimensional tensor; Where p×p represents the local image patch whose internal spatial size remains unchanged after performing channel-dimensional average pooling; (H'×W') / p 2 This represents the total number of local image patches, i.e., the number of patches that can be divided in the entire feature map (H' / p)·(W' / p)=(H'×W') / p 2 This three-dimensional tensor can be understood as a spatial feature representation of each local image patch after channel compression.

[0015] S223: Input the "image patch" features after channel average pooling into a feedforward neural network (FFN) and apply an activation function to generate a set of spatial probability distributions for weighting the features of each local region; S224: Apply a feature selection mechanism to the above weighted results to extract task-related features from two dimensions: Token (each token corresponds to a semantic representation of a channel, which aggregates the responses of that channel across all patches in its vector dimension) and channel. Let: d = (H' × W') / p 2 ; The formula means that the character 'd' represents the total number of local image patches formed after the input feature map is divided into non-overlapping parts; The weighted result is denoted as: (t) i ) i=1 C' ,t i ∈R d Where i=1,2,…,C' represents the index of the token in the channel dimension; C' is the number of channels in the feature map; t i This represents the semantic token of the i-th channel, so t i ∈R d Indicates t i It is a matrix of length d (1 row, d columns); further, the representation matrix of Token is Token∈R C'×d , is t1…t C' A stacked matrix of C' rows and d columns; S225: Apply the following formula to reweight each token for task guidance:

[0016] Among them, t' i t represents the feature vector of the i-th Token after task-guided reweighting; i Let ξ represent the original feature vector of the i-th token; ξ∈R C' Let ξ represent the task embedding vector, where ξ is an initially random vector that can be learned during training; P∈R C'×C' represents a learnable linear projection matrix used for remapping and feature selection of token features along the channel dimension; sim(,) represents the cosine similarity function used to measure the token feature t. i The similarity between the task embedding vector ξ and the task embedding vector ξ, with values ​​ranging from [0,1]. When t i When the similarity with the task embedding vector ξ is high, the corresponding token is enhanced during the reweighting process; when the similarity is low, the token is suppressed; thus achieving task-related feature selection and attention guidance.

[0017] S226: For all tokens i Perform linear projection (through) The reconstruction and interpolation operations ultimately restore the local enhanced feature map to the original feature map spatial resolution: F local ∈R C'×H'×W' ; Among them, F local The local branch output represents the final local enhanced feature map. The subscript "local" indicates that this is a local area, and it integrates local spatial structure information with task-related attention weights. R C'×H'×W'The feature map is defined as a three-dimensional feature tensor in the real number domain, with the same number of channels and spatial dimensions as the original input feature map. Here, C' represents the number of channels in the feature map, consistent with the number of channel-level tokens; H' represents the spatial resolution of the feature map in the height direction; and W' represents the spatial resolution of the feature map in the width direction.

[0018] Furthermore, in step S23, the specific processing procedure is as follows: S231: Input feature map F'∈R C'×H'×W' Non-overlapping partitions are performed along the spatial dimension at a local large scale p×p (e.g., 4×4) to form a four-dimensional tensor with dimensions p×p, C', H' / p, W' / p; S232: Perform average pooling along the channel dimension on each "image patch" to obtain a shape of p×p,(H'×W') / p 2 The characteristic tensor; S233: The above tensor is input into the feedforward neural network (FFN) and linearly transformed, and an activation function is applied to generate an attention probability distribution map in the spatial dimension, which is used to weight and adjust the features.

[0019] S234: Apply a feature selection mechanism to the above weighted results to extract task-related features from both the Token and Channel dimensions. Let: d = (H' × W') / p 2 ; The formula means that the character 'd' represents the total number of local image patches formed after the input feature map is divided into p×p non-overlapping sections. The weighted result is represented as a set of tokens {ti} i=1 C' , where t i ∈R d This represents the i-th token.

[0020] S235: Perform the following selection operation for each token:

[0021] Among them, t' i t represents the feature vector of the i-th Token after task-guided reweighting; i Let ξ represent the original feature vector of the i-th token; ξ∈R C' Let ξ represent the task embedding vector, where ξ is an initially random vector that can be learned during training; P∈R C'×C' represents a learnable linear projection matrix used for remapping and feature selection of token features along the channel dimension; sim(,) represents the cosine similarity function used to measure the token feature t. iThe similarity between the task embedding vector ξ and the task embedding vector ξ, with values ​​ranging from [0,1]. When t i When the similarity with the task embedding vector ξ is high, the corresponding token is enhanced during the reweighting process; when the similarity is low, the token is suppressed; thus achieving task-related feature selection and attention guidance.

[0022] S236: Each token i Reweighting is performed based on its similarity to the task vector, and a linear transformation is applied. Select the channel.

[0023] S237: Reassemble the token into a feature map, and reconstruct it into a feature map of size H'×W'×C' through interpolation to obtain the global enhanced feature: F global ∈R C'×H'×W' ; Among them, F global This represents a global enhanced feature map, with the subscript "global" indicating that it is a global feature map that integrates channel-level semantic information obtained from global context modeling and task-guided feature selection and reweighting results; R C'×H'×W' This indicates that the feature map is a three-dimensional feature tensor defined in the real number domain, and its number of channels and spatial dimensions are consistent with the original input feature map. Here, C' represents the number of channels in the feature map, consistent with the number of channel-level tokens; H' represents the spatial resolution of the feature map in the height direction; and W' represents the spatial resolution of the feature map in the width direction. Furthermore, in step S24, the specific processing procedure is as follows: S241: Apply three convolutional layers sequentially to the input feature map to obtain: F conv1 ∈R C'×H'×W' ;F conv2 ∈R C'×H'×W' ;F conv3 ∈R C'×H'×W' ; In this step, the feature map undergoes a sequential convolution process, F conv1 F represents the output feature map of the first convolutional layer in the sequential convolutional branch; conv2 F represents the output feature map of the second convolutional layer in the sequential convolutional branch; conv3 This represents the output feature map of the third convolutional layer in the sequential convolutional branch. R C'×H'×W' The above feature maps are all three-dimensional feature tensors defined in the real number domain, where C' is the number of channels in the feature map; H' and W' are the spatial resolutions of the feature map in the height and width directions, respectively. All three convolutional layers use the same stride and padding strategy, thus ensuring that their output features maintain consistency in spatial size and number of channels.

[0024] S242: Perform element-wise addition on the outputs of the above three convolutional layers: F conv =F conv1 +F conv2 +F conv3 ∈R C '×H'×W' ; + indicates element-wise addition, which means adding the outputs of the three convolutional layers at corresponding channels and spatial locations. F conv This represents the final output feature map of the serial convolution branch.

[0025] This allows us to obtain the output features of the serial convolution branch.

[0026] Furthermore, in step S25, the specific processing procedure is as follows: S261: Merging features Input Channel Attention Module M c ∈R C'×1×1 The channel-weighted features are obtained as follows: ; M C This represents the channel attention weight tensor, which assigns a scalar weight to each channel along the channel dimension to measure the importance of different channels to the current task. The spatial dimension is 1×1, indicating that this attention only acts on the channel dimension and does not change the spatial structure. This represents the fused feature map resulting from the summation of the feature maps output from the three branches.

[0027] M C ( The channel weights are calculated by the channel attention module based on the input features. F represents element-wise multiplication, which broadcasts the channel weights along the channel dimension to the spatial dimension and multiplies them element-wise with the input features; c This represents the channel-weighted feature map, used to highlight channel features that are more critical for target detection, while suppressing redundant or noisy channels.

[0028] S262: Input spatial attention module M s ∈R H'×W'×1 Spatial weighted features are obtained:

[0029] M s This represents the spatial attention weight tensor, which assigns a scalar weight to each spatial location in the spatial dimension to characterize the importance of different spatial locations. The channel dimension is 1, indicating that this attention is shared across all channels. F c This represents the channel-weighted feature map; M s (F c) indicates that the spatial attention module is based on the channel-weighted feature F c The calculated spatial weight distribution; This represents element-wise multiplication, with spatial weights broadcast along the channel dimension and associated with F. c Element-wise multiplication; F s This indicates that the final feature map, after being weighted by both channel attention and spatial attention, has stronger target saliency and background suppression capabilities.

[0030] S271: Obtain the input feature map F after branch fusion s F s ∈R C'×H'×W' And calculate the number of tokens N = H' × W', and put F s As input to the HiLo attention mechanism.

[0031] S272: Determine the head allocation strategy for HiLo attention, assuming the total number of attention heads is N. h And set the scaling factor α∈(0,1); divide the attention heads into two groups according to the scaling factor: The Hi-Fi branch uses (1-α)N h One point of attention; Lo-F branch uses αN h There are two attention heads. The Hi-Fi branch is used to emphasize high-frequency local details, while the Lo-Fi branch is used to model the low-frequency global context.

[0032] S273: In the Hi-Fi branch, window-based local self-attention computation is performed on the input feature map; specifically, the spatial dimension is divided into non-overlapping s×s windows, and self-attention interactions are performed within each window to obtain locally dependent enhancement features, resulting in: H i -F i (F s )=WSA(F s ;s); Among them, F s This represents the feature map input to the HiLo module, with a tensor dimension of F. s ∈R C'×H'×W' ; C' Number of channels; H'·W' spatial resolution; subscript s indicates that this feature is related to the window partitioning scale s×s. H i -F i (F s ) represents the output characteristics of the Hi-Fi branch, which mainly models local spatial details and high-frequency texture information; WSA(·) represents window self-attention operation.

[0033] S274: In the Lo-Fi branch, to obtain a low-frequency representation, for F...s Intra-window average pooling is performed using the same s×s window partitioning method as in the Hi-Fi branch to obtain low-frequency signals. The average pooling reduces the spatial resolution to 1 / s of the original, resulting in: ∈R C'×H' / s×W' / s ; in, This represents the low-frequency feature representation obtained through in-window average pooling. The average pooling operation first divides the window into s×s windows, similar to the Hi-Fi branch; then, it averages the features within each window spatially to reduce the resolution to 1 / s of the original. R C'×H' / s×W' / s This means that the input F, originally with a resolution of H'·W', is reduced through average pooling. s Convert to an output with a resolution of H' / s × W' / s The next step is to input F into the low-frequency branch.

[0034] S275: In the Lo-Fi branch, global attention is constructed based on the strategy of "queries from high resolution and key-value pairs from low resolution"; specifically: From the original high-resolution feature F s Generate query Q(F) s ); Due to low frequency characteristics Generate key K(F) and value V(F). Then perform attention operation to obtain the Lo-Fi branch output: LO-F. i (F s )=Attn(Q(F s ),K(F),V(F)); Using the above method, the number of key and value tokens is reduced from N to N / s. 2 Since the original number of tokens for global attention is N = H' × W', after average pooling, the number of tokens is reduced to N / s. 2 =H' / s×W' / s, thereby reducing the computational overhead of global dependency modeling. express Output characteristics of the branch; This represents the original high-resolution feature F s The query matrix generated by the linear mapping is used to maintain a fine-grained perception of spatial location; K( ) Due to low-frequency characteristics The generated key matrix; V( ) Due to low-frequency characteristics The generated Value matrix; Attn(·) represents the standard scaled dot product attention operation, used to calculate the relevance between the Query and the Key, and to perform weighted aggregation on the Value.

[0035] S276: Concatenate the Hi-Fi branch output and the Lo-Fi branch output along the feature dimension to form a joint representation that simultaneously includes local detail responses and global context information: [H i -F i (F s );LO-F i (F s )]; Among them, Hi-Fi (F s ) represents the output characteristics of the Hi-Fi branch. express The output characteristics of the branch This indicates that the output features of the Hi-Fi branch and the Lo-Fi branch are concatenated along the channel dimension.

[0036] S277: Apply a linear projection to the concatenated joint representation to perform channel blending and dimension alignment, obtaining the HiLo module output feature F. out :F out =W O [H i -F i (F s );L O -F i (F s )]; Among them, F out This indicates the final output characteristics of the HiLo module; out represents the output. O This output projection matrix is ​​used for channel blending, feature dimension alignment, and remapping of the fused representation. The goal is to integrate the concatenated high- and low-frequency information into a unified feature representation space for use by subsequent network layers.

[0037] S278: Output F out As the final result of the HiLo attention mechanism, it is passed on (e.g., channel attention, spatial attention, or pointwise convolution) to further improve the discriminativeness and robustness of infrared small target feature representation.

[0038] Furthermore, in step S3, the specific processing procedure is as follows: S31: Divide the input of this module into four subgroups along the channel dimension, denoted as a1, a2, a3, a4 ∈ R. C / 4×H×W This provides input for subsequent dilated convolution processing with different dilation rates; S32: For each set of sub-feature maps a i (where i = 1, 2, 3, 4) are respectively subjected to an expansion rate of d iThe dilated convolution operation is used to obtain the dilated convolution output feature a. i '∈R C / 4×H×W The process expression is: a i '∈DDWConv(a i ), i∈{1,2,3,4}; Among them, a i This represents the i-th sub-feature map, which is obtained by dividing the input feature map along the channel dimension in the previous processing step. Each sub-feature map contains 1 / 4 of the number of channels of the original feature map. i The spatial resolution is H×W, and the number of channels is C / 4; DDWConv() represents the depthwise separable dilated convolution operator, which includes depthwise dilated convolution, i.e., applying a dilation rate d independently to each input channel. i The dilated convolution operation; pointwise convolution (1×1 convolution). i ' represents the output feature map after the convolution of the i-th branch; H: spatial resolution of the output feature map in the height direction; W: spatial resolution of the output feature map in the width direction; C / 4 is the number of channels of the output feature map, which is equal to the input a. i Maintain consistency.

[0039] S33: For each set of sub-feature maps a i (where i=1,2,3,4) are simultaneously fed into the frequency domain branch based on Fast Fourier Transform (FFT) in parallel, and the frequency enhancement feature f is obtained through the FFT–Gate–IFFT processing pipeline. i ∈R H×W×C / 4 ,i∈{1,2,3,4}.

[0040] S34: Convolve all dilated features a i ',f i Fine-grained slicing and recombination are performed according to the channel dimension to form a fusion feature tensor H. By alternately fusing spatial features with corresponding frequency domain enhancement features at different expansion scales within the channel group, the full mixing and complementary expression of cross-scale information is achieved, thereby improving the interaction strength of multi-scale features and channel diversity. S35: Apply a 1×1 point convolution to the fusion tensor Hout to perform channel integration, and then perform batch normalization sequentially. ReLU activation function Processing yields the final output feature F. out :F out =δ(B(H out )),F out ∈R(C'×H'×W'); in, This indicates ReLU activation function processing; the output is the final result of the SFMCE module. F out ∈RC '×H'×W' The size of the final output tensor of this module is denoted as C'×H'×W'.

[0041] Furthermore, in step S33, the specific processing procedure is as follows: S331: For each set of sub-feature maps a i (where i=1,2,3,4) First, frequency domain transformation (FFT) is performed; Sub-feature map a i Input Fast Fourier Transform operator Its frequency domain representation is obtained: ; in, The expression represents the Fast Fourier Transform operator operation, and Ai represents the complex spectral features (including amplitude and phase information) output after performing the Fast Fourier Transform operator operation on the feature map.

[0042] S332: Based on complex spectral characteristics A i Through learnable gating functions Generate band modulation weights (gating mask): M i =G(A i ); Where i represents the i-th branch, i = 1, 2, 3, 4; A i The characteristic representation of the i-th branch in the frequency domain can be obtained by Fast Fourier Transform (FFT), and its form is a complex spectrum; This represents a learnable gating function, typically implemented by a lightweight neural network (1×1 convolution, MLP, or attention weight generation module), used to adaptively generate modulation weights based on the frequency domain response. i This indicates that for frequency domain feature A i The generated frequency band modulation weights (gating masks) are used to selectively enhance or suppress different frequency components.

[0043] S333: Gating weight M i Acting on frequency domain feature A i The frequency domain enhancement result after modulation is obtained:

[0044] in, This indicates the frequency domain enhancement characteristics after gated modulation; This represents element-wise multiplication, equivalent to applying frequency-wise linear modulation to the frequency domain features; through gated weights M i Frequency-wise adjustment of the spectrum enables feature reweighting at the frequency domain level.

[0045] S334: Modulated frequency domain characteristics Applying the inverse fast Fourier transform To obtain the frequency enhancement features in the spatial domain: f i =F -1 (A i ) in, f represents the inverse fast Fourier transform, used to map frequency domain features back to the spatial domain; i Indicates the enhancement features in the frequency domain The spatial domain frequency enhancement features obtained by the inverse transform realize the mapping from frequency domain enhancement to spatial domain representation, enabling frequency-selective enhancement to directly affect the spatial feature representation.

[0046] S335: f i Perform size / channel alignment and numerical normalization (maintaining the C / 4×H×W shape consistent with the original branch, taking the real part, amplitude mapping, or normalization if necessary) to obtain the final frequency enhancement feature: f i ∈R C / 4×H×W ,i∈{1,2,3,4}; Among them, f i To represent the output of the frequency channel, f needs to be adjusted. i The following necessary processing is performed: First, size alignment is performed to ensure that the spatial resolution is H×W; second, channel alignment is performed and the number of channels is adjusted to C / 4, consistent with the corresponding dilated convolution branch. Furthermore, in step S34, the specific processing procedure is as follows: S341: Convolve all dilated features a i '(i∈{1,2,3,4}) and frequency domain enhancement feature f i (i∈{1,2,3,4}) are concatenated along the channel dimension to obtain a preliminary fusion feature tensor H.

[0047] H = [a1', a2', a3', a4', f1, f2, f3, f4]; where a i ' represents the spatial domain feature output by the i-th dilated convolution branch, used to capture multi-scale spatial context information under different dilation rates; f i This represents the frequency domain enhancement feature corresponding to the i-th branch; This indicates a concatenation operation along the channel dimension; H represents the feature tensor after initial fusion, with dimension R. 8 ×H×W .

[0048] S342: On each fusion tensor H, utilize a channel-shared learnable matrix W∈R 4×8 By fusing dilated convolutional features with frequency domain enhancement features, a cross-scale information fusion matrix is ​​obtained. ; Where (h,w) represents the spatial location index; The learnable fusion matrix shared between channels is equivalent to a 1×1 convolution kernel; u(h,w) represents the cross-scale fusion feature vector obtained by linear combination at spatial location (h,w), with 4 channels.

[0049] S343: After step S342, the cross-scale features have been fused into four output channels through 1×1 convolution. Features at each spatial location will be recombined as needed to ensure effective fusion of information from different scales and frequency domains. The final fused feature tensor H f The shape is: H f ∈R 4×H×W ; H f This represents the cross-scale feature tensor after 1×1 convolution fusion, which contains interwoven information from multiple scales and frequency domain features.

[0050] S344: Finally, the adjusted feature tensor H... f A linear transformation is performed using 1×1 convolution to restore the channel dimensions, resulting in the final fused feature output H. out Its shape is: H out ∈R C×H×W ; The output H out This represents the final fused output features. It is a feature tensor that has undergone fine-grained slicing, staggered fusion, and adjustment, containing rich information across multiple scales and domains.

[0051] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention constructs a multi-scale densely nested U-shaped encoder-decoder backbone network and designs a multi-scale skip connection structure to achieve layer-by-layer fusion of low-level spatial detail features of the encoder and high-level semantic features of the decoder. This effectively alleviates the loss of fine-grained structural information of small targets caused by multiple downsampling, and fully preserves the fine structural features of infrared small targets. This invention embeds parallel contextual image block attention modules at each scale layer of the encoder, extracts local, global, and structural features in parallel, and combines the HiLo high-frequency-low-frequency attention mechanism to achieve cross-branch feature interaction. This fully integrates local and global multi-scale features, effectively models the contrast characteristics of targets and backgrounds at different scales, and enhances the perception ability of dim and weak small targets in complex backgrounds. This invention embeds a hole-frequency multi-scale channel enhancement module at the decoder layer, combines depth-separable hole convolution with different dilation rates to extract multi-receptive field spatial features, and achieves frequency domain feature enhancement through FFT-Gate-IFFT frequency domain branches, completing fine-grained interleaved fusion of spatial and frequency features. This further strengthens target structural information, suppresses clutter interference in complex backgrounds, and improves background suppression effects. Attached Figure Description

[0052] Figure 1 This is an architecture diagram of the infrared small target detection method based on a multi-scale nested attention network according to the present invention; Figure 2 This is a schematic diagram of the parallel context image block attention module structure of the infrared small target detection method based on a multi-scale nested attention network according to the present invention; Figure 3 This is a schematic diagram of the hole-frequency multi-scale channel enhancement mode structure of the infrared small target detection method based on multi-scale nested attention network of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Example: Figure 1-3 As shown, this invention provides a technical solution for infrared small target detection based on a multi-scale nested attention network, comprising the following steps: Step S1: Construct an infrared small target detection method based on a multi-scale densely nested attention network of an improved Transformer; Step S2: Establish a parallel context image patch attention module. The module includes multiple parallel attention branches. Each branch constructs attention weights for different ranges of image regions, integrates local window attention, global average attention, and spatial dimension channel attention, and finally processes them through a high-low frequency attention mechanism to obtain the final output features. Step S3: Establish a hole-frequency multi-scale channel enhancement module. The module contains multiple hole convolution branches with different dilation rates and frequency branches to extract and process target edge and shape feature information under different receptive fields. Step S4: A multi-scale jump connection structure is used to establish connections between each layer of the encoder and the corresponding layer of the decoder, so that the spatial detail features contained in the lower layers of the encoder can be fully integrated with the higher semantic features in the decoding and reconstruction stage. Step S5: The final output layer introduces the Sigmoid activation function to normalize the output results and generate a saliency map with values ​​between 0 and 1. Step S6: Use a pixel-level segmentation algorithm to extract regions from the saliency map and separate small infrared targets from the complex background.

[0055] Specifically, in this embodiment, the following steps are included: Step S1: Construct a multi-scale nested U-shaped network structure with dense jump connections. The network consists of multiple encoder and decoder layers. By performing cross-scale connections between each layer, multi-level feature aggregation is achieved, significantly alleviating the problem of loss of target detail information caused by downsampling.

[0056] Step S2: Embed the PCP-HF module in each scale layer of the encoder. This module contains three parallel sub-paths, which are used to extract local structural information, global context information, and continuous convolutional structural information, respectively. The processing flow is as follows: The input feature map is compressed into F' using a 1×1 convolution; First sub-path: Divide F' into local fragments using a 2×2 method, apply channel-average pooling, and calculate local attention weights through a feedforward network to obtain F. local ; The second sub-path: Divide F′ into 4×4 sections, extract global semantic features, and generate task relevance weights through a similarity-guided mechanism to obtain F. global ; Third sub-path: Apply three consecutive 3×3 convolutions to F′, fusing them into a continuous structural feature F. conv ; Finally, F local F global With F conv Element-wise summation And apply channel attention M in sequence c Spatial attention M s Generate enhanced features F s , will F s The input feature F generated as the HiLo attention mechanism out Finally, after processing by Dropout, batch normalization, and activation functions, the final output F'' is obtained.

[0057] Step S3: Embed the DMCE module in the decoder stage to enhance structural detail awareness. The decoder stage processes the feature map F. local Processing: The channel is divided into four sub-feature maps a1 to a4; Applying depth-separable convolutions with dilation rates d1 to d4 to each group yields a. i '; Then input each group into the frequency channel to obtain f. i ; a i ', f i After being segmented into single-channel features, they are interleaved and combined to form a tensor H; All tensors H are convolved through a matrix W∈R with a 1×1 kernel. 4×8The tensor H is merged into 4 output channels. f Then H f Applying a 1×1 convolution yields the output H. o ; Finally, H o After nonlinear enhancement operations, the final output F is generated. o .

[0058] Step S4: In the skip connection structure of the backbone network, connect the corresponding layers of the encoder and decoder to ensure that the low-level spatial details and high-level semantic features are fully integrated during the decoding process, thereby improving the saliency modeling capability of infrared small targets. Step S5: Apply the Sigmoid activation function to the output feature map of the last layer of the model to generate a saliency map; Step S6: Extract regions from the saliency map using a pixel-level segmentation algorithm to separate small infrared targets from the complex background, thereby achieving accurate detection of small target regions.

[0059] In this embodiment, the experimental setup may include a graphics processing unit (GPU), such as a GeForce GTX 4060 Laptop GPU manufactured by NVIDIA. Where appropriate, the GPU can work in conjunction with a central processing unit (CPU) to execute the deep learning model training and inference process of this application embodiment.

[0060] In a specific embodiment, to optimize the multi-scale densely nested attention network, this embodiment employs an adaptive moment estimation (Adam) optimizer to achieve efficient iterative updates of model parameters.

[0061] During model training, the batch size is set to 16, and the input images are uniformly adjusted to a resolution of 256×256 to ensure the consistency of the training data. Where appropriate, the learning rate can be set to 0.05 to balance training convergence speed and stability. The training process can last for 500 epochs to ensure the model learns and converges sufficiently on a large number of samples.

[0062] At the implementation level, this embodiment is built and deployed based on the open-source deep learning framework PyTorch. The PyTorch framework can run on the aforementioned GPU and its supporting hardware environment, and implements functions such as tensor operations, gradient calculations, and backpropagation by calling the corresponding computing libraries.

[0063] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. An infrared small target detection method based on a multi-scale nested attention network, characterized in that: The method includes the following steps: Construct a multi-scale densely nested U-shaped encoding-decoding backbone network; Parallel context image block attention modules are embedded in each scale layer of the encoder in the backbone network to extract and fuse multi-dimensional features of the feature maps, generating enhanced features for each scale layer of the encoder. A hole-frequency multi-scale channel enhancement module is embedded in the decoder layer of the backbone network to extract and fuse multi-receptive field spatial features and frequency domain features of the feature map. A multi-scale jump connection structure is constructed in the backbone network to fuse the low-level spatial detail features of the encoder with the high-level semantic features of the decoder layer by layer. Apply the Sigmoid activation function to the feature map of the final output of the backbone network to generate a normalized saliency map; A pixel-level segmentation algorithm is used to extract the target region from the saliency map, thereby enabling the detection and localization of small infrared targets.

2. The infrared small target detection method based on a multi-scale nested attention network according to claim 1, characterized in that: The processing flow of the parallel context image patch attention module is as follows: First, the input feature map of the module is compressed by convolution. Then, the compressed feature map is fed into the local branch, global branch and serial branch in parallel to extract the corresponding features. The output features of the three branches are added element by element to obtain the fused features. Channel attention and spatial attention are applied sequentially to the fused features for weighted enhancement. The weighted enhanced features are then input into the HiLo high-frequency-low-frequency attention mechanism for cross-branch feature interaction. Finally, after random deactivation, batch normalization, and ReLU activation function processing, the enhanced features output by the module are obtained.

3. The infrared small target detection method based on a multi-scale nested attention network according to claim 2, characterized in that: The specific processing steps for the local branch are as follows: The feature map after dimensionality compression is divided into non-overlapping blocks in the spatial dimension at a fixed small scale to form a local image block tensor. Perform average pooling on each local image patch tensor along the channel dimension to achieve channel dimension compression; The pooled features are input into the feedforward neural network and an activation function is applied to generate a block-level attention probability distribution in the spatial dimension, and the features of each local region are weighted. Based on the weighted results, task-related features are extracted from two dimensions: Token and Channel. The Token features are reweighted in a task-guided manner using task embedding vectors and similarity functions, and then Channel is remapped using a learnable linear projection matrix. The reweighted features are sequentially subjected to linear projection, reconstruction, and interpolation operations to restore them to local enhanced features with the same resolution as the feature map after convolutional dimension compression.

4. The infrared small target detection method based on a multi-scale nested attention network according to claim 2, characterized in that: The specific processing steps for the global branch are as follows: The feature map after dimensionality compression is divided into non-overlapping blocks in the spatial dimension according to a fixed large scale to form a global image block tensor. Perform average pooling on each global image patch tensor along the channel dimension to obtain a three-dimensional feature tensor; The three-dimensional feature tensor is input into a feedforward neural network for linear transformation, and an activation function is applied to generate an attention probability distribution map of the spatial dimension, and the features are weighted and adjusted. Extract task-related features from the weighted results in two dimensions: token and channel. Transform the weighted results into a set of channel-level token features to determine the total number of image blocks formed after non-overlapping block segmentation. Based on learnable task embedding vectors and cosine similarity functions, combined with learnable linear projection matrices, task-guided reweighting is applied to each token feature. Token features with similarity higher than a set threshold are enhanced, while token features with similarity lower than a set threshold are suppressed. After linear projection and reconstruction of all reweighted token features, interpolation is used to rebuild them into global enhanced features with the same resolution and number of channels as the feature map after convolutional dimension compression.

5. The infrared small target detection method based on a multi-scale nested attention network according to claim 2, characterized in that: The specific processing steps for the serial branch are as follows: The feature map after dimensionality compression is subjected to three 3×3 convolution operations in sequence. Each convolution layer adopts the same stride and padding strategy to obtain the output features of the three convolution layers. Each output feature maintains the same resolution and number of channels as the input feature map of the module. The output features of the three convolutional layers are added element-wise to obtain the structurally enhanced features of the serial convolution.

6. The infrared small target detection method based on a multi-scale nested attention network according to claim 2, characterized in that: The specific processing steps of the HiLo high-frequency-low-frequency attention mechanism are as follows: The features enhanced by channel attention and spatial attention are used as the input features of the mechanism, and the total attention head is divided into a high-frequency Hi-Fi branch attention head and a low-frequency Lo-Fi branch attention head according to a preset ratio; The high-frequency Hi-Fi branch divides the input features into spatially non-overlapping segments according to a fixed window scale, performs window self-attention operations within each window, and extracts local high-frequency detail features. The low-frequency Lo-Fi branch divides the input features into spatially non-overlapping segments using the same window scale as the Hi-Fi branch. Within each window, average pooling is performed to obtain low-frequency features. Then, based on a preset strategy, scaling dot product attention operations are performed to model global low-frequency dependent features. The output features of the Hi-Fi branch and the Lo-Fi branch are concatenated along the channel dimension, and then channel blending and dimension alignment are performed using a learnable output projection matrix to obtain intermediate output features. The intermediate output features are sequentially processed by random deactivation, batch normalization, and ReLU activation function to obtain the final output features of the attention mechanism, which serve as the final enhanced features of the parallel context image block attention module.

7. The infrared small target detection method based on a multi-scale nested attention network according to claim 1, characterized in that: The specific processing steps of the hole-frequency multi-scale channel enhancement module are as follows: The module input feature map is evenly divided into four groups of sub-feature maps along the channel dimension. Each group of sub-feature maps has the same number of channels and maintains the same resolution as the original feature map. For each group of sub-feature maps, depthwise separable dilated convolution operations with different dilation rates are applied to extract spatial domain enhancement features under each receptive field. The output features of each group maintain the same resolution and number of channels as the input sub-feature maps. Each group of sub-feature maps is fed into the frequency domain branch in parallel, and FFT transformation, learnable gated modulation, and inverse IFFT transformation are performed sequentially to extract frequency domain enhanced features. The output features of each group maintain the same resolution and number of channels as the input sub-feature maps. Each set of corresponding spatial domain enhancement features and frequency domain enhancement features is finely sliced ​​and interleaved according to the channel dimension to obtain the fused feature tensor; The fusion feature tensor is subjected to convolutional fusion and channel integration of 1×1 point convolution, and then nonlinear enhancement is performed by batch normalization and ReLU activation function. The final fusion enhanced feature of the output module maintains the same resolution and number of channels as the input feature map of the module.

8. The infrared small target detection method based on a multi-scale nested attention network according to claim 7, characterized in that: The specific processing steps for the frequency domain branch are as follows: Perform a Fast Fourier Transform (FFT) on the input sub-feature map to map the spatial domain features into complex spectral features containing amplitude and phase information; The complex spectral features are input into a learnable gating function composed of a lightweight neural network to generate a frequency-wise band modulation weight mask. The modulation weight mask is multiplied element-wise with the complex spectral features to obtain the modulated complex spectral features; Perform an inverse fast Fourier transform (IFFT) on the modulated complex spectral features to map the frequency domain features back to the spatial domain, and then take the real part of the result and perform numerical normalization. The spatially restored features are aligned in size and channels to obtain frequency domain enhanced features with the same resolution and number of channels as the input sub-feature map.

9. The infrared small target detection method based on a multi-scale nested attention network according to claim 1, characterized in that: The multi-scale jump connection structure is a dense cross-level connection structure. In the decoding stage, the output features of each layer of the encoder are directly fused with the input features of the corresponding layer of the decoder, so as to achieve the layer-by-layer complementarity between low-level spatial detail features and high-level semantic features.

10. The infrared small target detection method based on a multi-scale nested attention network according to claim 1, characterized in that: The pixel-level segmentation algorithm combines connected component analysis with false alarm suppression rules; the false alarm suppression rules eliminate false targets based on the area, shape, and local contrast features of the target region.

Images