Infrared weak and small target detection method based on meta-learning decoding

Abstract

The invention provides an infrared weak and small target detection method based on meta learning decoding. By introducing a meta learning decoding normal form, a dynamic network which can be adaptively adjusted according to the state of an input image is constructed, the dynamic network comprises an image encoder, an image-decoder converter and a meta decoder, and the network can customize the most suitable decoding strategy for each input image, so that the problem of mode drift is effectively solved, and the decoding efficiency is improved. And infrared image target detection is carried out by using the trained network, so that the robustness and accuracy of target detection can be remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision and infrared image processing, and particularly relates to an infrared small target detection method based on meta-learning decoding, which is suitable for military reconnaissance, public security monitoring, disaster rescue and the like, and can realize high-robustness detection under extreme conditions such as low signal-to-noise ratio, complex background (air, sea and land cross-scene) and target texture-free characteristics. BACKGROUND

[0002] Infrared small target detection (IRSTD) aims to accurately identify and locate targets with low signal-to-noise ratio, small size and lack of texture features from infrared images. Since infrared imaging relies on thermal radiation, images are easily affected by factors such as atmospheric noise, time and dynamic background, resulting in dramatic changes in target characteristics. Traditional methods such as filtering, low-rank decomposition or local contrast-based algorithms are computationally efficient, but are easily disturbed by background clutter in complex scenes and have poor generalization ability.

[0003] With the development of deep learning, the encoder-decoder framework (such as U-Net and its variants) has become mainstream, which improves target perception ability through multi-scale feature fusion (such as UIU-Net and ISNet). In recent years, Transformer-based methods (such as SCTransNet and TCI-Former) further strengthen global context modeling and improve low-contrast target detection performance.

[0004] However, the existing methods have key limitations: static mode learning leads to scene drift problem. The characteristics of infrared targets are significantly different in different scenes, and the fixed decoder parameters after training cannot adapt to the dynamically changing image state, thereby limiting its generalization ability in diversified scenes, resulting in decreased detection accuracy and increased false alarm rate.

[0005] Therefore, the existing technology lacks an infrared small target detection method that can automatically perceive the state of the input image and dynamically adjust its detection strategy accordingly to overcome mode drift and achieve infrared small target detection that balances multi-scene adaptability and detection accuracy. SUMMARY

[0006] In order to overcome the shortcomings of the prior art, the present application provides an infrared small target detection method based on meta-learning decoding. By introducing the "meta-learning decoding" paradigm, a dynamic network that can adaptively adjust according to the input image state is constructed, including an image encoder, an image-decoder transformer and a meta-decoder. This network can tailor the most suitable decoding strategy for each input image, thereby effectively addressing the mode drift problem. Using the trained network for infrared image target detection can significantly improve the robustness and accuracy of target detection.

[0007] A method for detecting weak infrared targets based on meta-learning decoding, characterized by the following steps:

[0008] S1: Preprocess the input infrared image to obtain a standardized infrared image;

[0009] S2: Construct a target detection network based on meta-learning decoding, including an image encoder, an image-decoder transformer, and a meta-decoder. The image encoder extracts multi-scale visual features from the input image and fuses high-frequency components. The image-decoder transformer dynamically generates decoder parameters using the extracted multi-scale visual features. The meta-decoder is constructed based on the generated decoder parameters, taking the deep features output by the encoder as input and upsampling them level by level. At each level, the shallow features output by the encoder are fused through skip connections. The meta-decoder processes the features level by level and finally outputs a single-channel target segmentation mask through a shared 1×1 convolutional layer.

[0010] S3: Input the preprocessed infrared image into the target detection network constructed in step S2, and train the network in an end-to-end manner to obtain the trained network.

[0011] S4: After preprocessing the single infrared image to be detected according to step S1, input it into the trained target detection network. Binarize the output target segmentation mask with a fixed threshold of 0.5. The resulting binary mask is the final target detection result.

[0012] Specifically, the preprocessing described in step S1 includes image size normalization, grayscale normalization, and data augmentation processing, wherein the data augmentation processing includes random flipping and random cropping.

[0013] Specifically, the process of fusing high-frequency components in step S2 is as follows: First, the input image is subjected to Fourier transform to obtain a frequency domain representation. Then, a Gaussian high-pass filter mask is used to filter it. Finally, the filtered frequency domain signal is subjected to inverse Fourier transform to obtain a high-frequency component image in the spatial domain. This image is then stitched together with the original input image along the channel dimension as the final input of the encoder.

[0014] Specifically, the image encoder adopts a multi-level UNet structure, with each level of convolutional block containing two 3×3 convolutional layers, which are then connected to a multi-core aggregation module, and redundant information is suppressed through a channel attention mechanism.

[0015] Specifically, the process by which the image-decoder transformer dynamically generates decoder parameters using extracted multi-scale visual features is as follows:

[0016] Step 1: Construct image perception conditions using the following formula:

[0017] c = c im +c hf +c pe

[0018] Where c represents the image perception conditional vector, c im This represents multi-scale visual feature embedding, which is obtained by patch embedding encoder features through specific convolutional layers, and then concatenating them along the channel dimension; c hf This indicates high-frequency signal embedding, obtained by projecting the high-frequency signal; c pe The cosine-sine position embedding is obtained using cosine-sine position encoding.

[0019] Step 2: Initialize a set of learnable decoder tokens, refine the image perception conditions through multiple Transformer layers to obtain decoder tokens; each Transformer layer contains a self-attention module, a cross-attention module and a feedforward network module. The self-attention module models the inter-layer dependencies between decoder tokens, the cross-attention module establishes the alignment interaction between image perception conditions and tokens, and the feedforward network module implements nonlinear transformation.

[0020] Step 3: Project and reshape the decoder tokens to obtain a structured two-dimensional tensor of decoder parameters. The row dimension of the two-dimensional tensor of decoder parameters corresponds to the level of the meta-decoder, and the column dimension corresponds to the number of parameters required by each layer of the decoder, including the convolution parameter column and the BN parameter column.

[0021] Specifically, the meta-decoder includes three dynamically generated structures: a basic decoder, a multi-scale decoder, and a spatial attention decoder. Each layer of the basic decoder contains a 3×3 convolution, batch normalization, and a ReLU activation function. The multi-scale decoder adds parallel branches of 3×3 and 5×5 depth-separable convolutions to the basic decoder. The output is summed and then processed by two layers of 1×1 convolutions. The spatial attention decoder adds a spatial attention branch to the multi-scale decoder, models spatial information along the vertical and horizontal directions respectively, and generates spatial weights that are weighted to the multi-scale branch output in a residual manner.

[0022] Specifically, the following joint loss function is used during network training:

[0023] L total =L BCE (m,m')+λ·L Dicg (m,m') (1)

[0024] Among them, L total L represents the total loss. BCE For binary cross-entropy loss, L DiceLet λ be the Dice loss, λ = 0.5 be the balance coefficient, m be the prediction mask, and m′ be the ground truth mask.

[0025] Specifically, the multi-core aggregation block groups the input channels and processes them in parallel through 1×1, 3×3, 5×5, and 7×7 convolutions. After the channels are concatenated, they are output with channel attention weights. The channel attention is input into the MLP layer through the difference between global max pooling (GMP) and global average pooling (GAP), and weights are generated by sigmoid activation.

[0026] Specifically, the Transformer has 6 layers and uses 6 attention heads, each with a hidden dimension of 64 and a dropout probability of 0.1.

[0027] The beneficial effects of this invention are as follows: By employing an image-decoder converter to dynamically generate decoder parameters, an input-adaptive detection strategy is achieved, fundamentally solving the pattern fixation problem caused by scene drift. Because the network can dynamically adjust the decoding strategy based on input images from different scenes such as sky, ocean, and land, it can significantly improve target detection accuracy and robustness. Since the network only requires a single forward propagation to complete the construction of the dynamic decoder and target detection, no additional computational overhead is required, resulting in high computational efficiency and strong applicability. It can be applied to infrared imaging platforms with stringent requirements for computational resources and real-time performance, such as airborne, missile-borne, and spaceborne systems. Due to the use of structured decoder parameter representation, it can flexibly support meta-decoder structures of varying complexity, offering a flexible architecture that is easy to expand and facilitates future optimization for specific scenarios. Attached Figure Description

[0028] Figure 1 This is a flowchart of an infrared weak target detection method based on meta-learning decoding according to the present invention;

[0029] Figure 2 This is a schematic diagram of the target detection network structure based on meta-learning decoding constructed in this invention;

[0030] Figure 3 This is a schematic diagram of the structure of the multi-core aggregation block in an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of the spatial attention meta-decoder in an embodiment of the present invention;

[0032] Figure 5 This is a visual comparison chart of the detection results of the embodiments of the present invention on the IRSTD-1K dataset. Detailed Implementation

[0033] The present invention will be further described below with reference to the accompanying drawings and embodiments. The present invention includes, but is not limited to, the following embodiments.

[0034] The example was run on a server with four NVIDIA RTX 4090 graphics cards (24GB each), CUDA version 12.4, Ubuntu 22.04 operating system, Python version 3.8, and PyTorch version 2.4.1. The experiment used three public datasets: NUAA-SIRST, NUDT-SIRST, and IRSTD-1K, all with a uniform resolution of 256×256. The training / test ratio was 5:5 for NUAA and NUDT datasets, and 8:2 for the IRSTD-1K dataset. Figure 1 As shown, the specific implementation process of the infrared weak target detection method based on meta-learning decoding of the present invention is as follows:

[0035] S1: Preprocess the input infrared image to obtain a standardized infrared image.

[0036] The preprocessing includes image size normalization (256×256), grayscale normalization, and data augmentation, whereby data augmentation includes random flipping, random cropping, etc.

[0037] S2: Construct an object detection network based on meta-learning decoding, such as Figure 2 As shown, it mainly includes an image encoder, an image-to-decoder converter, and a meta-decoder.

[0038] The image encoder extracts multi-scale visual features from the input image and fuses high-frequency components. The specific processing steps are as follows:

[0039] (1) The input image is subjected to Fourier transform to obtain its frequency domain representation. A Gaussian high-pass filter mask is then applied for filtering, followed by inverse Fourier transform to obtain the high-frequency component image in the spatial domain. The high-frequency components can supplement the target edge and location information. The specific process can be represented by the following formula:

[0040]

[0041] x hp =F -1 (M hp ⊙F(x)) (3)

[0042] Among them, M hp This represents a Gaussian mask, where (u, v) are frequency coordinates, (u0, v0) are the spectral center, σ is the cutoff frequency (which can be set to σ = 5), and F / F -1 For the forward / inverse Fourier transform, x hp For high-frequency components, ⊙ indicates element-wise multiplication.

[0043] (2) The high-frequency component image is concatenated with the original input image along the channel dimension and used as the final input of the encoder;

[0044] (3) The encoder adopts a multi-level UNet structure, with each level containing two 3×3 convolutional layers (ReLU activation), and then connected to a multi-kernel aggregation module. Redundant information is suppressed through a channel attention mechanism to output multi-level feature maps. Specifically, for an infrared image with an input size of 256×256, the encoder first extracts basic features through two 3×3 convolutional layers, and then connects to a 4-level downsampling structure to perform progressive downsampling. The number of input channels in the first level is 32, and the number of channels in each subsequent level doubles (i.e., the number of output channels in the first to fourth levels are 64, 128, 256, and 512, respectively).

[0045] Multi-core aggregation block modules such as Figure 3 As shown, the specific operations include: grouping the input channels and processing them in parallel through 1×1, 3×3, 5×5, and 7×7 convolutions to capture multi-scale context; concatenating the channels and then weighting the output with channel attention to achieve feature recalibration in the channel dimension; inputting the channel attention into the MLP layer through the difference between global max pooling (GMP) and global average pooling (GAP), and generating weights through sigmoid activation. This process can be expressed by the following formula:

[0046]

[0047] F (i) =f (i) ·(1+σ(MLP(GMP(f (i) )-GAP(f (i) (5)

[0048] Among them, F (i) f represents the output characteristics of a multi-core aggregate block. (i) This represents the features after parallel convolution processing. Indicates channel cascading. Let σ be the channel feature of the i-th layer and k-th group, and σ be the Sigmoid function.

[0049] The image-decoder transform dynamically generates complete decoder parameters that match the current image state based on the extracted multi-scale visual features. Specifically, it includes the following three steps:

[0050] (1) Constructing image perception conditions: Multi-scale visual feature embedding, high-frequency signal embedding, and cosine-sine position embedding are fused to form a comprehensive image perception condition vector c, which is expressed by the formula:

[0051] c = c im +c hf +c pe (6)

[0052] Among them, c imThis represents multi-scale visual feature embedding, where encoder features are processed through a specific convolutional layer and patched to obtain patch blocks of size 16 and 96 channels each, which are then concatenated along the channel dimension. hf This indicates high-frequency signal embedding, obtained by projecting the high-frequency signal; c pe This indicates that cosine-sine position embedding is used, and cosine-sine position encoding is used to preserve spatial information. Together, these three elements constitute a complete image state condition.

[0053] (2) Initialize a set of learnable decoder tokens Q with a dimension of (Nq+1)×Ct, where Nq is the total number of convolutional kernels (208), Ct is the hidden dimension (384), and "+1" is the normalized token used to represent the parameters of the BN layer. Construct a 6-layer Transformer with 6 attention heads, each with a hidden dimension of 64 and a dropout probability of 0.1. Each Transformer layer contains three sub-modules: self-attention, cross-attention, and a feedforward network. Self-attention models the dependencies between the parameters of each layer of the decoder; cross-attention uses the image perception condition c as the key and value, and the decoder token Q as the query, to establish an alignment interaction between the image state and the decoder parameters; the feedforward network performs nonlinear transformations.

[0054] After 6 layers of Transformer refinement, a decoder token with image state information is obtained.

[0055] (3) The final output decoder token is adjusted in dimension by a projection layer, and the hidden dimension is mapped from 384 to 759 for subsequent parameter reconstruction, resulting in a structured two-dimensional tensor D of decoder parameters. This tensor has the following characteristics: a) The row dimension corresponds to the level of the meta-decoder, preserving the vertical structural relationship between parameters between layers; b) The column dimension corresponds to the number of parameters required by each layer of the decoder, including the convolution parameter column and the BN parameter column, ensuring the consistency of parameter level.

[0056] A meta-decoder is constructed based on the generated parameters. It takes the deep features output by the encoder as input and upsamples them level by level. At each level, corresponding shallow features from the encoder are fused through skip connections to preserve spatial details. The meta-decoder processes features level by level and finally outputs a single-channel infrared weak target segmentation mask through a shared 1×1 convolutional layer.

[0057] The meta-decoder contains three dynamically generated structures: a basic decoder, a multi-scale decoder, and a spatial attention decoder. The basic decoder contains 3×3 convolutions, batch normalization, and ReLU activation functions in each layer. The multi-scale decoder adds parallel branches of 3×3 and 5×5 depthwise separable convolutions to the basic decoder. The output is summed and then processed by two layers of 1×1 convolutions to capture multi-scale spatial features. The spatial attention decoder adds a spatial attention branch to the multi-scale decoder, models spatial information along the vertical and horizontal directions respectively, and generates spatial weights that are weighted to the multi-scale branch output in a residual manner.

[0058] like Figure 4 As shown, the meta-decoder in this embodiment is a spatial attention decoder. Its processing is as follows: First, the deep features are processed through a 3×3 convolutional layer, and then through parallel branches of 3×3 and 5×5 depth-separable convolutions. The output is summed and then processed through two 1×1 convolutional layers to capture multi-scale spatial features. At the same time, the deep features are also fed into the spatial attention branch for processing. First, the channels are processed through 1×1 convolutions, and then average pooling is performed along the vertical and horizontal directions to obtain 1×W and H×1 features. Then, the features are transposed and concatenated to obtain fused features, which are fed into 1×1 convolutions for fusion. Finally, the features are split along the channel dimension, mapped separately, and multiplied to obtain spatial attention weights. Finally, spatial attention is applied to the features after multi-scale processing, and then the sum is residually connected with the original features to obtain the output features of this layer's meta-decoder.

[0059] S3: Input the preprocessed infrared image into the target detection network constructed in step S2, and train the network in an end-to-end manner to obtain the trained network.

[0060] The loss function is set as follows during training:

[0061] L total =L BCE (m,m')+λ·L Dicg (m,m') (7)

[0062] Among them, L total L represents the total loss. BCE Binary cross-entropy loss is used to optimize pixel-level classification error, L Dice λ is the Dice loss, used to optimize the overlap of region-level segmentation. λ = 0.5 is the balance coefficient, m is the prediction mask, and m′ is the ground truth mask.

[0063] In this embodiment, the optimizer and learning rate strategy during training are as follows: the Adam optimizer is used in conjunction with a cosine annealing learning rate scheduling strategy. The initial learning rate is set to 8e-4 on NUAA-SIRST and IRSTD-1K, and to 5e-4 on NUDT-SIRST. The model is trained for a total of 800 epochs.

[0064] S4: After preprocessing the single infrared image to be detected as in step S1, it is input into the trained target detection network. The network automatically performs image encoding, extracts multi-scale features, and senses the image state. Then, the image-decoder transformer dynamically generates a set of dedicated decoder parameters based on the sensed image state. The meta-decoder constructs a network based on the generated parameters, upsamples the deep features output by the encoder, and fuses shallow features through skip connections, finally outputting a target segmentation mask. The output target segmentation mask is binarized using a fixed threshold of 0.5, and the resulting binary mask is the final target detection result.

[0065] In this embodiment, the infrared weak target detection method based on meta-learning decoding outperforms the state-of-the-art methods on three standard datasets, verifying the effectiveness of the present invention.

[0066] 1. On the IRSTD-1K dataset, this invention achieves an IoU of 72.08% and a Fa of 9.641, which is significantly better than the previous best TCI-Former (70.14% IoU) and CFD-Net (69.97% IoU), demonstrating its powerful ability to handle low signal-to-noise ratio and complex backgrounds.

[0067] 2. On the NUDT-SIRST dataset, this invention significantly outperforms the others with an IoU of 95.74% and a Fa of 0.367, especially achieving a near-perfect level in terms of false alarm rate, highlighting the excellent performance of the dynamic decoding strategy in distinguishing between the target and the background.

[0068] 3. On the NUAA-SIRST dataset, this invention also achieved a leading result with an IoU of 81.77%.

[0069] Figure 5Visual comparison charts of detection results on the IRSTD-1K dataset are provided. In the chart, Image represents four infrared images with different scenes, GT represents the ground truth mask for small targets, DNANet represents the detection results on Image using DNANet, SCTransNet represents the detection results on Image using SCTransNet, L2SKNet represents the detection results on Image using L2SKNet, MMLNet represents the detection results on Image using MMLNet, and IrisNet represents the target detection results for different scene images obtained using this invention.

[0070] In summary, this embodiment fully demonstrates that the infrared weak target detection method based on meta-learning decoding of the present invention can effectively solve the mode drift problem and has outstanding advantages in terms of accuracy, robustness and practicality.

Claims

1. A method for detecting weak infrared targets based on meta-learning decoding, characterized in that... The steps are as follows: S1: Preprocess the input infrared image to obtain a standardized infrared image; S2: Construct a target detection network based on meta-learning decoding, including an image encoder, an image-decoder transformer, and a meta-decoder. The image encoder extracts multi-scale visual features from the input image and fuses high-frequency components. The image-decoder transformer dynamically generates decoder parameters using extracted multi-scale visual features. The meta-decoder is constructed based on the generated decoder parameters. It takes the deep features output by the encoder as input, upsamples them level by level, and fuses the shallow features output by the encoder through skip connections at each level. The meta-decoder processes the features level by level and finally outputs a single-channel target segmentation mask through a shared 1×1 convolutional layer. S3: Input the preprocessed infrared image into the target detection network constructed in step S2, and train the network in an end-to-end manner to obtain the trained network. S4: After preprocessing the single infrared image to be detected according to step S1, input it into the trained target detection network. Binarize the output target segmentation mask with a fixed threshold of 0.

5. The resulting binary mask is the final target detection result.

2. The infrared weak target detection method based on meta-learning decoding as described in claim 1, characterized in that: The preprocessing described in step S1 includes image size normalization, grayscale normalization, and data augmentation processing, wherein the data augmentation processing includes random flipping and random cropping.

3. The infrared weak target detection method based on meta-learning decoding as described in claim 1, characterized in that: The process of fusing high-frequency components in step S2 is as follows: First, the input image is subjected to Fourier transform to obtain the frequency domain representation. Then, a Gaussian high-pass filter mask is used to filter it. Finally, the filtered frequency domain signal is subjected to inverse Fourier transform to obtain the high-frequency component image in the spatial domain. This image is then stitched together with the original input image along the channel dimension as the final input of the encoder.

4. The infrared weak target detection method based on meta-learning decoding as described in claim 1, characterized in that: The image encoder described above adopts a multi-level UNet structure, with each level of convolutional block containing two 3×3 convolutional layers, which are then connected to a multi-core aggregation module, and redundant information is suppressed through a channel attention mechanism.

5. The infrared weak target detection method based on meta-learning decoding as described in claim 1, characterized in that: The specific process by which the image-decoder transformer dynamically generates decoder parameters using extracted multi-scale visual features is as follows: Step 1: Construct image perception conditions using the following formula: c = c im +c hf +c pe Where c represents the image perception conditional vector, c im This represents multi-scale visual feature embedding, which is obtained by patch embedding encoder features through specific convolutional layers, and then concatenating them along the channel dimension; c hf This indicates high-frequency signal embedding, obtained by projecting the high-frequency signal; c pe The cosine-sine position embedding is obtained using cosine-sine position encoding. Step 2: Initialize a set of learnable decoder tokens, refine the image perception conditions through multiple Transformer layers to obtain decoder tokens; each Transformer layer contains a self-attention module, a cross-attention module and a feedforward network module. The self-attention module models the inter-layer dependencies between decoder tokens, the cross-attention module establishes the alignment interaction between image perception conditions and tokens, and the feedforward network module implements nonlinear transformation. Step 3: Project and reshape the decoder tokens to obtain a structured two-dimensional tensor of decoder parameters. The row dimension of the two-dimensional tensor of decoder parameters corresponds to the level of the meta-decoder, and the column dimension corresponds to the number of parameters required by each layer of the decoder, including the convolution parameter column and the BN parameter column.

6. The infrared weak target detection method based on meta-learning decoding as described in claim 1, characterized in that: The meta-decoder includes three dynamically generated structures: a basic decoder, a multi-scale decoder, and a spatial attention decoder. Each layer of the basic decoder contains a 3×3 convolution, batch normalization, and a ReLU activation function. The multi-scale decoder adds parallel branches of 3×3 and 5×5 depth-separable convolutions to the basic decoder. The output is summed and then processed by two layers of 1×1 convolutions. The spatial attention decoder adds a spatial attention branch to the multi-scale decoder, models spatial information along the vertical and horizontal directions respectively, and generates spatial weights that are weighted to the multi-scale branch output in a residual manner.

7. The infrared weak target detection method based on meta-learning decoding as described in claim 1, characterized in that: The following joint loss function is used during network training: L total =L BCE (m,m′)+λ·L Dice (m,m′) Among them, L total L represents the total loss. BCE For binary cross-entropy loss, L Dice Let λ be the Dice loss, λ = 0.5 be the balance coefficient, m be the prediction mask, and m′ be the ground truth mask.

8. The infrared weak target detection method based on meta-learning decoding as described in claim 4, characterized in that: The multi-core aggregation block groups the input channels and processes them in parallel through 1×1, 3×3, 5×5, and 7×7 convolutions. After the channels are concatenated, they are output with channel attention weights. The channel attention is input into the MLP layer through the difference between global max pooling (GMP) and global average pooling (GAP), and weights are generated by sigmoid activation.

9. The infrared weak target detection method based on meta-learning decoding as described in claim 5, characterized in that: The Transformer has 6 layers and uses 6 attention heads, each with a hidden dimension of 64 and a dropout probability of 0.1.

Images