Infrared small target detection method based on double alignment and transparency map optimization

Abstract

The application provides an infrared small target detection method based on double alignment and transparency map optimization, and the implementation steps are as follows: acquiring a training sample set and a test sample set; constructing an infrared small target detection network model and iteratively training the same; and acquiring an infrared small target detection result. In the process of training the infrared small target detection model and acquiring the infrared small target detection result, the double alignment module fuses multi-modal information to generate a unified visual text prompt, calculates the similarity when the image and the text are aligned, enhances the utilization rate of different modal information, solves the problems of limited infrared small target data set and highly repeated scene, and improves the detection precision; the transition optimization module dynamically captures target-background transition details through a transparency map, fully utilizes the image and text information to separate the target and the background, reduces edge blur and false detection, significantly reduces the false positive rate, and effectively improves the accuracy of infrared small target detection.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and relates to an infrared small target detection method, specifically an infrared small target detection method based on double alignment and transparency map optimization, which can be applied to fields such as traffic management and maritime rescue. Background Technology

[0002] Small infrared targets refer to targets that are small in size and weak in brightness in infrared images, lacking effective information such as shape and texture, and typically have an area of ​​no more than 9×9 pixels. For these reasons, small infrared targets are easily obscured by complex backgrounds and random noise.

[0003] Infrared small target detection aims to identify small targets in low signal-to-noise ratio infrared images, with the core challenge being target extraction against complex backgrounds. Therefore, target enhancement, background suppression, and feature extraction are crucial factors affecting detection accuracy in infrared small target detection. Existing methods primarily rely on single-modal image input, ignoring the contextual information provided by the text modality, thus limiting detection accuracy. While deep learning methods (such as CNN and Transformer models) have made improvements, they are still limited by insufficient single-modal information. Multimodal models (such as CLIP) improve performance on natural image tasks by combining text and image modalities, demonstrating excellent performance in natural image tasks, but their direct application to infrared images faces challenges such as insufficient alignment and complexity of transition regions.

[0004] For example, Tianjin University disclosed an infrared small target detection method in its patent document "An Infrared Small Target Detection Method Based on Semantic Cueing" (Patent Application No.: CN202510572193.6, Publication No.: CN120495820A). This invention extracts image features through a 5-layer image encoder, where each layer consists of multiple CNN-Transformer fusion modules. Each encoder layer gradually reduces the image spatial resolution while doubling the number of channels, thus obtaining a low-resolution hidden layer representation. Next, a CLIP-pretrained Transformer text encoder is used to extract corresponding text embeddings to provide semantic guidance for image features. Based on the obtained image and text features, text features rich in category semantics are fused and interacted with the last two layers of image features through a two-stage feature fusion module, thereby enhancing the semantic expressive power of the image features. Finally, the semantically enhanced features are used to detect infrared small targets through a decoder. However, the invention has the following drawbacks: when extracting text and image features, it does not consider the limited infrared small target dataset and the high degree of scene repetition, resulting in indistinguishable sample pairs within the modality, making it impossible to fully explore the information in the image and text modalities; and it is difficult to extract the target boundary in the transition region between the target and the background. As a result, the accuracy of the model's infrared small target detection remains low. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and propose an infrared small target detection method based on double alignment and transparency map optimization, which is used to solve the technical problems of low detection accuracy caused by the limited infrared small target dataset and highly repetitive scenes in the prior art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention includes the following steps:

[0007] (1) Obtain the training sample set and the test sample set:

[0008] Get including A training sample set of preprocessed infrared images and their textual descriptions, and including A test sample set of preprocessed infrared images, in which, , Indicates the number of infrared images. , ;

[0009] (2) Construct an infrared small target detection network model:

[0010] Construct an infrared small target detection network model that includes a parallel double alignment module and a transition optimization module, and a mask decoding module cascaded with the aforementioned two modules. The dual alignment module includes a cascaded contrast alignment submodule for encoding image and text modalities and calculating similarity, and a generation alignment submodule for generating unified visual text cues; the transition optimization module is used to optimize the transparency map of the target-background transition region in the infrared image; and the mask decoding module is used to perform target detection based on the text cues and the optimized transparency map.

[0011] (3) Iteratively train the infrared small target detection network model:

[0012] The infrared small target detection network model was trained using the sample set. Iterative training is performed to obtain a well-trained infrared small target detection network model. ;

[0013] (4) Obtain infrared small target detection results:

[0014] The test sample set is used as the trained infrared small target detection network model. The input is propagated forward to obtain the detection result for each test sample.

[0015] Compared with the prior art, the present invention has the following advantages:

[0016] (1) In the process of iteratively training the infrared small target detection network model and obtaining the infrared small target detection results, the present invention integrates the multimodal information of infrared small targets by comparing the encoding of image and text modalities by the alignment submodule and generating unified visual text prompts generated by the alignment submodule. This can enhance the utilization rate of different modal information, improve the detection robustness in complex backgrounds compared with the prior art, and thus improve the detection accuracy.

[0017] (2) The present invention optimizes the transparency map of the target-background transition area in the infrared image through the transition optimization module, which can dynamically capture the details of the target-background transition, so as to achieve the separation of the target and the background, reduce the probability of edge blurring and false detection, and further improve the detection accuracy. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the implementation of the present invention.

[0019] Figure 2 This is a schematic diagram of the infrared small target detection network model in this invention.

[0020] Figure 3 This is a schematic diagram of the structure of the comparison alignment submodule in this invention. Detailed Implementation

[0021] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] Reference Figure 1 The present invention includes the following steps:

[0023] Step 1) Obtain the training sample set and the test sample set. The specific steps are as follows:

[0024] 1a) Dataset acquisition and partitioning:

[0025] Obtain data from publicly available infrared small target detection datasets NUAA-SIRST, NUDT-SIRST, and IRSTD-1k, including multiple categories. The system contains 1024 infrared images; NUAA-SIRST contains 420 images, NUDT-SIRST contains 1327 images, and IRSTD-1k contains 1001 images. All images are uniformly center-cropped to 1024×1024 pixels to fit the model input. A corresponding text description is generated for each image: the GPT-4V model is used to automatically generate a background description, including the background type, structural features, and overall scene semantics; and semantic accuracy is ensured through manual verification.

[0026] 1b) Data Augmentation and Standardization:

[0027] Images are randomly horizontally flipped and rotated 90° to expand the dataset size and reduce overfitting; the flip probability is set to 0.5 and the rotation angle is an integer multiple of 90°; and the image pixel values ​​are normalized to the [0,1] interval; the text description is converted into a 256-dimensional embedding vector by CLIP's text encoder, and the vocabulary size is based on the GPT-4V output configuration.

[0028] 1c) Partitioning of training and test sets

[0029] The preprocessed data was randomly divided into a training set at a 7:3 ratio. and test set This ensures a balanced distribution of samples across all categories; the training set is used for model learning, and the test set is used for performance validation.

[0030] Step 2) Construct an infrared small target detection network model Its structure is as follows Figure 2 As shown:

[0031] The double alignment module, whose contrast alignment submodule structure is as follows: Figure 3 As shown, it includes a parallel CLIP text encoder and a CLIP image encoder, and a similarity calculation unit cascaded with these two encoders; the similarity calculation unit includes a stacked vector dot product operation layer, a vector norm calculation layer, and an exponential operation layer; the alignment generation submodule includes a cascaded dual query transform (DQ-DETR) and Divergence calculation layer;

[0032] The transition optimization module includes a cascaded SAM image encoder, a feature gradient calculation submodule, a Laplacian transform submodule, a cross-attention fusion unit, and a transparency prediction layer. The feature gradient calculation submodule includes a foreground feature gradient calculation unit, a background feature gradient calculation unit, and an input feature gradient calculation unit arranged in parallel. The cross-attention fusion unit consists of a cross-attention layer, which internally includes a linear projection layer, an attention weight calculation layer, and a feature weighted summation layer.

[0033] The mask decoding module includes a cascaded feature fusion unit, a cue injection interface, and a mask decoder. The feature fusion unit includes two fully connected layers and a ReLU layer in between. The cue injection interface structure consists of a linear mapping layer and a feature fusion node. The mask decoder is composed of multiple decoder layers stacked on top of each other. Each decoder layer contains stacked attention computation layers and feedforward fully connected layers. Adjacent decoder layers are connected in sequence to transposed convolutional layers and multilayer perceptron decoder heads.

[0034] Step 3) Develop an infrared small target detection network model. Perform iterative training:

[0035] The infrared small target detection network model was trained using the sample set. Iterative training is performed to obtain a well-trained infrared small target detection network model. The specific steps are as follows:

[0036] (3a) Initialize the number of iterations to be The maximum number of iterations is , , No. The reconstruction network model of the next iteration The learnable weights are The bias parameter is and order ;

[0037] (3b) The contrast alignment submodule in the dual alignment module encodes the image and text modalities in each training sample to obtain the feature vectors of the image and text and calculates the similarity weights, and uses them to calculate the contrast alignment loss; at the same time, the generation alignment submodule performs cross-modal feature fusion on each training sample to obtain a unified visual-text cue vector; the transition optimization module calculates the gradient of the preprocessed infrared small target image and combines it with the visual-text cue vector output by the generation alignment module to optimize the transparency map and obtain the optimized transparency map parameters; the mask decoding module performs target-background separation and target detection on the optimized transparency map parameters output by the transition optimization module to obtain an accurate infrared small target segmentation mask;

[0038] (3b1) In the comparison alignment submodule, the CLIP image encoder performs image feature encoding and embedding mapping on the infrared images in the training samples to obtain visual feature vectors. The CLIP text encoder performs text feature encoding and embedding mapping on the text descriptions in the training samples to obtain text feature vectors. The similarity calculation unit calculates the similarity weights within a text modality. Similarity weights within image modalities The formula is:

[0039] ;

[0040] ;

[0041] in , ; This is the similarity scaling factor; These are the weighting coefficients for the distance term; Represents an exponential function; Represents the L2 norm;

[0042] (3b2) Generate an alignment submodule, which uses a text-to-image semantic guidance function in the dual-query transformer. , integrate text query vectors and visual query vector Generate prompts that adapt to the visual model The formula is:

[0043] ;

[0044] in, Represents the real number field. This represents the feature dimension of each query vector;

[0045] The SAM image encoder encodes the preprocessed infrared image and outputs a high-dimensional image embedding feature vector; the feature gradient calculation submodule processes the infrared image... and its target components Background components Calculate the gradient and construct it with the parameters of the transparency map. The constraints are used for transparency map optimization; the Laplace transform submodule performs a Laplace transform on the gradient; the cross-attention fusion unit uses transparency-related features as query vectors, and uses the gradient calculated by the feature calculation submodule and the Laplace feature obtained by the Laplace transform submodule as keys and values. It then weights and fuses the transparency map parameters, gradient features, and Laplace features through attention weights to obtain the gradient mixture term. The transparency prediction layer uses the fused features to optimize the transparency map parameters to obtain the optimal transparency map parameters. Constraint formulas and gradient mixture terms The calculation formula and the optimization calculation formula are as follows:

[0046] ;

[0047] ;

[0048] ;

[0049] in Represents the gradient; Indicates parameters of the transparency map; It is a gradient mixing term; This represents the parameters of the optimized transparency map; The parameter representing the minimum value; Indicates a transition region; This is the scaling factor; Indicates at pixel point The gradient mixing term at the location; Represents the square norm 2; This represents the integral over the transition region.

[0050] The feature fusion unit embeds features from the multi-scale image from the transition optimization module to form fused features for decoding; the cue injection interface receives the visual text cue vector output by the alignment module. The mask decoder decodes the fused features layer by layer under the constraints of the cue vector and transparency map parameters, resulting in an accurate infrared small target segmentation mask.

[0051] (3c) Calculate the contrast alignment loss using the similarity weights obtained from the contrast alignment module and the contrast learning loss function. By generating the matching probability between the image and text in the alignment module and Divergence alignment loss function calculates the alignment loss. The transition optimization module optimizes the objective function using transparency maps and calculates the transparency map loss value. Then through , and Weighted computation detection network model Total loss value Weights and bias parameters The detection network model for this iteration is updated. ;

[0052] The alignment loss And generating alignment loss Transparency map loss value and reconstructing the network model Total loss value The calculation formulas are as follows:

[0053] ;

[0054] ;

[0055] ;

[0056] ;

[0057] ;

[0058] ;

[0059] in For image-to-text contrast loss; For text-to-image contrast loss; For temperature parameters; Indicates the Kullback–Leibler divergence; The probability distribution from image to text. The probability distribution from text to image; This represents the summation operation; and These are the weighting coefficients;

[0060] For weights and bias parameters , The update is performed using the chain rule, and the update formulas are as follows:

[0061] ;

[0062] ;

[0063] in, Indicates bias parameters The update results express The update results Indicates the learning rate. express right Take the partial derivative, express right Take the partial derivative;

[0064] (3d) judgment If true, then the trained detection network model is obtained. Otherwise , Then proceed with step (3b).

[0065] Step 4) Target detection and output:

[0066] The test sample set is used as the trained infrared small target detection network model. The input is propagated forward to obtain The infrared small target detection results corresponding to each test sample.

[0067] The technical effects of the present invention will be further explained below with reference to simulation results:

[0068] 1. Experimental conditions and contents:

[0069] The hardware for the simulation experiment consisted of a single NVIDIA A800 GPU (80GB of memory). The software platform for the simulation experiment was Ubuntu 20.04 operating system, Python version 3.7, and PyTorch version 1.7.1. The initial learning rate was 0.0001, and the batch size was 4. The cross-union ratio, detection rate, and false alarm rate of the present invention and the prior art were compared using the simulation software PyCharm v.2023.1. The results are shown in Table 1.

[0070] Intersection and Union It is used to measure the degree of overlap between the predicted bounding box and the ground truth bounding box, and its calculation formula is as follows:

[0071]

[0072] in, This indicates the number of positive samples predicted to be positive. This indicates the number of samples that were correctly predicted. This represents the total number of positive samples.

[0073] Detection rate This represents the proportion of correctly classified positive and negative samples, and its calculation formula is:

[0074]

[0075] in, This indicates the number of positive samples predicted to be positive. This indicates the number of negative samples that were predicted to be positive. This indicates the number of negative samples that were predicted to be negative. This indicates the number of negative samples that are predicted to be positive.

[0076] False alarm rate This represents the proportion of samples that were predicted to be positive but actually were negative. The formula for calculating this proportion is:

[0077]

[0078] in, This indicates the number of positive samples predicted to be positive. This indicates the number of negative samples that are predicted to be positive.

[0079] 2. Analysis of experimental results:

[0080] Table 1

[0081]

[0082] Referring to Table 1, compared with the prior art, the present invention has the following advantages: and On the dataset, intersection-union ratio and detection rate All indicators have improved, and the false alarm rate has decreased. The decrease in the indicator indicates that the accuracy of the test has been effectively improved.

Claims

1. A method for detecting small infrared targets based on double alignment and transparency map optimization, characterized in that, Includes the following steps: (1) Obtain the training sample set and the test sample set: Get including A training sample set of preprocessed infrared images and their textual descriptions, and including A test sample set of preprocessed infrared images, in which, , Indicates the number of infrared images. , ; (2) Construct an infrared small target detection network model: Construct an infrared small target detection network model that includes a parallel double alignment module and a transition optimization module, and a mask decoding module cascaded with the aforementioned two modules. The dual alignment module includes a cascaded contrast alignment submodule for encoding image and text modalities and calculating similarity, and a generation alignment submodule for generating unified visual text cues; the transition optimization module is used to optimize the transparency map of the target-background transition region in the infrared image; and the mask decoding module is used to perform target detection based on the text cues and the optimized transparency map. (3) Iteratively train the infrared small target detection network model: The infrared small target detection network model was trained using the sample set. Iterative training is performed to obtain a well-trained infrared small target detection network model. ; (4) Obtain infrared small target detection results: The test sample set is used as the trained infrared small target detection network model. The input is propagated forward to obtain the detection result for each test sample.

2. The method according to claim 1, characterized in that, The method for obtaining the preprocessed infrared image mentioned in step (1) is as follows: Data augmentation is performed on each infrared small target image, and all augmented images are cropped to a uniform size to obtain the preprocessed image. A series of infrared images.

3. The method according to claim 1, characterized in that, The infrared small target detection network model described in step (2) ,in: The dual alignment module includes a comparison alignment submodule comprising parallel CLIP text encoders and CLIP image encoders, and a similarity calculation unit cascaded with these two encoders; the generation alignment submodule comprises a cascaded dual query transform (DQ-DETR) and... Divergence calculation layer; The transition optimization module includes a cascaded SAM image encoder, a feature gradient calculation submodule, a Laplacian transform submodule, a cross-attention fusion unit, and a transparency prediction layer. The mask decoding module includes a cascaded feature fusion unit, a hint injection interface, and a mask decoder.

4. The method according to claim 3, characterized in that, The iterative training of the infrared small target detection network model described in step (3) is implemented as follows: (3a) Initialize the number of iterations to be The maximum number of iterations is , , No. The reconstruction network model of the next iteration The learnable weights are The bias parameter is and order ; (3b) The contrast alignment submodule in the double alignment module encodes the image and text modalities in each training sample to obtain the feature vectors of the image and text and calculates the similarity weights; at the same time, the generation alignment submodule performs cross-modal feature fusion on each training sample to obtain a unified visual-text cue vector; the transition optimization module calculates the gradient of the preprocessed infrared small target image and combines it with the visual-text cue vector output by the generation alignment module to optimize the transparency map and obtain the optimized transparency map parameters; the mask decoding module performs target-background separation and target detection on the optimized transparency map parameters output by the transition optimization module to obtain an accurate infrared small target segmentation mask; (3c) Calculate the contrast alignment loss using the similarity weights obtained from the contrast alignment module and the contrast learning loss function. By generating the matching probability between the image and text in the alignment module and Divergence alignment loss function calculates the alignment loss. The transition optimization module optimizes the objective function using transparency maps and calculates the transparency map loss value. Then through , and Weighted computation detection network model Total loss value Weights and bias parameters The detection network model for this iteration is updated. ; (3d) judgment If true, then the trained detection network model is obtained. Otherwise , Then proceed with step (3b).

5. The method according to claim 4, characterized in that, The comparison alignment submodule described in step (3b) encodes the image and text modalities and calculates their similarity, and the generation alignment submodule generates a unified visual text prompt for each infrared small target image. The implementation steps are as follows: (3b1) In the comparison alignment submodule, the CLIP image encoder performs image feature encoding and embedding mapping on the infrared images in the training samples to obtain visual feature vectors. The CLIP text encoder performs text feature encoding and embedding mapping on the text descriptions in the training samples to obtain text feature vectors. The similarity calculation unit calculates the similarity weights within a text modality. Similarity weights within image modalities The formula is: ； ； in , ; This is the similarity scaling factor; These are the weighting coefficients for the distance term; Represents an exponential function; Represents the L2 norm; (3b2) The alignment submodule is generated in the dual query transformer via a text-to-image semantic guidance function. , integrate text query vectors and visual query vector Generate prompts that adapt to the visual model : ； in, Represents the real number field. This represents the feature dimension of each query vector.

6. The method according to claim 4, characterized in that, The transition optimization module described in step (3b) combines the preprocessed infrared small target image and the visual-text cue vector output by the alignment module to calculate the gradient and perform transparency map optimization. The implementation steps are as follows: The SAM image encoder encodes the preprocessed infrared image and outputs a high-dimensional image embedding feature vector. The feature gradient calculation submodule for infrared images and its target components Background components Calculate the gradient and construct it with the parameters of the transparency map. Constraints are used for transparency map optimization; The Laplace transform submodule performs a Laplace transform on the gradient; the cross-attention fusion unit uses transparency-related features as the query vector, and the gradient calculated by the feature calculation submodule and the Laplace feature obtained by the Laplace transform submodule as the key and value, respectively. It then weights and fuses the transparency map parameters, gradient features, and Laplace features using attention weights to obtain the gradient mixture term. The transparency prediction layer uses the fused features to optimize the transparency map parameters to obtain the optimal transparency map parameters. ; Constraint formulas, gradient mixture terms The calculation formula and the optimization calculation formula are as follows: ； ； ； in Represents the gradient; This represents the parameters of the optimized transparency map; The parameter representing the minimum value; Indicates a transition region; This is the scaling factor; This represents the 2-norm operation; Represents pixels exist The points on the scale.

7. The method according to claim 4, characterized in that, The mask decoding module described in step (3b) performs target-background separation and target detection on the optimized transparency map parameters output by the transition optimization module. The implementation steps are as follows: The feature fusion unit embeds features from the multi-scale image from the transition optimization module to form fused features for decoding; the cue injection interface receives the visual text cue vector output by the alignment module. The mask decoder decodes the fused features layer by layer under the constraints of the cue vector and transparency map parameters to obtain an accurate infrared small target segmentation mask.

8. The method according to claim 4, characterized in that, The alignment loss described in step (3c) And generating alignment loss Transparency map loss value and reconstructing the network model Total loss value The calculation formulas are as follows: The contrast alignment loss is calculated using the similarity weights obtained from the double alignment module and the contrast learning loss function. ,pass Divergence alignment loss function calculates the alignment loss. The transition optimization module optimizes the objective function using transparency maps and calculates the transparency map loss value. ; ； ； ； ； ； ； in For image-to-text contrast loss; For text-to-image contrast loss; For temperature parameters; Indicates the Kullback–Leibler divergence; The probability distribution from image to text. The probability distribution from text to image; This represents the summation operation; and These are the weighting coefficients.

9. The method according to claim 4, characterized in that, The steps (3d) described above for weights and bias parameters , The update is performed using the chain rule, and the update formulas are as follows: ； ； in, Indicates bias parameters The update results express The update results Indicates the learning rate. express right Take the partial derivative, express right Take the partial derivative.

Images