Single-point supervision infrared small target detection method based on mixed pseudo label generation

Abstract

The invention discloses a single-point supervision infrared small target detection method based on mixed pseudo tag generation, and the method comprises the steps: obtaining an infrared small target image data set containing a single-point supervision tag, and obtaining an initial pseudo tag marked by a small target pixel level through the single-point supervision tag according to the relative difference of pixel thresholds; constructing an infrared small target detection model, performing preliminary training by using the infrared small target image data set and the initial pseudo label supervision model, and fusing model output with the initial pseudo label after model convergence to obtain a mixed pseudo label; and training again by using the infrared small target image data set and the mixed pseudo label supervision model, and outputting a prediction result of the infrared small target detection task by using the trained model. The method provided by the invention is suitable for an infrared small target detection task which lacks pixel level labeling but only has single-point labeling, can help a model to detect a small target in an infrared image by using less labeling cost, and obtains a detection effect comparable with that of a model trained under a full supervision condition.

Description

Technical Field

[0001] The invention belongs to the field of computer vision, and in particular relates to a single-point supervised infrared small target detection method based on hybrid pseudo-label generation. Background Art

[0002] Recently, deep learning-based methods have shown excellent performance on pixel-level and instance-level metrics for infrared small target detection tasks. However, currently popular deep learning techniques usually rely on a large amount of labeled data for model training, and pixel-level annotation is time-consuming and laborious. In addition, considering the tiny scale and poor separability of a single pixel, each pixel has a significant impact on the final result.

[0003] To address these challenges, weakly supervised models for image segmentation and detection using imprecise annotations have become an effective solution and have gradually occupied an important position. The core of single-point supervised infrared small target detection lies in generating high-quality pixel-level masks through single-point annotation. Some studies have attempted to gradually expand point labels into mask labels through label evolution mechanisms. However, this approach still has a significant performance gap compared to fully supervised models. This gap mainly stems from the implicit nature of the label evolution mechanism, which relies entirely on the capabilities of neural networks. In addition, other methods generate masks by manually designing algorithms to circumvent the above limitations, but often face challenges when integrating high-level information beyond the algorithmic framework. These issues show that although weakly supervised learning has shown potential in the field of image segmentation and detection, more effective strategies still need to be further explored to improve its performance and applicability. Summary of the invention

[0004] To solve the above problems, the purpose of the present invention is to provide a single-point supervised infrared small target detection method based on hybrid pseudo-label generation to provide more abundant and accurate supervision information for the infrared small target detection model, so as to reduce the annotation cost and help the model adapt to the data-scarce environment. The method combines the advantages of manual methods and deep learning technology. Specifically, the method is divided into two stages, aiming to recover high-quality pseudo-labels from point labels.

[0005] In the initial stage, based on the characteristic that the pixel difference between the target area and the background area of ​​the infrared image is smaller than the pixel difference between the two areas, the present invention designs a mask generation strategy from single-point supervised annotation to initial pseudo-label. The process follows sequential steps to estimate the bounding box of the small target from the single-point supervised annotation and then generate detailed initial pseudo-label. Afterwards, the initial pseudo-label is used to supervise the training of the neural network and guide its prediction to converge to the label.

[0006] In the subsequent stage, the present invention combines the model's small target probability prediction probability map with the initial pseudo-label generated by the manual strategy to obtain a hybrid pseudo-label. The manual strategy helps to correct the model's prediction, while the model can integrate high-level information that may be ignored by the manual strategy. This complementary mechanism enables the present invention to generate higher-quality hybrid pseudo-labels. Finally, the hybrid pseudo-label is used to guide the model to achieve more accurate detection of infrared small targets.

[0007] To achieve the above object, the technical solution of the present invention is:

[0008] A single-point supervised infrared small target detection method based on hybrid pseudo-label generation includes the following steps:

[0009] S1. Obtain an infrared small target image dataset containing a single-point supervision label, and use the single-point supervision label to obtain an initial pseudo label for the small target pixel level annotation based on pixel differences;

[0010] S2. Build an infrared small target detection model, use the infrared small target image dataset and the initial pseudo-label supervision model for preliminary training, and after the model converges, fuse the model output with the initial pseudo-label to obtain a mixed pseudo-label;

[0011] S3. Retrain the model using the infrared small target image dataset and the hybrid pseudo-label supervision model, and use the trained model to output the prediction results of the infrared small target detection task.

[0012] Furthermore, in step S1, the infrared small target image dataset containing single-point supervision labels is represented as {(x1,s1),(x2,s2),...,(x N ,s N )}, where N is the total number of data in the data set, x i and i Respectively represent the i-th infrared image and its corresponding single-point supervision label, where x i is an infrared image with 1 channel and resolution H*W, and s i =(h i ,w i ), 0≤h i ≤H,0≤w i ≤W.

[0013] Furthermore, in step S1, obtaining the initial pseudo label of the small target pixel level annotation using the single point supervision label according to the relative pixel difference specifically includes:

[0014] S11. For each single-point supervision label s i , in the infrared image x i Taken from s iPixels in the four directions of up, down, left and right are used to estimate s based on pixel differences. i Corresponding to the boundary of the small target, the boundaries in the four directions are combined to obtain the estimated small target bounding box, and the pixel set in the bounding box is recorded as b i ;

[0015] S12, calculate pixel set b i The average value of the pixels in calculate and i Position pixel value The weighted value of is used as the small target judgment threshold σ i :

[0016]

[0017] Among them, α is the weighted adjustment coefficient;

[0018] S13, according to σ i For pixel set b i The pixels in are judged to generate a mask as the initial pseudo label

[0019]

[0020] in, is a tensor of dimension H*W, Represents the value of the tensor at coordinates (h,w), 1≤h≤H, 1≤w≤W.

[0021] Furthermore, in step S11, coordinate s i The boundaries of the small target in the four directions of up / down / left / right are estimated using the following method:

[0022] Taken from s i T pixels up / down / left / right form a vector v of length t+1, v contains s i ;

[0023] Calculate the absolute value of the difference between adjacent pixels in v to obtain a vector d = {d1, d2, ..., d t};

[0024] Calculate the mean of vector d and record it as

[0025] Start looping from i=2 and if the condition is not met, and When i≤t, execute i+1 and continue to judge; when this condition is met, i The position i units up / down / left / right is used as the estimated small object boundary.

[0026] Furthermore, in step S2, an infrared small target detection model is constructed based on a deep neural network, and the infrared small target detection model is trained using a batch processing method.

[0027] Furthermore, the initial training using the infrared small target image dataset and the initial pseudo-label supervision model includes:

[0028] S21. Randomly sample a batch of infrared images from the infrared small target image dataset The batch size N bs Given in advance; obtain the corresponding initial pseudo label according to the sampled infrared image set X

[0029] S22: Input the infrared image set X into the model and predict the probability of small targets existing at each pixel position of the batch of infrared images. where pred i is the predicted probability map of the i-th infrared image, expressed as a floating-point tensor with a resolution of H*W and a value range of 0-1;

[0030] S23, use the soft cross-combination function to calculate the predicted probability map pred of the model i With the initial pseudo label The loss function between

[0031] S24. According to the obtained loss function, use the Adam optimizer or the stochastic gradient descent optimizer with momentum and the back propagation algorithm to train the model;

[0032] S25. Repeat steps S21 to S24 until the model converges.

[0033] Furthermore, in step S2, fusing the model output with the initial pseudo-label to obtain a hybrid pseudo-label specifically includes:

[0034] Each infrared image x in the infrared small target image dataset i Input into the converged model to obtain the small target prediction probability map pred of the image i , pred i and Fusion to obtain mixed pseudo labels

[0035]

[0036] Furthermore, in step S3, retraining using the infrared small target image dataset and the hybrid pseudo-label supervision model specifically includes:

[0037] S31. Randomly sample a batch of infrared images from the infrared small target image dataset The batch size N bs Given in advance; find the corresponding hybrid pseudo-label based on the sampled infrared image set X′

[0038] S32: Input the infrared image set X′ into the model and predict the probability of small targets existing at each pixel position of the batch of infrared images.

[0039] S33, use the soft cross-combination function to calculate the predicted probability map pred of the model i With mixed pseudo labels The loss function between

[0040] S34. According to the obtained loss function, use the Adam optimizer or the stochastic gradient descent optimizer with momentum and the back propagation algorithm to train the model;

[0041] S35. Repeat steps S31 to S34 until the model converges.

[0042] Furthermore, in steps S23 and S33, the soft intersection-over-unit loss function between the predicted probability map and the pseudo-label is:

[0043]

[0044] Furthermore, in step S3, the prediction result of the infrared small target detection task outputted by the trained model specifically includes:

[0045] x i Input the trained model to get the small target prediction probability map pred i , prediction result y i for:

[0046]

[0047] The single-point supervised infrared small target detection method based on hybrid pseudo-label generation of the present invention has the following beneficial effects:

[0048] First, compared with the fully supervised annotation paradigm, the single-point supervision method applicable to the present invention significantly reduces the complexity and cost of data annotation. Compared with traditional full-image annotation, this method only needs to mark one point at the target location, reducing the workload of manual annotation. In some specific applications, it may be difficult to obtain a large number of labeled samples. The single-point supervision method can still effectively perform target detection when data is scarce, and is particularly suitable for the detection of small targets in infrared images.

[0049] Secondly, the present invention combines the advantages of both manual methods and deep neural networks, and can generate higher quality pseudo-labels. The manually designed pseudo-label generation method can make good use of the characteristics of small targets in infrared images as prior information, but it is difficult to flexibly mine high-level information outside the method. Using a deep neural network as a model to obtain pseudo-labels can use powerful feature learning capabilities and efficient processing capabilities to automatically learn complex feature representations from data, but lacks sufficient interpretability. The present invention achieves a fusion of the two to obtain a hybrid pseudo-label, which can effectively improve the quality of the pseudo-label and thus improve the overall performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flow chart of the single-point supervised infrared small target detection method based on hybrid pseudo-label generation of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0052] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.

[0053] refer to Figure 1 In a preferred embodiment of the present invention, a single-point supervised infrared small target detection method based on hybrid pseudo-label generation includes the following steps:

[0054] Firstly, an infrared small target image dataset with single-point supervision labels is obtained. The initial pseudo-labels of small target pixel-level annotations are obtained using the single-point supervision labels according to the relative difference of pixel thresholds.

[0055] In this embodiment, the infrared small target image dataset with single-point supervision labels is represented as {(x1,s1),(x2,s2),...,(x N ,s N )}, where N is the total number of data in the data set, x i and i They represent the i-th infrared image and its corresponding single-point supervision label respectively. i is an infrared image with 1 channel and resolution H*W, and s i =(h i ,wi ), 0≤h i ≤H,0≤w i ≤W.

[0056] In this embodiment, the initial pseudo label for pixel-level annotation of small objects is obtained by using single-point supervision labels according to the relative difference of pixel thresholds, which specifically includes the following steps:

[0057] The first step is to take each single-point supervision label s i , in the infrared image x i Taken from s i The pixels in the four directions of up, down, left and right are used to estimate s based on the pixel differences. i The boundary of the corresponding small target. The boundaries in the four directions are combined to obtain the estimated small target bounding box, and the pixel set in the bounding box is recorded as b i .

[0058] In this embodiment, the coordinate s is estimated based on the pixel difference i The boundaries corresponding to the small targets specifically include:

[0059] Take the above boundary estimation as an example, taken from s i The vector v of length t+1 is formed by going up t pixels. up (Including i ), then calculate v up The absolute value of the difference between adjacent pixels is a vector of length t. Calculate the vector d up The average value is recorded as Start looping from i=2 and if the condition is not met, and When i≤t, execute i+1 and continue to judge; when this condition is met, i The position i units upward is used as the estimated upper boundary of the small target. Similarly, the boundaries of the small targets in the other three directions are estimated.

[0060] The second step is to calculate the pixel set b in the estimated small target bounding box. i The average value of the pixels in calculate and i Position pixel value The weighted value of is used as the small target judgment threshold σ i :

[0061]

[0062] Among them, α is the weighted adjustment coefficient;

[0063] The third step is toi For pixel set b i Make judgments to generate masks as initial pseudo labels

[0064]

[0065] in, is a tensor of dimension H·W, Represents the value of the tensor at coordinates (h,w), 1≤h≤H, 1≤w≤W.

[0066] Secondly, an infrared small target detection model is constructed based on a deep neural network, and preliminary training is performed using an infrared small target image dataset and an initial pseudo-label supervision model. After the model converges, the model output is fused with the initial pseudo-label to obtain a hybrid pseudo-label. The specific steps are as follows:

[0067] The first step is to randomly sample a batch of infrared images from the infrared small target image dataset. The batch size N bs Given in advance; obtain the corresponding initial pseudo label according to the sampled infrared image set X

[0068] The second step is to input the infrared image set X into the model and predict the probability of small targets existing at each pixel position of the batch of infrared images. where pred i is the predicted probability map of the i-th infrared image, expressed as a floating-point tensor with a resolution of H*W and a value range of 0-1;

[0069] The third step is to use the soft intersection and ratio function to calculate the predicted probability map pred of the model i With the initial pseudo label The loss function between

[0070] In this embodiment, the soft intersection-over-union loss function between the predicted probability map and the pseudo-label is:

[0071]

[0072] Step 4: Based on the obtained loss function, use the Adam optimizer or the stochastic gradient descent optimizer with momentum and the back-propagation algorithm to train the model.

[0073] Step 5: Repeat steps 1 to 4 until the model converges.

[0074] The sixth step is to convert each infrared image x in the infrared small target image dataset i Input into the converged model to obtain the small target prediction probability map pred of the imagei . i and Fusion to obtain mixed pseudo labels

[0075]

[0076] Finally, the infrared small target image dataset and the hybrid pseudo-label supervision model are used for retraining, and the trained model is used to output the prediction results of the infrared small target detection task. The specific steps include:

[0077] The first step is to use batch processing to train the infrared small target detection model. A batch of infrared images is randomly sampled from the infrared small target image dataset. The batch size N bs Given in advance; find the corresponding hybrid pseudo-label based on the sampled infrared image set X′

[0078] The second step is to input the infrared image set X′ into the model and predict the probability of small targets existing at each pixel position of the batch of infrared images. where pred i is the predicted probability map of the i-th infrared image, expressed as a floating-point tensor with a resolution of H*W and a value range of 0-1;

[0079] The third step is to use the soft intersection and ratio function to calculate the predicted probability map pred of the model i With mixed pseudo labels The loss function between

[0080] Step 4: Based on the obtained loss function, use the Adam optimizer or the stochastic gradient descent optimizer with momentum and the back-propagation algorithm to train the model.

[0081] Step 5: Repeat steps 1 to 4 until the model converges.

[0082] Step 6: x i Input the trained model to get the small target prediction probability map pred i , prediction result y i for:

[0083]

[0084] Through the above technical scheme, the embodiment of the present invention is a single-point supervised infrared small target detection method based on hybrid pseudo-label generation. The present invention combines the advantages of manual methods and deep neural networks, obtains high-quality hybrid pseudo-labels by fusing the initial pseudo-labels generated by the manual method and the small target prediction probability map output by the model, and guides the model to achieve infrared small target detection performance comparable to full supervised labeling under the condition of only single-point supervised labeling.

[0085] Obviously, those skilled in the art should understand that each module or each step of the present invention can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0086] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A single-point supervised infrared small target detection method based on hybrid pseudo-label generation, characterized in that: The following steps are involved: S1. Obtain an infrared small target image dataset containing a single-point supervision label, and use the single-point supervision label to obtain an initial pseudo label for the small target pixel level annotation based on pixel differences; S2. Build an infrared small target detection model, use the infrared small target image dataset and the initial pseudo-label supervision model for preliminary training, and after the model converges, fuse the model output with the initial pseudo-label to obtain a mixed pseudo-label; S3. Retrain the model using the infrared small target image dataset and the hybrid pseudo-label supervision model, and use the trained model to output the prediction results of the infrared small target detection task.

2. The single-point supervised infrared small target detection method based on hybrid pseudo-label generation as claimed in claim 1, characterized in that: In step S1, the infrared small target image dataset containing single-point supervision labels is represented as {(x1,s1),(x2,s2),...,(x N ,s N )}, where N is the total number of data in the data set, x i and i Respectively represent the i-th infrared image and its corresponding single-point supervision label, where x i is an infrared image with 1 channel and resolution H*W, and s i =(h i ,w i ), 0≤h i ≤H,0≤w i ≤W.

3. The single-point supervised infrared small target detection method based on hybrid pseudo-label generation as claimed in claim 2 is characterized in that: In step S1, the initial pseudo label of the pixel level annotation of the small target is obtained by using the single-point supervision label according to the relative difference of the pixels, which specifically includes: S11. For each single-point supervision label s i , in the infrared image x i Taken from s i Pixels in the four directions of up, down, left and right are used to estimate s based on pixel differences. i Corresponding to the boundary of the small target, the boundaries in the four directions are combined to obtain the estimated small target bounding box, and the pixel set in the bounding box is recorded as b i ; S12, calculate pixel set b i The average value of the pixels in calculate and i Position pixel value The weighted value of is used as the small target judgment threshold σ i : Among them, α is the weighted adjustment coefficient; S13, according to σ i For pixel set b i The pixels in are judged to generate a mask as the initial pseudo label in, is a tensor of dimension H*W, Represents the value of the tensor at coordinates (h,w), 1≤h≤H, 1≤w≤W.

4. The single-point supervised infrared small target detection method based on hybrid pseudo-label generation as claimed in claim 3 is characterized in that: In step S11, coordinate s i The boundaries of the small target in the four directions of up / down / left / right are estimated using the following method: Taken from s i T pixels up / down / left / right form a vector v of length t+1, v contains s i ; Calculate the absolute value of the difference between adjacent pixels in v to obtain a vector d = {d1, d2, ..., d t }; Calculate the mean of vector d and record it as Start looping from i=2 and if the condition is not met, and When i≤t, execute i+1 and continue to judge; when this condition is met, i The position i units up / down / left / right is used as the estimated small object boundary.

5. The single-point supervised infrared small target detection method based on hybrid pseudo-label generation as claimed in claim 4 is characterized in that: In step S2, an infrared small target detection model is constructed based on a deep neural network, and the infrared small target detection model is trained using a batch processing method.

6. The single-point supervised infrared small target detection method based on hybrid pseudo-label generation as claimed in claim 5 is characterized in that: The initial training using the infrared small target image dataset and the initial pseudo-label supervision model specifically includes: S21. Randomly sample a batch of infrared images from the infrared small target image dataset The batch size N bs Given in advance; obtain the corresponding initial pseudo label according to the sampled infrared image set X S22: Input the infrared image set X into the model and predict the probability of small targets existing at each pixel position of the batch of infrared images. where pred i is the predicted probability map of the i-th infrared image, expressed as a floating-point tensor with a resolution of H*W and a value range of 0-1; S23, use the soft cross-combination function to calculate the predicted probability map pred of the model i With the initial pseudo label The loss function between S24. According to the obtained loss function, use the Adam optimizer or the stochastic gradient descent optimizer with momentum and the back propagation algorithm to train the model; S25. Repeat steps S21 to S24 until the model converges.

7. The single-point supervised infrared small target detection method based on hybrid pseudo-label generation as claimed in claim 6 is characterized in that: In step S2, the model output is fused with the initial pseudo-label to obtain a hybrid pseudo-label, specifically including: Each infrared image x in the infrared small target image dataset i Input into the converged model to obtain the small target prediction probability map pred of the image i , pred i and Fusion to obtain mixed pseudo labels 8. The single-point supervised infrared small target detection method based on hybrid pseudo-label generation as claimed in claim 7, characterized in that: In step S3, retraining using the infrared small target image dataset and the hybrid pseudo-label supervision model specifically includes: S31. Randomly sample a batch of infrared images from the infrared small target image dataset The batch size N bs Given in advance; find the corresponding hybrid pseudo-label based on the sampled infrared image set X′ S32: Input the infrared image set X′ into the model and predict the probability of small targets existing at each pixel position of the batch of infrared images. S33, use the soft cross-combination function to calculate the predicted probability map pred of the model i With mixed pseudo labels The loss function between S34. According to the obtained loss function, use the Adam optimizer or the stochastic gradient descent optimizer with momentum and the back propagation algorithm to train the model; S35. Repeat steps S31 to S34 until the model converges.

9. The single-point supervised infrared small target detection method based on hybrid pseudo-label generation as claimed in claim 8, characterized in that: In steps S23 and S33, the soft intersection-over-unit loss function between the predicted probability map and the pseudo-label is:

10. The single-point supervised infrared small target detection method based on hybrid pseudo-label generation according to claim 9, characterized in that: In step S3, the prediction results of the infrared small target detection task are outputted using the trained model, specifically including: x i Input the trained model to get the small target prediction probability map pred i , prediction result y i for:

Images