Single-point supervised infrared small target detection method based on hybrid pseudo-label generation

Abstract

The application discloses a kind of single-point supervision infrared small target detection method based on hybrid pseudo label generation, comprising: obtaining the infrared small target image dataset containing single-point supervision label, obtains the initial pseudo label of small target pixel level label according to pixel threshold relative difference using single-point supervision label;Construct infrared small target detection model, use infrared small target image dataset and initial pseudo label to supervise model and carry out preliminary training, after model convergence, the output of model is fused with initial pseudo label to obtain hybrid pseudo label;Using infrared small target image dataset and hybrid pseudo label to supervise model and retrain, and using the trained model to output the prediction result of infrared small target detection task.The method of the application is suitable for the infrared small target detection task that lacks pixel level label and only has single-point label, can help model to detect small target in infrared image using less annotation cost, obtain the detection effect comparable with the model trained under full supervision condition.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision, and particularly relates to a single-point supervised infrared small target detection method based on hybrid pseudo-label generation. BACKGROUND

[0002] Recently, deep learning-based methods have shown excellent performance on both pixel-level and instance-level metrics in the task of infrared small target detection. However, current popular deep learning techniques usually rely on a large amount of labeled data for model training, while pixel-level labeling is both time-consuming and labor-intensive. Moreover, considering the tiny scale and poor separability of individual pixels, each pixel has a significant impact on the final result.

[0003] To address these challenges, weakly supervised models that utilize imprecise labeled image segmentation and detection have emerged as an effective solution and gradually gained prominence. The core of single-point supervised infrared small target detection lies in generating high-quality pixel-level masks from single-point annotations. Some studies attempt to extend point labels to mask labels gradually through label evolution mechanisms. However, compared to fully supervised models, this approach still has a significant performance gap. This gap mainly stems from the implicit nature of the label evolution mechanism, which relies entirely on the capabilities of neural networks. Additionally, other methods generate masks through hand-designed algorithms to circumvent the aforementioned limitations, but often face challenges in integrating advanced information beyond the algorithmic framework. These issues indicate that although weakly supervised learning has shown potential in image segmentation and detection, further exploration of more effective strategies is needed to enhance its performance and applicability. SUMMARY

[0004] To address the above problems, the purpose of the present application is to provide a single-point supervised infrared small target detection method based on hybrid pseudo-label generation, which provides more abundant and accurate supervision information for infrared small target detection models, reduces labeling costs, and helps models adapt to data-scarce environments. This method combines the advantages of hand-crafted methods and deep learning techniques. Specifically, this method is divided into two stages, aiming to recover high-quality pseudo-labels from point labels.

[0005] In the initial stage, based on the characteristic that the pixel difference within the target region and the background region of the infrared image is smaller than the pixel difference between the two regions, the present application designs a mask generation strategy from single-point supervised labeling to initial pseudo-labels. This process follows sequential steps, estimating the bounding box of the small target from single-point supervised labeling, and then generating detailed initial pseudo-labels. Subsequently, the initial pseudo-labels are used to supervise the training of neural networks, guiding their prediction to converge towards the labels.

[0006] In the subsequent stage, the model small target probability prediction probability atlas is combined with the initial pseudo label generated by the manual strategy to obtain a hybrid pseudo label. The manual strategy helps to correct the model prediction, and the model can integrate high-level information that may be ignored by the manual strategy. The complementary mechanism enables the present application to generate a hybrid pseudo label with higher quality. Finally, the hybrid pseudo label is used to guide the model to achieve more accurate detection effect on the infrared small target.

[0007] To achieve the above object, the technical scheme of the present application is:

[0008] A single-point supervised infrared small target detection method based on hybrid pseudo label generation, comprising the following steps:

[0009] S1, obtaining an infrared small target image dataset containing single-point supervision labels, and obtaining initial pseudo labels of small target pixel-level labels by using single-point supervision labels according to pixel differences;

[0010] S2, constructing an infrared small target detection model, and using the infrared small target image dataset and the initial pseudo labels to supervise the model for preliminary training, and fusing the model output and the initial pseudo labels after the model converges to obtain a hybrid pseudo label;

[0011] S3, using the infrared small target image dataset and the hybrid pseudo label to supervise the model for retraining, and using the trained model to output the prediction result of the infrared small target detection task.

[0012] Further, in step S1, the infrared small target image dataset containing single-point supervision labels is represented as {(x1, s1), (x2, s2),..., (xN, sN)}, where N is the total number of data in the dataset, xi and si respectively represent the i-th infrared image and its corresponding single-point supervision label, xi is an infrared image with a channel number of 1 and a resolution of H*W, and si=(h, w), 0≤h≤H, 0≤w≤W. N N i i i i i i i i

[0013] Further, in step S1, the initial pseudo label of the small target pixel-level label is obtained by using the single-point supervision label according to the pixel relative difference, which specifically includes:

[0014] S11, for each single-point supervision label s i , taking s i from the infrared image x i , respectively,​​​​​​​​​The pixels in the four directions are estimated according to the pixel difference i The boundaries of the small target are combined to obtain an estimated small target boundary box, and the pixels in the boundary box are recorded as b i ;

[0015] S12, calculate the average value of the pixels in the set b i and record it as Calculate and s i The position pixel value The weighted value of s is taken as the small target judgment threshold σ i :

[0016]

[0017] Wherein, α is a weighting adjustment coefficient;

[0018] S13, according to σ i Judge the pixels in the set b i to generate a mask as an initial pseudo label

[0019]

[0020] Wherein, is a tensor with dimensions H*W, represents the value of the tensor at coordinates (h,w), 1≤h≤H, 1≤w≤W.

[0021] Further, in step S11, the coordinates s i The boundaries of the small target in the four directions of up / down / left / right are estimated by the following method:

[0022] Take s i The pixels in the four directions of up / down / left / right form a vector v with length t+1, v contains s i ;

[0023] Calculate the absolute value of the difference between adjacent pixels in v to obtain a vector d with length t={d1,d2,...,d t};

[0024] Calculate the average value of the vector d and record it as

[0025] From i=2, the judgment is made in a loop, when the condition and and i≤t, execute i+1, continue to determine; when the condition is met, the position from s i Up / down / left / right i units are taken as the estimated small target boundary.

[0026] Further, in step S2, an infrared small target detection model is constructed based on a deep neural network, and the infrared small target detection model is trained in a batch processing manner.

[0027] Further, the preliminary training of the infrared small target image dataset and the initial pseudo label supervised model specifically includes:

[0028] S21, a batch of infrared image sets are randomly sampled from the infrared small target image dataset Where the batch size N bs The initial pseudo label is obtained according to the sampled infrared image set X

[0029] S22, the infrared image set X is input into the model to predict the existence probability of the small target at each pixel position of the batch of infrared images Where pred i is the prediction probability graph of the ith infrared image, and the form is a floating point tensor with a resolution of H*W and a value range of 0-1;

[0030] S23, the soft intersection over union function is used to calculate the loss function between the prediction probability graph pred i and the initial pseudo label ;

[0031] S24, according to the obtained loss function, the Adam optimizer or the momentum-based stochastic gradient descent optimizer, and the back propagation algorithm are used to train the model;

[0032] S25, steps S21 to S24 are repeated until the model converges.

[0033] Further, in step S2, the model output and the initial pseudo label are fused to obtain a hybrid pseudo label, which specifically includes:

[0034] Each infrared image x i in the infrared small target image dataset is input into the converged model to obtain the small target prediction probability graph pred i of the image, pred i is fused with to obtain a hybrid pseudo label

[0035]

[0036] Further, in step S3, the infrared small target image dataset and the hybrid pseudo label supervised model are used to retrain the model, which specifically includes:

[0037] S31, randomly sampling a batch of infrared image sets from the infrared small target image data set Wherein the batch size N bs Predefined; find the corresponding mixed pseudo label according to the sampled infrared image set X'

[0038] S32, inputting the infrared image set X' into the model to predict the existence probability of small targets at each pixel position of the batch of infrared images

[0039] S33, calculating the prediction probability map pred of the model using the soft intersection over union function i Loss function between the mixed pseudo label

[0040] S34, training the model according to the obtained loss function using the Adam optimizer or the momentum-based stochastic gradient descent optimizer, and the back propagation algorithm;

[0041] S35, repeating steps S31 to S34 until the model converges.

[0042] Further, in steps S23 and S33, the soft intersection over union loss function between the prediction probability map and the pseudo label is:

[0043]

[0044] Further, in step S3, the prediction result of the infrared small target detection task output by the trained model specifically includes:

[0045] Inputting x i into the trained model to obtain a small target prediction probability map pred i , and the prediction result y i is:

[0046]

[0047] The single-point supervised infrared small target detection method based on mixed pseudo label generation has the following beneficial effects:

[0048] Firstly, compared with the full supervision labeling paradigm, the single-point supervision method suitable for the application significantly reduces the complexity and cost of data labeling. Compared with traditional full image labeling, this method only needs to label a point at the target position, reducing the workload of manual labeling. In some specific applications, it may be difficult to obtain a large number of labeled samples. The single-point supervision method can still effectively perform target detection in the case of data scarcity, and is especially suitable for small target detection in infrared images.

[0049] ​Secondly, this invention combines the advantages of both manual methods and deep neural networks, enabling the generation of higher-quality pseudo-labels. Manually designed pseudo-label generation methods can effectively utilize the characteristics of small targets in infrared images as prior information but struggle to flexibly extract higher-level information beyond the method's scope. While using deep neural networks as the model to obtain pseudo-labels leverages powerful feature learning capabilities and efficient processing to automatically learn complex feature representations from data, it lacks sufficient interpretability. This invention, by fusing the two methods to obtain hybrid pseudo-labels, effectively improves the quality of pseudo-labels, thereby enhancing the overall performance of the model. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating the single-point supervised infrared small target detection method based on hybrid pseudo-label generation according to the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0052] Conversely, this invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of the invention as defined in the claims. Furthermore, to provide a better understanding of the invention, certain specific details are described in detail below. However, those skilled in the art will fully understand the invention even without these detailed descriptions.

[0053] refer to Figure 1 In a preferred embodiment of the present invention, the single-point supervised infrared small target detection method based on hybrid pseudo-label generation includes the following steps:

[0054] First, an infrared small target image dataset containing single-point supervised labels is obtained. Based on the relative differences in pixel thresholds, initial pseudo-labels with pixel-level annotations for small targets are obtained using single-point supervised labels.

[0055] In this embodiment, the infrared small target image dataset containing single-point supervision labels is represented as {(x1,s1),(x2,s2),...,(x N ,s N )}, where N is the total number of data points in the dataset, x i and s i Let x represent the i-th infrared image and its corresponding single-point supervision label, respectively. i An infrared image with 1 channel and a resolution of H*W, and s i =(h i ,wi ), 0≤h i ≤H,0≤w i ≤W.

[0056] In this embodiment, obtaining initial pseudo-labels for small target pixel-level annotations using single-point supervised labels based on the relative differences in pixel thresholds specifically includes the following steps:

[0057] The first step is to perform a single-point supervision label s. i In infrared image x i The middle is taken from s respectively i The pixels in the four directions (up, down, left, and right) are estimated based on pixel differences. i The boundary of the corresponding small target. The estimated bounding box of the small target is obtained by combining the boundaries in four directions, and the set of pixels within the bounding box is recorded as b. i .

[0058] In this embodiment, coordinates s are estimated based on pixel differences. i The boundaries corresponding to the smaller goals specifically include:

[0059] Taking the above boundary estimation as an example, it is taken from s i The vector v is formed by moving t pixels upwards and having a length of t+1. up (including s) i Next, calculate v. up The absolute value of the difference between adjacent pixels is given as a vector of length t. Calculate vector d up The average value and recorded as Starting from i=2, the loop checks if the condition is not met. and If i ≤ t, execute i+1 and continue the judgment; when this condition is met, start from s. i The position i units upwards is used as the estimated upper boundary of the small target. Similarly, the boundaries of the small targets in the other three directions are estimated.

[0060] The second step is to calculate the set of pixels b in the estimated small target bounding box. i The average value of the middle pixels and recorded as calculate and s i Location pixel value The weighted value is used as the threshold σ for judging small targets. i :

[0061]

[0062] Where α is the weighted adjustment coefficient;

[0063] Third step, according to σi a pixel set b i judgment is made to generate a mask as an initial pseudo label

[0064]

[0065] wherein, is a tensor with dimensions H·W, represents the value of the tensor at coordinates (h, w), 1≤h≤H, 1≤w≤W.

[0066] Secondly, an infrared small target detection model is constructed based on a deep neural network, the model is preliminarily trained by using an infrared small target image dataset and an initial pseudo label to supervise the model, and after the model converges, the model output is fused with the initial pseudo label to obtain a mixed pseudo label. Specifically, the following steps are included:

[0067] Firstly, a batch of infrared image sets is randomly sampled from the infrared small target image dataset wherein the batch size N bs is predefined; the corresponding initial pseudo label is obtained according to the sampled infrared image set X

[0068] Secondly, the infrared image set X is input into the model to predict the existence probability of a small target at each pixel position of the batch of infrared images wherein pred i is a prediction probability graph of the i-th infrared image, and the form is a floating-point tensor with a resolution of H*W and a value range of 0-1;

[0069] Thirdly, a soft intersection over union function is used to calculate the loss function between the prediction probability graph pred i of the model and the initial pseudo label ;

[0070] In this embodiment, the soft intersection over union loss function between the prediction probability graph and the pseudo label is:

[0071]

[0072] Fourthly, according to the obtained loss function, the model is trained by using an Adam optimizer or a stochastic gradient descent optimizer with momentum, and a back propagation algorithm;

[0073] Fifthly, steps 1 to 4 are repeated until the model converges.

[0074] Sixthly, each infrared image x i in the infrared small target image dataset is input into the converged model to obtain a small target prediction probability graph pred of the image.i . The pred i is fused with the to obtain a mixed pseudo label

[0075]

[0076] Finally, the infrared small target image dataset and the mixed pseudo label supervised model are retrained, and the trained model is used to output the prediction result of the infrared small target detection task. Specifically, the following steps are included:

[0077] First, when training the infrared small target detection model, a batch processing method is used to randomly sample a batch of infrared image sets from the infrared small target image dataset Where the batch size N bs is given in advance; find the corresponding mixed pseudo label

[0078] Second, input the infrared image set X' into the model to predict the existence probability of small targets at each pixel position of the batch of infrared images Where pred i is the prediction probability map of the i-th infrared image, represented by a floating-point tensor with resolution H*W and value range 0-1.

[0079] Third, use the soft intersection over union function to calculate the loss function between the prediction probability map pred i of the model and the mixed pseudo label

[0080] Fourth, according to the obtained loss function, use the Adam optimizer or the momentum-based stochastic gradient descent optimizer, and the back propagation algorithm to train the model.

[0081] Fifth, repeat steps one to four until the model converges.

[0082] Sixth, input x i into the trained model to obtain the small target prediction probability map pred i , and the prediction result y i is:

[0083]

[0084] ​By the above technical solution, the embodiment of the present application is based on a single-point supervised infrared small target detection method generated by mixed pseudo labels. The present application combines the advantages of both manual methods and deep neural networks, obtains high-quality mixed pseudo labels by fusing the initial pseudo labels generated by the manual method and the small target prediction probability atlas output by the model, and guides the model to achieve infrared small target detection performance comparable to full supervision annotation under the condition of only single-point supervision annotation.

[0085] Obviously, those skilled in the art should understand that each module or each step of the present application can be realized by a general computing device, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, and optionally, each module or each step can be realized by program code executable by a computing device, so that each module or each step can be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described can be executed in different order, or each module can be manufactured into an integrated circuit module, or multiple modules or steps can be manufactured into a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.

[0086] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A single-point supervised infrared small target detection method based on hybrid pseudo-label generation, characterized in that, The method comprises the following steps: S1, acquiring an infrared small target image dataset containing a single-point supervision label, obtaining an initial pseudo label of a small target pixel level label according to pixel differences using the single-point supervision label, specifically comprising: S11, For each single-point supervision label s i In infrared image x i The middle is taken from s respectively i s is estimated based on pixel differences in the four directions: up, down, left, and right. i For the boundary of the small target, the boundaries in the four directions are combined to obtain the estimated bounding box of the small target. The set of pixels within the bounding box is recorded as b. i ; S12, calculate the pixel set b i The average value of the middle pixels is calculated and recorded as Calculate And s i The weighted value of the position pixel value As a small target judgment threshold σ i : Wherein, alpha is a weighting adjustment coefficient; S13, according to σ i Pixels in the pixel set b i are judged to generate a mask as an initial pseudo label wherein, is a tensor of dimension H*W, denotes the value of the tensor at coordinates (h, w), 1≤h≤H, 1≤w≤W; S2, constructing an infrared small target detection model, using the infrared small target image dataset and the initial pseudo label to supervise the model for preliminary training, and fusing the model output and the initial pseudo label after the model converges to obtain a mixed pseudo label; S3, using the infrared small target image dataset and the mixed pseudo label to supervise the model for retraining, and using the trained model to output a prediction result of an infrared small target detection task.

2. The method of claim 1, wherein the method is based on hybrid pseudo-labeling generation for single-point supervised infrared small target detection. In step S1, the infrared small target image dataset containing single-point supervised labels is represented as {(x1,s1),(x2,s2),...,(x...} N ,s N )}, where N is the total number of data points in the dataset, x i and s i Let x represent the i-th infrared image and its corresponding single-point supervision label, respectively. i For an infrared image with 1 channel and a resolution of H*w, and s i =(h i ,w i ), 0≤h i ≤H,0≤w i ≤W.

3. The method of claim 1, wherein the method is based on hybrid pseudo-labeling generation for single-point supervised infrared small target detection. In step S11, the coordinates s i The boundaries of the four directions (up, down, left, right) corresponding to the small target are estimated by the following method: taken from s i up / down / left / right t pixels into a vector v of length t+1, v contains s i ; The absolute value of the difference between adjacent pixels in v gives a vector d = {d1, d2,..., dt} of length t t}; The mean of the vector d is calculated and recorded as Looping through the judgment from i = 2, when the condition is not met And And i ≤ t, execute i + 1, continue to determine; when the condition is met, from s i Up / down / left / right i units of position as the estimated small target boundary.

4. The method of claim 3, wherein the method is based on hybrid pseudo-labeling generation for single-point supervised infrared small target detection. In step S2, the infrared small target detection model is constructed based on a deep neural network, and the infrared small target detection model is trained in a batch processing manner.

5. The method of claim 4, wherein the method is based on hybrid pseudo-labeling generation for single-point supervised infrared small target detection. The preliminary training using the infrared small target image dataset and the initial pseudo label to supervise the model specifically comprises: S21, randomly sample a batch of infrared image sets X = from the infrared small target image dataset where the batch size N bs Pre-given; obtain the corresponding initial pseudo label according to the sampled infrared image set X S22, input the infrared image set X into the model to predict the existence probability of small targets at each pixel position of the batch of infrared images predi= predi+1+ predi-1+ 2 * predi i prediis the predicted probability map of the ith infrared image, represented as a floating-point tensor with a resolution of H*W and a value range of 0-1; S23, calculate the prediction probability map pred of the model using the soft intersection over union function i loss function between the initial pseudo labels loss function between the initial pseudo labels S24, training the model according to the obtained loss function using an Adam optimizer or a momentum-based stochastic gradient descent optimizer, and a back propagation algorithm; S25, repeating steps S21 to S24 until the model converges.

6. The method of claim 5, wherein the method is based on hybrid pseudo-labeling generation for single-point supervised infrared dim target detection. In step S2, the model output is fused with the initial pseudo label to obtain the mixed pseudo label, specifically comprising: Each infrared image x in the infrared small target image dataset is input into the converged model to obtain a small target prediction probability atlas pred of the image i Input into the converged model to obtain a small target prediction probability atlas pred of the image i pred is fused to obtain a mixed pseudo label i With ​ 7. The method of claim 6, wherein the method is based on hybrid pseudo-labeling generation for single-point supervised infrared dim target detection. In step S3, the infrared small target image dataset and the mixed pseudo label are used to supervise the model for retraining, specifically comprising: S31, randomly sample a batch of infrared image sets X' = from the infrared small target image dataset where batch size N bs Pre-given; find the corresponding mixed pseudo-label according to the sampled infrared image set X' S32, input the infrared image set X' into the model to predict the existence probability of small targets at each pixel position of the batch of infrared images after the model S33, calculate the prediction probability map pred of the model using the soft intersection over union function i with the hybrid pseudo label loss function between S34, training the model according to the obtained loss function using an Adam optimizer or a momentum-based stochastic gradient descent optimizer, and a back propagation algorithm; S35, repeating steps S31 to S34 until the model converges.

8. The method of claim 7, wherein the method is based on hybrid pseudo-labeling generation for single-point supervised infrared dim target detection. In steps S23 and S33, the soft intersection over union loss function between the prediction probability graph and the pseudo label is:

9. The method of claim 8, wherein the method is based on hybrid pseudo-labeling generation for single-point supervised infrared dim target detection. In step S3, the trained model is used to output the prediction result of the infrared small target detection task, specifically comprising: x i Input the trained model to obtain a small target prediction probability atlas pred i The prediction result y i is:

Images