A contrast-based single-point supervised infrared weak target detection method and system
By generating initial extended pseudo-labels and updating them using the contrast of the network output, the problem of unstable label updates in single-point supervised infrared weak target detection is solved, achieving high-quality target detection results and reaching the performance level of fully supervised methods.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2023-12-29
- Publication Date
- 2026-05-26
AI Technical Summary
Existing single-point supervised infrared weak target detection algorithms can only achieve about 50% of the average cross-union ratio of the real labels when updating labels. Moreover, the network state is unstable in the early stage of training, making it difficult to cope with targets of different shapes and sizes, and the class imbalance problem is serious.
By generating initial expanded pseudo-labels and updating them using the contrast of the network output, the intrinsic behavior of the network in the early stages of training is coordinated. Confidence ensemble is used to promote network convergence, and contrast is used as the benchmark for label updates to achieve a stable iterative process.
It improves the detection rate and average crossover ratio of infrared weak target detection, reaching the performance level of fully supervised methods. The network training is more stable and adaptable to different target shapes and sizes.
Smart Images
Figure CN117809097B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of pattern recognition and machine learning technology, and particularly to weakly supervised training and infrared weak target detection, and more specifically to a contrast-based single-point supervised infrared weak target detection method and system. Background Technology
[0002] In recent years, with the development of deep learning technology, infrared weak target detection algorithms based on convolutional neural networks have been proposed. However, these methods are all fully supervised and require a large amount of labeled datasets. Moreover, since infrared weak target detection networks often treat this problem as an image segmentation problem, accurate pixel-level mask labels are required. Due to the sensitivity and complexity of infrared data, obtaining a large-scale training dataset is quite resource-intensive.
[0003] Weakly supervised learning has been gradually introduced into the field of infrared weak target detection in recent years. Since it does not require complete labels, it can reduce data constraints and introduce a more effective detection paradigm in infrared weak target detection. Single-point supervision is an important form of weak supervision. The method of single-point supervision for infrared weak target detection utilizes the training characteristics of convolutional neural networks. In the early stage, a cluster of pixels around the target is segmented with low confidence, and as training progresses, it gradually approaches the given single-point supervision label. The label is updated by using the network output at specific stages of training.
[0004] While current label evolution strategies can generate pseudo-labels using single-point labels to some extent, current single-point supervised infrared weak target detection algorithms can only achieve about 50% of the average cross-union ratio (CUI) of the true labels in updating labels. This is because the size of each target cannot be known in advance before training, and using a uniform pseudo-label stopping criterion is very complex when facing targets with different shapes and scales. In addition, the infrared weak target detection problem is an imbalanced classification problem. Due to the randomness of the network initialization in the early stage and the unavoidable randomness during training, the network state is unstable in the early stage of single-point supervised training, and using a uniform update start time cannot achieve the best performance.
[0005] Therefore, how to provide a contrast-based single-point supervised infrared weak target detection method and system is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0006] In view of this, the present invention provides a contrast-based single-point supervised infrared weak target detection method and system to solve the problems mentioned in the background art.
[0007] To achieve the above objectives, the present invention adopts the following technical solution:
[0008] A contrast-based single-point supervised infrared target detection method includes the following steps:
[0009] S1. Acquire an infrared image, and for each target in each single-point label of the infrared image, expand the single-point label by using the contrast in the image to generate an initial expanded pseudo-label;
[0010] S2. Train the convolutional neural network using the initial expanded pseudo-labels and the acquired infrared images;
[0011] S3. During training, the pseudo-labels are updated based on the contrast of confidence in the network output. The updated pseudo-labels are then used for iterative training to obtain the infrared weak target detection network model.
[0012] S4. Use an infrared weak target detection network model to perform target detection on the infrared image under test and output target labels.
[0013] Preferably, the specific content of generating the initial extended pseudo-tag in step S1 includes:
[0014] S11. Record the grayscale value at the single-point label as I. center And define a target box at that location;
[0015] S12. Gradually expand the target box adaptively until the gray value of a pixel on the boundary of the target box is less than the preset gray value threshold or the boundary size of the target box reaches the preset maximum size, and obtain the estimated bounding box.
[0016] S13. The initial pseudo-labels are extracted by thresholding the bounding box, which are the generated initial extended pseudo-labels.
[0017] Preferably, the preset grayscale threshold in step S12 is specifically as follows:
[0018] T p =0.8×I center
[0019] Among them, I center This represents the grayscale value at the single label point.
[0020] Preferably, the segmentation threshold in step S13 is as follows:
[0021] T box =I center -k box ×σ box
[0022] Among them, I center σ represents the grayscale value at a single label point. box k is the standard deviation of the grayscale values of the pixels inside the bounding box. box For parameters.
[0023] Preferably, the specific content for updating the pseudo-tags is as follows:
[0024] S31. For each target in the nth round of pseudo-labels, extract the d×d neighborhood label L′. n Subtract neighborhood label L′ n The smallest gray value is used to obtain the neighborhood label.
[0025] S32. Expand the target box in the nth round of pseudo-labels to m times the maximum length and width to obtain the expanded box;
[0026] S33. Calculate the segmentation threshold based on the contrast between the confidence representation of pixels within the target box and the confidence representation of pixels between the target box and the expanded box in the network output, and set the pixels with confidence values lower than the threshold in the output label to 0.
[0027] S34. Discard pixels in the output label that are not 8-connected to the original pseudo-label for false alarm cancellation, denoted as... Based on the label after false alarm elimination Extracting the neighborhood label L′ of d×d n Update pseudo tags.
[0028] Preferably, the specific method for calculating the threshold in step S33 based on the grayscale values of the pixels within the target box and the grayscale values of the pixels between the target box and the expanded box is as follows:
[0029]
[0030]
[0031]
[0032] Among them, I core I represents the confidence level of pixels within the target bounding box. surr This represents the confidence level of pixels between the target bounding box and the expanded bounding box. The confidence score of the i-th pixel with the highest grayscale value within the target bounding box, where p is the number of pixels within the target bounding box used to represent the confidence score. represents the confidence level of the i-th pixel with the largest gray value within the region between the target box and the expanded box, q is the number of pixels used to represent the confidence level within the region between the target box and the expanded box, k is the minimum threshold, and λ is the balance parameter.
[0033] Preferably, the method for updating the tag in step S34 is as follows:
[0034]
[0035] Where r is the label update speed, L′ n+1 This is the updated pseudo-label for the (n+1)th round.
[0036] A contrast-based single-point supervised infrared weak target detection system, based on the aforementioned contrast-based single-point supervised infrared weak target detection method, includes: an infrared image acquisition module, an infrared weak target detection network model, and a training module;
[0037] Infrared image acquisition module, used to acquire infrared images;
[0038] The training module includes an initial expanded pseudo-label generation unit, a pseudo-label update unit, and a model training unit;
[0039] The initial extended pseudo-label generation unit is used to generate an initial extended pseudo-label for each target in each single-point label of the infrared image by expanding the single-point label in the image through contrast.
[0040] The pseudo-label update unit is used to update the pseudo-labels during training based on the contrast of confidence in the network output.
[0041] The model training unit is used to iteratively train the convolutional neural network using the initial expanded pseudo-labels, the updated pseudo-labels, and the infrared image to obtain the infrared weak target detection network model.
[0042] An infrared weak target detection network model is used to detect targets in infrared images and output target labels.
[0043] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the contrast-based single-point supervised infrared weak target detection method.
[0044] A processing terminal includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the contrast-based single-point supervised infrared weak target detection method.
[0045] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a contrast-based single-point supervised infrared weak target detection method and system. By expanding the initial pseudo-labels, pseudo-labels suitable for network training can be obtained, which can coordinate the inherent behavior of the network in the early stage of training, promote convergence and alleviate the class imbalance problem. Using the contrast of the neural network output as the benchmark for label update can achieve a stable iterative process to obtain high-quality pseudo-labels. Integrating the prediction confidence into the label helps the network convergence process. Attached Figure Description
[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0047] Figure 1 The attached figure is a schematic diagram of a contrast-based single-point supervised infrared weak target detection method provided by the present invention;
[0048] Figure 2 The attached figure is a schematic diagram of the initial extended pseudo-tag generation provided by the present invention;
[0049] Figure 3 The attached figure is a schematic diagram of the pseudo-tag update provided by the present invention;
[0050] Figure 4 The attached figure is a comparative diagram of target detection results when the target scale is large, as provided by the present invention.
[0051] Figure 5 The attached figure is a comparative diagram of target detection results provided by the present invention when the target is easily submerged in the background in complex situations;
[0052] Figure 6 The attached figure is a schematic diagram comparing the target detection results when the targets are relatively dense, as provided by the present invention. Detailed Implementation
[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0054] This invention discloses a contrast-based single-point supervised infrared weak target detection method, such as... Figure 1 This includes the following steps:
[0055] S1. Acquire an infrared image, and for each target in each single-point label of the infrared image, expand the single-point label by using the contrast in the image to generate an initial expanded pseudo-label;
[0056] S2. Train the convolutional neural network using the initial expanded pseudo-labels and the acquired infrared images;
[0057] S3. During training, the pseudo-labels are updated based on the contrast of confidence in the network output. The updated pseudo-labels are then used for iterative training to obtain the infrared weak target detection network model.
[0058] S4. Use an infrared weak target detection network model to perform target detection on the infrared image under test and output target labels.
[0059] To further implement the above technical solutions, such as Figure 2 The specific content of generating the initial extended pseudo-tag in step S1 includes:
[0060] S11. Record the grayscale value at the single-point label as I. center And define a 1×1 target box at that location;
[0061] S12. Gradually expand the target box adaptively until the gray value of a pixel on the boundary of the target box is less than the preset gray value threshold or the boundary size of the target box reaches the preset maximum size, and obtain the estimated bounding box.
[0062] S13. The initial pseudo-labels are extracted by thresholding the bounding box, which are the generated initial extended pseudo-labels.
[0063] To further implement the above technical solution, the preset grayscale threshold in step S12 is specifically as follows:
[0064] T p =0.8×I center
[0065] Among them, I center This represents the grayscale value at the single label point.
[0066] To further implement the above technical solution, the segmentation threshold in step S13 is specifically as follows:
[0067] T box =I center -k box ×σ box
[0068] Among them, I center σ represents the grayscale value at a single label point. box k is the standard deviation of the grayscale values of the pixels inside the bounding box. box The parameter is set to 0.9.
[0069] In practical applications, the maximum size of the target bounding box is set to 15×15.
[0070] To further implement the above technical solutions, such as Figure 3 The specific content of updating pseudo-tags is as follows:
[0071] S31. For each target in the nth round of pseudo-labels, extract the d×d neighborhood label L′. n Subtract neighborhood label L′ n The smallest gray value is used to obtain the neighborhood label.
[0072] S32. Expand the target box in the nth round of pseudo-labels to m times the maximum length and width to obtain the expanded box;
[0073] S33. Calculate the segmentation threshold based on the contrast between the confidence representation of pixels within the target box and the confidence representation of pixels between the target box and the expanded box in the network output, and set the pixels with confidence values lower than the threshold in the output label to 0.
[0074] S34. Discard pixels in the output label that are not 8-connected to the original pseudo-label for false alarm cancellation, denoted as... Based on the label after false alarm elimination Extracting the neighborhood label L′ of d×d n Update pseudo tags.
[0075] In practical applications, the labels are updated every 5 rounds or so after the first 50 rounds of training, based on the contrast of confidence in the network output.
[0076] To further implement the above technical solution, the specific method for calculating the threshold in step S33 based on the grayscale values of the pixels within the target box and the grayscale values of the pixels between the target box and the expanded box is as follows:
[0077]
[0078]
[0079]
[0080] Among them, I core I represents the confidence level of pixels within the target bounding box. surr This represents the confidence level of pixels between the target bounding box and the expanded bounding box. The confidence score of the i-th pixel with the highest grayscale value within the target bounding box, where p is the number of pixels within the target bounding box used to represent the confidence score. represents the confidence level of the i-th pixel with the largest gray value within the region between the target box and the expanded box, q is the number of pixels used to represent the confidence level within the region between the target box and the expanded box, k is the minimum threshold, and λ is the balancing parameter, set to 0.9 and 0.3 respectively.
[0081] To further implement the above technical solution, the method for updating the tag in step S34 is as follows:
[0082]
[0083] Where r is the tag update speed, set to 0.5, L′ n+1 This is the updated pseudo-label for the (n+1)th round.
[0084] During training, as pseudo-labels are generated and updated, the network model also learns the characteristics of the target. The two promote each other, and eventually the network training converges, resulting in the final infrared weak target detection network model.
[0085] In this embodiment, three publicly available infrared weak target detection datasets were used for training and testing, with a training set to test set ratio of 7:3. The datasets were NUAA-SIRST, NUDT-SIRST, and IRSTD-1k. DNA-Net, UIU-Net, and ALCNet were used as benchmark models, and their detection rates (Pd) and mean intersection-over-union ratios (mIoU) were statistically analyzed. The results are shown in Tables 1-3.
[0086] Table 1 shows the DNA-Net detection results:
[0087]
[0088] Table 2 shows the UIU-Net detection results:
[0089]
[0090] Table 3 shows the ALCNet detection results:
[0091]
[0092] The results in Tables 1-3 show that the single-point supervised training strategy proposed in this invention can achieve a fully supervised performance of 95% detection rate on multiple datasets and with multiple models, and a fully supervised performance of about 80% average intersection-union ratio.
[0093] In another embodiment, taking the DNANet neural network as an example, a contrast-based single-point supervised infrared weak target detection method of the present invention is compared with the traditional Top-Hat, Max-Mean, and TTLCM methods, as well as the fully supervised results and single-point supervised detection results of deep learning-based UIU-Net, ALCNet, and DNANet methods. Figures 4-6 , Figure 4 This indicates detection results with a larger target scale. Figure 5 This indicates the detection result when the target is easily submerged in a complex background. Figure 6The results show that the method of this invention is closest to fully supervised detection and can achieve better target segmentation performance.
[0094] A contrast-based single-point supervised infrared weak target detection system, based on a contrast-based single-point supervised infrared weak target detection method, includes: an infrared image acquisition module, an infrared weak target detection network model, and a training module;
[0095] Infrared image acquisition module, used to acquire infrared images;
[0096] The training module includes an initial expanded pseudo-label generation unit, a pseudo-label update unit, and a model training unit;
[0097] The initial extended pseudo-label generation unit is used to generate an initial extended pseudo-label for each target in each single-point label of the infrared image by expanding the single-point label in the image through contrast.
[0098] The pseudo-label update unit is used to update the pseudo-labels during training based on the contrast of confidence in the network output.
[0099] The model training unit is used to iteratively train the convolutional neural network using the initial expanded pseudo-labels, the updated pseudo-labels, and the infrared image to obtain the infrared weak target detection network model.
[0100] An infrared weak target detection network model is used to detect targets in infrared images and output target labels.
[0101] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a contrast-based single-point supervised infrared weak target detection method.
[0102] A processing terminal includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements a contrast-based single-point supervised infrared weak target detection method.
[0103] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0104] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A contrast-based single-point supervised infrared dim small target detection method, characterized in that, Includes the following steps: S1. Acquire an infrared image, and for each target in each single-point label of the infrared image, expand the single-point label by using the contrast in the image to generate an initial expanded pseudo-label; S2. Train the convolutional neural network using the initial expanded pseudo-labels and the acquired infrared images; S3. During training, the pseudo-labels are updated based on the contrast of confidence in the network output. The updated pseudo-labels are then used for iterative training to obtain the infrared weak target detection network model. S4. Use an infrared weak target detection network model to perform target detection on the infrared image under test and output target labels; The specific content of updating pseudo-tags is as follows: S31. For each target in the n-th round of pseudo-labels, extract the neighborhood label , subtract the minimum grayscale value in the neighborhood label to obtain the neighborhood label ; S32. Expand the target box in the nth round of pseudo-labels to m times the maximum length and width to obtain the expanded box; S33. Calculate the segmentation threshold based on the contrast between the confidence representation of pixels within the target box and the confidence representation of pixels between the target box and the expanded box in the network output, and set the pixels with confidence values lower than the threshold in the output label to 0. S34. Discard the pixels which are not eight-connected with the pseudo-label before updating in the output label for false alarm elimination, denoted as S35. Update the pseudo-label according to the false alarm eliminated label and the extracted neighborhood label ; The specific method for calculating the segmentation threshold in step S33 based on the contrast between the confidence representation of pixels within the target box and the confidence representation of pixels between the target box and the expanded box in the network output is as follows: wherein, is a confidence representation of pixels within the target box, is a confidence representation of pixels between the target box and the enlarged box, is a confidence of the i-th largest gray value pixel within the target box region, p is the number of pixels used to represent the confidence within the target box region, is a confidence of the i-th largest gray value pixel within the region between the target box and the enlarged box, q is the number of pixels used to represent the confidence within the region between the target box and the enlarged box, k is a minimum threshold, and λ is a balance parameter. The method for updating the tags in step S34 is as follows: Where r is the tag update speed, For extraction neighborhood tags, The label after false alarms have been cleared. This is the updated pseudo-label for the (n+1)th round.
2. The single-point supervised infrared target detection method based on contrast according to claim 1, characterized in that, The specific content of generating the initial extended pseudo-tags in step S1 includes: S11. Record the grayscale value at the single-point label as... And define a target box at that location; S12. Gradually expand the target box adaptively until the gray value of a pixel on the boundary of the target box is less than the preset gray value threshold or the boundary size of the target box reaches the preset maximum size, and obtain the estimated bounding box. S13. The initial pseudo-labels are extracted by thresholding the bounding box, which are the generated initial extended pseudo-labels.
3. The single-point supervised infrared weak target detection method based on contrast according to claim 2, characterized in that, The preset grayscale threshold in step S12 is as follows: in, This represents the grayscale value at the single label point.
4. The single-point supervised infrared target detection method based on contrast according to claim 2, characterized in that, The segmentation threshold in step S13 is as follows: in, This represents the grayscale value at the single label point. The standard deviation of the grayscale values of the pixels inside the bounding box. For parameters.
5. A contrast-based single-point supervised infrared weak target detection system, characterized in that, A contrast-based single-point supervised infrared weak target detection method according to any one of claims 1-4 includes: an infrared image acquisition module, an infrared weak target detection network model, and a training module. Infrared image acquisition module, used to acquire infrared images; The training module includes an initial expanded pseudo-label generation unit, a pseudo-label update unit, and a model training unit; The initial extended pseudo-label generation unit is used to generate an initial extended pseudo-label for each target in each single-point label of the infrared image by expanding the single-point label in the image through contrast. The pseudo-label update unit is used to update the pseudo-labels during training based on the contrast of confidence in the network output. The model training unit is used to iteratively train the convolutional neural network using the initial expanded pseudo-labels, the updated pseudo-labels, and the infrared image to obtain the infrared weak target detection network model. An infrared weak target detection network model is used to detect targets in infrared images and output target labels.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements a contrast-based single-point supervised infrared weak target detection method as described in any one of claims 1-4.
7. A processing terminal, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements a contrast-based single-point supervised infrared weak target detection method as described in any one of claims 1-4.