A training method for a weakly semi-supervised infrared small target detection model
By employing a weak semi-supervised approach and utilizing iterative updates of pixel-level mask labels and pseudo-labels, the training instability and crash issues in infrared weak target detection are resolved, thereby improving the network's generalization ability and detection performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2024-05-09
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies suffer from instability in the early training stages and network crashes in infrared weak target detection, especially when using single-point labels versus fully supervised labels, where class imbalance is a serious problem.
We adopt a weak semi-supervised method, which performs one iteration of training on the CNN network using pixel-level mask labels. The weakly supervised labels are then updated multiple times based on the network output to gradually refine the pseudo-labels until the network converges. Finally, a second iteration of training is performed using the complete pixel-level mask labels and the evolved pseudo-labels.
It effectively avoids instability and network crashes in the early stages of training, improves the network's generalization ability, and achieves detection performance similar to that of fully supervised training with very little labeled data.
Smart Images
Figure CN118247606B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of pattern recognition and machine learning technology, particularly to the fields of weakly supervised training, semi-supervised training, and infrared small target detection technology, and more specifically to a training method for a weakly and semi-supervised infrared small target detection model. Background Technology
[0002] Infrared detection technology operates by sensing the radiation characteristics of a target, using infrared sensors for reconnaissance. This method overcomes the limitations of lighting conditions, and its passive detection significantly reduces the likelihood of detection. Furthermore, infrared imaging can penetrate clouds and fog, playing a crucial role in maritime search and rescue and civilian applications. Infrared small target detection is a key task in infrared detection. However, because infrared targets typically lack color and texture features and have low signal-to-noise ratios, they are easily obscured in complex backgrounds. Traditional algorithms often perform poorly in detecting weak infrared targets and are prone to false alarms.
[0003] With the advancement of deep learning technology, infrared weak target detection methods based on convolutional neural networks (CNNs) have emerged. Most of these methods are based on fully supervised learning and rely on large amounts of labeled datasets. These algorithms typically treat infrared weak target detection as an image segmentation task, requiring precise pixel-level mask annotations. However, given the sensitivity and complexity of infrared data, obtaining a large training dataset is a resource-intensive process.
[0004] In recent years, weakly supervised learning has begun to be applied to infrared weak target detection. This method does not rely on complete labels, reducing data limitations in infrared weak target detection and introducing a more efficient detection mechanism. Single-point supervision is a key type of weakly supervised learning. In 2023, the first infrared weak target detection algorithm using single-point supervision was proposed. It leverages the training characteristics of convolutional neural networks, initially identifying a set of pixels around the target with low confidence, and gradually refining to the single-point labels provided by the supervisor as training progresses. This method dynamically updates the labels through the network output at specific stages of training.
[0005] However, the problem of detecting weak targets in infrared images (i.e., weak targets in infrared images) is an imbalanced class problem. Supervised training using single-point labels exacerbates this imbalance. This leads to instability in CNN training in the early stages. Updating labels under these circumstances sometimes yields poor results. Furthermore, conventional weakly semi-supervised frameworks, when using both single-point labels and fully labeled data simultaneously, can cause network training to collapse due to the discrepancy between the two.
[0006] Therefore, how to effectively avoid instability in the early stages of training, and how to avoid network collapse when using single-point labels and fully supervised labels simultaneously, are problems that urgently need to be solved by those skilled in the art. Summary of the Invention
[0007] In view of the above problems, the present invention provides a training method for a weak semi-supervised infrared weak target detection model, so as to at least solve some of the technical problems mentioned in the background art.
[0008] To achieve the above objectives, the present invention adopts the following technical solution:
[0009] This invention provides a training method for a weakly semi-supervised infrared weak target detection model, which is applied to a deep learning-based infrared weak target detection algorithm, and includes the following steps:
[0010] Obtain pixel-level mask labels and weakly supervised labels for the image set;
[0011] The CNN network is trained in one iteration using the pixel-level mask labels to obtain the initial detection model.
[0012] During the training iteration, after each preset number of iterations, the weakly supervised label is updated once based on the output of the CNN network.
[0013] The initial detection model is trained iteratively using the updated weakly supervised labels and the pixel-level mask labels until the network converges, thus obtaining the final detection model.
[0014] Furthermore, during the second iteration training process, after each preset number of iterations, the weakly supervised labels updated once are updated a second time using the output of the initial detection model; the initial detection model is then trained again using the second-updated weakly supervised labels and the pixel-level mask labels until the network converges, thus obtaining the final detection model.
[0015] Furthermore, during one iteration of training the CNN network using the pixel-level mask labels, the image corresponding to the pixel-level mask labels is used as the input to the CNN network, and the pixel-level mask labels are used as supervision information.
[0016] Furthermore, when the number of iterations in a single training iteration reaches a preset threshold, the single training iteration ends.
[0017] Furthermore, the update of the weakly supervised label based on the output of the CNN network specifically includes:
[0018] Extract a neighborhood of a preset size from the output of the CNN network, denoted as the first neighborhood; subtract the smallest gray value in the first neighborhood to obtain the second neighborhood;
[0019] In the current round, the weak supervision target area corresponding to the unupdated weak supervision label is expanded by a preset factor to obtain the expanded target area;
[0020] Based on the output of the CNN network and the first neighborhood, the pixel grayscale values of the weakly supervised target region and the enlarged target region are obtained respectively; represented as:
[0021]
[0022] in, This represents the pixel grayscale value of the weakly supervised target region; represents the pixel grayscale value of the expanded target region; p represents the number of pixels selected within the weakly supervised target region; q represents the number of pixels selected within the expanded target region; This represents the i-th largest value in the output of the CNN network; This represents the i-th largest value in the first neighborhood;
[0023] The threshold is calculated based on the pixel grayscale values of the weakly supervised target region and the enlarged target region; expressed as:
[0024]
[0025] in, Indicates the threshold to be calculated; The second neighborhood is represented by k; the minimum threshold is represented by k. Indicates the balance parameters;
[0026] Based on the calculated threshold, the output of the CNN network is preprocessed to obtain the third neighborhood.
[0027] Based on the third neighborhood and the second neighborhood, an update of the weakly supervised label is implemented, represented as:
[0028]
[0029] Where n represents the nth update of the weakly supervised label; This represents the weakly supervised label after the first update in the nth round; Indicates the first neighborhood; 'r' represents the third neighborhood; 'r' represents the speed of weak supervision label updates.
[0030] Furthermore, the preprocessing specifically includes: if the pixel value to be updated in the target region is not located in the eight-neighborhood of any positive pixel value in the previous pseudo-label, then the pixel is excluded from the pixels to be updated.
[0031] Furthermore, during the second iteration training of the initial detection model using the updated weak supervision label and the pixel-level mask label, the image containing the pixel-level mask label uses the pixel-level mask label as supervision information during training; the image containing the weak supervision label uses the weak supervision label as supervision information during training.
[0032] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a training method for a weak semi-supervised infrared weak target detection model, which has the following beneficial effects:
[0033] This invention initially uses a preset number of fully labeled pixel-level mask labels to perform an iterative training of the CNN network, thus avoiding instability issues that occur in the early stages of training.
[0034] This invention uses the output of a CNN network to iteratively update the weakly supervised labels a certain number of times, which can make the pseudo labels closer to the real labels before training, avoiding the problem of network training crashing due to the large difference between single-point labels and pixel-level mask labels.
[0035] This invention uses complete pixel-level mask labels and pseudo-labels that have undergone a certain degree of evolution to train the network, which can enhance the network's generalization ability and significantly improve the network's performance on the entire dataset, avoiding the problem of weak generalization ability caused by insufficient labeled data in semi-supervised learning.
[0036] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention.
[0037] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0039] Figure 1 This is a schematic diagram of the training method for the weak semi-supervised infrared weak target detection model provided in Embodiment 1 of the present invention.
[0040] Figure 2 This is a schematic diagram of the training method framework for the weak semi-supervised infrared weak target detection model provided in Embodiment 1 of the present invention.
[0041] Figure 3 This is a schematic diagram of the weakly supervised label update framework provided in Embodiment 1 of the present invention.
[0042] Figure 4 This is a schematic diagram of the training method for the weak semi-supervised infrared weak target detection model provided in Embodiment 2 of the present invention.
[0043] Figure 5 This is a schematic diagram illustrating the comparison results provided in an embodiment of the present invention. Detailed Implementation
[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0045] The invention will now be illustrated through two embodiments.
[0046] Example 1:
[0047] See Figure 1 and Figure 2 As shown in the figure, this invention discloses a training method for a weakly semi-supervised infrared weak target detection model, including the following steps: obtaining pixel-level mask labels and weak supervision labels for infrared weak targets; performing one iteration training on a CNN network using the pixel-level mask labels to obtain an initial detection model; during one iteration training process, after each preset number of iterations, updating the weak supervision labels based on the output of the CNN network; and performing a second iteration training on the initial detection model using the updated weak supervision labels and pixel-level mask labels until the network converges to obtain the final detection model.
[0048] This method can effectively avoid instability issues in the early stages of training, as well as network crashes when using single-point labels and fully supervised labels simultaneously.
[0049] The training method for a weak semi-supervised infrared small target detection model provided by the present invention will be described in detail in the following three parts.
[0050] 1. One iteration of training:
[0051] In the training method provided in this embodiment of the invention, the training data of the image set involved is divided into two parts. One part consists of a small number (approximately 5%-30%) of fully labeled pixel-level mask labels, and the remaining labels are weakly supervised labels. The initial form of the weakly supervised labels is a single-point label, which will form a pseudo-label after being updated. The label is the target region marked in the image.
[0052] First, the CNN network is trained iteratively using pixel-level mask labels. That is, the image corresponding to the pixel-level mask label is used as the input of the CNN network, and the pixel-level mask label is used as the supervision information. In other words, the image corresponding to the pixel-level mask label is used as the input of the CNN network, and the pixel-level mask label is used as the output of the CNN network. This is how the CNN network is trained.
[0053] When the number of iterations in a single training iteration reaches a preset threshold, the training iteration ends, and an initial detection model is obtained. In this embodiment of the invention, the preset threshold is set to 50 rounds. If the preset threshold is selected to be higher than 50, the network will approach the given label, resulting in a deterioration in performance. If it is selected to be lower than 50, the prior knowledge learned by the network is insufficient to distinguish between the background and the target. In this training iteration, the CNN network will learn initial prior information about the target and gain the ability to initially separate the target from the background. Since fully labeled pixel-level mask labels are used, the training instability caused by class imbalance in the early stages of training can be effectively avoided.
[0054] 2. Update the weak supervision label:
[0055] During an iterative training of a CNN network, after each preset number of iterations, the weakly supervised label is updated based on the output of the CNN network. In this embodiment of the invention, the preset number of iterations is set to 5. That is, when the number of iterations in a training iteration is 5, the weakly supervised label (at this time, it is a single-point label) is updated to generate an updated weakly supervised label (i.e., a pseudo-label). When the number of iterations in a training iteration is 10, the weakly supervised label (at this time, it is a pseudo-label) is updated again. This process is repeated until a training iteration reaches 50 iterations.
[0056] It is important to note that the weakly supervised labels updated in this step are not used in the training of the CNN network; they are only updated iteratively using the output of the CNN network. Since the output of the CNN network in one iteration of training is a pixel-level mask label, the output of the CNN network is actually the target region marked in the input image. Based on this, the following steps are used as an example to illustrate the update of the weakly supervised labels in the nth iteration:
[0057] (1) Extract the preset size from the output of the CNN network (i.e., from the pixel-level mask target area corresponding to the pixel-level mask label). The neighborhood of ) is denoted as the first neighborhood. Subtract the first neighborhood The second neighborhood is obtained by finding the smallest gray value in the middle. ;
[0058] (2) Expand the weak supervision target region corresponding to the unupdated weak supervision label in the current round by a preset multiple to obtain the expanded target region; see the embodiment of the present invention. Figure 3 As shown, in the nth round of updating the weak supervision labels, the weak supervision target regions identified among the unupdated weak supervision labels are as follows: Figure 3 As shown in the solid line box, the solid line box is then expanded to twice the maximum length and width to obtain the expanded target area as shown in the dashed line box.
[0059] (3) Based on the output of the CNN network and the first neighborhood, obtain the pixel grayscale values of the weakly supervised target region and the expanded target region respectively; expressed as:
[0060]
[0061] in, This represents the pixel grayscale value of the weakly supervised target region; represents the pixel grayscale value of the expanded target region; p represents the number of pixels selected within the weakly supervised target region; q represents the number of pixels selected within the expanded target region; This represents the i-th largest value in the output of the CNN network; This represents the i-th largest value in the first neighborhood, i.e. Figure 3 The i-th largest value between the dashed and solid bounding boxes; where the i-th largest value refers to the value in the i-th position after sorting the CNN outputs of the pixel positions in the selected region from largest to smallest.
[0062] (4) Obtain the calculation threshold based on the pixel grayscale values of the weakly supervised target region and the expanded target region; expressed as:
[0063]
[0064] in, Indicates the threshold to be calculated; The second neighborhood is represented by k; the minimum threshold is represented by k. In this embodiment of the invention, k and represent the balance parameters. Set to 0.9 and 0.3 respectively; pixels below the threshold in the output are set to 0;
[0065] (5) Based on the calculated threshold, the output of the CNN network is preprocessed to obtain the third neighborhood; the purpose of the preprocessing is to eliminate false alarms generated during the threshold calculation process; the preprocessing specifically includes: removing pixels from the target region that are not octagonally connected to the weak supervision label before this update, that is, if the pixel value to be updated in the target region is not located in the octagonal neighborhood of any positive pixel value in the previous pseudo label, then the pixel is excluded from the pixels to be updated.
[0066] (6) Based on the third neighborhood and the second domain, a single update of the weakly supervised label is achieved, represented as:
[0067]
[0068] Where n represents the nth update of the weakly supervised label; This represents the weakly supervised label after the first update in the nth round; Indicates the first neighborhood; Indicates the third neighborhood; This indicates the speed at which the weakly supervised labels are updated.
[0069] 3. Second iteration training:
[0070] After updating the weakly supervised labels once, relatively reliable pseudo-labels are obtained. Then, the pseudo-labels (i.e., the weakly supervised labels after the first update) and pixel-level mask labels are used to train the initial detection model for a second iteration. At this time, the training set includes all images. Images containing pixel-level mask labels are trained using pixel-level mask labels as supervision information, and images containing weakly supervised labels are trained using weakly supervised labels as supervision information.
[0071] In summary, this invention provides a training method for a weakly semi-supervised infrared weak target detection model, which can be plugged and played as a module in any infrared weak target detection deep learning network. First, in the initial stage, a preset number of fully labeled pixel-level mask labels are used to train the CNN network, avoiding instability issues in the early stages of training. In the middle stage of training, the network output is used to iteratively update the single-point supervision labels a certain number of times, making the pseudo-labels closer to the real labels before training, avoiding the network training collapse problem caused by excessive differences between single-point labels and pixel-level mask labels. In the later stage of training, the network is trained using complete pixel-level mask labels and pseudo-labels that have undergone a certain degree of evolution, enhancing the network's generalization ability and significantly improving its performance on the entire dataset, avoiding the weak generalization ability problem caused by insufficient labeled data in semi-supervised training.
[0072] Example 2:
[0073] See Figure 4As shown, based on the first embodiment above, during the second iteration training process, after each preset number of iterations, the weakly supervised labels updated once are updated a second time using the output of the initial detection model. This second update effectively prevents excessive label differences caused by insufficient updates in the first iteration, which could hinder the next training step. The initial detection model continues to be trained using the second-updated weakly supervised labels and pixel-level mask labels until the network converges, obtaining the final detection model. In this embodiment, the preset number of iterations is set to 5.
[0074] Next, the technical effects of the training method for a weak semi-supervised infrared small target detection model provided by the present invention will be explained through a specific embodiment.
[0075] In this invention, the NUAA-SIRST dataset was used as the test dataset, with a training set to test set ratio of 7:3. DNA-Net was used as the benchmark model, and its detection rate (Pd) and mean intersection-union ratio (mIoU) were calculated. The model was trained with 10%, 20%, and 30% of fully labeled data plus the remaining single-point supervised data, and compared with the fully supervised model. The results are shown in Table 1.
[0076] Table 1 Quantitative Analysis of Detection Results
[0077] <![CDATA[Pd(×10 2 )]]> <![CDATA[mIoU(×10 2 )]]> Full supervision 94.10 67.84 10% fully labeled data 24.82 7.97 20% fully labeled data 56.61 9.61 30% fully labeled data 66.43 41.44 10% fully annotated data + this invention 87.86 59.64 20% fully annotated data + this invention 90.35 63.12 30% fully annotated data + this invention 93.04 67.89
[0078] Experimental results demonstrate that in conventional weakly semi-supervised training frameworks, where both fully labeled and single-point labels are used to train neural networks, the training performance significantly declines due to the large difference between the two types of labels. The method in this invention avoids this problem. It achieves performance similar to fully supervised training using 100% fully labeled data, even with only 30% fully labeled data. Furthermore, the method used in this invention still achieves over 85% of the performance of fully supervised training when using only a very small amount of labeled data, i.e., 10% fully labeled data.
[0079] Figure 5 This is a visualization of the detection results in this embodiment of the invention compared with other methods. Under the condition of using only a common weak semi-supervised framework, the confidence of the target is too low and the shape is unstable. The method in this invention can achieve better results than other weakly supervised detection methods using only 10% of fully labeled data.
[0080] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0081] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A training method for a weakly semi-supervised infrared weak target detection model, applied to a deep learning-based infrared weak target detection algorithm, characterized in that, Includes the following steps: Obtain pixel-level mask labels and weakly supervised labels for the image set; The CNN network is trained in one iteration using the pixel-level mask labels. When the number of iterations in the one iteration reaches a preset threshold, the one iteration training ends and the initial detection model is obtained. During the training iteration, after each preset number of iterations, the weakly supervised label is updated once based on the output of the CNN network. The initial detection model is trained in a second iteration using the updated weak supervision labels and the pixel-level mask labels. During the second iteration training, after each preset number of iterations, the output of the initial detection model is used to update the updated weak supervision labels a second time. The initial detection model is trained again using the weakly supervised labels updated twice and the pixel-level mask labels until the network converges to obtain the final detection model. During one iteration of training the CNN network using the pixel-level mask labels, the image corresponding to the pixel-level mask labels is used as the input to the CNN network, and the pixel-level mask labels are used as supervision information. Extract a neighborhood of a preset size from the output of the CNN network, and denote it as the first neighborhood; Subtract the smallest gray value in the first neighborhood to obtain the second neighborhood; The output of the CNN network is preprocessed based on a calculated threshold to obtain a third neighborhood. The preprocessing specifically includes: if the pixel value to be updated in the target region is not located in the eight-neighbor area of any positive pixel value in the previous pseudo-label, then the pixel is excluded from the pixels to be updated; Based on the third neighborhood and the second neighborhood, an update of the weakly supervised label is performed, represented as follows: ； Where n represents the nth update of the weakly supervised label; This represents the weakly supervised label after the first update in the nth round; Indicates the first neighborhood; Indicates the third neighborhood; This indicates the speed at which the weakly supervised labels are updated.
2. The training method for a weak semi-supervised infrared weak target detection model according to claim 1, characterized in that, The update of the weakly supervised label based on the output of the CNN network specifically includes: In the current round, the weak supervision target area corresponding to the unupdated weak supervision label is expanded by a preset factor to obtain the expanded target area; Based on the output of the CNN network and the first neighborhood, the pixel grayscale values of the weakly supervised target region and the enlarged target region are obtained respectively; represented as: ； ； in, This represents the pixel grayscale value of the weakly supervised target region; represents the pixel grayscale value of the expanded target region; p represents the number of pixels selected within the weakly supervised target region; q represents the number of pixels selected within the expanded target region; This represents the i-th largest value in the output of the CNN network; This represents the i-th largest value in the first neighborhood; The threshold is calculated based on the pixel grayscale values of the weakly supervised target region and the enlarged target region; expressed as: ； in, Indicates the threshold to be calculated; Indicates the second neighborhood; Indicates the minimum threshold; Represents the equilibrium parameters 3. The training method for a weak semi-supervised infrared weak target detection model according to claim 1, characterized in that, During the second iteration training of the initial detection model using the updated weak supervision label and the pixel-level mask label, the image containing the pixel-level mask label is trained using the pixel-level mask label as supervision information. Images containing the weak supervision labels are used as supervision information during training.