Semi-supervised open set learning method based on progressive universum and contrast learning

Abstract

The invention provides a unified semi-supervised open set learning method based on progressive universum augmentation and comparative learning in order to overcome the defects existing in existing semi-supervised open set processing. Specifically, the method comprises the following steps of: firstly, extracting data of which a predicted value is higher than a set threshold value from unmarked data, and distributing a pseudo tag to the data; secondly, combining the data with the pseudo labels with the existing marked data to form an expanded marked data set; then, generating corresponding universum data according to the category of the extended mark data, taking the universum data as a negative sample with semantic information, and training the model by utilizing a comparative learning framework; and finally, dividing positive and negative samples according to the sample labels, and further optimizing and training the contrast learning model by taking a product of sample predicted values as a weight. According to the method, the performance bottleneck caused by scarcity of marked data and existence of abnormal classes in the semi-supervised open set problem is effectively solved, and the classification precision and the reliability of abnormal detection are remarkably improved.

Description

Technical Field

[0001] The embodiment of the present invention relates to the field of open set problems in semi-supervised learning, and specifically to a semi-supervised open set learning method based on progressive universum and contrastive learning. Background Art

[0002] In recent years, deep learning technology has made significant breakthroughs in various fields. One of the keys to its success is that it relies on a large amount of high-quality labeled data. These data provide sufficient supervision signals for the model, enabling the model to perform well in complex tasks. However, in real applications, obtaining large-scale labeled data usually requires a lot of time and manpower costs, which is a time-consuming and laborious process. In order to meet this challenge, semi-supervised learning came into being. Semi-supervised learning combines a small amount of labeled data with a large amount of unlabeled data, and uses the potential information in unlabeled data to improve the generalization ability of the model, thereby alleviating the problem of scarce labeled data to a certain extent. This method significantly reduces the dependence on labeled data and provides an effective solution for many practical scenarios where data is scarce. However, traditional semi-supervised learning methods usually rely on the closed set assumption, that is, it is assumed that the label space of labeled data and unlabeled data is completely consistent, and the sample categories in the unlabeled data can be covered by the categories in the labeled data. However, in many real scenarios, this assumption is often difficult to hold. Unlabeled data may contain some new category samples that have not been involved in the labeled data. These samples are called out-of-distribution (OOD) samples, also known as outliers. These outliers may contain unknown categories or interference information, which can easily interfere with the model's learning process and have a serious impact on the model's performance. To address this problem, semi-supervised open set learning was proposed. The goal of semi-supervised open set learning is to effectively identify and isolate outliers in unlabeled data while combining a small amount of labeled data with a large amount of unlabeled data for learning, thereby reducing their negative impact on model training.

[0003] Since outliers can have a significant impact on the performance of the model, an intuitive solution is to detect them and filter them out. Based on this idea, a large number of studies have designed different methods to filter by building detectors or designing OOD scores. For example, OpenMatch[9] proposed a one-vs-all (OVA) classifier trained on labeled data, which compares the output confidence with the set threshold to filter out outliers. In a similar way, ProSub[5] considered the statistical characteristics of the problem and proposed an OOD score based on the angle between the data in the feature space and the in-distribution subspace to distinguish OOD. These methods have improved the performance of the model by eliminating the influence of outliers.

[0004] However, there are two significant problems with this design. First, existing methods usually rely only on labeled data to train the detector. However, in the open set semi-supervised learning scenario, labeled data is extremely limited, and relying only on labeled data can lead to significant deviations in the performance of the detector. More importantly, in the subsequent training process, the model relies heavily on the results of the detector (for example, by using the inliers identified by the detector for self-training), which causes errors to gradually accumulate and propagate throughout the model, further reducing the accuracy and robustness of the model. Second, such methods usually adopt a multi-head structure, that is, an architecture containing a classifier and a detector, and the research focus is mainly on how to optimize each head separately based on the model output. However, they often ignore the impact of the feature space shared by the classifier and the detector on the overall performance. In this design, due to the presence of outliers (i.e., open set samples), the representation of the feature space will inevitably be disturbed, which will affect the synergy between the classifier and the detector and the final task performance.

[0005] In view of this, the present invention is proposed. Summary of the invention

[0006] Purpose of the invention: In view of the above problems, the purpose of the present invention is to propose improvements to the shortcomings of the prior art and design a new unified open set semi-supervised learning method, called PSUMatch. Unlike traditional methods, PSUMatch not only significantly improves the classification performance by introducing an additional data set, but also enhances the detection ability of outliers. Specifically, the present invention designs a progressive semantic universal set PSU (Progressive Semantic Universe), which is dynamically constructed during the training process and contains pseudo-labeled inlier samples in unlabeled data and labeled data, but these data do not belong to any labeled category. By combining pseudo-labeled inliers and labeled data, the construction of the PSU data set is progressive and is continuously optimized and improved as the training progresses. In terms of method implementation, PSU is regarded as a negative sample, and a contrastive learning strategy is introduced to further optimize the feature representation. Contrastive learning can improve the expression ability of the feature space by comparing the features of positive and negative samples, thereby providing better shared features for multiple heads such as classifiers and detectors at the same time. In addition, in order to adapt to the complexity of open set scenarios, a soft contrast loss is also designed. This loss function effectively alleviates the interference of outliers on model training while gradually optimizing pseudo labels. Through this design, PSUMatch can not only improve the performance of the model in classification tasks, but also enhance the ability to detect outliers in the open set, achieving unified optimization and performance improvement.

[0007] Technical solution: In order to achieve the above objectives, the following technical solutions are provided:

[0008] The PSUMatch method includes at least:

[0009] Step S1: Filter out high-confidence samples from the unlabeled data and assign pseudo labels, and combine them with the labeled data to form expanded new labeled data;

[0010] Step S2: Generate the corresponding progressive universe according to the category of the expanded labeled data, and use the universe as a negative sample to learn features using contrast loss;

[0011] Step S3: Divide the positive and negative samples according to the similarities and differences of the labels, use the product of the predicted values ​​of the two samples as the weight, and optimize and adjust the contrastive learning;

[0012] Step S4: The learned features are used to train the classification head and the detection head at the same time to classify known classes and detect abnormal classes in the testing phase.

[0013] Beneficial effects: Compared with the prior art, the technical solution of the present invention has at least the following beneficial effects:

[0014] 1) This paper proposes a unified feature learning method for open set semi-supervised learning, which can improve the accuracy and classification performance of the detector at the same time;

[0015] 2) The present invention introduces the Progressive Semantic Universal Dataset PSU to make up for the lack of labeled data and negative samples, and to assist in the optimization of the training process;

[0016] 3) The present invention designs a softened contrast loss and a method combining labeled data to alleviate the impact of outliers and improve the robustness of the model;

[0017] 4) Through these innovative designs, this paper conducts comprehensive experimental verification on multiple data sets. On most data sets, the present invention improves their respective performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a framework diagram of the overall training of a semi-supervised open set learning method based on progressive universum and contrastive learning proposed in the present invention.

[0019] Figure 2 for Figure 1 A flow chart of a semi-supervised open set learning method based on progressive universum and contrastive learning is shown in FIG.

[0020] Figure 3 The following is a schematic diagram showing construction of a progressive semantic universal dataset (PSU) and softened contrastive learning according to the present embodiment. DETAILED DESCRIPTION

[0021] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The present invention provides a semi-supervised open set learning method based on progressive universum and contrastive learning, such as Figure 2 As shown, the method at least includes:

[0023] Step S1: Filter out high-confidence samples from the unlabeled data and assign pseudo labels, and combine them with the labeled data to form expanded new labeled data;

[0024] Step S2: Generate the corresponding progressive universe according to the category of the expanded labeled data, and use the universe as a negative sample to learn features using contrast loss;

[0025] Step S3: Divide the positive and negative samples according to the similarities and differences of the labels, use the product of the predicted values ​​of the two samples as the weight, and optimize and adjust the contrastive learning;

[0026] Step S4: The learned features are used to train the classification head and the detection head at the same time to classify known classes and detect abnormal classes in the testing phase.

[0027] In step S1, Figure 1 As shown, it specifically includes using the labeled data D l The initialized OOD detector d trained on the unlabeled data D u Filter out high confidence samples above the dynamic threshold The category corresponding to the maximum value in the predicted value on classifier C is used as the pseudo label And combine these samples and labeled samples into the expanded labeled data D a Through continuous training iterations, D a It will continue to change and eventually stabilize.

[0028] In the step S2, the corresponding progressive universe is generated according to the label category, and the universe is used as a negative sample to learn features using contrast loss, which specifically includes:

[0029] like Figure 3 As shown in (a), for a given anchor sample x i ∈D a and randomly select samples x from different categories i ∈D a In the first step, the present invention uses the mixup technique to mix two different samples at the image level to generate the corresponding progressive universe, which is expressed as:

[0030]

[0031] In the formula It means x i The corresponding universe, λ represents the mixing coefficient, and the present invention sets λ to 0.5, and N represents D a The total number of samples contained, due to D a will change, so the generated universe changes gradually with the training time;

[0032] The second step is to i and Both of them are strongly augmented in two different ways, and then features are extracted from them, expressed as:

[0033]

[0034] Where Aug s Represents two different augmentation methods, F represents the encoder, and P represents the projector used for mapping the feature space;

[0035] The third step is to soften the contrast loss of the model's predicted value to improve robustness. For the labeled data, all are set to 1, and for the pseudo-labels, they are set to the corresponding predicted values, expressed as:

[0036]

[0037] Where p i =C(F(x i )) represents the predicted value of classifier C;

[0038] The fourth step will As the feature of the corresponding negative sample, the sample is moved closer to the center of the prototype, and the prototype-based contrast loss function is used, which is expressed as:

[0039]

[0040] In the formula It means that all the corresponding The average value of , τ represents the temperature coefficient. In the present invention, τ=0.1 is set. . represents the dot product of two vectors used to calculate the similarity between the two vectors, and |·| represents the total number of samples.

[0041] Finally, the features are learned by continuously optimizing and iterating the prototype-based contrast loss.

[0042] In step S3, Figure 3As shown in (b), it specifically includes dividing positive and negative samples according to the similarities and differences of labels, taking the product of the predicted values ​​of the two samples as the weight, optimizing and adjusting the contrastive learning, which is specifically expressed as constructing x according to whether the labels are the same. i ∈D a The positive and negative pairs are then input into the encoder F and the projector P to obtain their respective feature representations. The corresponding contrast loss function is defined as:

[0043]

[0044] Where τ = 0.1, The product of the predicted values ​​of the two samples is used as the weight to optimize L scon , specific The construction is expressed as follows:

[0045]

[0046] 5. In step S4, the learned features are used to train the classification head and the detection head at the same time, and the known classes are classified and the abnormal classes are detected in the test phase, which specifically includes: using the overall loss function L = λ a (L psu +L scon ) to optimize feature learning, where λ a As the training time decreases exponentially, we can focus on the classification and detection tasks, and use the learned features for the classifier and detector respectively. In the test phase, we first use the detector to determine whether it is an abnormal category, and then use the classifier to classify it.

[0047] like Figure 1 As shown, for unlabeled data, an OOD detector is first used to identify inliers, and pseudo labels are assigned to these high-confidence data according to the classifier, and combined with the labeled data to form expanded labeled data; then, the expanded labeled data is passed through the PSU module to generate the corresponding PSU dataset; then, the expanded labeled dataset and the PSU dataset are passed through the encoder and projection head respectively to obtain the corresponding features and use the psu contrast loss to learn better features; finally, the features corresponding to the expanded labeled data are passed through the soft contrast module, and the scon contrast loss is used to alleviate the impact of outliers on the model and improve the robustness of the model. Figure 3 (a) shows the construction process of PSU, by selecting two samples x of different categories i and x j , and then use the mixup technique to mix the two samples psu, and use contrastive learning as negative samples to pull them apart and move closer to the center of the prototype generated by psu to obtain better feature expression. Figure 3(b) shows the training method of softened contrast loss, which uses labeled data to guide unlabeled data to learn better feature expressions, and also uses the product of predicted values ​​to soften the contrast loss to obtain robust expression.

[0048] As described in the review, the semi-supervised open set learning method based on progressive universum and contrastive learning proposed in the present invention is actively helpful for studying the open set problem of semi-supervised learning in reality. How to design an efficient learning algorithm to solve this problem is very challenging. In order to meet this challenge, the present invention constructs a data set called PSU, which does not belong to any labeled category. By combining the pseudo-annotated inliers in the unlabeled data with the labeled data, the data set is gradually constructed as the training proceeds. Then, PSU is used as a negative sample for contrastive learning to obtain a better feature representation, so that it can support multiple task heads at the same time. During the learning process, the present invention introduces a soft contrast loss for gradually optimizing pseudo labels while reducing the influence of outliers. The present invention is verified by experiments on the data sets CIFAR-10 and CIFAR-100, with the number of labeled data being 50 and 100 respectively. The results are shown in Table 1.

[0049] The results show that the present invention achieves the best performance on all data sets, indicating that the present invention can detect outliers better than other methods, thereby proving the effectiveness of the present invention.

[0050] The above specific embodiments are only for explaining the principle and technical method of the method proposed in the present invention, and are not intended to limit the implementation of the technical solution of the present invention. For those skilled in the art, the technical solution of the method proposed in the present invention can be reasonably adjusted and replaced according to needs, and these adjustments and replacements are protected by the claims of the present invention.

[0051] Table 1 Comparison of the accuracy of different methods in outlier detection

[0052]

Claims

1. A semi-supervised open set learning method based on progressive universum and contrastive learning, characterized in that: The method at least comprises: Step S1: Filter out high-confidence samples from the unlabeled data and assign pseudo labels, and combine them with the labeled data to form expanded new labeled data; Step S2: Generate the corresponding progressive universe according to the category of the expanded labeled data, and use the universe as a negative sample to learn features using contrast loss; Step S3: Divide the positive and negative samples according to the similarities and differences of the labels, take the product of the predicted values ​​of the two samples as the weight, and optimize and adjust the contrastive learning; Step S4: The learned features are used to train the classification head and the detection head at the same time to classify known classes and detect abnormal classes in the testing phase.

2. The semi-supervised open set learning process based on progressive universum and contrastive learning according to claim 1, characterized in that Step S1 specifically includes using the marked data D l The initialization detector d trained on the unlabeled data D u Filter out high confidence samples above the dynamic threshold The category corresponding to the maximum value in the predicted value on classifier C is used as the pseudo label And combine these samples and labeled samples into the expanded labeled data D a Through continuous training iterations, D a It will continue to change and eventually stabilize.

3. The process of expanding the original marked data according to claim 2, characterized in that: The step S2 generates the corresponding progressive universe according to the label category, and uses the universe as a negative sample to learn features using contrast loss, specifically including: For a given anchor sample x i ∈D a and randomly select samples x from different categories j ∈D a In the first step, the present invention uses the mixup technique to mix two different samples at the image level to generate the corresponding progressive universe, which is expressed as: In the formula It means x i The corresponding universe, λ represents the mixing coefficient, and the present invention sets λ to 0.5, and N represents D a The total number of samples contained, due to D a will change, so the generated universe changes gradually with the training time; The second step is to i and Both of them are strongly augmented in two different ways, and then features are extracted from them, expressed as: Where Aug s Represents two different augmentation methods, F represents the encoder, and P represents the projector used for mapping the feature space; The third step is to soften the contrast loss of the model's predicted value to improve robustness. For the labeled data, all are set to 1, and for the pseudo-labels, they are set to the corresponding predicted values, expressed as: Where p i =C(F(x i )) represents the predicted value of classifier C; The fourth step will As the feature of the corresponding negative sample, the sample is then moved closer to the center of the prototype, using the prototype-based contrast loss function, expressed as: In the formula It means that all the corresponding The average value of , τ represents the temperature coefficient, in the present invention, τ=0.1 is set, · represents the dot product of two vectors used to calculate the similarity between the two, and |·| represents the total number of samples. Finally, the features are learned by continuously optimizing and iterating the prototype-based contrast loss.

4. The method for constructing a progressive universe and establishing negative samples for contrastive learning according to claim 3, characterized in that: Step S3 divides positive and negative samples according to the similarities and differences of labels, and uses the product of the predicted values ​​of the two samples as the weight to optimize and adjust the contrastive learning. Specifically, x is constructed according to whether the labels are the same. i ∈D a The positive and negative pairs are then input into the encoder and projector P to obtain their respective feature representations. The corresponding contrast loss function is defined as: Where τ = 0.1, The product of the predicted values ​​of the two samples is used as the weight to optimize L n , specific The construction is expressed as follows:

5. According to claim 4, the method of using the product of the difference of the label and the predicted value to adjust and optimize the contrastive learning to learn the feature representation is characterized in that: Step S4 uses the learned features to train the classification head and the detection head at the same time, and classifies the known classes and detects the abnormal classes in the test phase, specifically including: using the overall loss function L = λ a (L psu +L scon ) to optimize feature learning, where λ a As the training time decreases exponentially, we can focus on the classification and detection tasks, and use the learned features for the classifier and detector respectively. In the test phase, we first use the detector to determine whether it is an abnormal category, and then use the classifier to classify it.

Images