A small sample semi-supervised learning method and device based on pseudo-label noise filtering

By alternately optimizing the main neural network and pseudo-label filtering neural network, combining semi-supervised learning and noise tag learning loss function, the problem of pseudo-label noise in small sample semi-supervised learning is solved, high-precision image recognition is achieved, and the model is enhanced General use.

CN114444687BActive Publication Date: 2024-11-12TONGJI UNIV
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202210015285.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-07
Publication Date
2024-11-12
Estimated Expiration
2042-01-07

AI Technical Summary

Technical Problem

In small sample semi-supervised learning, a high proportion of pseudo-label noise will affect model performance, especially in the case of insufficient sampling bias and quantity of labeled data, it is difficult for the prior art to effectively filter noise, resulting in inaccurate prediction results.

Method used

By alternately optimizing the main neural network model and pseudo-label filtering neural network model, using semi-supervised learning loss function and noise tag learning loss function, combining data augmentation and data set expansion, gradually filtering pseudo-label noise, avoiding local optimal solutions, and through Logarithmic growth function optimization dataset.

Benefits of technology

It effectively reduces the impact of pseudo-label noise, improves the generalization and accuracy of the image recognition model, and can accurately classify image recognition even when the number of data sets is small, avoiding the model from falling into local optimal solutions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114444687B_ABST
    Figure CN114444687B_ABST
Patent Text Reader

Abstract

The present invention relates to a small sample semi-supervised learning method and device based on pseudo-label noise filtering, wherein during the execution of the method, a main model and a pseudo-label filtering model are alternately optimized, the main model is trained using a semi-supervised learning method, and the pseudo-label filtering model is trained using a noise label learning method. The optimization of the main model and the pseudo-label filtering model is alternately performed for several rounds, and after each round, part of the unlabeled data together with its current pseudo-label will be classified into a labeled data set until the optimization stagnates, and the optimized model is output. Compared with the prior art, the model obtained by the present invention has the advantages of better accuracy in image recognition and the like.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data mining and machine learning, and in particular to a small sample semi-supervised learning method and device based on pseudo-label noise filtering. Background Art

[0002] Data is one of the basic elements supporting artificial intelligence technology, especially in the field of deep learning, the quantity and quality of data are directly related to the final generalization performance of the model. At present, most mainstream research and applications rely on the support of big data, and the number of labeled samples is often in the millions. However, with the continuous penetration of deep learning technology in various vertical fields, the specificity and professionalism of target tasks are also increasing. In actual scenarios, considering the cost of expert labeling, the labeled data that can be collected may be extremely limited. Semi-supervised learning is aimed at scenarios where only some of the samples are labeled. For such small sample problems caused by limited labeling costs, semi-supervised learning is an important solution.

[0003] Semi-supervised learning algorithms are mainly used in the field of computer image recognition. They use labeled image data to train neural networks and then realize image recognition and classification. Most mainstream semi-supervised learning algorithms directly use the prediction results of the main model on unlabeled samples as the supervision information of the sample. Considering the sampling bias effect of the initial labeled data, when the supervised data sampling is biased or too small, the empirical distribution of the prediction results will not be sufficient to approximate the true potential sample distribution. At this time, the prediction recognition results given by the main model will contain a lot of pseudo-label noise. If the proportion of pseudo-label noise is too high, it will affect the final performance of the main model. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a small sample semi-supervised learning method and device based on pseudo-label noise filtering.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A small sample semi-supervised learning method based on pseudo-label noise filtering includes the following steps:

[0007] S1. Get the initial labeled dataset L s and the initial unlabeled dataset U s , L s ={(x i ,y i )},U s ={x j};

[0008] S2. Initialize the main neural network model and pseudo-label filtering neural network model

[0009] S3, the initial labeled dataset L s and the initial unlabeled dataset U s Input the main neural network model Combined with semi-supervised learning loss function Optimize the main neural network model;

[0010] S4, the initial unlabeled dataset U s Input the optimized main neural network model The category with the highest confidence in the prediction result is defined as the pseudo label y′ corresponding to the unlabeled data sample j , get the pseudo-label dataset U′ s , U′ s ={(x j ,y′ j )};

[0011] S5. The initial labeled dataset L s and pseudo-label dataset U′ s Input pseudo-label filtering neural network model Learning loss function with noisy labels Optimize the pseudo-label filtering neural network model;

[0012] S6. The initial labeled dataset L s and pseudo-label dataset U′ s Input optimized pseudo-label filtering neural network model Calculate the noise label learning loss function at this time The average value is used to determine whether the training is stagnant. If so, the number of stagnation rounds is set to s. stop Add 1; if not stagnant, make it stagnant for s rounds stop Return to 0;

[0013] S7, determine the number of stagnation rounds s stop Whether the maximum threshold is reached, if so, the optimized main neural network model is output And the optimized pseudo-label filtering neural network model If not, execute step S8;

[0014] S8. Determine the extension amount of the labeled dataset according to the growth function LScale, and convert the pseudo-labeled dataset U′ s The samples and labels with the smallest training loss are added to the initial labeled dataset L s , and from the initial unlabeled dataset U s Delete the corresponding sample and execute step S3 again.

[0015] Furthermore, when step S3 is re-executed, the main neural network model and the pseudo-label filtering neural network model are initialized.

[0016] Furthermore, in step S7, the pseudo-label dataset U′ s An equal number of low training loss samples are selected from each category of pseudo-label samples and added to the initial labeled dataset.

[0017] Furthermore, the method for determining training stagnation is:

[0018] If the average loss of this round is lower than the lowest value of the average loss of the previous rounds or this round is the first round of training, the training is not stagnant; if the average value of this round is higher than the lowest value of the average loss of the previous rounds, the training is stagnant.

[0019] Furthermore, the semi-supervised learning loss function The expression is as follows:

[0020]

[0021]

[0022]

[0023]

[0024] In the formula, λ u is the weight of the unlabeled loss function; CrossEntropy(*) is the cross entropy loss, RelativeEntropy(* is the relative entropy loss, subscript k is the category number, K is the total number of categories in the current task, and They are the current unlabeled samples x j Augmented samples after weak random augmentation and strong random augmentation.

[0025] Furthermore, when optimizing the main neural network model in step S3, the model parameters are optimized using the stochastic gradient descent method.

[0026] Furthermore, the noise label learning loss function The expression is as follows:

[0027]

[0028]

[0029]

[0030] In the formula The term is the reverse cross entropy loss, λ rceis the corresponding weight, CrossEntropy(*) is the cross entropy loss.

[0031] Furthermore, the growth function LScale is a logarithmic growth function.

[0032] Furthermore, when executing step S3, a convex combination of samples and labels may be used for data enhancement.

[0033] Furthermore, the two source samples of the convex combination should all come from the initial labeled dataset L s Or the initial unlabeled dataset U s .

[0034] A small sample semi-supervised learning device based on pseudo-label noise filtering comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement a small sample semi-supervised learning method based on pseudo-label noise filtering as described above when executing the computer program.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] 1. The present invention alternately optimizes the main model and the pseudo-label filtering model through a semi-supervised learning loss function and a noisy label learning loss function, and performs cyclic optimization by expanding the labeled data set. Compared with the prior art that directly uses the prediction results of unlabeled samples as supervisory information, the present invention filters the noise in the prediction of unlabeled samples to obtain an image recognition model with little influence of pseudo-label noise. When the main neural network model obtained by the present invention is applied to image recognition, it can accurately identify and classify images even when a small number of image data sets are selected because the noise is filtered, and it has strong versatility.

[0037] 2. The present invention will initialize the main model and the pseudo-label filtering model during loop optimization to avoid falling into a local optimal solution.

[0038] 3. The present invention applies convex combination data enhancement in executing step S3 to make the model optimization more efficient.

[0039] 4. The present invention uses a logarithmic growth function to expand the data set, which is more in line with the changing trend of model training optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION

[0041] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0042] This embodiment provides a small sample semi-supervised learning method based on pseudo-label noise filtering, which is applied to image classification problems in the field of computer vision. The process is as follows Figure 1 As shown, the specific steps include:

[0043] Step S1: Get the initial labeled data set L s and the initial unlabeled dataset U s , L s ={(x i ,y i )},U s ={x j Among them, these data sets are all derived from the miniimagenet image data set, and a total of 50,000 training images of 100 categories are selected as the initial labeled data set L s and the initial unlabeled dataset U s In this embodiment, a random sampling method is used to simulate the data labeling conditions of semi-supervised learning. Specifically, some images are randomly selected from the training images of each category as labeled samples, and the remaining images are hidden as unlabeled samples. Determine the main model The number of optimization steps per stage is T m ; Pseudo-label filtering model The number of optimization steps per stage is T a ; The amount of labeled data increases with the execution stage, and the maximum number of stall rounds is S stop The growth function LScale(s) should be logarithmic. The characteristic of this type of growth function is that the growth rate is fast at first, then slows down, and finally enters the saturation zone. This design fully considers the changing trend of model performance growth during the training process.

[0044] Step S2: Randomly initialize the main neural network model and pseudo-label filtering neural network model And initialize the number of stagnation rounds s stop And the total number of rounds s=1.

[0045] Step S3: Initial labeled dataset L s and the initial unlabeled dataset U s Input the main neural network model Combined with semi-supervised learning loss function Optimize the main neural network model.

[0046] Among them, the semi-supervised learning loss function The expression is as follows:

[0047]

[0048]

[0049]

[0050]

[0051] In the formula, λ u is the weight of the unlabeled loss function; CrossEntropy(*) is the cross entropy loss, RelativeEntropy(*) is the relative entropy loss, which is used to measure the difference from one distribution to another. The subscript k is the category number, K is the total number of categories in the current task, and They are the current unlabeled samples x j The augmented samples after weak random enhancement and strong random enhancement, the former only participates in the forward operation and does not participate in the back propagation. The model parameters are optimized by stochastic gradient descent method, and a total of T m step.

[0052] Among them, a convex combination of samples and labels can be used for data enhancement. The two source samples of the convex combination should all come from the initial labeled dataset L s Or the initial unlabeled dataset U s , the purpose is to prevent U s The noisy pseudo labels in are transferred to more enhanced samples.

[0053] Step S4: Initial unlabeled dataset U s Input the optimized main neural network model The category with the highest confidence in the prediction result is defined as the pseudo label y′ corresponding to the unlabeled data sample j , get the pseudo-label dataset U′ s , U′ s ={(x j ,y′ j )}.

[0054] Step S5: Initial labeled dataset L s and pseudo-label dataset U′ s Input pseudo-label filtering neural network model Learning loss function with noisy labels Optimize the pseudo-label filtering neural network model and execute T a step.

[0055] Among them, the noise label learning loss function The expression is as follows:

[0056]

[0057]

[0058]

[0059] In the formula The term is the reverse cross entropy loss, λ rce is the corresponding weight, and CrossEntropy(*) is the cross entropy loss. According to the relevant theory of noisy label learning, when the correct labels account for the largest proportion of samples in this category (since wrong labels are often scattered in multiple other categories, this condition is easier to meet), the above loss function will accelerate the convergence of the model on correctly labeled samples and correspondingly slow down the convergence of the model on wrongly labeled samples. Therefore, in the subsequent steps, we can use the final training loss of the sample in this step as an indicator to measure the credibility of the pseudo-label.

[0060] Step S6: Initial labeled dataset L s and pseudo-label dataset U′ s Input optimized pseudo-label filtering neural network model Calculate the noise label learning loss function for all samples at this time The average value is used to determine whether the training is stagnant. If so, the number of stagnation rounds is set to s. stop Add 1; if not stagnant, make it stagnant for s rounds stop Return to 0. The criteria for judging training stagnation are:

[0061] If the average loss of this round is lower than the lowest value of the average loss of the previous round, or this round of training is the first round of training, the training has not stagnated; if the average value of this round is higher than the lowest value of the average loss of the previous round, it means that the model of this round is mature and the training is stagnant.

[0062] Step S7: Determine the number of stagnation rounds stop Whether the maximum threshold S is reached stop If yes, the training is completed and the optimized main neural network model is output. And the optimized pseudo-label filtering neural network model If not, loop optimization is still needed, and step S8 is executed;

[0063] Step S8: Determine the extension amount of the labeled data set according to the growth function LScale, and set all U′ s The samples in the training set are arranged from small to large according to the training loss, and the amount of labeled data in round s+1 is determined according to the growth function LScale |L s+1|=LScale(s+1), from U′ s Extract the noise label learning loss function with the smallest value |L s+1 |-|L s | samples and corresponding pseudo labels are taken as credible data and classified into the labeled dataset, s is incremented by 1, and the initial unlabeled dataset U s Delete the corresponding sample and execute step S3 again.

[0064] Among them, U′ s The samples are sorted according to the categories indicated by the pseudo labels, and an equal number of credible samples are selected from each category and included in the labeled dataset to prevent the model from being overly biased towards easy-to-learn categories.

[0065] The expression of the growth function LScale is as follows:

[0066]

[0067] Among them, α and β are hyperparameters in the interval (0,1]. α controls the growth rate of the labeled dataset. The larger the value, the faster the growth rate. β is the upper limit of the proportion of the labeled dataset in the total data. The value depends on the current task performance baseline. If the baseline is low, it proves that the current task is more difficult and there may be more noise in the pseudo-label, then β should also take a lower value.

[0068] In addition, each time step S3 is returned and re-executed, only the data set is updated, that is, the main neural network model must be initialized before each training. and pseudo-label filtering neural network model The reason is that the model may be trapped in a local optimum after the previous stage of training. Reinitialize at each stage and appropriately extend the number of optimization steps T m and T a , which can reduce the risk of training stagnation and help the model better fit the current stage data set.

[0069] Finally, the optimized main neural network model is obtained And the optimized pseudo-label filtering neural network model Then, the corresponding 10,000 test images in the miniimagenet image dataset were used for testing to verify the accuracy of the main neural network model obtained by the present invention in image recognition and classification. Compared with the prior art method of directly using the prediction results of unlabeled samples as supervisory information, the present invention filters the noise of the prediction of unlabeled samples and obtains an image recognition model with little influence of pseudo-label noise. When the image recognition model obtained by the present invention is applied to image recognition, due to the filtering of noise, it can accurately recognize images even when the number of selected image datasets is small, and the model has high accuracy.

[0070] This embodiment further provides a small sample semi-supervised learning device based on pseudo-label noise filtering, including a memory and a processor; the memory is used to store computer programs; the processor executes a small sample semi-supervised learning method based on pseudo-label noise filtering mentioned in the above embodiment.

[0071] This embodiment also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by the processor, a small sample semi-supervised learning method based on pseudo-label noise filtering as mentioned in the embodiment of the present invention can be implemented, and any combination of one or more computer-readable media can be used. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0072] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A small sample semi-supervised learning method based on pseudo-label noise filtering, characterized in that: Image classification, as applied to computer vision, includes the following steps: S1. Obtain an initial labeled data set and an initial unlabeled data set, wherein some images are randomly selected from the training images of each category as the labeled data set, and the remaining images are hidden as the unlabeled data set; S2, initializing the main neural network model and the pseudo-label filtering neural network model; S3, inputting the initial labeled data set and the initial unlabeled data set into the main neural network model, optimizing the main neural network model in combination with the semi-supervised learning loss function, and obtaining the optimized main neural network model; S4, inputting the initial unlabeled data set into the optimized main neural network model, defining the category with the highest confidence in the prediction result as the pseudo label corresponding to the unlabeled data sample, and obtaining the pseudo label data set; S5, inputting the initial labeled data set and the pseudo-label data set into the pseudo-label filtering neural network model, optimizing the pseudo-label filtering neural network model in combination with the noise label learning loss function, and obtaining an optimized pseudo-label filtering neural network model; S6, input the initial labeled data set and the pseudo-label data set into the optimized pseudo-label filtering neural network model, calculate the average value of the noise label learning loss function at this time, and judge whether the training is stagnant according to the average value, if stagnant, increase the number of stagnant rounds by 1; if not stagnant, return the number of stagnant rounds to 0; S7, determining whether the number of stagnation rounds reaches the maximum threshold, if so, outputting the optimized main neural network model and the optimized pseudo-label filtering neural network model; if not, executing step S8; S8, determining the amount of expansion of the labeled data set according to the growth function, adding the samples and labels with the smallest training loss in the pseudo-labeled data set to the initial labeled data set, and deleting the corresponding samples from the initial unlabeled data set, and re-performing step S3; The semi-supervised learning loss function The expression is as follows: In the formula, (x i ,y i ) is the data and label in the labeled dataset, L s is a labeled dataset, is the main neural network model, U s is an unlabeled dataset, λ u is the weight of the unlabeled loss function; CrossEntropy(*) is the cross entropy loss, RelativeEntropy(*) is the relative entropy loss, subscript k is the category number, K is the total number of categories in the current task, and They are the current unlabeled samples x j Augmented samples after weak random enhancement and strong random enhancement; The noisy label learning loss function The expression is as follows: In the formula, U′ s is a pseudo-label dataset, is a pseudo-label filtering neural network model, (x i ,y i ) is the data and label in the labeled dataset, L s is a labeled dataset, The term is the reverse cross entropy loss, λ rce is the corresponding weight, CrossEntropy(*) is the cross entropy loss.

2. According to claim 1, a small sample semi-supervised learning method based on pseudo-label noise filtering is characterized in that: When step S3 is re-executed, the main neural network model and the pseudo-label filtering neural network model are initialized.

3. The small sample semi-supervised learning method based on pseudo-label noise filtering according to claim 1 is characterized in that: In step S7, an equal number of low training loss samples are selected from each type of pseudo-label samples in the pseudo-label dataset and added to the initial labeled dataset.

4. The small sample semi-supervised learning method based on pseudo-label noise filtering according to claim 1, characterized in that: The method to determine training stagnation is: If the average loss of this round is lower than the lowest value of the average loss of the previous rounds or this round is the first round of training, the training is not stagnant; if the average value of this round is higher than the lowest value of the average loss of the previous rounds, the training is stagnant.

5. The small sample semi-supervised learning method based on pseudo-label noise filtering according to claim 1, characterized in that: The growth function is a logarithmic growth function.

6. The small sample semi-supervised learning method based on pseudo-label noise filtering according to claim 1, characterized in that: When executing step S3, a convex combination of samples and labels may be used for data augmentation.

7. The small sample semi-supervised learning method based on pseudo-label noise filtering according to claim 6, characterized in that: The two source samples of the convex combination should all come from the initial labeled dataset or the initial unlabeled dataset.

8. A small sample semi-supervised learning device based on pseudo-label noise filtering, characterized in that: It comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement a small sample semi-supervised learning method based on pseudo-label noise filtering as described in any one of claims 1 to 7 when executing the computer program.

Citation Information

Patent Citations

  • Single sample image segmentation method based on category prototype learning

    CN113569865A

  • Neural network training method and device

    CN113705769A

  • Model training using a teacher-student learning paradigm

    US20200410388A1