Semi-supervised learning method, system, terminal and storage medium based on feature fusion

Abstract

The present invention relates to the field of deep learning technology, and specifically provides a semi-supervised learning method, system, terminal and storage medium based on feature fusion, including: randomly selecting two initial training subsets from a labeled data set. Classifier 1 and classifier 2 are trained respectively using the labeled data of the two training subsets. Then, in each iteration process, classifier 1 and classifier 2 respectively predict the unlabeled samples that have undergone feature fusion and the unlabeled samples that have not undergone feature fusion in subset 1 and subset 2, and measure the confidence of the model prediction by comparing the changes before and after fusion, thereby determining the pseudo labels of a part of the unlabeled samples. The classifier of each subset selects several unlabeled samples with the highest prediction confidence, adds them to the labeled data set of the other subset, and retrains the classifier using the amplified labeled samples. The present invention greatly improves the training efficiency and accuracy of the classifier.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning technology, and specifically relates to a semi-supervised learning method, system, terminal and storage medium based on feature fusion. Background Art

[0002] Classifiers are often used in fields such as equipment fault detection and image recognition, and are an important technology for the development of industrial intelligence. Classifiers need to be trained with data before they can be used. The training data should be samples marked with corresponding labels, so that the classifier can train its own weights during the training process, and the weight values ​​affect the recognition accuracy of the classifier.

[0003] The acquisition of labeled data is usually a very expensive and time-consuming process. In contrast, the acquisition cost of unlabeled data is lower. Therefore, semi-supervised learning, as a method of learning with limited labeled data and a large amount of unlabeled data, has received widespread attention. The goal of semi-supervised learning is to improve the performance of the classifier by making full use of the information of unlabeled data. Traditional methods usually use the confidence of the model output as the basis for judging the pseudo-label, that is, if the confidence exceeds the threshold, the network prediction result is used as the pseudo-label of the sample.

[0004] However, in the early stages of model training, the model cannot make correct judgments on unlabeled samples. Therefore, there will be a large number of samples where even with high confidence, the model prediction results are still wrong. This error will continue to increase with the model training process, resulting in extremely poor model performance.

[0005] Therefore, when the proportion of labeled data is low, the accuracy of the classifier using semi-supervised learning is poor and cannot meet application requirements. Summary of the invention

[0006] In view of the above-mentioned deficiencies in the prior art, the present invention provides a semi-supervised learning method, system, terminal and storage medium based on feature fusion to solve the above-mentioned technical problems.

[0007] In a first aspect, the present invention provides a semi-supervised learning method based on feature fusion, comprising:

[0008] Constructing a labeled data set, dividing the labeled data set into a first subset and a second subset, using the first subset to train a first classifier, and using the second subset to train a second classifier;

[0009] Generate pseudo labels for the first group of unlabeled samples using the trained first classifier, and generate pseudo labels for the second group of unlabeled samples using the trained second classifier;

[0010] Using feature fusion technology to obtain the confidence of the pseudo-label of the unlabeled sample, screen out the first candidate sample whose confidence reaches a preset threshold from the first group of unlabeled samples, and screen out the second candidate sample whose confidence reaches the preset threshold from the second group of unlabeled samples;

[0011] Convert the pseudo label of the first candidate sample into a label, and update the first candidate sample with the label to the second subset, convert the pseudo label of the second candidate sample into a label, and update the second candidate sample with the label to the first subset;

[0012] The first classifier and the second classifier are trained iteratively.

[0013] In an optional implementation, constructing a labeled data set, dividing the labeled data set into a first subset and a second subset, using the first subset to train a first classifier, and using the second subset to train a second classifier, comprises:

[0014] Divide the labeled data set into the first subset and the second subset evenly according to the number of samples;

[0015] Constructing a first classifier, wherein the first classifier includes WideResNet and Shake_Drop2;

[0016] Constructing a second classifier, wherein the second classifier includes Densenet_cifar and Shake_Drop2;

[0017] The first subset is set as the training set of the first classifier, and the second subset is set as the training set of the second classifier.

[0018] In an optional embodiment, using the trained first classifier to generate pseudo labels for the first group of unlabeled samples, and using the trained second classifier to generate pseudo labels for the second group of unlabeled samples, includes:

[0019] Collect unlabeled samples in advance and save them to an unlabeled dataset;

[0020] Setting the number of samples in the unlabeled sample group, randomly extracting a corresponding number of unlabeled sample groups from the unlabeled data set based on the number of samples, setting the unlabeled sample group as the first group of unlabeled samples, and setting the copies of the unlabeled sample group as the second group of unlabeled samples;

[0021] Using the first classifier to identify the first group of unlabeled samples one by one, and setting the identification results as pseudo labels of the corresponding unlabeled samples, and saving the unlabeled samples with added pseudo labels into the first intermediate sample set;

[0022] The second group of unlabeled samples are identified one by one using the second classifier, and the identification results are set as pseudo labels of the corresponding unlabeled samples, and the unlabeled samples with added pseudo labels are saved in the second intermediate sample set.

[0023] In an optional implementation, the confidence of the pseudo label of the unlabeled sample is obtained by using the feature fusion technology, including:

[0024] Classify the labeled samples according to the label category, use the feature extraction network to extract the feature map of each category of labeled samples, and set the average value of all feature maps of each category of labeled samples as the feature anchor point of the corresponding category of labeled samples;

[0025] Selecting an unlabeled sample with a pseudo label, and extracting a feature map of the unlabeled sample using a feature extraction network;

[0026] Interpolating and fusing the feature map with the feature anchor points of each category to obtain a plurality of corresponding fused feature maps;

[0027] Using corresponding classifiers to generate corresponding classification results based on the feature graph of the unlabeled sample and the fused feature graph of the target category, and calculating the norm between the classification results;

[0028] Traversing all categories, and taking the sum of the multiple norms obtained as the total distance of the prediction results before and after the unlabeled sample feature fusion;

[0029] The total distance is converted into a confidence level, and the total distance is inversely proportional to the confidence level.

[0030] In an optional implementation, the feature map is interpolated and fused with the feature anchor points of each category to obtain a fused feature map

[0031] Use interpolation ratio α ∈ [0, 1) D ;

[0032] The interpolation formula is: α = αz ∗ + (1 − α)z u , where z α is the fusion feature map, z u Feature map of unlabeled samples, z ∗ is the feature anchor of the target category.

[0033] In an optional embodiment, the first classifier and the second classifier are iteratively trained, including:

[0034] The first classifier and the second classifier are iteratively trained respectively using the updated first subset and the second subset until there are no unprocessed unlabeled samples in the unlabeled data set.

[0035] In a second aspect, the present invention provides a semi-supervised learning system based on feature fusion, comprising:

[0036] A basic training module, used for constructing a labeled data set, dividing the labeled data set into a first subset and a second subset, training a first classifier using the first subset, and training a second classifier using the second subset;

[0037] A label prediction module, used to generate pseudo labels for the first group of unlabeled samples using the trained first classifier, and to generate pseudo labels for the second group of unlabeled samples using the trained second classifier;

[0038] A label identification module is used to obtain the confidence of the pseudo label of the unlabeled sample by using the feature fusion technology, select the first candidate sample whose confidence reaches the preset threshold from the first group of unlabeled samples, and select the second candidate sample whose confidence reaches the preset threshold from the second group of unlabeled samples;

[0039] A sample updating module, configured to convert the pseudo-label of the first candidate sample into a label, and update the first candidate sample with the label to the second subset, convert the pseudo-label of the second candidate sample into a label, and update the second candidate sample with the label to the first subset;

[0040] The iterative control module is used to iteratively train the first classifier and the second classifier.

[0041] In an optional embodiment, the basic training module includes:

[0042] A sample partitioning unit, used for evenly partitioning the labeled data set into a first subset and a second subset according to the number of samples;

[0043] A first construction unit, used to construct a first classifier, wherein the first classifier includes WideResNet and Shake_Drop2;

[0044] A second construction unit is used to construct a second classifier, wherein the second classifier includes Densenet_cifar and Shake_Drop2;

[0045] The training setting unit is used to set the first subset as a training set of the first classifier and set the second subset as a training set of the second classifier.

[0046] In a third aspect, a terminal is provided, including:

[0047] processor, memory, wherein:

[0048] The memory is used to store computer programs.

[0049] The processor is used to call and run the computer program from the memory, so that the terminal executes the above-mentioned terminal method.

[0050] According to a fourth aspect, a computer storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the methods described in the above aspects.

[0051] The beneficial effect of the present invention lies in that the semi-supervised learning method, system, terminal and storage medium based on feature fusion provided by the present invention draw on the idea of ​​feature fusion to convert unlabeled samples with relatively certain estimated pseudo-labels into labeled samples, thereby expanding the number of labeled samples and increasing the proportion of labeled samples, thereby greatly improving the training efficiency and accuracy of the classifier.

[0052] In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention.

[0055] Figure 2 is another schematic flow chart of a method according to an embodiment of the present invention.

[0056] Figure 3 It is a rendering of a method according to an embodiment of the present invention.

[0057] Figure 4 is a schematic block diagram of a system according to an embodiment of the present invention.

[0058] Figure 5 A schematic diagram of the structure of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0061] The key terms appearing in the present invention are explained below.

[0062] Semi-supervised learning: Semi-supervised learning is an important research direction in the field of machine learning. It uses both labeled and unlabeled data to improve the performance of classifiers. In traditional supervised learning, we only use labeled data to train classifiers, while in semi-supervised learning, we can use the information of unlabeled data to enhance the generalization ability of classifiers. Semi-supervised learning has great application potential for problems with limited labeled data or where labels are difficult to obtain, such as image recognition, text classification, and social network analysis.

[0063] Feature fusion: Feature fusion helps discover features that the model cannot recognize in unlabeled instances by finding interpolations that intervene in feature representations. By comparing the recognition results of the model for samples before and after fusion, we can identify unlabeled instances with sufficiently obvious features and treat them as labeled samples with pseudo labels based on the model prediction results. This will increase the number of labeled samples and improve the performance of the classifier.

[0064] Interpolation parameters: Interpolation parameters are parameters used in the interpolation process to control the interpolation method and results. Interpolation is a method of inferring the value of an unknown location through known data points, estimating or filling in between data points. In interpolation, interpolation parameters are usually used to determine the form of the interpolation function and the relationship between the interpolation points. The specific interpolation parameters depend on the interpolation method and algorithm used.

[0065] Entropy: Entropy is a measure of the random variable A measure of uncertainty.

[0066] Entropy Regularization: Entropy regularization is a regularization technique commonly used in machine learning to adjust the learning process and results of the model. It introduces an entropy term in the loss function to encourage the model to generate a smoother and more balanced prediction distribution. In machine learning, entropy regularization pushes the model to produce a more balanced prediction probability distribution by minimizing the entropy of the model. This can help prevent the model from being overconfident or overfitting the training data and improve the generalization ability of the model.

[0067] WideResNet: WideResNet is a variant of a deep residual network for image classification tasks. It is an improvement and extension of ResNet. The main goal of WideResNet is to improve the performance and expressiveness of the model, especially when dealing with complex datasets and large-scale image data. It does this by increasing the width of the network (i.e., increasing the number of channels in each layer). Compared with traditional deep convolutional neural networks, WideResNet uses more convolutional channels in each layer, thereby increasing the capacity of the model. This enables WideResNet to handle more complex features and more category information, improving the accuracy of classification tasks. The core of WideResNet is the residual block, which solves the problem of gradient vanishing and model degradation in deep networks by introducing cross-layer connections). The residual block allows information to skip a layer directly in the network, thereby more effectively propagating and learning feature representations.

[0068] DenseNet-CIFAR: DenseNet-CIFAR is a model based on DenseNet, which is specially designed for image classification tasks on the CIFAR dataset. DenseNet is a densely connected deep residual network structure that promotes information flow and reuse of features by introducing dense connections. The design of DenseNet-CIFAR on the CIFAR dataset takes into account the characteristics of the dataset, including small image size and relatively few training samples. Its core idea is to improve the performance and generalization ability of the model through dense connections and deep networks. Unlike traditional deep convolutional neural networks, each layer in DenseNet-CIFAR is directly connected to all previous layers, allowing information to flow freely in the network. This dense connection method helps to solve the gradient vanishing problem, improve the efficiency of feature propagation, and reduce the number of parameters.

[0069] The semi-supervised learning method based on feature fusion provided in the embodiment of the present invention is executed by a computer device, and accordingly, the semi-supervised learning system based on feature fusion runs in the computer device.

[0070] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention. Figure 1 The execution subject may be a semi-supervised learning system based on feature fusion. According to different requirements, the order of the steps in the flowchart may be changed, and some may be omitted.

[0071] like Figure 1 As shown, the method includes:

[0072] Step 110, constructing a labeled data set, dividing the labeled data set into a first subset and a second subset, using the first subset to train a first classifier, and using the second subset to train a second classifier;

[0073] Step 120, using the trained first classifier to generate pseudo labels for the first group of unlabeled samples, and using the trained second classifier to generate pseudo labels for the second group of unlabeled samples;

[0074] Step 130, using feature fusion technology to obtain the confidence of the pseudo label of the unlabeled sample, screening out the first candidate sample whose confidence reaches a preset threshold from the first group of unlabeled samples, and screening out the second candidate sample whose confidence reaches a preset threshold from the second group of unlabeled samples;

[0075] Step 140, converting the pseudo label of the first candidate sample into a label, and updating the first candidate sample with the label to the second subset, converting the pseudo label of the second candidate sample into a label, and updating the second candidate sample with the label to the first subset;

[0076] Step 150: iteratively train the first classifier and the second classifier.

[0077] To facilitate understanding of the present invention, the semi-supervised learning method based on feature fusion provided by the present invention is further described below based on the principle of the semi-supervised learning method based on feature fusion of the present invention and in combination with the process of training the classifier in the embodiment.

[0078] For details, please refer to Figure 2 , semi-supervised learning methods based on feature fusion include:

[0079] S1. Construct a labeled data set, divide the labeled data set into a first subset and a second subset, use the first subset to train a first classifier, and use the second subset to train a second classifier.

[0080] The labeled data set is evenly divided into a first subset and a second subset according to the number of samples; a first classifier is constructed, and the first classifier includes WideResNet and Shake_Drop2; a second classifier is constructed, and the second classifier includes Densenet_cifar and Shake_Drop2; the first subset is set as a training set of the first classifier, and the second subset is set as a training set of the second classifier.

[0081] Specifically, the dataset includes 10 different types of samples, totaling 50,000 images, to ensure the diversity and robustness of the experiment.

[0082] S2. Generate pseudo labels for the first group of unlabeled samples using the trained first classifier, and generate pseudo labels for the second group of unlabeled samples using the trained second classifier.

[0083] Collect unlabeled samples in advance and save the unlabeled samples to an unlabeled data set; set the number of samples in the unlabeled sample group, randomly extract a corresponding number of unlabeled sample groups from the unlabeled data set based on the sample number, set the unlabeled sample group to be the first group of unlabeled samples, and set the copy of the unlabeled sample group to be the second group of unlabeled samples; use the first classifier to identify the first group of unlabeled samples one by one, set the identification results as pseudo labels of the corresponding unlabeled samples, and save the unlabeled samples with pseudo labels added to the first intermediate sample set; use the second classifier to identify the second group of unlabeled samples one by one, set the identification results as pseudo labels of the corresponding unlabeled samples, and save the unlabeled samples with pseudo labels added to the second intermediate sample set.

[0084] Specifically include:

[0085] (1) Collect unlabeled samples in advance:

[0086] During the data collection phase, a large amount of unlabeled raw data is obtained through crawlers, database queries, sensor collection, etc.

[0087] The collected unlabeled samples are preliminarily cleaned and preprocessed to remove noise, errors or irrelevant data.

[0088] The cleaned unlabeled samples are saved to an unlabeled dataset for subsequent use.

[0089] (2) Set the number of samples in the unlabeled sample group:

[0090] According to the actual application scenario and model training requirements, set an appropriate unlabeled sample group size (that is, the number of unlabeled samples processed each time).

[0091] This number should ensure the stability of model training and the effective use of computing resources.

[0092] (3) Randomly extract unlabeled sample groups:

[0093] Based on the preset number of samples, a corresponding number of unlabeled samples are randomly selected from the unlabeled dataset to form an unlabeled sample group.

[0094] Set this group of unlabeled samples as the first group of unlabeled samples.

[0095] At the same time, for subsequent data enhancement, copies of the first set of unlabeled samples are set as the second set of unlabeled samples.

[0096] (4) Generate pseudo labels using the first classifier:

[0097] If you already have some labeled data, you can use this data to train an initial classifier (i.e., the first classifier).

[0098] The first group of unlabeled samples are identified one by one using the trained first classifier, and the identification results are used as pseudo labels of the corresponding unlabeled samples.

[0099] The unlabeled samples with pseudo labels added are saved in the first intermediate sample set, which will be used for subsequent model training or further data enhancement.

[0100] (2) Generate pseudo labels using the second classifier:

[0101] The second classifier is used to generate pseudo labels for the second set of unlabeled samples.

[0102] Similarly, the second classifier is used to identify the second group of unlabeled samples one by one, and the identification results are used as pseudo labels of the corresponding unlabeled samples.

[0103] The unlabeled samples with pseudo labels added are saved in the second intermediate sample set.

[0104] S3. Use feature fusion technology to obtain the confidence of the pseudo label of the unlabeled sample, screen out the first candidate sample whose confidence reaches the preset threshold from the first group of unlabeled samples, and screen out the second candidate sample whose confidence reaches the preset threshold from the second group of unlabeled samples.

[0105] The confidence of the sample pseudo labels in the first intermediate sample set is calculated using the first classifier and the first subset, and the confidence of the sample pseudo labels in the second intermediate sample set is calculated using the second classifier and the second subset.

[0106] Confidence calculation methods include:

[0107] (1) Classify the labeled samples according to the label category, use the feature extraction network to extract the feature map of each category of labeled samples, and set the average value of all feature maps of each category of labeled samples as the feature anchor point of the corresponding category of labeled samples;

[0108] All labeled samples are classified according to their categories. For example, the "Dog" category includes all samples labeled "Dog". Next, we traverse all different categories and input all samples belonging to this category into the feature extraction part of the model to obtain the feature map corresponding to each sample. Then, we take the average of all feature maps of this category to obtain the average feature map representing this category, which is called the anchor. At this point, we have the anchors of all categories.

[0109] Specifically, the method to obtain the anchor point is as follows:

[0110] All labeled samples are classified according to the category to which the label belongs, and all samples of the same category are used to calculate the anchor point of the category. All M samples of the same category are input into the feature extraction network to extract the feature map (FeatureMap), and the M feature maps corresponding to the M labeled samples of the same category are averaged to obtain the fused feature map, which is the anchor point (Anchor) of the category.

[0111] This process can be viewed as a way to sample a new instance without explicitly modeling the joint probability of labeled and unlabeled instances:

[0112]

[0113] Consider interpolating an unlabeled instance with all anchor points representing different classes to reveal sufficiently different features by considering how the model's predictions change. To this end, the change in the pseudo-labels of the unlabeled instance (i.e., y∗) and the loss due to interpolation should be studied. Obviously, making sufficiently small interpolations on the labeled data should not have a corresponding impact on the predicted labels of each unlabeled point.

[0114] You can u The first-order Taylor expansion of the interpolation loss is written as:

[0115]

[0116] Considering ,have:

[0117]

[0118] By replacing ,have:

[0119] ;

[0120] In summary, using the u The first-order Taylor expansion of , the loss of the model in predicting pseudo labels for unlabeled instances when interpolating with labeled instances can be rewritten as Eq. In fact, this statement is true for any given instance and any convex combination of points in the latent space. But for semi-supervised learning, we are particularly interested in unlabeled instances.

[0121] For a sufficiently small α, such as α ≤ ϵ, it is almost exact. Therefore, for the complete labeled set, by choosing the minimum loss, we have:

[0122] ;

[0123] Here, ϵ is a hyperparameter that controls the size of the mixture.

[0124] Based on the Cauchy-Schwartz inequality, we know that:

[0125] ;

[0126] It can be inferred that:

[0127] ;

[0128] Therefore, the optimization problem can be changed to the optimization equation:

[0129] ;

[0130] The dual norm of the above optimization equation can be used to approximate The optimal value of should be:

[0131] ;

[0132] After replacing u and After the actual value, we have:

[0133] .

[0134] (2) Selecting unlabeled samples with pseudo labels and extracting feature maps of the unlabeled samples using a feature extraction network;

[0135] For each unlabeled sample, it is input into the feature extraction network to obtain a feature map.

[0136] (3) interpolating and fusing the feature map with the feature anchor points of each category to obtain a plurality of corresponding fused feature maps;

[0137] It is interpolated and fused with the feature anchor points of each other category to obtain a fused feature map. The interpolation methods include:

[0138] In this paper, convex combination of features (i.e., interpolation) is used as a method to explore new features near each unlabeled point. Formally, consider using an interpolation ratio α ∈ [0, 1) D , with z α = αz ∗ + (1 − α)z u represents the interpolation between, where z u is the feature map of the unlabeled sample and z ∗ Feature anchors of a certain category.

[0139] (4) using corresponding classifiers to generate corresponding classification results based on the feature map of the unlabeled sample and the fused feature map of the target category, and calculating the norm between the classification results;

[0140] Suppose we fuse the unlabeled sample with the anchor of the xth category, input the feature maps before and after fusion into the classifier of the model, and obtain the classification results. Each classification result is in the form of p1, p2, …, pn, which indicates the probability that the sample belongs to category 1, category 2, …, category n.

[0141] By calculating the L2 norm of the classification results before and after fusion, we can get the "distance" between the two results. The L2 norm can be used to measure the change in the prediction results before and after fusion.

[0142] (5) Traversing all categories, and taking the sum of the multiple norms obtained as the total distance of the prediction results before and after the unlabeled sample feature fusion;

[0143] Calculate the "distance" between the prediction results of the unlabeled sample and all categories of anchor points before and after fusion. Add up all the "distances" to get the "total distance" of the prediction results of the unlabeled sample before and after fusion.

[0144] (6) The total distance is converted into confidence, and the total distance is inversely proportional to the confidence.

[0145] If the "total distance" is small, it means that the unlabeled samples are input into the classifier respectively before and after fusion, and the classification results given by the classifier are slightly different. This means that even after the interference caused by the interpolation of "anchor points" of other categories, the classifier still roughly maintains the original classification result, indicating that the classifier has a high confidence in the sample, which means that the prediction result can be used as a pseudo label.

[0146] S4. Convert the pseudo-label of the first candidate sample into a label, and update the first candidate sample with the label to the second subset; convert the pseudo-label of the second candidate sample into a label, and update the second candidate sample with the label to the first subset.

[0147] In each round of iteration, the 100 samples with the smallest "total distance" are selected, the pseudo labels of the samples are converted into labels, and then the 100 samples selected from this subset are added to the labeled sample training set of the other subset, so as to continuously expand the training set, supplement more feature information, improve the accuracy of the model, and make the model have stronger generalization ability.

[0148] S5. Perform iterative training on the first classifier and the second classifier.

[0149] The first classifier and the second classifier are iteratively trained respectively using the updated first subset and the second subset until there are no unprocessed unlabeled samples in the unlabeled data set.

[0150] Figure 3 These are the experimental results of the FeatureMix semi-supervised learning algorithm on the CIFAR10 dataset using the WideResNet&Shake_Drop2 architecture and the densenet_cifar&Shake_Drop2 architecture. In addition, we also designed a comparative experiment. The comparison method is the traditional Co-Training model architecture. In each round of iteration, we sort all samples from small to large according to the model prediction probability confidence, select the first 100 samples that are consistent with the anchor point fusion of the category to which they belong, and add them to the other party's labeled dataset. It can be found that the total number of samples in the dataset is 50,000. When the initial number of samples is 100, 100 samples are added to the labeled dataset in each iteration. After only 11 iterations, the accuracy rates of 42.70% and 45.66% were achieved, which are 4.08% and 7.04% higher than the highest of other methods. The experimental results show that the proposed semi-supervised learning algorithm FeatureMix based on feature fusion has achieved good performance on CIFAR10 and has practical value.

[0151] In some embodiments, the semi-supervised learning system based on feature fusion may include multiple functional modules composed of computer program segments. The computer program of each program segment in the semi-supervised learning system based on feature fusion may be stored in a memory of a computer device and executed by at least one processor to perform (see Figure 1 Description) Function of semi-supervised learning based on feature fusion.

[0152] In this embodiment, the semi-supervised learning system based on feature fusion can be divided into multiple functional modules according to the functions it performs, such as Figure 4 As shown. The functional modules of the system 400 may include: a basic training module 410, a label prediction module 420, a label identification module 430, a sample update module 440 and an iteration control module 450. The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0153] A basic training module, used for constructing a labeled data set, dividing the labeled data set into a first subset and a second subset, training a first classifier using the first subset, and training a second classifier using the second subset;

[0154] A label prediction module, used to generate pseudo labels for the first group of unlabeled samples using the trained first classifier, and to generate pseudo labels for the second group of unlabeled samples using the trained second classifier;

[0155] A label identification module is used to obtain the confidence of the pseudo label of the unlabeled sample by using the feature fusion technology, select the first candidate sample whose confidence reaches the preset threshold from the first group of unlabeled samples, and select the second candidate sample whose confidence reaches the preset threshold from the second group of unlabeled samples;

[0156] A sample updating module, configured to convert the pseudo-label of the first candidate sample into a label, and update the first candidate sample with the label to the second subset, convert the pseudo-label of the second candidate sample into a label, and update the second candidate sample with the label to the first subset;

[0157] The iterative control module is used to iteratively train the first classifier and the second classifier.

[0158] Optionally, as an embodiment of the present invention, the basic training module includes:

[0159] A sample partitioning unit, used for evenly partitioning the labeled data set into a first subset and a second subset according to the number of samples;

[0160] A first construction unit, used to construct a first classifier, wherein the first classifier includes WideResNet and Shake_Drop2;

[0161] A second construction unit is used to construct a second classifier, wherein the second classifier includes Densenet_cifar and Shake_Drop2;

[0162] The training setting unit is used to set the first subset as a training set of the first classifier and set the second subset as a training set of the second classifier.

[0163] Figure 5 A schematic diagram of the structure of a terminal 500 provided in an embodiment of the present invention. The terminal 500 can be used to execute the semi-supervised learning method based on feature fusion provided in an embodiment of the present invention.

[0164] The terminal 500 may include: a processor 510, a memory 520 and a communication unit 530. These components communicate via one or more buses. Those skilled in the art will appreciate that the server structure shown in the figure does not limit the present invention, and it may be a bus structure or a star structure, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0165] The memory 520 can be used to store the execution instructions of the processor 510, and the memory 520 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. When the execution instructions in the memory 520 are executed by the processor 510, the terminal 500 can execute some or all of the steps in the following method embodiments.

[0166] The processor 510 is the control center of the storage terminal, and uses various interfaces and lines to connect various parts of the entire electronic terminal. It runs or executes software programs and / or modules stored in the memory 520, and calls data stored in the memory to perform various functions of the electronic terminal and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of multiple packaged ICs with the same or different functions. For example, the processor 510 can only include a central processing unit (CPU). In the embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0167] The communication unit 530 is used to establish a communication channel so that the storage terminal can communicate with other terminals, receive user data sent by other terminals or send user data to other terminals.

[0168] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0169] Therefore, the present invention draws on the idea of ​​feature fusion and converts the estimated unlabeled samples with relatively certain pseudo-labels into labeled samples, thereby expanding the number of labeled samples and increasing the proportion of labeled samples, thereby greatly improving the training efficiency and accuracy of the classifier. The technical effects that can be achieved in this embodiment can be found in the description above and will not be repeated here.

[0170] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes, including several instructions for enabling a computer terminal (which can be a personal computer, a server, or a second terminal, a network terminal, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention.

[0171] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the terminal embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0172] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or modules, which can be electrical, mechanical or other forms.

[0173] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0174] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0175] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.

Claims

1. A semi-supervised learning method based on feature fusion, characterized in that: include: Constructing a labeled data set, dividing the labeled data set into a first subset and a second subset, using the first subset to train a first classifier, and using the second subset to train a second classifier; Generate pseudo labels for the first group of unlabeled samples using the trained first classifier, and generate pseudo labels for the second group of unlabeled samples using the trained second classifier; Using feature fusion technology to obtain the confidence of the pseudo-label of the unlabeled sample, screen out the first candidate sample whose confidence reaches a preset threshold from the first group of unlabeled samples, and screen out the second candidate sample whose confidence reaches the preset threshold from the second group of unlabeled samples; Convert the pseudo label of the first candidate sample into a label, and update the first candidate sample with the label to the second subset, convert the pseudo label of the second candidate sample into a label, and update the second candidate sample with the label to the first subset; Iteratively training the first classifier and the second classifier; Generating pseudo labels for the first group of unlabeled samples using the trained first classifier, and generating pseudo labels for the second group of unlabeled samples using the trained second classifier, including: Collect unlabeled samples in advance and save them to an unlabeled dataset; Setting the number of samples in the unlabeled sample group, randomly extracting a corresponding number of unlabeled sample groups from the unlabeled data set based on the number of samples, setting the unlabeled sample group as the first group of unlabeled samples, and setting the copies of the unlabeled sample group as the second group of unlabeled samples; Using the first classifier to identify the first group of unlabeled samples one by one, and setting the identification results as pseudo labels of the corresponding unlabeled samples, and saving the unlabeled samples with added pseudo labels into the first intermediate sample set; Using the second classifier to identify the second group of unlabeled samples one by one, and setting the identification results as pseudo labels of the corresponding unlabeled samples, and saving the unlabeled samples with added pseudo labels into the second intermediate sample set; The confidence of the pseudo-label of the unlabeled sample is obtained by using feature fusion technology, including: Classify the labeled samples according to the label category, use the feature extraction network to extract the feature map of each category of labeled samples, and set the average value of all feature maps of each category of labeled samples as the feature anchor point of the corresponding category of labeled samples; Selecting an unlabeled sample with a pseudo label, and extracting a feature map of the unlabeled sample using a feature extraction network; Interpolating and fusing the feature map with the feature anchor points of each category to obtain a plurality of corresponding fused feature maps; Using corresponding classifiers to generate corresponding classification results based on the feature graph of the unlabeled sample and the fused feature graph of the target category, and calculating the norm between the classification results; Traversing all categories, and taking the sum of the multiple norms obtained as the total distance of the prediction results before and after the unlabeled sample feature fusion; Converting the total distance into a confidence level, wherein the total distance is inversely proportional to the confidence level; The feature map is interpolated and fused with the feature anchor points of each category to obtain a fused feature map Use interpolation ratio α ∈ [0, 1) D ; The interpolation formula is: α = αz ∗ + (1 − α)z u , where z α is the fusion feature map, z u Feature map of unlabeled samples, z ∗ is the feature anchor of the target category.

2. The method according to claim 1, characterized in that Constructing a labeled data set, dividing the labeled data set into a first subset and a second subset, using the first subset to train a first classifier, and using the second subset to train a second classifier, including: Divide the labeled data set into the first subset and the second subset evenly according to the number of samples; Constructing a first classifier, wherein the first classifier includes WideResNet and Shake_Drop2; Constructing a second classifier, wherein the second classifier includes Densenet_cifar and Shake_Drop2; The first subset is set as the training set of the first classifier, and the second subset is set as the training set of the second classifier.

3. The method according to claim 1, characterized in that The first classifier and the second classifier are iteratively trained, including: The first classifier and the second classifier are iteratively trained respectively using the updated first subset and the second subset until there are no unprocessed unlabeled samples in the unlabeled data set.

4. A semi-supervised learning system based on feature fusion, characterized in that: include: A basic training module, used for constructing a labeled data set, dividing the labeled data set into a first subset and a second subset, training a first classifier using the first subset, and training a second classifier using the second subset; A label prediction module, used to generate pseudo labels for the first group of unlabeled samples using the trained first classifier, and to generate pseudo labels for the second group of unlabeled samples using the trained second classifier; A label identification module is used to obtain the confidence of the pseudo label of the unlabeled sample by using the feature fusion technology, select the first candidate sample whose confidence reaches the preset threshold from the first group of unlabeled samples, and select the second candidate sample whose confidence reaches the preset threshold from the second group of unlabeled samples; A sample updating module, configured to convert the pseudo-label of the first candidate sample into a label, and update the first candidate sample with the label to the second subset, convert the pseudo-label of the second candidate sample into a label, and update the second candidate sample with the label to the first subset; An iterative control module, used for iteratively training the first classifier and the second classifier; Generating pseudo labels for the first group of unlabeled samples using the trained first classifier, and generating pseudo labels for the second group of unlabeled samples using the trained second classifier, including: Collect unlabeled samples in advance and save them to an unlabeled dataset; Setting the number of samples in the unlabeled sample group, randomly extracting a corresponding number of unlabeled sample groups from the unlabeled data set based on the number of samples, setting the unlabeled sample group as the first group of unlabeled samples, and setting the copies of the unlabeled sample group as the second group of unlabeled samples; Using the first classifier to identify the first group of unlabeled samples one by one, and setting the identification results as pseudo labels of the corresponding unlabeled samples, and saving the unlabeled samples with added pseudo labels into the first intermediate sample set; Using the second classifier to identify the second group of unlabeled samples one by one, and setting the identification results as pseudo labels of the corresponding unlabeled samples, and saving the unlabeled samples with added pseudo labels into the second intermediate sample set; The confidence of the pseudo-label of the unlabeled sample is obtained by using feature fusion technology, including: Classify the labeled samples according to the label category, use the feature extraction network to extract the feature map of each category of labeled samples, and set the average value of all feature maps of each category of labeled samples as the feature anchor point of the corresponding category of labeled samples; Selecting an unlabeled sample with a pseudo label, and extracting a feature map of the unlabeled sample using a feature extraction network; Interpolating and fusing the feature map with the feature anchor points of each category to obtain a plurality of corresponding fused feature maps; Using corresponding classifiers to generate corresponding classification results based on the feature graph of the unlabeled sample and the fused feature graph of the target category, and calculating the norm between the classification results; Traversing all categories, and taking the sum of the multiple norms obtained as the total distance of the prediction results before and after the unlabeled sample feature fusion; Converting the total distance into a confidence level, wherein the total distance is inversely proportional to the confidence level; The feature map is interpolated and fused with the feature anchor points of each category to obtain a fused feature map Use interpolation ratio α ∈ [0, 1) D ; The interpolation formula is: α = αz ∗ + (1 − α)z u , where z α is the fusion feature map, z u Feature map of unlabeled samples, z ∗ is the feature anchor of the target category.

5. The system according to claim 4, characterized in that The basic training modules include: A sample partitioning unit, used for evenly partitioning the labeled data set into a first subset and a second subset according to the number of samples; A first construction unit, used to construct a first classifier, wherein the first classifier includes WideResNet and Shake_Drop2; A second construction unit is used to construct a second classifier, wherein the second classifier includes Densenet_cifar and Shake_Drop2; The training setting unit is used to set the first subset as a training set of the first classifier and set the second subset as a training set of the second classifier.

6. A terminal, characterized in that: include: A memory for storing a semi-supervised learning program based on feature fusion; A processor, used to implement the steps of the semi-supervised learning method based on feature fusion as described in any one of claims 1 to 3 when executing the semi-supervised learning program based on feature fusion.

7. A computer-readable storage medium storing a computer program, characterized in that: The readable storage medium stores a semi-supervised learning program based on feature fusion, and when the semi-supervised learning program based on feature fusion is executed by a processor, the steps of the semi-supervised learning method based on feature fusion as described in any one of claims 1-3 are implemented.

Images