Image classification method and system based on label propagation contrastive semi-supervised learning

Abstract

The present invention discloses an image classification method and system based on label propagation contrast semi-supervised learning. The present invention first learns meaningful potential features of self-supervised contrast learning for image data. Secondly, a label propagation algorithm is used to propagate known labels on unsupervised features to effectively label the entire image data set in an automatic manner. Then, a similarity alignment method is used to select matching image samples and form a reliable image sample data set. Finally, a semi-supervised network model is trained on the matching reliable image sample data set. Unsupervised contrast learning is performed on unmatched image samples to extract features and predict labels for classification. The present invention can improve the performance of semi-supervised image classification under extremely low supervision settings, and can solve the problem of serious degradation of model performance caused by data mismatch during labeling due to unbalanced image data distribution during image classification, thereby improving the accuracy of image data classification and effectively reducing the work of manual annotation.

Description

Technical Field

[0001] The present invention belongs to the field of image processing and relates to an image classification method and system based on label propagation contrast semi-supervised learning. Background Art

[0002] Image classification has always been a hot topic and a difficult topic in research. Since labeling data requires a lot of manpower and financial resources, the introduction of semi-supervised learning solves the problem of large amounts of labeled data. However, there is sometimes a mismatch between labeled data and unlabeled data. Data that has not appeared in the labeled data will appear in the unlabeled dataset, which causes data mismatch problems and makes the labeled data unreliable. Unreliable sample points may mislead the model learning process, thereby reducing the performance of the algorithm.

[0003] Semi-supervised learning has received a lot of attention recently because it alleviates the need for large amounts of labeled data, which is often expensive, requires expert knowledge, and is time-consuming to collect. Recent developments in deep semi-supervised classification have achieved unprecedented performance, narrowing the gap between supervised and semi-supervised learning. This performance improvement is based on a complex optimization scheme that incorporates many technical tricks, powerful augmentation techniques, and multiple loss functions. Semi-supervised learning aims to combine small amounts of labeled data to extract information from unlabeled data and produce results comparable to fully supervised methods. In recent years, advances in deep learning have driven a new direction in semi-supervised learning (SSL) for image classification. The main benefit of these new deep methods is their ability to learn feature representations rather than relying on hand-crafted features. In the past few years, deep SSL papers have achieved unprecedented performance, and the gap between supervised and semi-supervised models is much smaller than it was even five years ago, with semi-supervised methods outperforming some supervised techniques.

[0004] Before the present invention was made, deep semi-supervised image recognition and image classification currently use noise-free and interference-free data, and the labeled data and unlabeled data are also balanced. When encountering data that is unbalanced with the labeled data, the part that matches the labeled data is directly discarded. In practical applications, it is usually difficult to obtain noise-free unlabeled data. The training data is randomly selected, that is, the labeled samples with classes and the samples without class labels are independently distributed. The samples without class labels may come from different distribution scenarios of the labeled samples with classes and are noisy, resulting in mismatches between the labeled data and the unlabeled data, which also leads to unreliable data. The accuracy calculation of the data cannot be well grasped, which reduces the performance of the model. Summary of the Invention

[0005] Purpose of the invention: The purpose of the present invention is to overcome the above-mentioned defect of unbalanced distribution of image data, and to provide an image classification method and system based on label propagation contrast semi-supervised learning, so as to solve the problem of serious degradation of model performance caused by mismatch of data during labeling due to unbalanced distribution of image data during image classification, and to improve the accuracy of image data classification.

[0006] Technical solution: To achieve the above-mentioned purpose, the present invention adopts the following technical solution:

[0007] An image classification method based on label propagation contrastive semi-supervised learning, comprising the following steps:

[0008] (1) Partially label the original image dataset, where the number of labeled images is less than that of unlabeled images, and preprocess the image dataset;

[0009] (2) Constructing a semi-supervised image classification framework including a contrastive learning model and a label propagation model, narrowing the distance between similar features of unlabeled image data and labeled image data through contrastive learning; the contrastive learning model maximizes the similarity of positive sample pairs and minimizes the similarity of negative sample pairs, and optimizes the weight parameters of the contrastive learning model through the contrastive loss of instance mixing;

[0010] (3) Use the image features obtained by contrastive learning to perform label propagation, and use the similarity alignment method to find labels that match the labeled image data and labels that do not match the labeled image data; for matching labels, use the propagated labels to construct a reliable image dataset using image samples whose cross entropy loss value is greater than the cosine similarity distribution value; for mismatched labels, use the unsupervised contrastive learning method to extract image features and predict their labels and categories;

[0011] (4) Use reliable image datasets and labeled data to train the semi-supervised network model, use the cross entropy of the ground truth label and the predicted label to form the supervised loss, the prediction of the unlabeled data constitutes the cross entropy unsupervised loss, and use the supervised loss and unsupervised loss to form the total loss of the semi-supervised network model. Optimize the weight parameters of the semi-supervised network model through the back propagation algorithm;

[0012] (5) Use the semi-supervised network model to classify the test image data and obtain the classification results.

[0013] Furthermore, the contrastive learning model in step (2) uses an instance mixing strategy to improve contrastive feature learning. First, contrastive learning is transformed into training a classifier to assign a unique virtual label to a batch of data, indicating their identity in the batch; then, image instances are mixed in the input space and the virtual label space to provide more augmented data during training.

[0014] Furthermore, the instance mixing strategy is a data-driven enhancement strategy for contrastive learning, specifically: is a batch of image data pairs, where N p is the batch size, for each anchor point, are two different degrees of enhanced versions of the same image data. For each input image sample x i , and They are respectively called x i positive and negative samples; the contrastive learning model f(·) learns to maximize the similarity of positive sample pairs and minimize the similarity of negative sample pairs. The output of f(·) is L2 normalized. It is in batch B and The virtual label of the same image is 1, and the mixed virtual label value of different images is 0. For general samples with virtual labels (x i ,v i )、(x j ,v j ), the instance mixing is as follows:

[0015] mix=(λx i +(1-λ)x j ,λv i +(1-λ)v j )

[0016] where λ is the mixing coefficient.

[0017] Furthermore, step (2) uses the contrastive loss of instance-mixed contrastive learning for:

[0018]

[0019] A virtual label is set for each batch, and the total loss is calculated, where N represents the total number of samples and τ t represents the temperature parameter, |B| represents the number of image samples in batch B, represents the inner product of two L2 normalized vectors, Respectively represent feature extraction under two different enhanced versions, v i,j represents a virtual label, and v i,i =1,v i,j≠i =0.

[0020] Furthermore, the label propagation model in step (2) uses the graph diffusion algorithm, specifically: let the image dataset D = D l +D u , N=Nl +N u Indicates the total sample size, N l ,N u Respectively represent the number of labeled image samples and the number of unlabeled image samples, y i is the image sample x i The label propagation task is to propagate labels from labeled images to estimate the labels of unlabeled image samples. First, use the descriptor set V = (v1, v2, ..., v N ) defines the affinity matrix, where v i For the nodes of the graph, i=1,2,...,N, construct the affinity matrix according to the data relationship: Where D=diag(A1 N ) represents the degree matrix, 1 N Represents the all-one N vector, the adjacency matrix A i,j :

[0021]

[0022] γ is the affinity parameter, ||·|| is the two-norm, and the estimation process of label propagation diffusion is:

[0023] F=(I-αS) -1 Y

[0024] F is the estimation matrix, α∈[0,1] represents the probability of the adjacent vertex, Y is the N×C label matrix, C is the number of classes; if the image sample x i ∈D l , and y i =c, then Y ic =1, c represents the class, the final estimated one-hot label It is expressed as follows:

[0025]

[0026] in is the value of the i-th row and c-th column in the matrix Y, indicating the estimated label of the i-th row and c-th column, F ic Represents the estimated value of the i-th row and c-th column in the matrix F.

[0027] Furthermore, the similarity distribution S on the class in the similarity alignment method used in step (3) d It is obtained as follows: Given a list of c classes in a set of labeled image samples, use the encoder to calculate the labeled image sample x i The feature representation z i , the feature representation of class c is obtained by averaging, that is, Where L is the normalization constant value, N cis the number of labeled image samples of category c; given a normalized feature For an unlabeled image sample x′, we use C predetermined prototypes to obtain its cosine similarity distribution S d =(s1,s2,...,s C ), where s c express similarity, c = 1, 2, ..., C, C is the number of classes.

[0028] Furthermore, in step (3), a reliable image dataset D is constructed. r Specifically, we use cross entropy loss and cosine similarity distribution to identify reliable image samples: Use softmax normalization l i Represents the i-th image sample x i The cross entropy of is the estimated label obtained after label propagation, Represents the contrastive learning model parameters; select the l with the smallest loss among the labeled images and unlabeled images of class c c , when l c >s c Select a reliable sample set for class c Creating a reliable image dataset N r >N l , N r is the number of samples in the reliable image dataset, N l is the number of labeled image samples.

[0029] Furthermore, in step (4), the supervision loss Use the ground truth labels y i The cross entropy between the predicted value and the labeled data is expressed as:

[0030]

[0031] Where H represents the cross entropy between two values, Denotes the semi-supervised model as a labeled image x i The resulting predicted class distribution;

[0032] Unsupervised loss Expressed as:

[0033]

[0034] in represents the predicted class distribution generated by the model under the enhancement operation for the unlabeled image, Represents the pseudo label generated by the model, 1 is the indicator function, when max(pi )≥τ is true, then it is 1, false, then it is 0, and the maximum class probability is higher than the threshold τ. for optimization.

[0035] Based on the same inventive concept, the present invention provides an image classification system based on label propagation contrastive semi-supervised learning, comprising:

[0036] A preprocessing module is used to partially label the original image dataset, wherein the number of labeled images is less than that of unlabeled images, and to preprocess the image dataset;

[0037] A contrastive learning feature extraction module is used to build a contrastive learning model to narrow the distance between similar features of unlabeled image data and labeled image data through contrastive learning. The contrastive learning model maximizes the similarity of positive sample pairs while minimizing the similarity of negative sample pairs, and optimizes the weight parameters of the contrastive learning model through the contrastive loss of instance mixing.

[0038] The label propagation module is used to build a label propagation model and use the image features obtained by contrastive learning to perform label propagation;

[0039] The image screening module uses similarity alignment to find labels that match or mismatch the labeled image data. For matching labels, the propagated labels use image samples with cross-entropy loss values ​​greater than the cosine similarity distribution value to construct a reliable image dataset. For mismatched labels, an unsupervised contrastive learning method is used to extract image features and predict their labels and categories.

[0040] The classification model training module is used to train a semi-supervised network model using a reliable image dataset and labeled data. The supervised loss is formed by the cross entropy of the ground truth label and the predicted label, and the cross entropy unsupervised loss is formed by the prediction of the unlabeled data. The supervised loss and the unsupervised loss are used to form the total loss of the semi-supervised network model. The weight parameters of the semi-supervised network model are optimized through the back-propagation algorithm.

[0041] And a testing module, which is used to classify the test image data using the semi-supervised network model to obtain classification results.

[0042] Based on the same inventive concept, the present invention provides a computer system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, the image classification method based on label propagation contrast and semi-supervised learning is implemented.

[0043] Beneficial effects: The advantages and effects of the present invention are to overcome the mismatch defects between labeled image data and unlabeled image data, incorporate a few labeled prior knowledge into contrastive learning, and aim to seek more discriminative representations with the help of mined image samples in the pseudo-labeled branch. The distance between the similar features of unlabeled image data and labeled image data is shortened. The feature representation is more discriminative, which is conducive to semi-supervised learning. Label propagation is performed using the features obtained by contrastive learning, and reliable image samples are selected by similar alignment for subsequent semi-supervised learning. For mismatched data, unsupervised contrastive learning is used to predict its label. Specifically, it is manifested as follows:

[0044] (1) The matching data and the unmatched data are discussed separately. The features of the labeled image data are learned using contrastive learning. The learned labeled features are then used for label propagation. Unsupervised learning is performed using a small number of labeled image samples and unlabeled image samples to obtain discriminative representations. Label propagation on unsupervised representations propagates a small number of labeled examples to all image samples. This can result in correct and incorrect labels. Finally, sample selection is performed to select reliable image samples with high similarity. The selected labels are more reliable to avoid noisy labels and create a reliable extended labeled image dataset. Finally, the reliable extended labeled image dataset and the unlabeled image dataset are used to perform semi-supervised training of the model for semi-supervised image classification.

[0045] (2) For labels that do not match the labeled data, an unsupervised contrastive learning method is used to extract the features of the image and predict its label and category.

[0046] The present invention adopts a semi-supervised learning method based on label propagation contrast. The semi-supervised classification of label propagation contrast learning features provides a method with the advantages of using the original less labeled set. Expanded (larger) reliable marker set A semi-supervised algorithm. arrive This extension is accomplished in a fully self-supervised manner. We utilize contrastive learning within self-supervision to investigate this approach, demonstrating that it can significantly reduce the error rate of SSL algorithms when given few labels. In contrastive learning, positive samples are simply restricted to a single image and subjected to various data transformations, while all other images are treated as negative samples and pushed aside. We incorporate prior knowledge of a small number of labels into contrastive learning, aiming to find more discriminative representations using mined image samples in the pseudo-labeled branch. We extend the original contrastive learning loss and align the mined image samples with predefined prototypes. For image labels obtained through label propagation, we use similarity distribution alignment to calculate image similarity, identifying labels that match the labeled data and labels that do not. We then use different classification methods for the two different labels: semi-supervised classification is performed on the matching data along with the labeled data, while unsupervised contrastive learning is used to extract features from the mismatched data, predicting and classifying the labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Detailed framework diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The solution of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] like Figure 1 As shown, an image classification method based on label propagation and contrastive semi-supervised learning is disclosed in an embodiment of the present invention. Contrastive learning is used to extract features from image data. Feature information is passed through a label propagation algorithm. Similarity alignment is used to select labels from unlabeled image data that match and do not match the labeled data. Classification is then performed using different methods. A semi-supervised model is trained on both the matched and labeled data to classify the images. Contrastive learning using a mixture of instances with an effective regularization strategy is used to extract features from the unmatched data and predict category information. Contrastive learning is performed using a small number of labeled and unlabeled image samples to obtain discriminative representations. Label propagation on the unsupervised representation propagates the few labeled image examples to all image samples. This can result in both correct and incorrect labels. Finally, image sample selection is performed to select reliable image samples with high similarity. This ensures more reliable labels, avoids noisy labels, and creates a reliable extended labeled image dataset. Finally, the reliable extended labeled image dataset and the unlabeled image dataset are used to semi-supervisedly train the model for semi-supervised image classification.

[0050] The present invention adopts semi-supervised learning based on label propagation contrast to perform image classification, which overcomes the defect of general semi-supervised methods that have no good feedback on unlabeled image data, resulting in low classification accuracy. At the same time, it overcomes the defect of fully supervised methods that require more labeled information and consumes huge materials, as well as the problem of data mismatch caused by uneven data distribution, which leads to decreased model performance. The present invention adopts an image classification method based on semi-supervised learning based on label propagation contrast, adopts contrastive learning to extract image data features, uses label propagation and similarity alignment methods to determine whether it matches the labeled data, fully studies the correlation between data, and improves the accuracy of image classification.

[0051] An image classification method based on label propagation contrast and semi-supervised learning disclosed in an embodiment of the present invention specifically includes the following steps:

[0052] (1) Partially label the original image dataset, where the number of labeled images is less than that of unlabeled images, and preprocess the image dataset;

[0053] (2) Constructing a semi-supervised image classification framework including a contrastive learning model and a label propagation model, narrowing the distance between similar features of unlabeled image data and labeled image data through contrastive learning; the contrastive learning model maximizes the similarity of positive sample pairs and minimizes the similarity of negative sample pairs, and optimizes the weight parameters of the contrastive learning model through the contrastive loss of instance mixing;

[0054] (3) Use the image features obtained by contrastive learning to perform label propagation, and use the similarity alignment method to find labels that match the labeled image data and labels that do not match the labeled image data; for matching labels, use the propagated labels to construct a reliable image dataset using image samples whose cross entropy loss value is greater than the cosine similarity distribution value; for mismatched labels, use the unsupervised contrastive learning method to extract image features and predict their labels and categories;

[0055] (4) Use reliable image datasets and labeled data to train the semi-supervised network model, use the cross entropy of the ground truth label and the predicted label to form the supervised loss, the prediction of the unlabeled data constitutes the cross entropy unsupervised loss, and use the supervised loss and unsupervised loss to form the total loss of the semi-supervised network model. Optimize the weight parameters of the semi-supervised network model through the back propagation algorithm;

[0056] (5) Use the semi-supervised network model to classify the test image data and obtain the classification results.

[0057] In step (1), the image dataset is divided into a training image dataset and a test image dataset. The image dataset is preprocessed including data enhancement and normalization. The data enhancement includes flipping, rotating a certain angle, and performing random whitening operations on the images in the training dataset to expand the training image dataset.

[0058] The contrastive learning model in step (2) uses an instance mixing strategy to improve contrastive feature learning. First, contrastive learning is transformed into training a classifier to assign a unique virtual label to a batch of data, indicating their identity in the batch; then, image instances are mixed in the input space and the virtual label space to provide more augmented data during training.

[0059] Furthermore, the instance mixing strategy is a data-driven enhancement strategy for contrastive learning, specifically: is a batch of image data pairs, where N p is the batch size, for each anchor point, are two different degrees of enhanced versions of the same image data. For each input image sample x i , and They are respectively called x i positive and negative samples; the contrastive learning model f(·) learns to maximize the similarity of positive sample pairs and minimize the similarity of negative sample pairs. The output of f(·) is L2 normalized. It is in batch B and The virtual label of the same image is 1, and the mixed virtual label value of different images is 0. For general samples with virtual labels (x i ,v i )、(x j ,v j ), the instance mixing is as follows:

[0060] mix=(λx i +(1-λ)x j ,λv i +(1-λ)v j )

[0061] where λ is the mixing coefficient.

[0062] Contrastive loss using instance-mixed contrastive learning in step (2) for:

[0063]

[0064] A virtual label is set for each batch, and the total loss is calculated, where N represents the total number of samples and τ trepresents the temperature parameter, |B| represents the number of image samples in batch B, represents the inner product of two L2 normalized vectors, Respectively represent feature extraction under two different enhanced versions, v i,j represents a virtual label, and v i,i =1,v i,j≠i =0.

[0065] In step (2), the label propagation model uses the graph diffusion algorithm, specifically: let the image dataset D = D l +D u , N=N l +N u Indicates the total sample size, N l ,N u Respectively represent the number of labeled image samples and the number of unlabeled image samples, y i is the image sample x i The label propagation task is to propagate labels from labeled images to estimate the labels of unlabeled image samples. First, use the descriptor set V = (v1, v2, ..., v N ) defines the affinity matrix, where v i For the nodes of the graph, i=1,2,...,N, construct the affinity matrix according to the data relationship: Where D=diag(A1 N ) represents the degree matrix, 1 N Represents the all-one N vector, the adjacency matrix A i,j :

[0066]

[0067] γ is the affinity parameter, ||·|| is the two-norm, and the estimation process of label propagation diffusion is:

[0068] F=(I-αS) -1 Y

[0069] F is the estimation matrix, α∈[0,1] represents the probability of the adjacent vertex, Y is the N×C label matrix, C is the number of classes; if the image sample x i ∈D l , and y i =c, then Y ic =1, c represents the class, the final estimated one-hot label It is expressed as follows:

[0070]

[0071] in is the value of the i-th row and c-th column in the matrix Y, indicating the estimated label of the i-th row and c-th column, F ic Represents the estimated value of the i-th row and c-th column in the matrix F.

[0072] The similarity distribution S on the class in the similarity alignment method used in step (3) d It is obtained as follows: Given a list of c classes in a set of labeled image samples, use the encoder to calculate the labeled image sample x i The feature representation z i , the feature representation of class c is obtained by averaging, that is, Where L is the normalization constant value, N c is the number of labeled image samples of category c; given a normalized feature For an unlabeled image sample x′, we use C predetermined prototypes to obtain its cosine similarity distribution S d =(s1,s2,...,s C ), where s c express similarity, c = 1, 2, ..., C, C is the number of classes.

[0073] In step (3), a reliable image dataset D is constructed r The method is as follows: using cross entropy loss and cosine similarity distribution to identify reliable image samples: Use softmax normalization l i Represents the i-th image sample x i The cross entropy of is the estimated label obtained after label propagation, Represents the contrastive learning model parameters; select the l with the smallest loss among the labeled images and unlabeled images of class c c , when l c >s c Select a reliable sample set for class c Creating a reliable image dataset N r >N l , N r is the number of samples in the reliable image dataset, N l is the number of labeled image samples.

[0074] In step (4), the supervision loss Use the ground truth labels y i The cross entropy between the predicted value and the labeled data is expressed as:

[0075]

[0076] Where H represents the cross entropy between two values, Denotes the semi-supervised model as a labeled image x i The resulting predicted class distribution;

[0077] Unsupervised loss Expressed as:

[0078]

[0079] in represents the predicted class distribution generated by the model under the enhancement operation for the unlabeled image, Represents the pseudo label generated by the model, 1 is the indicator function, when max(p i )≥τ is true, then it is 1, false, then it is 0, and the maximum class probability is higher than the threshold τ. for optimization.

[0080] To demonstrate the effectiveness of the present invention, the method of the present invention was compared with existing methods on existing image datasets.

[0081] We conduct experiments for our proposed method (LPCSSL) using two image classification datasets: CIFAR10 and CIFAR100. The CIFAR dataset consists of 60K 32×32 (84×84) RGB images, divided into 50K training samples and 10K testing samples. CIFAR10 samples are classified into 10 categories, while CIFAR100 samples are classified into 100 categories. Following common practice in image retrieval, CIFAR10 and CIFAR100 are labeled with 1, 4, 10, and 25 labels, respectively, and PCA whitening and L2 normalization are applied to the features before diffusion. A reliable set is constructed by training for 60 epochs with a high learning rate (0.1) to prevent label noise from being memorized. At the end of training, the sample with the lowest loss per class is selected. The per-sample loss is averaged over the last T = 30 training epochs. For semi-supervised learning experiments, a standard WideResNet-28-2 is used for fair comparison with related work. Our method is combined with state-of-the-art pseudo-labeling and consistency regularization-based semi-supervised methods to demonstrate the stability of our inventive method when applied to different semi-supervised strategies. The default configuration of pseudo-labeling is used, except for network initialization, which utilizes a rotation self-supervised target. The network is warmed up for 200 epochs on the labeled set and then trained for 400 epochs on the entire dataset. The use of extended supervision from a reliable set enables the semi-supervised algorithm to achieve significant and stable accuracy on a standard dataset with very few labeled samples. From the tabular data, it can be seen that regardless of the amount of labeled data, the effect of the present invention is the best, which shows the effectiveness of the present invention for image classification tasks.

[0082] Table 1: Classification results of 6 algorithms on the CIFAR10 image dataset

[0083]

[0084] Table 2: Classification results of 6 algorithms on the CIFAR100 image dataset

[0085]

[0086] Based on the same inventive concept, an embodiment of the present invention discloses an image classification system based on label propagation contrast semi-supervised learning, including: a preprocessing module, used to partially label the original image data set, wherein the number of labeled images is less than that of unlabeled images, and preprocess the image data set; a contrastive learning feature extraction module, used to construct a contrastive learning model, and shorten the distance between similar features between unlabeled image data and labeled image data through contrastive learning; the contrastive learning model maximizes the similarity of positive sample pairs and minimizes the similarity of negative sample pairs, and optimizes the weight parameters of the contrastive learning model through the contrast loss of instance mixing; a label propagation module, used to construct a label propagation model, and use the image features obtained by contrastive learning to perform label propagation; an image screening module, used to find and label images using a similarity alignment method Data matching and labels that do not match the labeled image data; for matched labels, the propagated labels use image samples whose cross entropy loss values ​​are greater than the cosine similarity distribution value to construct a reliable image dataset; for unmatched labels, an unsupervised contrastive learning method is used to extract image features and predict their labels and categories; a classification model training module is used to train a semi-supervised network model using a reliable image dataset and labeled data, the supervised loss is formed by the cross entropy of the ground truth label and the predicted label, the cross entropy unsupervised loss is formed by the prediction of the unlabeled data, the supervised loss and the unsupervised loss are used to form the total loss of the semi-supervised network model, and the weight parameters of the semi-supervised network model are optimized by the back propagation algorithm; and a testing module is used to classify the test image data using the semi-supervised network model to obtain the classification results.

[0087] The specific working process of each module described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here. The division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as combining multiple modules or integrating them into another system.

[0088] Based on the same inventive concept, an embodiment of the present invention discloses a computer system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, the image classification method based on label propagation contrast semi-supervised learning is implemented.

[0089] Those skilled in the art will appreciate that the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes instructions for causing a computer system (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present invention. Storage media include various media capable of storing computer programs, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random-access memories (RAMs), magnetic disks, or optical disks.

Claims

1. An image classification method based on label propagation contrastive semi-supervised learning, characterized in that: The steps include: (1) Partially label the original image dataset, where the number of labeled images is less than that of unlabeled images, and preprocess the image dataset; (2) Construct a semi-supervised image classification framework including a contrastive learning model and a label propagation model, and narrow the distance between similar features of unlabeled image data and labeled image data through contrastive learning; the contrastive learning model maximizes the similarity of positive sample pairs and minimizes the similarity of negative sample pairs, and optimizes the weight parameters of the contrastive learning model through the contrastive loss of instance mixing; wherein, let is a batch of image data pairs, where N p is the batch size, for each anchor point, are two different degrees of enhanced versions of the same image data. For each input image sample x i , and They are respectively called x i Positive and negative samples; It is in batch B and The virtual label of the same image is 1, and the mixed virtual label value of different images is 0. For samples with virtual labels (x i ,v i )、(x j ,v j ), the instance mixing method is mix=(λx i +(1-λ)x j ,λv i +(1-λ)v j ), λ is the mixing coefficient; (3) Use the image features obtained by contrastive learning to perform label propagation, and use the similarity alignment method to find labels that match the labeled image data and labels that do not match the labeled image data; for matching labels, use the propagated labels to construct a reliable image dataset using image samples whose cross entropy loss value is greater than the cosine similarity distribution value; for mismatched labels, use the unsupervised contrastive learning method to extract image features and predict their labels and categories; (4) Use reliable image datasets and labeled data to train the semi-supervised network model, use the cross entropy of the ground truth label and the predicted label to form the supervised loss, the prediction of the unlabeled data constitutes the cross entropy unsupervised loss, and use the supervised loss and unsupervised loss to form the total loss of the semi-supervised network model. Optimize the weight parameters of the semi-supervised network model through the back propagation algorithm; (5) Use the semi-supervised network model to classify the test image data and obtain the classification results.

2. The image classification method based on label propagation contrast semi-supervised learning according to claim 1, characterized in that: The contrastive learning model in step (2) uses an instance mixing strategy to improve contrastive feature learning. First, contrastive learning is transformed into training a classifier to assign a unique virtual label to a batch of data, indicating their identity in the batch; then, image instances are mixed in the input space and the virtual label space to provide more augmented data during training.

3. The image classification method based on label propagation contrast semi-supervised learning according to claim 1, characterized in that: Contrastive loss using instance-mixed contrastive learning in step (2) for: A virtual label is set for each batch, and the total loss is calculated, where N represents the total number of samples and τ t represents the temperature parameter, |B| represents the number of image samples in batch B, represents the inner product of two L2 normalized vectors, Respectively represent feature extraction under two different enhanced versions, v i,j represents a virtual label, and v i,i =1,v i,j≠i =0.

4. The image classification method based on label propagation contrast semi-supervised learning according to claim 1, characterized in that: In step (2), the label propagation model uses the graph diffusion algorithm, specifically: let the image dataset D = D l +D u , N=N l +N u Indicates the total sample size, N l ,N u Respectively represent the number of labeled image samples and the number of unlabeled image samples, y i is the image sample x i The label propagation task is to propagate labels from labeled images to estimate the labels of unlabeled image samples. First, use the descriptor set V = (v1, v2, ..., v N ) defines the affinity matrix, where v i For the nodes of the graph, i=1,2,...,N, construct the affinity matrix according to the data relationship: Where D=diag(A1 N ) represents the degree matrix, 1 N Represents the all-one N vector, the adjacency matrix A i,j : γ is the affinity parameter, ||·|| is the two-norm, and the estimation process of label propagation diffusion is: F=(I-αS) -1 Y F is the estimation matrix, α∈[0,1] represents the probability of the adjacent vertex, Y is the N×C label matrix, C is the number of classes; if the image sample x i ∈D l , and y i =c, then Y ic =1, c represents the class, the final estimated one-hot label It is expressed as follows: in is the value of the i-th row and c-th column in the matrix Y, indicating the estimated label of the i-th row and c-th column, F ic Represents the estimated value of the i-th row and c-th column in the matrix F.

5. The image classification method based on label propagation contrast semi-supervised learning according to claim 1, characterized in that: The similarity distribution S on the class in the similarity alignment method used in step (3) d It is obtained as follows: Given a list of c classes in a set of labeled image samples, use the encoder to calculate the labeled image sample x i The feature representation z i , the feature representation of class c is obtained by averaging, that is, Where L is the normalization constant value, N c is the number of labeled image samples of category c; given a normalized feature For an unlabeled image sample x′, we use C predetermined prototypes to obtain its cosine similarity distribution S d =(s1,s2,...,s C ), where s c express similarity, c = 1, 2, ..., C, C is the number of classes.

6. The image classification method based on label propagation contrast semi-supervised learning according to claim 5, characterized in that: In step (3), a reliable image dataset D is constructed r Specifically, we use cross entropy loss and cosine similarity distribution to identify reliable image samples: Use softmax normalization l i Represents the i-th image sample x i The cross entropy of is the estimated label obtained after label propagation, Represents the contrastive learning model parameters; select the l with the smallest loss among the labeled images and unlabeled images of class c c , when l c >s c Select a reliable sample set for class c Creating a reliable image dataset N r >N l , N r is the number of samples in the reliable image dataset, N l is the number of labeled image samples.

7. The image classification method based on label propagation contrast semi-supervised learning according to claim 1, characterized in that: In step (4), the supervision loss Use the ground truth labels y i The cross entropy between the predicted value and the labeled data is expressed as: where N l represents the number of labeled image samples, H represents the cross entropy between two values, Denotes the semi-supervised model as a labeled image x i The resulting predicted class distribution; Unsupervised loss Expressed as: where N r is the number of samples in the reliable image dataset, represents the predicted class distribution generated by the model under the enhancement operation for the unlabeled image, Represents the pseudo label generated by the model, 1 is the indicator function, when max(p i )≥τ is true, then it is 1, false, then it is 0, and the maximum class probability is higher than the threshold τ. for optimization.

8. An image classification system based on label propagation contrastive semi-supervised learning, characterized in that: include: A preprocessing module is used to partially label the original image dataset, wherein the number of labeled images is less than that of unlabeled images, and to preprocess the image dataset; The contrastive learning feature extraction module is used to build a contrastive learning model, which shortens the distance between similar features of unlabeled image data and labeled image data through contrastive learning. The contrastive learning model maximizes the similarity of positive sample pairs and minimizes the similarity of negative sample pairs, and optimizes the weight parameters of the contrastive learning model through the contrastive loss of instance mixing. is a batch of image data pairs, where N p is the batch size, for each anchor point, are two different degrees of enhanced versions of the same image data. For each input image sample x i , and They are respectively called x i Positive and negative samples; It is in batch B and The virtual label of the same image is 1, and the mixed virtual label value of different images is 0. For samples with virtual labels (x i ,v i )、(x j ,v j ), the instance mixing method is mix=(λx i +(1-λ)x j ,λv i +(1-λ)v j ), λ is the mixing coefficient; The label propagation module is used to build a label propagation model and use the image features obtained by contrastive learning to perform label propagation; The image screening module uses similarity alignment to find labels that match or mismatch the labeled image data. For matching labels, the propagated labels use image samples with cross-entropy loss values ​​greater than the cosine similarity distribution value to construct a reliable image dataset. For mismatched labels, an unsupervised contrastive learning method is used to extract image features and predict their labels and categories. The classification model training module is used to train a semi-supervised network model using a reliable image dataset and labeled data. The supervised loss is formed by the cross entropy of the ground truth label and the predicted label, and the cross entropy unsupervised loss is formed by the prediction of the unlabeled data. The supervised loss and the unsupervised loss are used to form the total loss of the semi-supervised network model. The weight parameters of the semi-supervised network model are optimized through the back-propagation algorithm. And a testing module, which is used to classify the test image data using the semi-supervised network model to obtain classification results.

9. A computer system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is loaded into a processor, the image classification method based on label propagation contrast semi-supervised learning according to any one of claims 1 to 7 is implemented.

Images