Semi-supervised image classification method and system based on pseudo labels and embedded clustering matching

Abstract

The invention provides a semi-supervised image classification method and system based on pseudo labels and embedded clustering matching, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a marked image and an unmarked image, and performing enhancement processing on the unmarked image; for the marked image, calculating supervision loss based on a semi-supervised image classification model; inputting the enhanced unmarked image into a semi-supervised image classification model, screening false labels based on a dynamic threshold value and a learning state of a category, and calculating unsupervised loss; respectively mapping the weakly enhanced image and the strongly enhanced image to an embedding space to generate K-Means clustering results which are respectively used for generating a target image and a strongly enhanced embedded clustering image, and calculating image matching loss; total loss is established based on supervised loss, unsupervised loss and graph matching loss, and the performance of the semi-supervised image classification model is improved by minimizing the total loss. Consistency regularization is applied to a single sample and a sample affinity level, so that confirmation deviation is effectively relieved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a semi-supervised image classification method and system based on pseudo-label and embedded cluster matching. Background Art

[0002] Pseudo-labeling and consistency regularization are two main technical routes in the field of semi-supervised learning. Combining pseudo-labeling and consistency regularization with data augmentation is a new trend. However, these two methods still have obvious shortcomings: pseudo-labeling methods cannot avoid confirmation bias, because the models trained with this method tend to maintain or amplify the generated predictions, even if they are noisy predictions; and existing consistency regularization methods only emphasize the application of consistency assumptions on a single instance, but ignore the relationship between aligned samples. Summary of the invention

[0003] In order to solve the above problems, the present invention proposes a semi-supervised image classification method and system based on pseudo-label and embedded cluster matching, which includes two parts: predictive matching and graph matching. Predictive matching is to apply consistency regularization at the level of a single sample. A dynamic threshold adjustment strategy is introduced in this module to alleviate confirmation bias. The initial threshold is low, and the threshold is gradually increased as the training process progresses. Graph matching constructs a comparison graph and a target graph based on the clustering results on different enhanced sample embeddings, and achieves consistency constraints at the level of sample affinity by optimizing the graph matching loss.

[0004] In order to achieve the above object, the present invention adopts the following technical solution: In a first aspect, the present invention provides a semi-supervised image classification method based on pseudo-label and embedded cluster matching, comprising: Obtain a labeled image and an unlabeled image; for the unlabeled image, perform weak enhancement and strong enhancement to obtain a weakly enhanced image and a strongly enhanced image; For labeled images, the supervised loss is calculated based on the semi-supervised image classification model; The enhanced unlabeled images are input into the semi-supervised image classification model, and a dynamic threshold is generated based on the model's learning effect on the category to filter out pseudo labels and calculate the unsupervised loss; The weakly enhanced image and the strongly enhanced image are respectively mapped to the embedding space for K-Means clustering, and the clustering results are respectively used to generate the target graph and the strongly enhanced embedded clustering graph, and the graph matching loss is calculated; A total loss is established based on supervised loss, unsupervised loss and graph matching loss, and the performance of the semi-supervised image classification model is improved by minimizing the total loss.

[0005] Preferably, for the labeled image, the supervised loss is calculated based on a semi-supervised image classification model, specifically comprising: weakly enhancing the labeled image, and calculating the supervised loss of the prediction of the input weakly enhanced labeled image and its corresponding true label based on the semi-supervised image classification model.

[0006] Preferably, the method generates a dynamic threshold based on the learning effect of the model on the category to filter pseudo labels, specifically including: In this round, the unlabeled images are input into the semi-supervised image classification model to obtain the predicted classification labels; Using the dynamic threshold updated in the previous round, the predicted classification labels are screened to obtain the pseudo labels for this round; Based on the pseudo labels and predicted classification labels of this round, evaluate the learning effect of the model on the category in this round; The learning effect is normalized and used to update the dynamic threshold for use in the next round.

[0007] Preferably, the method of using the dynamic threshold updated in the previous round to screen the predicted classification labels to obtain the pseudo labels in this round specifically includes:

[0008] in, is the discriminant function, is an unlabeled image The maximum class probability prediction of , that is, the pseudo label of this round; is the dynamic threshold updated in the previous round; the screening process associates the threshold with the learning effect of the model on each class, which is used to provide a lower threshold in the early stage of training and gradually increase the threshold as the training process progresses.

[0009] Preferably, the step of evaluating the learning effect of the current round model on the category based on the current round pseudo labels and predicted classification labels specifically includes:

[0010] in, It reflects the learning effect of the model on class c at time step t; is the model for unlabeled images The weakly enhanced prediction vector, is an unlabeled image The maximum class probability prediction of Represents a fixed threshold, which is used to select samples with high confidence to train the model; represents the pseudo label belonging to class c; is the discriminant function; It is an unlabeled sample The probability of being in class c.

[0011] Preferably, the learning effect is normalized and used to update the dynamic threshold for use in the next round, specifically including:

[0012] Among them, Perform maximum normalization to obtain ;

[0013] Among them, the normalized category learning effect To scale the predefined threshold , generates a dynamic threshold for each category at each time step t; in order to avoid generating extremely low This results in many noise samples being involved in training. Further adjustments are made as follows:

[0014] Among them, C represents the number of categories.

[0015] Preferably, the weakly enhanced image and the strongly enhanced image are respectively mapped to the embedding space for K-Means clustering, and the clustering results are respectively used to generate the target graph and the strongly enhanced embedded clustering graph, and the graph matching loss is calculated, which specifically includes: The weakly enhanced image and the strongly enhanced image are mapped to the embedding space respectively, and K-Means clustering is performed respectively; the clustering result of the weakly enhanced image is used to generate the target graph, and the clustering result of the strongly enhanced image is used to generate the strongly enhanced embedded clustering graph, and the graph matching loss between the target graph and the strongly enhanced embedded clustering graph is calculated.

[0016] In a second aspect, the present invention provides a semi-supervised image classification system based on pseudo-label and embedded cluster matching, comprising: A data enhancement module is used to obtain a labeled image and an unlabeled image; for the unlabeled image, weak enhancement and strong enhancement are performed to obtain a weakly enhanced image and a strongly enhanced image; The prediction and matching module is used to calculate the supervised loss for the labeled image based on the semi-supervised image classification model; the enhanced unlabeled image is input into the semi-supervised image classification model, and a dynamic threshold is generated based on the model's learning state of the category to filter out pseudo labels and calculate the unsupervised loss; A graph matching module is used to map the weakly enhanced image and the strongly enhanced image to the embedding space for K-Means clustering, and the clustering results are used to generate the target graph and the strongly enhanced embedded clustering graph, and calculate the graph matching loss; The loss calculation module is used to establish the total loss based on the supervised loss, unsupervised loss and graph matching loss, and improve the performance of the semi-supervised image classification model by minimizing the total loss.

[0017] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a semi-supervised image classification method based on pseudo-labels and embedded cluster matching described in the first aspect.

[0018] In a fourth aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of a semi-supervised image classification method based on pseudo-labels and embedded cluster matching described in the first aspect are implemented.

[0019] Compared with the prior art, the present invention has the following beneficial effects: The semi-supervised learning framework ClusMatch proposed in the present invention includes two components: a prediction matching module and a graph matching module, which can simultaneously strengthen the prediction consistency at the instance level and the similarity consistency at the graph level. The prediction matching module is used to generate pseudo labels to strengthen the prediction consistency at the instance level; a simple dynamic threshold adjustment strategy is introduced in this module to provide a lower threshold at the beginning of training, and then gradually increase the threshold as the training process progresses, so as to adaptively screen sufficient and high-quality samples according to the training process, which can greatly improve the training efficiency. The graph matching module can utilize the relationship between data points to generate target graphs and comparison graphs based on the clustering results of low-dimensional embeddings to construct graph matching targets, so that the local distribution relationship of data points in the feature space remains consistent under different disturbances. In addition, the introduction of pseudo labels when constructing the target graph can improve the accuracy of the target and promote graph learning.

[0020] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The exemplary embodiments of the present disclosure and the description thereof are used to explain the present disclosure but do not constitute a limitation of the present disclosure.

[0022] Figure 1 A main flow chart of a semi-supervised image classification method based on pseudo-label and embedded cluster matching provided by an embodiment of the present invention; Figure 2 A schematic diagram of the ClusMatch framework provided by an embodiment of the present invention; Figure 3 A schematic diagram of dynamic threshold adjustment provided by an embodiment of the present invention; Figure 4A schematic diagram of the performance comparison of ClusMatch, FixMatch and FlexMatch in terms of confidence threshold provided by an embodiment of the present invention; Figure 5 A schematic diagram showing a performance comparison of ClusMatch, FixMatch and FlexMatch in terms of sampling rate provided by an embodiment of the present invention; Figure 6 A schematic diagram showing a performance comparison of ClusMatch, FixMatch and FlexMatch in terms of pseudo-label accuracy provided by an embodiment of the present invention; Figure 7 A schematic diagram showing the performance comparison of ClusMatch, CCSSL and CoMatch in terms of Top-1 accuracy provided in an embodiment of the present invention; Figure 8 A schematic diagram showing a performance comparison of ClusMatch, CCSSL and CoMatch in terms of graph loss provided by an embodiment of the present invention; Fig. 9 A schematic diagram of the performance comparison of ClusMatch, CCSSL and CoMatch in terms of overall loss provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0024] Technical term explanation (1) Pseudo-label-based semi-supervised learning: This is the simplest and most effective semi-supervised method. Its core idea is to use self-generated predictions as proxy labels for unlabeled data to expand the labeled training set. The model trained using this method cannot correct its own errors. Even if the threshold can filter out some low-confidence noise labels, high-confidence samples may also contain systematic errors. Such errors are accumulated continuously, and eventually confident but wrong pseudo-labels will appear on unlabeled sample points. This is confirmation bias.

[0025] (2) Consistency regularization: Based on the consistency assumption, the model should have the same or similar output for the same input before and after the perturbation. In other words, consistency regularization encourages similar samples to have similar predictions. Most current methods only perform consistency regularization in the probability space, while ignoring the consistency of the local relationship between sample points before and after the perturbation.

[0026] (3) Graph-based semi-supervised learning: Data points are usually represented as nodes in a graph, and the edges connecting each pair of nodes reflect their similarity or distance. The popular hypothesis holds that high-dimensional data can be represented as a low-dimensional manifold. Based on this hypothesis, two types of semi-supervised graph learning methods are derived. One type is label propagation, which propagates labels from labeled data to unlabeled data based on the popular structure of the data and the similarity of intermediate nodes. This type of method relies too much on node quality, and if there is noise in the neighboring nodes, it may reduce the effect of graph learning. The other type is node embedding, which aims to learn a low-dimensional vector representation for each node. These low-dimensional embeddings can effectively capture the characteristics of the node and the structural information of the graph, which is more conducive to downstream tasks. However, this type of method is usually computationally complex for measuring the similarity between nodes.

[0027] Embodiment 1 like Figure 1 As shown, this embodiment discloses a semi-supervised image classification method based on pseudo-label and embedded cluster matching, comprising the following steps: S1: Obtain a labeled image and an unlabeled image; for the unlabeled image, perform weak enhancement and strong enhancement to obtain a weakly enhanced image and a strongly enhanced image; S2: For labeled images, the supervised loss is calculated based on the semi-supervised image classification model; S3: Input the enhanced unlabeled image into the semi-supervised image classification model, generate a dynamic threshold based on the model's learning effect on the category to filter out pseudo labels and calculate the unsupervised loss; S4: respectively mapping the weakly enhanced image and the strongly enhanced image to the embedding space for K-Means clustering, wherein the clustering results are respectively used to generate the target graph and the strongly enhanced embedded clustering graph, and calculate the graph matching loss; S5: Establish a total loss based on supervised loss, unsupervised loss and graph matching loss, and improve the performance of the semi-supervised image classification model by minimizing the total loss.

[0028] Next, combine Figure 2 , a semi-supervised image classification method based on pseudo-label and embedded cluster matching disclosed in this embodiment is described in detail.

[0029] This embodiment proposes a semi-supervised learning framework, ClusMatch, which can effectively solve the above problems. ClusMatch consists of a prediction matching module and a graph matching module. The former is designed to generate pseudo labels in the iteration process and provide classification predictions in the reasoning process. The latter uses pseudo labels and clustering results on the embedding set to construct an embedding graph, which reflects the similarity relationship between samples. By using a variable threshold to obtain the classification loss, it can not only speed up the model convergence, but also alleviate the confirmation bias. At the same time, a simple method for measuring sample similarity is defined, and graph learning is carried out based on this to achieve multi-level consistency regularization.

[0030] The ClusMatch architecture provided in this embodiment is shown in the following figure Figure 2 For the convenience of explanation, the basic setting of semi-supervised classification problem is introduced here.

[0031] For a C-class semi-supervised image classification task, we usually let Represents a batch of labeled data, where This is the bth picture . represents a batch of unlabeled data, where µ is a hyperparameter used to control the relative size of X and U. A convolutional neural network based encoder is used to extract feature r, and a fully connected classifier is used to generate predictions for images. General threshold-based semi-supervised learning methods, such as FixMatch, aim to obtain encoders and classifiers with good performance by optimizing a supervised loss on labeled samples. and an unsupervised loss on unlabeled samples to achieve.

[0032]

[0033] in, represents the cross entropy function, Indicates a weak augmentation, such as flipping or cropping.

[0034] To obtain the unsupervised loss, FixMatch applies a weak augmentation to the unlabeled data. and a strong enhancement , and then calculate the cross entropy between the weakly enhanced maximum probability prediction (i.e., pseudo-label) and the prediction of the strongly enhanced version. According to the consistency assumption, the pseudo-label should be similar or consistent with the strongly enhanced prediction, so there is no supervised loss is defined as follows:

[0035] in, and They are and abbreviation of; .symbol Represents the threshold, which is used to select samples with high confidence to train the model. In algorithms using fixed thresholds, it is usually set , which is a very high value. Although it can filter out most low-quality samples, it will also result in low data utilization, especially in the early stages of training, making the model training cycle very long.

[0036] 1. Prediction and matching module By mapping image features to probability space to obtain pseudo labels, and then minimizing the cross entropy between the pseudo labels and the predictions of the strongly enhanced version of the same sample to achieve prediction alignment, this alignment applies the consistency assumption at the level of a single sample, which can promote the encoder to extract features that are robust to perturbations. Therefore, the quality of pseudo labels is crucial to the model training effect. In order to effectively improve the quality of pseudo labels, this embodiment proposes a prediction matching module.

[0037] First, following FixMatch, this example supervises the loss Defined as the cross entropy between the true labels and the model predictions.

[0038] Afterwards, in order to obtain unsupervised loss, a predefined high threshold is introduced in FixMatch to filter out pseudo labels. Although this ensures that the pseudo labels are confident, it also leads to slower training speed.

[0039] Therefore, this embodiment introduces a dynamic threshold adjustment strategy to improve equation (2). This strategy associates the threshold with the learning effect of the model on each class, and can provide a lower threshold at the beginning of training, and then gradually increase the threshold as the training progresses. This not only ensures that the samples selected in each iteration are relatively confident to reduce confirmation bias, but also avoids discarding most samples at the beginning of training to ensure training efficiency.

[0040] The process of dynamic class threshold adjustment is as follows Figure 3 As shown in Figure 1, at time step t, the model first outputs a batch of classification predictions for unlabeled data, and then uses the most recently updated dynamic class threshold to filter out low-confidence samples (represented by gray dots). The learning effect of each class is estimated as the mean of the sum of the probabilities of samples belonging to that class and exceeding the dynamic class threshold. Finally, the learning effect of each class is maximized and normalized, and the fixed threshold is scaled to obtain the dynamic class threshold to be used in the next time step.

[0041] Specifically, this embodiment uses a heuristic method to quantify the learning effect of the model for each class under the current threshold condition, and then uses the quantized value to scale the fixed threshold to obtain a new threshold for a specific class. In this module, the mean of the sum of the probabilities of all pseudo labels that exceed the threshold and belong to class c on class c is used to represent the learning state of the model for class c, formalized as:

[0042] in, reflects the learning effect of the model on class c at time step t, Is the model for unlabeled samples The weakly enhanced prediction vector, It is an unlabeled sample The maximum class probability prediction of is the pseudo label; symbol Represents a fixed threshold, which is used to select samples with high confidence to train the model; represents the pseudo label belonging to class c; is the discriminant function; It is an unlabeled sample The probability of being in class c.

[0043] In order to smooth out misclassifications and extreme samples, and to more finely represent the model’s confidence in each class, thus leading to more accurate adjustments, this implementation The maximum normalization was performed and the normalized To scale the fixed threshold . Formally:

[0044] However, in this case, it may happen This may cause a large number of low-quality samples to participate in the training, especially in the early stage of training. Therefore, this embodiment adds a minimum limit to the threshold:

[0045] Among them, C represents the number of categories.

[0046] The unsupervised loss of ClusMatch calculated using this threshold can be expressed as:

[0047] It can be seen that the cross entropy loss of a sample is calculated only when the maximum predicted probability of the sample exceeds the dynamic threshold of its corresponding category. In the early stage of training, since the model has poor learning effect on each class, the dynamic threshold is relatively low, allowing more samples to participate in training and ensuring training efficiency; as the training progresses, the model's learning effect on each class gradually improves, and the dynamic threshold also gradually increases, thus ensuring that the samples selected in each iteration are relatively confident (that is, have a higher prediction probability).

[0048] 2. Graph Matching Module In order to further alleviate the confirmation bias, this embodiment also introduces graph learning. The clustering results on the low-dimensional embedding of image features are used to generate a target graph and a comparison graph to construct a graph matching target.

[0049] The affinity graph is used to represent the similarity relationship between samples in order to apply consistency regularization in the low-dimensional embedding space. According to the popular hypothesis, high-dimensional data is roughly located on a low-dimensional manifold, so this embodiment generalizes the consistency hypothesis to the embedding space. The consistency hypothesis requires that similar samples have similar outputs. This embodiment takes into account that the relative distribution of a sample point with other sample points in the embedding space and the relative distribution of its similar points with other sample points in the embedding space are similar.

[0050] Formally speaking, a batch of unlabeled data U has a strongly enhanced version and a weakly enhanced version Assumptions Representation sample and The relationship vector of the sample points in Representation sample and The relationship vector of the data points in . Then, and The distance between them should be small.

[0051] For the measurement of similarity between samples, traditional methods use cosine similarity, which has high computational complexity, or use vector inner product. These two methods focus on fine-grained information and are easily affected by boundary values.

[0052] This embodiment believes that focusing on the global distribution structure of samples in the embedding space and ignoring unimportant details on a balanced data set may produce better results. Therefore, this embodiment expects to have such a method, which is simple, emphasizes the category boundaries in the embedding space, and ensures the stability of the value when calculating the loss.

[0053] Based on the above expectations, considering that K-Means clustering perfectly meets the above requirements, the division of clusters not only reflects the intra-class clustering of similar samples, but also generates clear category boundaries, which helps to enhance the reliability of pseudo-labels. At the same time, clustering can suppress noise samples and reduce the negative impact of erroneous pseudo-labels on model training. In addition, since the number of categories C is known, this embodiment does not require additional exploration of the number of clusters k, and k is directly determined by C.

[0054] Specifically, this embodiment sets the number of clusters K=C. of On and originated from of K-Means clustering is performed on . A strongly enhanced relationship graph constructed based on the clustering results on Can be written as:

[0055] in, express For the matrix elements in , for two strongly enhanced samples belonging to the same cluster, this embodiment sets the edge between them to 1, and the rest to 0. That is, for any pair of strongly enhanced samples, the embedding vector and , if their data class labels are the same, the similarity between them is 1, otherwise it is 0.

[0056] In graph matching, this embodiment hopes to use weakly enhanced graph Guidance for the goal The ultimate goal of the generation of is to obtain a more discriminative encoder and a more robust and efficient mapping head. Therefore, the more accurate The more it can improve the effect of graph learning. Therefore, this embodiment combines cluster label and pseudo label information to construct the target matrix , formalized as:

[0057] in, express The matrix elements in It is a weakly enhanced sample The low-dimensional embedding vector of It is a weakly enhanced sample The low-dimensional embedding vector represents the embedding vector for any pair of weakly enhanced samples and , if their cluster labels are the same and their respective pseudo labels exceed their corresponding dynamic class thresholds, the similarity between them is 1, otherwise it is 0.

[0058] The loss of the graph matching module is calculated by minimizing the cross entropy loss between the two graphs in Equation (9) and Equation (10). Can be defined as:

[0059] because and The elements in are 0 or 1, so this embodiment selects binary cross entropy as H.

[0060] like Figure 2 As shown, in the ClusMatch framework of this embodiment, given a batch of unlabeled data, the weakly enhanced images enter a dynamic threshold-based predictive matching module to generate pseudo labels. Pseudo labels are not only used to calculate classification losses, but also combined with clustering on weakly enhanced embeddings to generate an accurate target graph. The graph matching module strengthens the consistency of the relationship between samples by minimizing the cross entropy between the target graph and the strongly enhanced embedding cluster graph.

[0061] Finally, the ClusMatch provided in this embodiment jointly optimizes a supervised loss , an unsupervised loss , and a graph matching loss The overall training goal can be written as:

[0062] in, and are all loss weight coefficients.

[0063] 3. Experiment CIFAR10 is a balanced dataset consisting of 60K 32×32 color images from 10 classes. It contains 50K training images and 10K test images. This example performs experiments in a class-balanced manner with 40, 250, and 4000 labels.

[0064] CIFAR100 consists of 60K 32×32 color images, including 100 classes. Each class has 500 images for training and 100 images for testing. This example performs experiments in a class-balanced manner with 400, 2500, and 10000 labels.

[0065] STL10 is derived from ImageNet and consists of 100K unlabeled images and 13K labeled images. The labeled data contains 10 classes, each with 500 training images and 800 test images. All images in STL10 are 96×96 color images.

[0066] ImageNet-1k is a subset of the ImageNet dataset. It contains 1k categories, each with approximately 1.3k training images and 50 validation images.

[0067] Baseline methods: This example considers the methods based on pseudo-labels and consistency regularization in the past five years, including MixMatch, ReMixMatch, UDA, FixMatch, FlexMatch, Dash, FreeMatch, SimMatch. In addition, several state-of-the-art methods involving graph regularization, such as CoMatch, CCSSL, and CHMatch, are also included in the comparison. It should be stated that the method of this example is based on CCSSL. This example integrates the code into SemiCLS.

[0068] Experimental settings: For CIFAR10, this example uses WideResNet-28-2, and for CIFAR100, WideResNet-28-8. For STL10, this example uses ResNet-18. A 2-layer MLP mapping head is used to obtain a 64-dimensional embedding. The model is optimized using SGD with a momentum of 0.9 and a weight decay of 0.001. All methods use an initial learning rate of 0.03 and a cosine learning rate decay strategy, and train for 1024 rounds. In all experiments on these three datasets, this example uniformly sets , , , , K , , where µ is the sample coefficient, B is the batch size, is the unsupervised loss coefficient, is the graph loss coefficient, and K is the parameter of the k-means algorithm.

[0069] For ImageNet-1k, 1% or 10% of labeled images are randomly extracted in a class-balanced manner, and the rest of the images are unlabeled. ResNet-50 is used as the encoder, and a two-layer MLP is used as the projection head to obtain a 128-dimensional embedding. Except for B=160 and τ=0.6, other hyperparameters on ImageNet-1k are consistent with those on the previous three datasets.

[0070] As a specific implementation method, Figure 4 , Figure 5 , Figure 6 This figure shows the performance comparison of ClusMatch and two threshold-based methods FixMatch and FlexMatch on CIFAR10-40. They are the curves of the change of confidence threshold, sampling rate and pseudo-label accuracy during training. Figure 4 and Figure 5 It can be seen that ClusMatch shows superior threshold and sampling rate, which shows that the dynamic threshold adjustment strategy is effective. Figure 6 As shown in Figure 2, after 100k steps, the pseudo-label accuracy of the three methods fluctuates in a small range of 0.88 to 0.9.

[0071] As a specific implementation method, Figure 7 , Figure 8 , Fig. 9 The performance of ClusMatch and two graph-based semi-supervised methods CCSSL and CoMatch on CIFAR100-400 is shown in the figure. They are the curves showing the change of Top-1 accuracy, graph loss and overall loss during training. Figure 7 As shown in , due to the different ways of calculating graph loss, ClusMatch shows higher graph loss compared with CCSSL and CoMatch. Figure 8 and Fig. 9 As shown, ClusMatch achieves the highest accuracy and the lowest overall loss, which proves the effectiveness of the graph matching module described in this embodiment.

[0072] Table 1 below shows the error rates of ClusMatch and other benchmark methods provided in this embodiment under different settings of the number of labels for three small datasets: CIFAR10, CIFAR100, and STL10. ClusMatch achieved the best performance under four settings and was close to the best method in the other three settings. Even when the labels are extremely sparse, ClusMatch still maintains a very stable effect, which shows that ClusMatch is very suitable for processing classification tasks on small datasets.

[0073]

[0074]

[0075] Table 2 below shows the error rate of ClusMatch on ImageNet-1K provided by this embodiment. Under the setting of 1% and 10% of the number of markers, the performance of ClusMatch generally surpasses CoMatch, which shows that ClusMatch can handle classification tasks on large datasets.

[0076]

[0077] This embodiment uses WideResNet-28-2 to conduct experiments on the CIFAR10 dataset with 40 labeled samples to verify the effectiveness of each component. The results are shown in Table 3. According to the error rate comparison results of the 5th and 6th rows of the table, the dynamic threshold adjustment scheme proposed in this embodiment is significantly better than the predefined threshold. The 1st and 2nd rows are both fixed thresholds combined with the graph matching module of this embodiment. The difference is whether pseudo-label information is introduced to fine-tune the target graph; among them, the error rate of the 2nd row is lower than that of the 1st row, which shows that it is necessary to use pseudo-labels to fine-tune the target graph. The 3rd row represents the combination of the dynamic threshold adjustment scheme and the graph matching module. The 4th row is ClusMatch proposed in this embodiment, which introduces pseudo-labels on the basis of the 3rd row. Their comparison also illustrates the importance of fine-tuning the target graph with pseudo-labels. The error rate of the 4th row is lower than that of the 5th row, which illustrates the role of the graph matching module of this embodiment in promoting correct classification.

[0078]

[0079] In order to solve the challenges in semi-supervised image classification, this embodiment proposes a ClusMatch semi-supervised learning framework, which integrates two modules: prediction matching and graph matching. ClusMatch not only strengthens the prediction consistency at the instance level, but also ensures the similarity consistency at the graph level. The prediction matching module introduces a dynamic threshold adjustment strategy, sets a lower threshold at the beginning of training, and gradually increases it to adapt to the training process, effectively alleviating the confirmation bias and significantly improving the training efficiency. The graph matching module cleverly uses the relationship between data points to construct the target graph and the comparison graph based on the clustering results of low-dimensional embedding, ensuring the consistency of the local distribution relationship of data points in the feature space under different disturbances. In addition, by introducing pseudo-labels, the accuracy of the target graph is further improved, which promotes the effect of graph learning. The introduction of the ClusMatch framework not only provides a new solution for semi-supervised image classification, but also shows significant beneficial effects in improving classification accuracy and training efficiency.

[0080] Embodiment 2 This embodiment provides a semi-supervised image classification system based on pseudo labels and embedded cluster matching, including: A data enhancement module is used to obtain a labeled image and an unlabeled image; for the unlabeled image, weak enhancement and strong enhancement are performed to obtain a weakly enhanced image and a strongly enhanced image; The prediction and matching module is used to calculate the supervised loss for the labeled image based on the semi-supervised image classification model; the enhanced unlabeled image is input into the semi-supervised image classification model, and a dynamic threshold is generated based on the model's learning effect on the category to filter out pseudo labels and calculate the unsupervised loss; A graph matching module is used to map the weakly enhanced image and the strongly enhanced image to the embedding space for K-Means clustering, and the clustering results are used to generate the target graph and the strongly enhanced embedded clustering graph, and calculate the graph matching loss; The loss calculation module is used to establish the total loss based on the supervised loss, unsupervised loss and graph matching loss, and improve the performance of the semi-supervised image classification model by minimizing the total loss.

[0081] Embodiment 3 This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps in the semi-supervised image classification method based on pseudo-labels and embedded cluster matching as described in the first embodiment above are implemented.

[0082] Embodiment 4 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of a semi-supervised image classification method based on pseudo-labels and embedded cluster matching as described in the first embodiment are implemented.

[0083] The steps or modules involved in the above embodiments 2 to 4 correspond to those in embodiment 1. For the specific implementation, please refer to the relevant description of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0084] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A semi-supervised image classification method based on pseudo-label and embedded cluster matching, characterized in that: include: Obtain labeled images and unlabeled images; For unlabeled images, weak enhancement and strong enhancement are performed to obtain weakly enhanced images and strongly enhanced images; For labeled images, the supervised loss is calculated based on the semi-supervised image classification model; The enhanced unlabeled images are input into the semi-supervised image classification model, and a dynamic threshold is generated based on the model's learning effect on the category to filter out pseudo labels and calculate the unsupervised loss; The weakly enhanced image and the strongly enhanced image are respectively mapped to the embedding space for K-Means clustering, and the clustering results are respectively used to generate the target graph and the strongly enhanced embedded clustering graph, and the graph matching loss is calculated; A total loss is established based on supervised loss, unsupervised loss and graph matching loss, and the performance of the semi-supervised image classification model is improved by minimizing the total loss.

2. A semi-supervised image classification method based on pseudo-label and embedded cluster matching as claimed in claim 1, characterized in that: The method of calculating the supervised loss for the labeled image based on the semi-supervised image classification model specifically includes: weakly enhancing the labeled image, and calculating the supervised loss between the prediction of the input weakly enhanced labeled image and its corresponding true label based on the semi-supervised image classification model.

3. A semi-supervised image classification method based on pseudo-label and embedded cluster matching as claimed in claim 1, characterized in that: The dynamic threshold is generated based on the learning effect of the model on the category to filter the pseudo-labels, specifically including: In this round, the unlabeled images are input into the semi-supervised image classification model to obtain the predicted classification labels; Using the dynamic threshold updated in the previous round, the predicted classification labels are screened to obtain the pseudo labels for this round; Based on the pseudo labels and predicted classification labels of this round, evaluate the learning effect of the model on the category in this round; The learning effect is normalized and used to update the dynamic threshold for use in the next round.

4. A semi-supervised image classification method based on pseudo-label and embedded cluster matching as claimed in claim 3, characterized in that: The method uses the dynamic threshold updated in the previous round to filter the predicted classification labels to obtain the pseudo labels of this round, which specifically includes: in, is the discriminant function, is an unlabeled image The maximum class probability prediction of , that is, the pseudo label of this round; is the dynamic threshold updated in the previous round; the screening process associates the threshold with the learning effect of the model on each class, which is used to provide a lower threshold in the early stage of training and gradually increase the threshold as the training process progresses.

5. A semi-supervised image classification method based on pseudo-label and embedded cluster matching as claimed in claim 3, characterized in that: The learning effect of the model on the category is evaluated based on the pseudo labels and predicted classification labels of this round, specifically including: in, It reflects the learning effect of the model on class c at time step t; is the model for unlabeled images The weakly enhanced prediction vector, is an unlabeled image The maximum class probability prediction of Represents a fixed threshold, which is used to select samples with high confidence to train the model; represents the pseudo label belonging to class c; is the discriminant function; It is an unlabeled sample The probability of being in class c.

6. A semi-supervised image classification method based on pseudo-label and embedded cluster matching as claimed in claim 3, characterized in that: The learning effect is normalized and used to update the dynamic threshold for use in the next round, specifically including: Among them, Perform maximum normalization to obtain ; Among them, the normalized category learning effect To scale the predefined threshold , generates a dynamic threshold for each category at each time step t; in order to avoid generating extremely low This results in many noise samples being involved in training. Further adjustments are made as follows: Among them, C represents the number of categories.

7. A semi-supervised image classification method based on pseudo-label and embedded cluster matching as claimed in claim 1, characterized in that: The weakly enhanced image and the strongly enhanced image are respectively mapped to the embedding space for K-Means clustering, and the clustering results are respectively used to generate the target graph and the strongly enhanced embedded clustering graph, and calculate the graph matching loss, specifically including: The weakly enhanced image and the strongly enhanced image are mapped to the embedding space respectively, and K-Means clustering is performed respectively; the clustering result of the weakly enhanced image is used to generate the target graph, and the clustering result of the strongly enhanced image is used to generate the strongly enhanced embedded clustering graph, and the graph matching loss between the target graph and the strongly enhanced embedded clustering graph is calculated.

8. A semi-supervised image classification system based on pseudo-label and embedded cluster matching, characterized in that: include: Data augmentation module, used to obtain labeled images and unlabeled images; For unlabeled images, weak enhancement and strong enhancement are performed to obtain weakly enhanced images and strongly enhanced images; A prediction matching module is used to calculate the supervision loss for the labeled image based on the semi-supervised image classification model; The enhanced unlabeled images are input into the semi-supervised image classification model, and a dynamic threshold is generated based on the model's learning effect on the category to filter out pseudo labels and calculate the unsupervised loss; A graph matching module is used to map the weakly enhanced image and the strongly enhanced image to the embedding space for K-Means clustering, and the clustering results are used to generate the target graph and the strongly enhanced embedded clustering graph, and calculate the graph matching loss; The loss calculation module is used to establish the total loss based on the supervised loss, unsupervised loss and graph matching loss, and improve the performance of the semi-supervised image classification model by minimizing the total loss.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in a semi-supervised image classification method based on pseudo-labels and embedded cluster matching as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the semi-supervised image classification method based on pseudo-label and embedded cluster matching as described in any one of claims 1-7 are implemented.

Images