Semi-supervised small sample image classification method and system in cross-domain scene

Abstract

The invention relates to a semi-supervised small sample image classification method and system in a cross-domain scene, and relates to the technical field of deep learning and machine learning, and the system comprises a pre-training feature extractor and a self-supervised encoder; using a feature extractor and a self-supervised encoder to respectively obtain initial features and internal features, and fusing the initial features and the internal features to obtain fused features; generating initial pseudo labels through a clustering algorithm, and screening the initial pseudo labels according to a clustering validity index to obtain high-quality pseudo labels; constructing a mixed loss function, and updating the feature extractor on the mixed training sample by using the mixed loss function; and inputting a to-be-classified sample into the updated feature extractor, and outputting the category of the to-be-classified sample. According to the method, the technical problems of unstable model training and weak generalization ability caused by low pseudo label quality when small sample learning tasks with significant domain differences are processed in the prior art are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning and machine learning, and particularly relates to a semi-supervised small sample image classification method and system in a cross-domain scene. BACKGROUND

[0002] In recent years, deep learning has made remarkable achievements in image classification, and some models have performed close to or even better than human level in specific fields. However, this outstanding performance is highly dependent on a large amount of high-quality labeled data, which is difficult to achieve in real-world scenarios such as medical image analysis, industrial defect detection, and autonomous driving. The cost of obtaining labeled data is high, and the cycle is long, which has become one of the main obstacles to the landing of artificial intelligence technology.

[0003] To alleviate the problem of data scarcity, small sample learning has emerged, which aims to train a model with good generalization ability under the condition of only a small amount of labeled samples. Cross-domain scene refers to the significant difference in data distribution between the source domain of model training and the target domain of actual application, which will lead to a significant decline in model performance. Semi-supervised learning aims to use a large number of unlabeled samples to assist model training, and the pseudo-label strategy is a common method to generate labels for unlabeled samples to enhance training effectiveness.

[0004] Existing small sample learning methods mainly fall into two categories: one is the metric learning-based method, such as Siamese Networks and Prototypical Networks, which learns a discriminative feature space to make same-class samples close and different-class samples far apart; the other is the optimization-based meta-learning method, such as Model-Agnostic Meta-Learning (MAML), which learns the initial model parameters to achieve rapid adaptation to new tasks. However, existing small sample learning methods perform poorly in cross-domain scenarios, severely affecting the performance of cross-domain small sample image classification.

[0005] Although the above methods have achieved good performance, they still face many challenges in complex real-world applications, especially in cross-domain scenarios. To address the cross-domain problem, existing technologies align the feature distributions of the source and target domains by introducing a feature transformation network, or enhance the inter-domain correlation using an attention mechanism. However, these methods generally rely on complex network structure design and cannot effectively apply to semi-supervised scenarios where a large number of unlabeled samples exist in the target domain. To more effectively utilize target domain information, researchers have begun to incorporate semi-supervised learning ideas. In semi-supervised learning, the pseudo-label strategy combines consistency regularization and high-confidence screening mechanisms, which have shown significant effectiveness in the same-domain task.

[0006] However, there are limitations when directly applying existing pseudo-label strategies to this specific and more difficult scenario of cross-domain few-shot learning. Existing methods usually assume that the initial predictions of the model have a certain reliability, but in the scenario of huge domain difference, the initial prediction performance of the source domain model in the target domain is weak, resulting in low confidence of the pseudo-labels generated by it. At this time, if the model sets a high threshold, it will filter out a large number of unlabeled samples, which cannot be effectively utilized; setting a low threshold will introduce a large number of false pseudo-labels, which will interfere with the learning direction of the model, affect the stability of the model training process, and cause the performance of the model to decline. Therefore, the existing technology lacks an effective method to evaluate the quality of pseudo-labels under huge domain difference, and a mechanism to optimize the accuracy of pseudo-label generation from the feature level, which cannot effectively utilize the unlabeled data in the target domain. SUMMARY

[0007] The present application proposes a semi-supervised few-shot image classification method and system in a cross-domain scenario, aiming to solve the technical problems mentioned in the background art.

[0008] To achieve the above purpose, the present application adopts the following technical solutions:

[0009] In a first aspect, the present application provides a semi-supervised few-shot image classification method in a cross-domain scenario, comprising:

[0010] Based on labeled samples, a meta-learning model is used to pretrain a feature extractor;

[0011] Based on unlabeled samples, a self-supervised contrastive learning is used to pretrain a self-supervised encoder;

[0012] The unlabeled sample data is input into the feature extractor and the self-supervised encoder respectively to obtain initial features and intrinsic features respectively, and the initial features and the intrinsic features are fused to obtain fused features; based on the fused features, an initial pseudo-label is generated through a clustering algorithm, and the initial pseudo-label is filtered according to a clustering effectiveness index to obtain a high-quality pseudo-label;

[0013] The labeled samples and the high-quality pseudo-labels are combined to form a mixed training sample set, and a mixed loss function is constructed, and the feature extractor is updated on the mixed training samples using the mixed loss function;

[0014] The sample to be classified is input into the updated feature extractor, and the class of the sample to be classified is output.

[0015] Further, the pre-training feature extractor specifically comprises: firstly, constructing a training task comprising a support set and a query set; then calculating a feature prototype of each category through samples in the support set, and calculating the Euclidean distance between the feature vector of a sample in the query set and the feature prototype of each category to obtain the probability of the sample belonging to each category through normalization; then constructing a combined loss function, and updating the parameters of the feature extractor by minimizing the combined loss function.

[0016] Further, the feature prototype of each category is calculated as follows:

[0017] wherein, c k represents the feature prototype of the kth category in the support set, S k represents the sample set of the kth category in the support set, |S k represents the number of samples in the set S k , xi i represents the ith sample in the support set, yi i represents the label corresponding to the ith sample, f φ (xi i ) represents the feature vector extracted by the feature extractor on xi i .

[0018] The calculation formula for obtaining the probability of the sample belonging to each category is as follows:

[0019] wherein, p φ (y=k | x) represents the probability of the sample belonging to the kth category, d(f φ (x), c k ) represents the Euclidean distance between the feature vector of the sample x and the feature prototype of the kth category, d() represents a distance measurement function, and c j represents the sum of the denominators of the jth category prototype.

[0020] Further, the combined loss function is represented as:

[0021] L P = L ce + L d + L r ; wherein, L P represents the combined loss function constructed, L ce represents the classification cross-entropy loss, L d represents the loss term for adjusting the average distance between feature prototypes, and L r represents a regularization term.

[0022] The classification cross-entropy loss is represented as: The loss term of adjusting the inter-prototype average distance of features is expressed as: Wherein, represents the difference between the actual inter-prototype average distance of features and the expected distance.

[0023] Further technical solutions, the clustering effectiveness index is the silhouette coefficient, and the calculation formula of the silhouette coefficient is:

[0024] Wherein, s(i) represents the silhouette coefficient of the initial pseudo-labeled sample i, a(i) represents the average distance between sample i and other points in the cluster where it is located, and b(i) represents the average distance between sample i and all points in the nearest neighbor cluster.

[0025] The confidence threshold is preset, when the silhouette coefficient is greater than the confidence threshold, the initial pseudo-labeled sample is regarded as a high-confidence sample and is retained, and then a high-quality pseudo-label is obtained, otherwise it is rejected.

[0026] Further technical solutions, the mixed loss function includes confidence weighted cross-entropy loss, contrastive loss, inverse cross-entropy loss and sparse regularization; and the parameters of the feature extractor are updated by minimizing the mixed loss function using Adam optimizer.

[0027] Further technical solutions, the confidence weighted cross-entropy loss is expressed as:

[0028] Wherein, D new represents the mixed training sample set, |D new | represents the total number of samples in the mixed training sample set, w i represents the confidence weight, m i represents the sample in the mixed training sample set, n i represents m i corresponding label, logp φ (n=m i |n i ) represents the probability that sample m i belongs to label n i .

[0029] The contrastive loss is expressed as:

[0030] Wherein, P(m i ) represents the sample set in D new belonging to the same category as m i , A(m i ) represents all samples including m i , and parameter ξ takes value between 0 and 1.

[0031] The inverse cross-entropy loss is expressed as:

[0032] wherein q φ (y=k|x) represents the probability of the sample x belonging to the class k, q φ (y=k|x) represents the one-hot distribution represented by the pseudo label of the sample;

[0033] The sparse regularization is represented as: L sr = lambda * || omega || p ; wherein omega represents a weight parameter, and || || p represents a p - norm, and lambda represents a hyperparameter of regularization strength.

[0034] In a second aspect, the present application provides a semi-supervised small sample image classification system in a cross-domain scene, comprising the following modules:

[0035] The feature extractor pre-training model is configured to pre-train the feature extractor based on the labeled samples using a meta-learning model;

[0036] The self-supervised encoder pre-training model is configured to pre-train the self-supervised encoder based on the unlabeled samples using self-supervised contrastive learning;

[0037] The pseudo label generation module is configured to input the unlabeled sample data into the feature extractor and the self-supervised encoder respectively to obtain initial features and intrinsic features respectively, and fuse the initial features and the intrinsic features to obtain fused features; generate initial pseudo labels based on the fused features through a clustering algorithm, and filter the initial pseudo labels according to a clustering effectiveness index to obtain high-quality pseudo labels;

[0038] The feature extractor updating module is configured to combine and mix the labeled samples and the high-quality pseudo labels to obtain a mixed training sample set, construct a mixed loss function, and use the mixed loss function to update the feature extractor on the mixed training sample;

[0039] The classification module is configured to input the sample to be classified into the updated feature extractor to output the class of the sample to be classified.

[0040] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0041] The present application extracts the intrinsic features of the unlabeled data in the target domain through self-supervised learning, and then fuses them with the initial features extracted by the feature extractor, effectively capturing the characteristics of the target domain data and reducing the feature distribution difference between the source domain and the target domain. In multiple cross-domain tasks, the accuracy of the present application is more than 20% higher than that of the prior art, showing strong cross-domain small sample learning ability, making small sample learning perform better in a cross-domain scene, and effectively improving the performance of cross-domain small sample image classification.

[0042] The present application adopts a clustering effectiveness index (a profile coefficient) as a pseudo-label quality quantification index, provides a reliable quantitative evaluation standard for pseudo-label quality, and screens out high-quality pseudo-labels by setting a threshold, ensures the reliability of supervision information when fine-tuning the model by eliminating low-confidence samples, reduces training fluctuations caused by pseudo-label noise, and significantly enhances the stability of the model training process. BRIEF DESCRIPTION OF DRAWINGS

[0043] The drawings constituting a part of this specification are used to provide further understanding of the present application, the illustrative embodiments of the present application and the description thereof serve to explain the present application, and do not constitute a limitation on the present application.

[0044] Figure 1 is a flowchart of a semi-supervised small sample image classification method in a cross-domain scene according to the present application;

[0045] Figure 2 is a sample image schematic diagram of part of the data set used in the present application, wherein (a) is the Omniglot data set; (b) is the FC100 data set; (c) is the miniImageNet data set, and (d) is the CUB data set. DETAILED DESCRIPTION

[0046] Embodiment 1

[0047] It should be pointed out that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the present application belongs.

[0048] In order to achieve the above-mentioned purpose, the present embodiment provides a semi-supervised small sample image classification method in a cross-domain scene, as shown in Figure 1 The specific steps are as follows:

[0049] S1: Based on the labeled source domain data set D source , D source is a labeled sample, a meta-learning model is used to pre-train a feature extractor, so that it learns to have a preliminary generalization performance.

[0050] In step S1, a prototypical network is selected as a meta-learning model, and pre-training of a feature extractor specifically includes: first, a training task including a support set and a query set is constructed, that is, the training task construction follows an "N-way K-shot" strategy, in each training task, N-way represents that N classes are randomly selected from the source domain dataset, and K-shot represents that K samples are randomly sampled from each class as a support set. That is, in each training task, a specified number of samples are randomly sampled from the source domain dataset D source according to the N-way K-shot setting, to respectively form a support set S and a query set Q. Therefore, the support set S of each training task contains N*K samples, and the query set Q is composed of additional samples under the same N classes, for evaluating the generalization ability of the model on the task.

[0051] Then, the feature prototype of each class is calculated through the samples in the support set S, that is, the mean of all samples of the class in the feature space, and the Euclidean distance between the feature vector of the sample in the query set Q and the feature prototype of each class is calculated, and the probability that the sample belongs to each class is obtained by normalization, specifically:

[0052] The calculation method of the feature prototype of the class is as follows:

[0053] wherein, c k represents the prototype of the kth class in the support set, S k represents the sample set of the kth class in the support set, |S k | represents the number of samples in the set S k , x i represents a vector composed of the features of the ith sample in the support set, y i represents the label corresponding to the ith sample, f1(x i ) represents the feature vector extracted by the feature extractor on x i .

[0054] For the sample x q in the query set Q, the Euclidean distance between its feature vector and the feature prototype c k of each class is calculated, and the Softmax function is applied for normalization, so that the calculation formula of the probability that the sample belongs to each class is:

[0055]

[0056] wherein, p φ (y=k| x) represents the probability that the sample belongs to the class k, d(f φ (x), c krepresents the distance between the feature vector of the sample x and the prototype of the k-th class, d(·) represents the distance metric function, and the Euclidean distance is used by default, c j represents the sum of the denominators of the prototypes of the j-th class, and the sum is taken over all classes.

[0057] Finally, a combined loss function is constructed, and the parameters of the feature extractor are updated by minimizing the combined loss function, specifically: the combined loss function is represented as: L P = L ce + L d + L r ; wherein, L P represents the combined loss function constructed, L ce represents the classification cross-entropy loss, L d represents the loss term for adjusting the average distance between feature prototypes, L r represents the regularization term.

[0058] The classification cross-entropy loss is represented as: Supervised learning is used in classification tasks, and the network is trained by minimizing the classification cross-entropy loss, which can make the model learn more accurate classification boundaries and improve the performance of the classification task; the regularization term L r is used to control the complexity of the model to prevent overfitting on limited data.

[0059] The purpose of designing the loss term L d for adjusting the average distance between feature prototypes is to maintain a suitable degree of separation between feature prototypes of different classes in the feature space and enhance the distinguishability of the features, specifically: L d takes a preset hyperparameter threshold τ (representing the expected average distance between class prototypes) as a benchmark and compares it with the actual average prototype distance MeanDistance generated by the current model, wherein MeanDistance is calculated by the mean of the squared Euclidean distances between all pairs of feature prototypes (c i ,c j ):

[0060] wherein, N represents the number of classes.

[0061] The loss term for adjusting the average distance between feature prototypes is represented as: wherein, represents the difference between the actual average distance between feature prototypes and the expected distance, and

[0062] S2: Pre-training a self-supervised encoder f based on the unlabeled target domain dataset D U,target , D U,target is an unlabeled sample, and a self-supervised contrast learning method SimCLR is used to pre-train the self-supervised encoder fsimCLR The self-supervised encoder f simCLR Taking the target domain image (the image to be classified) as input, a contrastive learning method is used to learn and output a high-dimensional embedding vector to represent its structured features. The core function of this encoder is to extract discriminative features from the target domain. These features reflect the inherent distribution characteristics of the target domain data, which helps in subsequent pseudo-label generation and cluster analysis, thereby improving the reliability of pseudo-labels.

[0063] In step S2, the pre-trained self-supervised encoder is in D U,target The above method employs SimCLR as a self-supervised contrastive learning approach. Each sample undergoes two different random data augmentations to form a positive sample pair, while the remaining two different samples constitute a negative sample pair. The model learns its representation by maximizing the consistency of the positive sample pairs in the feature space, and its loss function can be expressed as:

[0064]

[0065] Where, sim(z) i ,z j ) represents two eigenvectors z i and z j The cosine similarity is defined as the dot product of two vectors divided by the product of their respective Euclidean norms, i.e.:

[0066]

[0067] Cosine similarity measures the similarity between two vectors along their direction, ranging from -1 to 1. A larger value indicates that the directions are consistent and the similarity is higher. ξ > 0 is a hyperparameter; a smaller ξ value makes the model more sensitive to differences in similarity, suppressing similarity between negative sample pairs and promoting similarity between positive sample pairs, thus making the learned features more discriminative. Conversely, a larger ξ value results in a smoother similarity distribution, weakening the model's ability to distinguish between positive and negative samples. The parameter ξ typically takes values ​​between (0, 1], with common settings such as 0.07 or 0.1. This step makes f... simCLR It can extract deeper features that are more discriminative to target domain data.

[0068] S3: Input the unlabeled sample data into the feature extractor and the self-supervised encoder respectively to obtain the initial features and intrinsic features. Fuse the initial features and intrinsic features to obtain the fused features. Generate initial pseudo-labels based on the fused features using a clustering algorithm. Then, filter the initial pseudo-labels according to the clustering effectiveness index to obtain high-quality pseudo-labels.

[0069] In step S3, for any sample x in the target domain u (Unlabeled sample data), its initial features fφ (x u ) and intrinsic characteristics f simCLR (x u The features are stitched together along the dimensions to form a fused feature f. uesd (x u Using the fusion features of all samples as input, the K-means clustering algorithm is run to assign each sample to the nearest cluster center, and its class label is the initial pseudo-label of the sample.

[0070] The initial pseudo-labels are screened based on a clustering effectiveness index to obtain high-quality pseudo-labels. Specifically, the clustering effectiveness index is the silhouette coefficient. For each sample with an initial pseudo-label, the formula for calculating its silhouette coefficient s(i) is as follows:

[0071]

[0072] Where s(i) represents the silhouette coefficient of sample i with initial pseudo-label, a(i) represents the average distance between sample i and other points in its cluster, and b(i) represents the average distance between sample i and all points in its nearest neighbor cluster. A confidence threshold is preset. When the silhouette coefficient s(i) is greater than the confidence threshold, the sample with initial pseudo-label is regarded as a high-confidence sample and is retained, thus obtaining a high-quality pseudo-label. Otherwise, it is discarded.

[0073] S4: Merge the labeled samples with high-quality pseudo-labels into a mixed training sample set, construct a mixed loss function, and use the mixed loss function to update the feature extractor on the mixed training samples.

[0074] In step S4, in the mixed training set D new Fine-tuning of the model was performed on the training set D. new By D source The model is composed of all labeled samples and high-quality pseudo-labels. The goal of model fine-tuning is to synergistically optimize its ability to distinguish between different class boundaries and to maintain consistent representation of intra-class features.

[0075] Using the constructed hybrid loss function L mix Fine-tune the model, using the mixture loss function L mix Including confidence-weighted cross-entropy loss L CE-conf , contrast loss L Contrastive Inverse cross-entropy loss L rce Sparse Regularization L sr Furthermore, the Adam optimizer is used to minimize the mixed loss function L. mix Update the parameters of the feature extractor.

[0076] Confidence-weighted cross-entropy loss L CE-conf: Targeting D new Each sample (m) i ,n i The confidence-weighted cross-entropy loss is calculated as follows: Among them, D new Denotes the mixed training sample set, |D new | represents the total number of samples in the mixed training sample set, w i m represents the confidence weight. i n represents the samples in the mixed training sample set. i m i The corresponding tag, p φ (n=m i |n i ) represents sample m i Belongs to label n i The probability of.

[0077] Comparison loss L Contrastive This loss learns more discriminative representations at the feature level, aiming to bring samples with the same label (true label or high-confidence pseudo label) closer together in the feature space, while pushing away samples with different labels. The contrastive loss is expressed as:

[0078]

[0079] in, z p z a Both are feature representations of the samples, P(m) i ) represents D new In and m i A(m) is a collection of samples belonging to the same category. i ) indicates that it includes m i For all samples, including ζ, the loss L sSL The parameters are the same.

[0080] Inverse cross-entropy loss L rce This loss function mitigates the negative impact of erroneous labels by reversing the roles of model prediction and label in standard cross-entropy. The inverse cross-entropy loss is expressed as:

[0081] Where, p φ (y=k|x) represents the probability that sample x belongs to class k, q φ (y=k|x) represents the one-hot distribution represented by the pseudo-label of the sample; this loss function predicts p by the model. φ As a weight, to penalize the label distribution q φ The logarithm of .

[0082] Sparse Regularization L srThe regularization term reduces the model complexity by penalizing the norm of the model parameters. The sparse regularization is represented as: L sr = λ · ‖ω‖ p ; where ω represents the weight parameters, ‖‖ p denotes the p-norm, and λ represents the hyperparameter of the regularization strength.

[0083] The sparse regularization L sr is combined with the confidence-weighted cross-entropy loss L , the contrastive loss L Contrastive , and the inverse cross-entropy loss L rce to obtain the final hybrid loss function L mix used for model fine-tuning:

[0084] L mix = L CE-conf + L Contrastive + L rce + L sr ;

[0085] The Adam optimizer is used to minimize L mix to update the parameters of the feature extractor f φ . The initial learning rate of the Adam optimizer is set to 0.001 at the beginning of training. In this embodiment, the dynamic learning rate adjustment strategy of ReduceLROnPlateau is also used. According to this strategy, if the validation performance of the model fails to achieve significant improvement within 10 consecutive iteration cycles, the optimizer will automatically trigger the decay of the learning rate.

[0086] S5: inputting the sample to be classified into the updated feature extractor, and outputting the class of the sample to be classified.

[0087] In step S5, the class of the sample to be classified is predicted and output by calculating the distance between the feature vector of the sample to be classified and the feature prototype of each class in the feature space.

[0088] To verify the performance of the method provided in this embodiment in the cross-domain semi-supervised few-shot learning task, four standard data sets widely used in the field of few-shot learning, Omniglot, miniImageNet, FC100 and CUB-200, are selected. These data sets differ in data features, class numbers and domain characteristics, so that the performance and generalization ability of the method can be verified from multiple dimensions. The experimental results will be compared with the prototype network method in the prior art. The core information of these data sets has been listed in Table 1 below, and some sample images are shown in Figure 2 .

[0089] Table 1 Core information of experimental data sets

[0090]

[0091]

[0092] Omniglot: This dataset contains 1,623 categories of handwritten characters from 50 different alphabets, with 20 samples for each category.

[0093] miniImageNet: As a subset of ImageNet, it contains 100 categories with 600 color images per category and is one of the benchmarks for few-shot classification.

[0094] FC100: Derived from CIFAR100, it also contains 100 categories, with 600 images per category, and is characterized by its lower image resolution.

[0095] CUB-200-2011 (CUB): A fine-grained bird image dataset containing 11,788 images of 200 bird species. It is often used to test the classification ability of models under subtle feature differences.

[0096] To verify the effectiveness of the method (SFL) in this embodiment, the ProtoNet network was selected as the baseline comparison method. ProtoNet is a few-shot learning method based on metric learning, which classifies data by calculating the distance between the query sample and each class's prototype. The comparative experimental results between the method in this embodiment and the proposed method will be used to illustrate the effectiveness of this invention in feature learning and pseudo-label utilization. All experiments were conducted in the same hardware and software environment, with specific configurations shown in Table 2.

[0097] Table 2 Experimental Environment Configuration

[0098]

[0099] To verify the effectiveness of the method in this embodiment, two commonly used feature extractors were selected as the backbone network:

[0100] (1) ConvNet4: This network is a shallow convolutional network and is a commonly used benchmark model in few-shot learning research. Its architecture consists of four convolutional modules, each consisting of a 3x3 convolutional layer (equipped with 64 filters), a batch normalization layer, a ReLU non-linear activation function, and a 2x2 max pooling layer. After feature extraction and dimensionality reduction by the four modules, the final output feature map is flattened into a one-dimensional global feature vector for subsequent metric learning or classification.

[0101] (2) ResNet-12: This network is a deep neural network based on residual learning, which provides stronger feature extraction capabilities than ConvNet4. The network architecture consists of four residual blocks, each containing three convolutional layers, supplemented by batch normalization layers and ReLU activation functions. After processing all residual blocks, the network uses a global average pooling layer to convert the high-dimensional feature map into a one-dimensional global feature vector, which is used for subsequent tasks of measurement and classification.

[0102] This embodiment selects two standard few-shot learning tasks for validation: 5-way 1-shot and 5-way 5-shot. In the 5-way 1-shot task, the support set contains 5 classes, with 1 labeled sample per class; the query set contains 15 unlabeled samples per class. In the meta-training stage, the Adam optimizer is used, with an initial learning rate set to 0.001. If the validation performance does not improve for 10 consecutive epochs, the learning rate is reduced to 0.3 times the original value, and the batch size is set to 64. In the self-supervised feature enhancement step, SimCLR is pre-trained for 30 epochs on unlabeled data in the target domain, with the temperature parameter τ set to 0.07. In the pseudo-label selection step, pseudo-labels are first generated using K-means clustering, with the number of clusters K being the same as the number of task classes (i.e., 5), and the confidence threshold for the silhouette coefficient is set to 0.01, retaining only high-confidence samples for subsequent training. In the fine-tuning stage, for the sparse regularization term L... sr We selected the L1 norm and set the hyperparameter λ for regularization strength to 1. The hybrid loss function uses a weighted combination of cross-entropy loss, contrastive loss, inverse cross-entropy loss, and a sparse regularization term, with each loss term having a weight of 1. In the meta-testing phase, the model's generalization ability was evaluated by randomly selecting 600 tasks from the test set, and the average classification accuracy of these tasks was calculated as the final performance metric.

[0103] To further illustrate the technical solution of the present invention and explain its beneficial effects, specific embodiments are provided. These embodiments will demonstrate the performance of the present invention in two key scenarios: semi-supervised (non-cross-domain) and cross-domain.

[0104] All embodiments described above are tested using standard datasets widely used in the field, and their performance is compared with existing comparative methods. This embodiment verifies the effectiveness of the method by comparing classification accuracy under different network architectures and task settings. The specific configurations of the datasets, hardware environment, and backbone network used in the experiments have been described in Tables 1 and 2 above.

[0105] Semi-supervised (non-cross-domain) learning on the Omniglot dataset:

[0106] This embodiment demonstrates the effectiveness of the method of the present invention on the Omniglot dataset. The experimental environment configuration is shown in Table 2. Experimental examples were performed on the Omniglot dataset using 5-way 1-shot and 5-way 5-shot semi-supervised learning tasks, and the results are shown in Table 3. Specifically, when ConvNet4 is used as the encoder, with 20 label categories, the accuracy of the 1-shot task improved from 68.3% to 91.4%; the accuracy of the 5-shot task improved from 93.2% to 98.3%. When ResNet-12 is used as the encoder, with 20 label categories, the accuracy of the 1-shot task improved from 83.3% to 95.4%; the accuracy of the 5-shot task improved from 94.2% to 98.5%. The results of this embodiment show that the method of the present invention can effectively improve model performance in handwritten character recognition tasks by utilizing unlabeled data.

[0107] Table 3 shows the performance of the method in this embodiment on the Omniglot dataset.

[0108]

[0109] Semi-supervised (non-cross-domain) learning on the MiniImageNet dataset:

[0110] This embodiment demonstrates the effectiveness of the method of the present invention on the MiniImageNet dataset. The results are shown in Table 4. When ConvNet4 is used as the encoder, with 25 labeled categories, the accuracy of the 1-shot task improves from 36.6% to 43.5%; the accuracy of the 5-shot task improves from 59.2% to 67.1%. When ResNet-12 is used as the encoder, with 55 labeled categories, the accuracy of the 1-shot task improves from 50.5% to 55.3%; the accuracy of the 5-shot task improves from 68.5% to 79.1%. The results of this embodiment show that the method of this embodiment can also improve model performance on general color image datasets by utilizing unlabeled data.

[0111] Table 4 shows the performance of the method in this embodiment on the MiniImageNet dataset.

[0112]

[0113]

[0114] Cross-domain learning using MiniImageNet as the source domain:

[0115] This embodiment demonstrates the performance of the method of the present invention when migrating from a general image dataset (MiniImageNet) to a target domain with different features. Experimental results are presented in Table 5. In the MiniImageNet→Omniglot task, there are significant feature differences between the source domain (natural images) and the target domain (handwritten characters). The method of the present invention achieves a significant performance gain in this task. When using ConvNet4 as the feature extractor, the 1-shot accuracy reaches 94.43%, an improvement of 20.17 percentage points compared to the baseline (ProtoNet) of 74.26%. This performance improvement is also verified on the deeper ResNet-12 network, with an accuracy improvement of 19.85 percentage points, demonstrating the effectiveness of the method of the present invention in handling large domain differences. Furthermore, when migrating to low-resolution images (MiniImageNet→FC100), the method of the present invention also shows a stable performance improvement, with accuracy improvements ranging from 10 to 13 percentage points. In scenarios involving migration to fine-grained images (MiniImageNet→CUB), while the performance on the 1-shot task is comparable to the baseline, the method of this invention can utilize information more effectively in a 5-shot setting with more available samples, improving accuracy by approximately 4.5 percentage points. In summary, the results of this embodiment demonstrate that when using a general dataset as the source domain, the method of this invention can effectively handle various domain migrations, especially in scenarios with significant domain differences and degraded image quality, effectively improving model performance.

[0116] Cross-domain learning with CUB as the source domain

[0117] This embodiment demonstrates the performance of the method of the present invention when transferring from the fine-grained image dataset (CUB) as the source domain to other target domains. Experimental results are cited in Table 5. When transferring from CUB to the Omniglot handwritten character set, which has a significantly different structure, the method of the present invention also achieves effective performance improvement. Using ConvNet4 as the feature extractor, the 1-shot accuracy reaches 93.43%, an improvement of 17.28 percentage points compared to the baseline (ProtoNet) of 76.15%. In tasks transferring to general images (CUB→MiniImageNet) and low-resolution images (CUB→FC100), the method of the present invention also shows stable performance improvement. In the 5-shot task of CUB→MiniImageNet, the accuracy improvement exceeds 11 percentage points, indicating that the method of the present invention can effectively generalize the features learned in the fine-grained task. In the CUB→FC100 task, performance gains ranging from 4 to 9 percentage points were also observed. The results of this embodiment show that when using the fine-grained CUB dataset as the source domain, the method of the present invention consistently improves performance compared to the baseline method when migrating to the general domain, low-resolution domain, and structurally different domain, demonstrating good robustness and versatility.

[0118] The performance of the proposed method (SFL) was compared with the baseline method (ProtoNet) on several typical cross-domain tasks. The experimental results are shown in Table 5.

[0119] Table 5. Comparison of classification accuracy (%) between the proposed method and the prototype network in cross-domain experimental scenarios.

[0120]

[0121]

[0122] The results show that:

[0123] 1. Performance Advantages: The proposed method (SFL) significantly outperforms the baseline prototype network in all tested cross-domain tasks, regardless of whether the settings are 1-shot or 5-shot. Particularly in tasks with significant domain differences, such as MiniImageNet→Omniglot, the proposed method achieves 94.43% accuracy in the 1-shot setting, a significant improvement of 20.17 percentage points compared to the prototype network's 74.26%. This demonstrates that the proposed method, through self-supervised learning and feature fusion, enhances the ability to capture the characteristics of target domain data, thereby reducing the impact of inter-domain distribution differences.

[0124] 2. Applicability to Different Feature Extractors: The method in this embodiment improves performance under feature extractors of different depths (ConvNet4 and ResNet-12). For example, in the CUB→FC100 5-shot task, the method of this invention achieved accuracies of 49.20% and 49.84% under ConvNet4 and ResNet-12, respectively, representing an improvement of more than 7 percentage points compared to the prototype network. This demonstrates that the framework design of this invention has good adaptability to different backbone networks.

[0125] To verify the effectiveness of the silhouette coefficient-based pseudo-label screening mechanism in this embodiment, an ablation experiment was designed. The experiment compared the performance difference between using the screening strategy of this embodiment (SFL(0.01), i.e., samples with a silhouette coefficient value greater than 0.01 are considered reliable pseudo-labels) and not using the screening strategy (comparison method, which directly uses all pseudo-labels generated by clustering).

[0126] Ablation experiment:

[0127] This embodiment aims to verify the effectiveness of the pseudo-label screening mechanism based on contour coefficients in the technical solution of this invention. The experimental results are all cited in Table 6. As can be seen from Table 6, the strategy of using contour coefficients to screen high-quality pseudo-labels (SFL(0.01)) has a classification accuracy that is slightly higher than or equal to that of the strategy using all pseudo-labels (SFL(0)) in most cross-domain tasks. For example, when using ResNet-12 as a feature extractor for the MiniImageNet→CUB task, the SFL(0.01) model achieved accuracies of 36.2% and 53.89% in 1-shot and 5-shot settings, respectively, which are higher than the 36.12% and 53.65% of the SFL(0) model. This performance improvement proves the effectiveness of the screening strategy. The results of this embodiment show that by eliminating pseudo-label samples with low confidence through contour coefficients, the quality of the pseudo-label set used for model training can be effectively improved, thereby enhancing the stability of model training and the final performance. Therefore, this pseudo-label screening strategy is an important part of the method of this invention.

[0128] Table 6. Comparative experimental results (%) of screening with and without profile coefficient threshold.

[0129]

[0130]

[0131] Example 2

[0132] This embodiment provides a semi-supervised few-sample image classification system for cross-domain scenarios, which specifically includes the following modules:

[0133] The feature extractor pre-training model is configured to: pre-train the feature extractor based on labeled samples using a meta-learning model;

[0134] The self-supervised encoder pre-trained model is configured to: pre-train a self-supervised encoder based on unlabeled samples using self-supervised contrastive learning;

[0135] The pseudo-label generation module is configured to: input unlabeled sample data into the feature extractor and the self-supervised encoder respectively to obtain initial features and intrinsic features respectively; fuse the initial features and intrinsic features to obtain fused features; generate initial pseudo-labels based on the fused features using a clustering algorithm; and then filter the initial pseudo-labels according to the clustering effectiveness index to obtain high-quality pseudo-labels.

[0136] The feature extractor update module is configured to: merge labeled samples with high-quality pseudo-labels into a mixed training sample set, construct a mixed loss function, and use the mixed loss function to update the feature extractor on the mixed training samples;

[0137] The classification module is configured to input the sample to be classified into the updated feature extractor and output the category of the sample to be classified.

[0138] The implementation of the specific modules in this embodiment refers to the steps of a semi-supervised few-sample image classification method in a cross-domain scenario described in Embodiment 1, and will not be described in detail here.

[0139] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A semi-supervised small sample image classification method under cross-domain scenes, characterized in that, The method comprises the following steps: Based on the labeled samples, a meta-learning model is used to pre-train a feature extractor; Based on the unlabeled samples, a self-supervised encoder is pre-trained using self-supervised contrastive learning; The unlabeled sample data is input into the feature extractor and the self-supervised encoder respectively to obtain initial features and intrinsic features, and the initial features and the intrinsic features are fused to obtain fused features; Based on the fused features, initial pseudo labels are generated through a clustering algorithm, and the initial pseudo labels are screened according to a clustering validity index to obtain high-quality pseudo labels; The labeled samples and the high-quality pseudo labels are combined to form a mixed training sample set, and a mixed loss function is constructed, and the feature extractor is updated on the mixed training samples by using the mixed loss function; The sample to be classified is input into the updated feature extractor, and the class of the sample to be classified is output.

2. The semi-supervised small sample image classification method in a cross-domain scene according to claim 1, characterized in that, The pre-trained feature extractor specifically comprises the following steps: firstly, a training task including a support set and a query set is constructed; then the feature prototypes of each class are calculated through the samples in the support set, and the Euclidean distance between the feature vectors of the samples in the query set and the feature prototypes of each class is calculated, and the probability that the samples belong to each class is obtained through normalization; then a combined loss function is constructed, and the parameters of the feature extractor are updated by minimizing the combined loss function. 3.The semi-supervised few-shot image classification method under cross-domain scenes according to claim 2, characterized in that, The calculation method of the feature prototype of each class is as follows: where c k represents the feature prototype of the kth class in the support set, S k represents the sample set of the kth class in the support set, |S k | represents the number of samples in the set S k ; x i represents the ith sample in the support set, y i represents the label corresponding to the ith sample, f φ (x i ) represents the feature vector extracted by the feature extractor on x i . The calculation formula of the probability that the sample belongs to each class is: where p φ (y = k | x) denotes the probability that a sample belongs to class k, d(f φ (x), c k ) denotes the Euclidean distance between the feature vector of sample x and the feature prototype of the k-th class, d() denotes the distance metric function, and c j denotes the summation over all classes except the j-th class in the denominator of the prototype. 4.The semi-supervised few-shot image classification method under cross-domain scenes of claim 1, wherein, The combined loss function is expressed as: L P = L ce + L d + L r ; wherein, L P represents the constructed combined loss function, L ce represents the classification cross-entropy loss, L d represents the loss term for adjusting the average distance between feature prototypes, L r represents the regularization term; wherein the classification cross-entropy loss is represented as: The loss term for adjusting the average distance between feature prototypes is represented as: wherein, represents the difference between the actual average distance between feature prototypes and the expected distance.

5. The semi-supervised small sample image classification method in a cross-domain scene according to claim 1, characterized in that, The clustering validity index is a silhouette coefficient, and the calculation formula of the silhouette coefficient is: where s(i) denotes silhouette coefficient with initial pseudo-label for sample i, a(i) denotes the average distance of sample i to other points in the cluster it belongs to, and b(i) denotes the average distance of sample i to all points in the nearest neighbor cluster. A confidence threshold is pre-set, when the silhouette coefficient is greater than the confidence threshold, the initial pseudo label sample is regarded as a high-confidence sample and is retained, thereby obtaining high-quality pseudo labels, otherwise it is excluded.

6. The semi-supervised small sample image classification method in a cross-domain scene according to claim 1, wherein, The mixed loss function includes confidence weighted cross-entropy loss, contrastive loss, inverse cross-entropy loss and sparse regularization; and the parameters of the feature extractor are updated by minimizing the mixed loss function using an Adam optimizer.

7. The semi-supervised small sample image classification method in a cross-domain scene according to claim 6, characterized in that, The confidence weighted cross-entropy loss is expressed as: where D new denotes the mixed training sample set, |D new denotes the total number of samples in the mixed training sample set, w i denotes the confidence weight, m i denotes a sample in the mixed training sample set, n i denotes i m i the corresponding label, logp φ (n = m i |n i ) denotes the probability that sample m i belongs to label n i ; The contrastive loss is expressed as: Wherein, P(m) i ) represents D new In and m i A(m) is a collection of samples belonging to the same category. i ) indicates that it includes m i For all samples including those included, the parameter ξ takes values ​​between 0 and 1. The inverse cross-entropy loss is expressed as: where p φ (y = k \ x) denotes the probability that sample x belongs to class k, q φ (y = k \ x) denotes the one-hot distribution represented by the pseudo label of the sample; The sparse regularization is represented as: L sr = λ · ‖ω‖ p ; wherein ω represents a weight parameter, ‖ ‖ p p-norm, and λ represents a hyperparameter of regularization strength.

8. A semi-supervised small sample image classification system under cross-domain scenarios, characterized in that, The method comprises the following steps: The feature extractor pre-training model is configured to pre-train the feature extractor based on the labeled samples using a meta-learning model; The self-supervised encoder pre-training model is configured to pre-train the self-supervised encoder based on the unlabeled samples using self-supervised contrastive learning; The pseudo label generation module is configured to input the unlabeled sample data into the feature extractor and the self-supervised encoder respectively to obtain initial features and intrinsic features, and fuse the initial features and the intrinsic features to obtain fused features; Based on the fused features, initial pseudo labels are generated through a clustering algorithm, and the initial pseudo labels are screened according to a clustering validity index to obtain high-quality pseudo labels; The feature extractor updating module is configured to combine the labeled samples and the high-quality pseudo labels to form a mixed training sample set, construct a mixed loss function, and update the feature extractor on the mixed training samples by using the mixed loss function; The classification module is configured to input the sample to be classified into the updated feature extractor, and output the class of the sample to be classified.

9. A computer-readable storage medium having stored thereon a program, characterized in that, The program is executed by the processor to implement the steps in the semi-supervised small sample image classification method in a cross-domain scene according to any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps in the semi-supervised small sample image classification method in a cross-domain scene according to any one of claims 1-7.

Images