Unsupervised domain adaptive image classification method for small sample, medium and equipment

Abstract

The invention discloses a small sample unsupervised domain adaptive image classification method, medium and equipment, and the method comprises the steps: S1, constructing a small sample unsupervised domain adaptive task data set, and transmitting labeled source domain data and unlabeled target domain data into an image feature extractor to extract image features; s2, amplifying a source domain small sample category by adopting a Mixup data enhancement strategy; s3, constructing a source domain similarity graph, a target domain similarity graph and a cross-domain similarity graph, executing label propagation on the constructed graphs, and generating a pseudo label of a target domain sample; s4, calculating classification loss and image regularization loss, and minimizing total loss to optimize the image classification model; and S5, based on the optimized image classification model, performing classification prediction on the target domain sample. The classification performance of the model in the target domain can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and in particular to a method for small-sample unsupervised domain adaptation image classification and a computer device. Background Art

[0002] Traditional deep learning relies on a large number of fully labeled and identically distributed samples for training. However, in practical applications, there are often distribution differences, i.e., inter-domain differences, between different data sets. This difference causes the performance of the model to drop significantly when it is transferred from the source domain to the target domain, namely the domain adaptation problem. As a method to improve the model performance in the case of no labels in the target domain, unsupervised domain adaptation has received extensive attention in recent years. However, most unsupervised domain adaptation methods assume that the source domain has sufficient labeled samples. However, in real-world tasks, there may also be small-sample classes in the source domain data, that is, the labeled samples of some classes are extremely limited.

[0003] Under the dual constraints of insufficient source domain data and no labels in the target domain, small-sample unsupervised domain adaptation image classification has become a new research hotspot. Its goal is to effectively improve the classification performance of the target domain when there are only a small number of labeled samples in the source domain and no labeled data in the target domain. Due to the scarcity of data in the small-sample classes of the source domain, it may be difficult for the model to learn stable class features and is vulnerable to domain shift during the transfer process. Therefore, there are two main challenges in small-sample unsupervised domain adaptation image classification: First, there is an inconsistency in the data distributions between the source domain and the target domain; second, there is a small-sample set in the source domain, that is, there are only a small number of labeled samples for some classes.

[0004] To address these challenges, some research efforts have started to explore methods to solve these problems in recent years. For example, Long et al. (Long M, Cao Z, Wang J, et al. Conditional adversarial domain adaptation[J]. Advances in neural information processing systems, 2018, 31) proposed the Conditional Domain Adversarial Network (CDAN). This method combines the cross-covariance of feature representation and classification prediction results through a multilinear mapping mechanism to enhance discriminability. At the same time, it uses entropy conditions to control classification uncertainty, thereby enhancing transferability. Saito et al. (Saito K, Watanabe K, Ushiku Y, et al. Maximum classifier discrepancy for unsupervised domain adaptation[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2018: 3723-3732) identified target samples far from the source domain by maximizing the prediction differences of two classifiers and used a feature generator to minimize this difference to generate more discriminative target features, thus achieving effective alignment of inter-domain distributions. However, neither of these two methods considered the problem of small sample classes in the source domain, resulting in poor performance in the small sample unsupervised domain adaptation image classification task. To solve this problem, Wang et al. (Wang T, Ding Z, Shao W, et al. Towards fair cross-domain adaptation via generative learning[C] / / Proceedings of the IEEE / CVF Winter Conference on Applications of Computer Vision. 2021: 454-463) introduced a generative adversarial network model to address small sample unsupervised domain adaptation.This method uses a generative model to synthesize the features of few-shot classes in the source domain, and minimizes the cross-domain maximum mean discrepancy to achieve feature alignment between the source domain and the target domain; Xu et al. (Xu B, Zeng Z, Lian C, et al. Few-shot domain adaptation via mixup optimal transport[J]. IEEE Transactions on Image Processing, 2022, 31: 2518-2528) proposed a few-shot unsupervised domain adaptation method based on mixup optimal transport. This method applies Mixup between source domain samples for data augmentation to generate more training samples, and combines optimal transport technology to align the representations of the source domain and the target domain to improve the generalization ability of the model; Wang et al. (Wang W, Li H, Shi K, et al. Optimal graph learning and nuclear norm maximization for deep cross-domain robust label propagation[C] / / Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence. 2024: 1407-1415) proposed a new label propagation method to enhance the robustness of label propagation, and combined graph embedding loss and nuclear norm maximization to improve domain invariance and class diversity, thus effectively alleviating the problems of distribution difference and class imbalance; however, traditional label propagation methods are often affected by class imbalance in the source domain in few-shot unsupervised domain adaptation tasks, resulting in insufficient feature representations of few-shot classes, further exacerbating the inter-domain gap, making label propagation more inclined to enhance normal classes while ignoring few-shot classes. Summary of the Invention

[0005] A method for few-shot unsupervised domain adaptation image classification and a computer device based on Mixup data augmentation robust label propagation method provided by the present invention can solve at least one of the above technical problems.

[0006] To solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A method for few-shot unsupervised domain adaptation image classification includes the following steps:

[0008] S1. Construct a few-shot unsupervised domain adaptation task dataset, and send the labeled source domain data and unlabeled target domain data into an image feature extractor to extract image features;

[0009] S2. Apply the Mixup data augmentation strategy to amplify the small-sample classes in the source domain to enhance the diversity of small-sample features in the source domain;

[0010] S3. Feed the source domain sample features and target domain sample features extracted by S1 and the enhanced small-sample features of the source domain after S2 into a classifier for prediction. At the same time, based on the above source domain sample features, target domain sample features, and enhanced small-sample features of the source domain, construct a source domain similarity graph, a target domain similarity graph, and a cross-domain similarity graph, and perform label propagation on the constructed graphs to generate pseudo-labels for the target domain samples;

[0011] S4. Calculate the classification loss and graph regularization loss according to the prediction results of the classifier, the pseudo-labels generated by label propagation, and the constructed similarity graphs, and minimize the total loss to optimize the image classification model;

[0012] S5. Based on the optimized image classification model, perform classification prediction on the target domain samples.

[0013] Furthermore, in S1, the purpose of the small-sample unsupervised domain adaptation image classification task is to transfer the knowledge learned from the source domain to the target domain with different data distributions, so as to correctly classify the unlabeled target domain samples. The small-sample unsupervised domain adaptation task dataset includes the source domain dataset D s and the target domain dataset D t :

[0014]

[0015] Among them, the source domain dataset includes a normal set and a small-sample set n s represents the size of the source domain dataset samples, n t represents the size of the target domain dataset samples, n i represents the size of the source domain normal set samples, f j represents the size of the source domain small-sample set samples.

[0016] Furthermore, in S1, the source domain data and target domain data are fed into an image feature extractor to extract features Z s / t , and the calculation formula is:

[0017] Z s / t =F image (x s / t )

[0018] Among them, F image (.) is the image feature extractor, and x s / t is the sample data of the source domain and the target domain.

[0019] Further, in S2, the Mixup data augmentation strategy expands the feature representation of the small sample categories in the source domain by means of linear interpolation, and the calculation method is as follows:

[0020]

[0021] where λ is a value randomly sampled from the Beta distribution Beta(σ, σ), σ ∈ (0, ∞), and are the sample features from the small sample categories in the source domain, and are the labels of the corresponding samples, Z m represents the feature of the small sample category in the source domain expanded by data augmentation, y m represents the label of the feature of the small sample category in the source domain expanded by data augmentation. For the sake of simplified representation, the feature of the small sample category in the source domain expanded by data augmentation and the corresponding label are directly represented as Z s and y s .

[0022] Further, S3 further includes:

[0023] S31. Based on the source domain sample features and target domain sample features extracted through S1 and the features of the small sample categories in the source domain enhanced through S2, construct a source domain similarity graph, a target domain similarity graph, and a cross-domain similarity graph through the following optimization method:

[0024]

[0025] where:

[0026] represents the similarity weight between the i-th sample feature and the j-th sample feature in the source domain;

[0027] represents the similarity weight between the i-th sample feature and the j-th sample feature in the target domain;

[0028] represents the similarity weight between the i-th sample feature in the source domain and the j-th sample feature in the target domain;

[0029] represents the similarity weight between the i-th sample feature in the target domain and the j-th sample feature The similarity weight between;

[0030] n s represents the number of original samples in the source domain;

[0031] represents the number of generated samples of the small sample class in the source domain;

[0032] n t represents the number of samples in the target domain;

[0033] W ss represents the similarity graph between source domain sample features;

[0034] W tt represents the similarity graph between target domain sample features;

[0035] W st represents the similarity graph from source domain sample features to target domain sample features;

[0036] W ts represents the similarity graph from target domain sample features to source domain sample features;

[0037] argmin means finding the optimal similarity graph W to minimize the value of the objective function;

[0038] S32. Combine the four formulas in the above S31 to obtain two intra-domain similarity graphs W ss and W tt as well as two cross-domain similarity graphs W st and W ts , and these two cross-domain similarity graphs are fused through an averaging operation to generate the final cross-domain similarity graph W cro , and the calculation method is as follows:

[0039]

[0040] where T represents the transpose of the matrix;

[0041] S33. Based on the constructed target domain similarity graph W tt and the final cross-domain similarity graph W cro , perform label propagation through the following optimization method to generate pseudo-labels for target domain samples:

[0042]

[0043] where, represents the label matrix generated for all samples in the target domain through label propagation, represents the pseudo-label of the j-th sample in the target domain, represents the similarity weight between the i-th sample and the j-th sample in the cross-domain similarity graph, Denote the label of the $i$-th sample in the source domain;

[0044] S34. Feed the source domain sample features and target domain sample features after extraction in S1 and the source domain small sample features after enhancement in S2 into the classifier for prediction, and the calculation method is as follows:

[0045] $p = F$ Classifier $(Z)$

[0046] where $F$ Classifier represents the classifier, $Z$ represents the features, and $p$ represents the predicted value after the model is processed by softmax.

[0047] Furthermore, the S4 further includes:

[0048] S41. Calculate the classification loss The calculation method is as follows:

[0049]

[0050] S42. Calculate the graph regularization loss The calculation method is as follows:

[0051]

[0052] where and respectively represent the intra-domain graph regularization loss and the cross-domain graph regularization loss;

[0053] For the intra-domain graph regularization loss the calculation method is as follows:

[0054]

[0055] where $Z$ d $(d\in\{s,t\})$ represents the feature matrices of the source domain and the target domain, $L$ d represents the Laplacian matrix of the intra-domain weighted graph, and $\text{Tr}(.)$ represents the trace operation;

[0056] For the cross-domain graph regularization loss the calculation method is as follows:

[0057]

[0058] where $Z$ s+t $=[Z$ s ; $Z$ t represents the concatenation of the source domain feature matrix $Z$ s and the target domain feature matrix $Z$ t $, and $L$ cro represents the Laplacian matrix of the cross-domain weighted graph;

[0059] The Laplacian matrix L is calculated as follows:

[0060] L = D - W ′

[0061] where D is the corresponding degree matrix, and W ′ is the adjacency matrix. When calculating , W ′ = W ss or W ′ = W tt . When calculating , W ′ = W cro ;

[0062] S43. Minimize the total loss to optimize the image classification model. The total loss is calculated as follows:

[0063]

[0064] where both α and β are hyperparameters.

[0065] Furthermore, in S5, based on the image classification model optimized in S4, in image classification prediction, first send the target domain samples into the feature extractor to extract features, and then send them into the classifier for classification prediction. The calculation method is as follows:

[0066] p = F Classifier (F image (x t ))

[0067]

[0068] where arg max represents finding the index corresponding to the maximum value in p, is the predicted class label of the image.

[0069] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor is caused to execute the steps of the above method for small-sample unsupervised domain adaptation image classification.

[0070] A computer device includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor is caused to execute the steps of the above method for small-sample unsupervised domain adaptation image classification.

[0071] The beneficial effects of the present invention are as follows:

[0072] 1. The present invention is based on the Mixup data enhancement strategy. Mixup generates diversified training samples through linear interpolation, which effectively alleviates the data imbalance problem of small sample categories in the source domain and enhances feature diversity, thereby improving the model's learning ability for domain-invariant features. At the same time, the smoothing effect of Mixup reduces the overfitting of the model to small sample categories and enhances the generalization ability of the model. In addition, Mixup promotes category alignment between the source domain and the target domain by generating more diverse feature distributions, narrowing the distribution gap between the two. It effectively solves the problem of image classification in small sample unsupervised domain adaptation tasks, due to the imbalance of categories in the source domain, which leads to the phenomenon of model overfitting with the majority of sample categories, and even makes the network model have poor classification ability for small sample categories.

[0073] 2. In image classification in small sample unsupervised domain adaptation tasks, label propagation relies on a graph structure based on sample similarity, where edge weights are used to quantify the degree of similarity between samples. The present invention constructs independent cross-domain and intra-domain similarity graphs, where the cross-domain similarity graph focuses on the similarity relationship between source domain and target domain samples. Even if the source domain small sample category data is scarce, it can effectively connect similar samples in the target domain to ensure smooth information propagation and alleviate the problem of low cross-domain edge weights caused by unified composition, thereby improving the label propagation effect. It effectively solves the problem that the source domain small sample category data is scarce, the feature distribution range is limited, and the feature distance from the target domain sample is large, resulting in extremely low cross-domain edge weights or even close to zero, which makes it difficult for the source domain label to be effectively propagated to the target domain, affecting the generation of pseudo labels.

[0074] 3. The present invention introduces graph regularization loss. Graph regularization loss encourages samples of the same category to be as close as possible in the feature space by modeling the feature similarity of samples in the source domain and the target domain, while widening the distance between samples of different categories. This constraint not only enhances the intra-class compactness, but also reduces the distribution difference between the source domain and the target domain. In addition, graph regularization can also capture the local structural information between samples, further improving the model's generalization ability for target domain data. By combining graph regularization loss, the model can better learn domain-invariant features and significantly improve the classification performance of the model in the target domain. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0076] Figure 1 It is a schematic diagram of the overall flow of the image classification method according to an embodiment of the present invention.

[0077] Figure 2It is a flowchart of the image classification method according to an embodiment of the present invention.

[0078] Figure 3 It is a schematic diagram of experimental comparison on the Office-31 dataset according to an embodiment of the present invention.

[0079] Figure 4 It is a comparison schematic diagram on the Office-Home dataset according to an embodiment of the present invention.

[0080] Figure 5 It is a structural block diagram of a computer device according to an embodiment of the present invention. Detailed implementation manners

[0081] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0082] It should be noted that the meaning of "and / or" appearing throughout the text includes three parallel solutions. Taking "A and / or B" as an example, it includes solution A, or solution B, or a solution that satisfies both A and B at the same time. In addition, "a plurality of" means two or more. In addition, the technical solutions between the embodiments can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement them. When the combination of technical solutions appears to be contradictory or unable to be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0083] See Figure 1 - Figure 2 , an embodiment of the present invention provides a method for small-sample unsupervised domain adaptation image classification, including the following steps:

[0084] S1. Construct a small-sample unsupervised domain adaptation task dataset, and send the labeled source domain data and the unlabeled target domain data into an image feature extractor to extract image features;

[0085] In this step S1, the purpose of the small-sample unsupervised domain adaptation image classification task is to transfer the knowledge learned from the source domain to the target domain with different data, so as to correctly classify the unlabeled target domain samples. The small-sample unsupervised domain adaptation task dataset includes the source domain dataset D s and the target domain dataset D t :

[0086]

[0087] Among them, the source domain dataset includes a normal set and a small sample set n s represents the size of the number of samples in the source domain dataset, n t represents the size of the number of samples in the target domain dataset, n i represents the size of the number of samples in the source domain normal set, f j represents the size of the number of samples in the source domain small sample set.

[0088] The data in the source domain and the target domain come from different distributions but share a consistent label space. In actual simulation experiments, the present invention evaluated this method on two well-known domain adaptation benchmark datasets, Office-31 and Office-Home. Office-31 contains three domains (Amazon, Webcam, and DSLR). The Amazon domain contains 2,817 images downloaded from Amazon.com. The Webcam domain consists of 795 images taken using a webcam. The DSLR domain contains 498 images taken using a digital single-lens reflex camera. Each domain contains 31 categories. Office-Home contains images of 65 categories from four different domains (Art, Clipart, Product, and Real-World). The Art domain contains artistic-style images. The Clipar domain consists of clipart images. The images in the Product domain are object-centered and have no background. The Real-World domain contains object images taken using a standard camera.

[0089] The present invention follows the small sample setting in previous studies, selects 10 categories from the Office-31 dataset and 20 categories from the Office-Home dataset as small sample categories. For each small sample category, only 3 samples are randomly selected from the Office-31 dataset or the Office-Home dataset, and other categories remain as normal categories with sufficient training samples.

[0090] In this step S1, the source domain data and the target domain data are sent into an image feature extractor to extract the feature Z s / t , and the calculation formula is:

[0091] Z s / t =F image (x s / t )

[0092] Among them, F image (.) is the image feature extractor, and x s / t is the sample data of the source domain and the target domain.

[0093] S2. Apply the Mixup data augmentation strategy to amplify the small-sample classes in the source domain to enhance the diversity of small-sample features in the source domain;

[0094] In this step S2, the Mixup data augmentation strategy expands the feature representation of the small-sample classes in the source domain by linear interpolation, and the calculation method is as follows:

[0095]

[0096] where λ is a value randomly sampled from the Beta distribution Beta(σ, σ), σ ∈ (0, ∞), and are the sample features from the small-sample classes in the source domain, and are the labels of the corresponding samples, Z m represents the features of the small-sample classes in the source domain after data augmentation, and y m represents the label of the features of the small-sample classes in the source domain after data augmentation. For the sake of simplicity, the features of the small-sample classes in the source domain after data augmentation and the corresponding labels are directly represented as Z s and y s ;

[0097] In the small-sample unsupervised domain adaptation task, there are usually differences in the distributions of the source domain data and the target domain data, and the number of samples in some classes in the source domain may be small. This problem of data imbalance will lead to the problem that the model has insufficient learning of the features of the small-sample classes during the training process, thus affecting the performance of the model on the target domain. Through the Mixup data augmentation strategy, the number of sample features in each class in the source domain can be balanced, avoiding the model from being overly biased towards the large-sample classes, thereby improving the feature learning ability for the small-sample classes. At the same time, it also avoids the problem that the pseudo-labels generated in the small-sample domain adaptation task by label propagation overfit the multi-sample classes, and improves the robustness of label propagation.

[0098] S3. Feed the source domain sample features, target domain sample features extracted through S1, and the small-sample features of the source domain enhanced through S2 into the classifier for prediction. At the same time, based on the above source domain sample features, target domain sample features, and the enhanced small-sample features of the source domain, construct a source domain similarity graph, a target domain similarity graph, and a cross-domain similarity graph, and perform label propagation on the constructed graphs to generate pseudo-labels for the target domain samples;

[0099] In this step S3, it further includes:

[0100] S31. Based on the source domain sample features, target domain sample features extracted through S1, and the small-sample class features of the source domain enhanced through S2, construct a source domain similarity graph, a target domain similarity graph, and a cross-domain similarity graph through the following optimization method:

[0101]

[0102] wherein:

[0103] represents the similarity weight between the i-th sample feature in the source domain and the j-th sample feature ;

[0104] represents the similarity weight between the i-th sample feature in the target domain and the j-th sample feature ;

[0105] represents the similarity weight between the i-th sample feature in the source domain and the j-th sample feature in the target domain ;

[0106] represents the similarity weight between the i-th sample feature in the target domain and the j-th sample feature in the source domain ;

[0107] n s represents the number of original samples in the source domain;

[0108] represents the number of generated samples of the small sample class in the source domain;

[0109] n t represents the number of samples in the target domain;

[0110] W ss represents the similarity graph between source domain sample features;

[0111] W tt represents the similarity graph between target domain sample features;

[0112] W st represents the similarity graph from source domain sample features to target domain sample features;

[0113] W ts represents the similarity graph from target domain sample features to source domain sample features;

[0114] argmin means finding the optimal similarity graph W to minimize the value of the objective function;

[0115] The refinement of the above step S31 includes two key optimization objectives: the first one hopes to ensure that for samples that are close to each other in the feature space, the similarity weights between them should be larger. This objective not only promotes the connection between similar samples but also strengthens the propagation efficiency of label information in the graph structure. Secondly, by introducing the second optimization term, a potential problem is avoided, that is, preventing the graph structure from being too concentrated on a few neighboring points, which may lead to overfitting or information loss. By balancing these two objectives, this method can effectively narrow the cross-domain information gap while maintaining coherence within the domain;

[0116] S32. Combine the four formulas in the above S31 to obtain two intra-domain similarity graphs W ss and W tt and two cross-domain similarity graphs W st and W ts . These two cross-domain similarity graphs are fused through an averaging operation to generate the final cross-domain similarity graph W cro . The calculation method is as follows:

[0117]

[0118] where T represents the transpose of the matrix;

[0119] S33. Based on the constructed target-domain similarity graph W tt and the final cross-domain similarity graph W cro , the following optimization method is used for label propagation to generate pseudo-labels for the target-domain samples:

[0120]

[0121] where, represents the label matrix generated for all samples in the target domain through label propagation, represents the pseudo-label of the j-th sample in the target domain, represents the similarity weight between the i-th sample and the j-th sample in the cross-domain similarity graph, represents the label of the i-th sample in the source domain;

[0122] The refinement of the above step S33 includes two key optimization objectives: optimizing the first term can make samples that are close in two feature spaces from different domains have similar labels; optimizing the second term, two samples that are close in the target-domain feature space have similar labels. Intuitively, the first term aims to propagate labels from the source domain to the target domain, while the second term uses the structure of the target domain itself to correct some label propagation errors in the first term;

[0123] S34. Feed the source domain sample features and target domain sample features after extraction in S1 and the source domain small sample features after enhancement in S2 into the classifier for prediction respectively. The calculation method is as follows:

[0124] p = F Classifier (z)

[0125] Among them, F Classifier represents the classifier, Z represents the features, and p represents the predicted value after the model is processed by softmax.

[0126] S4. Calculate the classification loss and graph regularization loss according to the prediction results of the classifier, the pseudo-labels generated by label propagation, and the constructed similarity graph, and minimize the total loss to optimize the image classification model;

[0127] In this step S4, S4 further includes:

[0128] S41. Calculate the classification loss The calculation method is as follows:

[0129]

[0130] During the optimization process, the source domain samples are supervised and learned through their true labels to ensure that the image classification model can fully learn the discriminative features of the known classes in the source domain, thus providing a reliable basis for cross-domain tasks. At the same time, the target domain samples are optimized using the pseudo-labels generated by label propagation. Label propagation constructs a graph structure based on the similarity between samples, which can effectively mine the local and global structure information of the target domain data, enabling the generated pseudo-labels to not only reflect the class information of the samples but also capture the intrinsic distribution characteristics of the data, thereby significantly enhancing the image classification model's understanding ability of the target domain data;

[0131] S42. Calculate the graph regularization loss The calculation method is as follows:

[0132]

[0133] Among them, and represent the intra-domain graph regularization loss and the cross-domain graph regularization loss respectively;

[0134] The graph regularization loss can optimize the intra-class compactness and inter-class separability. Among them, the intra-domain graph regularization loss ensures the discriminability of the features in the source domain and the target domain, and at the same time effectively reduces the intra-class distribution difference between the source domain and the target domain through the cross-domain graph regularization loss, thereby promoting the learning of domain-invariant features. In this way, the image classification model can learn more discriminative and domain-invariant embedding representations in cross-domain tasks, significantly enhancing the generalization ability and cross-domain adaptability of the model;

[0135] For the in - domain graph regularization loss the calculation method is as follows:

[0136]

[0137] Among them, Z d (d ∈ {s, t}) represents the feature matrices of the source domain and the target domain, L d represents the Laplacian matrix of the in - domain weighted graph, and Tr(.) represents the trace operation;

[0138] For the cross - domain graph regularization loss the calculation method is as follows:

[0139]

[0140] Among them, Z s+t =[Z s ; Z t represents the concatenation of the source - domain feature matrix Z s and the target - domain feature matrix Z t , and L cro represents the Laplacian matrix of the cross - domain weighted graph;

[0141] The Laplacian matrix L is calculated as follows:

[0142] L = D - W ′

[0143] Among them, D is the corresponding degree matrix, and W ′ is the adjacency matrix. When calculating , W ′ =W ss or W ′ =W tt . When calculating , W ′ =W cro ;

[0144] Among them, the construction methods of the adjacency - matrix graphs of W ss , W tt and W cro are the same as the method of constructing the graph by label propagation in the above - mentioned S3.

[0145] S43. Minimize the total loss to optimize the image - classification model. The total loss is calculated as follows:

[0146]

[0147] Among them, both α and β are hyperparameters.

[0148] S5. Classify and predict the target domain samples based on the optimized image classification model;

[0149] In this step S5, based on the image classification model optimized in S4, in image classification prediction, first send the target domain samples into the feature extractor to extract features, and then send them into the classifier for classification prediction. The calculation method is as follows:

[0150] p = F Classifier (F image (x t ))

[0151]

[0152] where arg max represents finding the index corresponding to the maximum value in p, which is the predicted class label of the image.

[0153] Next, this application will illustrate the technical effects of the embodiments of the present invention through comparative experiments on the Office-31 dataset and the Office-Home dataset:

[0154] Table 1 Comparative experiment results on the Office-31 dataset (%)

[0155]

[0156]

[0157] Table 2 Comparative experiment results on the Office-Home dataset (%)

[0158]

[0159] Table 1 and Table 2 are the results of comparing the experimental results of the present invention (Ours) with other mainstream methods. From Table 1 - Table 2, it can be seen that on the two mainstream datasets of Office-31 and Office-Home, the average accuracy of the embodiments of the present invention has achieved the best results almost. These results indicate that the present invention effectively solves two key problems in small sample unsupervised domain adaptation: (1) there are only a few samples in some categories in the source domain, making it difficult to learn sufficient features; (2) the data distributions of the source domain and the target domain are inconsistent, resulting in the model being vulnerable to distribution bias during the migration process.

[0160] At the same time, in terms of single-task performance, Figure 3 and Figure 4respectively demonstrate the excellent performance of the embodiments of the present invention in most tasks. It can be seen from the experimental results that the method of the present invention has achieved significant performance improvements in multiple single-task scenarios, and the classification accuracy is significantly better than that of traditional methods and existing advanced methods. Especially in the A→R and R→P tasks of the challenging Office-Home dataset, the embodiments of the present invention show stronger robustness and generalization ability, and can effectively cope with the challenges brought by the inter-domain distribution differences and data complexity. This result fully proves the effectiveness of the present invention in aspects such as feature extraction, domain adaptation optimization, and label propagation, and can significantly improve the discriminative ability and adaptability of the model in single tasks. Through Figure 3 and Figure 4 comparative analysis, it can be clearly observed that the method of the present invention has stability and consistency in different tasks, further verifying its wide applicability and technical advantages in practical applications, and providing an efficient and reliable solution for cross-domain tasks.

[0161] The embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to execute the steps of the small-sample unsupervised domain adaptation image classification method as described above.

[0162] See Figure 5 , the embodiments of the present invention also provide a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to execute the steps of the small-sample unsupervised domain adaptation image classification method as described above.

[0163] The embodiments of the present invention also provide a computer program product containing instructions, which, when running on a computer, causes the computer to execute the steps of the small-sample unsupervised domain adaptation image classification method as described above.

[0164] It is understandable that the systems, devices, and storage media provided by the embodiments of the present invention correspond to the methods provided by the embodiments of the present invention, and the explanations, examples, and beneficial effects of related content can refer to the corresponding parts in the small-sample unsupervised domain adaptation image classification method described above.

[0165] It should be noted that those of ordinary skill in the art can understand that all or part of the steps implemented in the embodiments of the present invention can be implemented in whole or in part through software, hardware, firmware, or any combination thereof. When implemented using hardware, it can be implemented in whole or in part in the form of purchasing standard parts or modified parts. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

[0166] In summary, the present invention proposes a robust label propagation method based on Mixup data augmentation for small-sample unsupervised domain adaptation image classification tasks. This method can effectively improve the classification performance of the model in the target domain when the source domain has scarce small-sample category data and the target domain has no labels.

[0167] Specifically, for the problem of source domain class imbalance, the present invention proposes a Mixup data augmentation strategy. First, by performing linear interpolation between small-sample classes in the source domain, more diverse small-sample class features are generated, thereby effectively alleviating the class imbalance problem. In addition, the present invention constructs independent cross-domain and intra-domain similarity graphs and performs label propagation on these graphs to generate high-quality target domain pseudo-labels. At the same time, a graph regularization loss is introduced to enhance intra-class compactness and reduce the distribution deviation of small-sample classes between the source domain and the target domain, ensuring that the model can still effectively improve the classification performance of the model in the target domain under the condition of class imbalance.

[0168] It should be understood that the examples and embodiments described herein are for illustrative purposes only and are not intended to limit the present invention. Those skilled in the art can make various modifications or changes based on it. Any modification, equivalent replacement, improvement, etc., made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for small sample unsupervised domain adaptation image classification, characterized in that: The following steps are involved: S1. Construct a small sample unsupervised domain adaptation task dataset, and send the labeled source domain data and the unlabeled target domain data to the image feature extractor to extract image features; S2, using the Mixup data enhancement strategy to expand the source domain small sample categories to improve the feature diversity of the source domain small samples; S3, the source domain sample features and target domain sample features extracted by S1 and the source domain small sample features enhanced by S2 are sent to the classifier for prediction. At the same time, based on the above source domain sample features, target domain sample features and enhanced source domain small sample features, a source domain similarity graph, a target domain similarity graph and a cross-domain similarity graph are constructed, and label propagation is performed on the constructed graph to generate pseudo labels for target domain samples; S4. According to the prediction results of the classifier, the pseudo labels generated by label propagation, and the constructed similarity graph, the classification loss and graph regularization loss are calculated, and the total loss is minimized to optimize the image classification model. S5. Based on the optimized image classification model, classify and predict the target domain samples.

2. The method for small sample unsupervised domain adaptation image classification according to claim 1, characterized in that: In S1, the purpose of the small sample unsupervised domain adaptation image classification task is to transfer the knowledge learned from the source domain to the target domain with different data, so as to correctly classify the target domain samples without labels. The small sample unsupervised domain adaptation task dataset includes the source domain dataset D s and the target domain dataset D t : Among them, the source domain dataset includes the normal set and small sample sets n s Indicates the number of samples in the source domain dataset, n t Indicates the number of samples in the target domain dataset, n i represents the number of samples in the normal set of the source domain, f j Represents the sample size of the small sample set in the source domain.

3. The method for small sample unsupervised domain adaptation image classification according to claim 2, characterized in that: In S1, the source domain data and the target domain data are sent to the image feature extractor to extract the feature Z s / t , the calculation formula is: Z s / t =F image (x s / t ) Among them, F image (.) is the image feature extractor, x s / t is the sample data of the source domain and the target domain.

4. The method for small sample unsupervised domain adaptation image classification according to claim 1, characterized in that: In S2, the Mixup data enhancement strategy expands the feature representation of the small sample category in the source domain by linear interpolation. The calculation method is as follows: where λ is a value randomly sampled from the Beta distribution Beta(σ,σ), σ∈(0,∞), and is the sample feature from the small sample category in the source domain, and is the label of the corresponding sample, Z m represents the source domain small sample category feature after data enhancement, y m Represents the label of the source domain small sample category feature after data enhancement. In order to simplify the representation, the source domain small sample category feature and the corresponding label after data enhancement are directly expressed as Z in the subsequent formulas. s and s .

5. The method for small sample unsupervised domain adaptation image classification according to claim 1, characterized in that: The S3 further comprises: S31, based on the source domain sample features and target domain sample features extracted by S1 and the source domain small sample category features enhanced by S2, construct the source domain similarity graph, target domain similarity graph and cross-domain similarity graph through the following optimization method: in: Represents the i-th sample feature in the source domain With the jth sample feature The similarity weight between them; Represents the i-th sample feature in the target domain With the jth sample feature The similarity weight between them; Represents the i-th sample feature in the source domain and the jth sample feature in the target domain The similarity weight between them; Represents the i-th sample feature in the target domain and the jth sample feature in the source domain The similarity weight between them; n s represents the number of original samples in the source domain; Indicates the number of generated samples of the minority sample category in the source domain; n t Represents the number of samples in the target domain; W ss Represents the similarity graph between source domain sample features; W tt Represents the similarity graph between the sample features of the target domain; W st Represents the similarity graph from the source domain sample features to the target domain sample features; W ts Represents the similarity graph between the target domain sample features and the source domain sample features; argmin means finding the optimal similarity graph W so that the value of the objective function is minimized; S32. Combine the four formulas in S31 to obtain two intra-domain similarity graphs W ss and W tt And two cross-domain similarity graphs W st and W ts , these two cross-domain similarity graphs are fused through an average operation to generate the final cross-domain similarity graph W cro , the calculation method is as follows: IN cro =[0,(W st +W tsT ) / 2;((In st +W tsT ) / 2) T ,0] Where T represents the transpose of the matrix; S33, based on constructing the target domain similarity graph W tt And the final cross-domain similarity graph W cro , the pseudo labels of target domain samples are generated by label propagation through the following optimization method: in, represents the label matrix generated for all samples in the target domain through label propagation, represents the pseudo label of the jth sample in the target domain, represents the similarity weight between the i-th sample and the j-th sample in the cross-domain similarity graph, Represents the label of the i-th sample in the source domain; S34, the source domain sample features and target domain sample features extracted by S1 and the source domain small sample features enhanced by S2 are respectively sent to the classifier for prediction. The calculation method is as follows: p=F Classifier (Z) Among them, F Classifier represents the classifier, Z represents the feature, and p represents the predicted value of the model after softmax processing.

6. The method for small sample unsupervised domain adaptation image classification according to claim 1, characterized in that: The S4 further comprises: S41. Calculate classification loss The calculation is as follows: S42. Calculate graph regularization loss The calculation is as follows: in, and They represent the intra-domain graph regularization loss and the cross-domain graph regularization loss respectively; For the in-domain graph regularization loss The calculation method is as follows: Among them, Z d (d∈{s,t}) represents the feature matrix of the source domain and the target domain, L d represents the Laplacian matrix of the weighted graph in the domain, Tr(.) represents the trace operation; For cross-domain graph regularization loss The calculation method is as follows: Among them, Z s+t =[Z s ; Z t ] represents the source domain feature matrix Z s and the target domain feature matrix Z t The splicing of L cro The Laplacian matrix representing the cross-domain weighted graph; The Laplace matrix L is calculated as follows: L=DW′ Among them, D is the corresponding degree matrix, W′ is the adjacency matrix, and in the calculation When W′=W ss Or W′=W tt , in the calculation When W′=W cro ; S43. Minimize the total loss to optimize the image classification model, the total loss The calculation is as follows: Among them, α and β are hyperparameters.

7. The method for small sample unsupervised domain adaptation image classification according to claim 1, characterized in that: In S5, based on the image classification model optimized in S4, in the image classification prediction, the target domain samples are first sent to the feature extractor to extract features, and then sent to the classifier for classification prediction. The calculation method is as follows: p=F Classifier (F image (x t )) Among them, arg max means finding the index corresponding to the maximum value in p. is the predicted category label of the image.

8. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the processor executes the steps of the small sample unsupervised domain adaptation image classification method as described in any one of claims 1 to 7.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the small sample unsupervised domain adaptation image classification method as described in any one of claims 1 to 7.

Images