Sample weighted domain adaptive image classification method based on source domain variance and Gaussian mixture distribution modeling
The sample weighted domain adaptation method, which models the source domain variance and the Gaussian mixture distribution, solves the performance degradation problem of deep learning image classification under cross-domain data distribution differences, and achieves higher accuracy and robustness.
Patent Information
- Application Number
- CN202511051552.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-29
- Publication Date
- 2025-11-07
AI Technical Summary
Existing deep learning image classification methods struggle to maintain ideal classification performance when target domain labels are scarce or missing. Traditional supervised learning methods show significant performance degradation when the distributions of training and test data differ, and they lack fine-grained feature relationships and probabilistic interpretations.
A sample-weighted domain adaptation image classification method based on source domain variance and Gaussian mixture distribution modeling is adopted. Through data augmentation, pseudo-label pre-training and formal training stages, the source domain sample weights are dynamically calculated. The cross-attention module and KL divergence minimization loss function are used to optimize the model to reduce the difference in feature distribution between the source and target domains.
It significantly improves the accuracy and robustness of image classification, can more finely characterize data distribution, and enhances the model's generalization ability and interpretability in complex cross-domain scenarios.
Smart Images
Figure CN120912979A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of image classification, and particularly relates to a sample weighted domain adaptation image classification method based on source domain variance and mixed Gaussian distribution modeling. BACKGROUND
[0002] As one of the core technologies in the field of computer vision, image classification has important industrial application value in multiple industries. In the field of medical diagnosis, the image classification algorithm based on deep learning can realize intelligent identification and classification of medical images such as X-ray films and CT scan images, assist doctors in lesion detection and disease diagnosis, and significantly improve the accuracy and work efficiency of medical diagnosis. In the intelligent transportation system, the technology classifies and identifies key targets such as pedestrians, vehicles and traffic signs in the road environment in real time, provides reliable perception ability for the automatic driving system, and effectively ensures driving safety. In the industrial manufacturing field, the image classification system can be applied to the automatic detection link of product quality, and through high-precision classification and identification, the automatic screening of defective products on the production line can be realized, which not only greatly reduces the misjudgment rate of manual detection, but also significantly saves labor costs.
[0003] At present, deep learning has become the mainstream method in image classification tasks, covering various technical routes such as convolutional neural network (CNN) and visual Transformer. However, these methods often have difficulty in maintaining ideal classification performance when facing the actual situation of extremely scarce or completely missing target domain labels. Traditional supervised learning methods highly depend on a large amount of labeled data, and when there is a distribution difference between the training data (source domain) and the test data (target domain), the model performance will decrease significantly. Domain adaptation technology can effectively alleviate this problem, and its core lies in reducing the distribution difference between the source domain and the target domain through feature distribution alignment and other means, thereby improving the generalization ability of the model on the target domain.
[0004] In existing domain adaptation methods, although the global alignment strategy based on maximum mean difference (MMD) has achieved certain results, its main limitations are: on the one hand, it is difficult to capture the fine-grained feature relationship at the sample level, and on the other hand, it lacks a clear probabilistic explanation mechanism.
[0005] Therefore, a method is needed to solve the above problems. SUMMARY
[0006] The application provides a sample weighted domain adaptation image classification method based on source domain variance and mixed Gaussian distribution modeling, which gives a more comprehensive adaptation limit that considers the variance of domain loss. By adjusting the weight of the source domain sample, its distribution is closer to the target domain data distribution, improving the generalization ability of the model, thereby improving the accuracy of image classification.
[0007] In order to achieve the above object, the application adopts the following technical scheme:
[0008] The application provides a sample weighted domain adaptation image classification method based on source domain variance and mixed Gaussian distribution modeling, comprising the following steps:
[0009] Step 1, data enhancement stage: after obtaining the source domain and target domain images, generating structure-enhanced source domain features on the source domain side and structure-enhanced target domain features on the target domain side, thereby forming an extended source domain and target domain dataset containing original features and their enhanced versions;
[0010] Step 2, pseudo-label pre-training stage: constructing a domain adaptation image classification model composed of a ResNet50-based feature extractor and a multilayer perceptron classifier, using the extended source domain dataset obtained in step 1 to supervise the training of the model, testing the extended target domain dataset using the model, and preliminarily establishing a target domain pseudo-label set;
[0011] Step 3, formal training stage: jointly training the target domain pseudo-label set and the source domain, dynamically calculating the source domain sample weight using the minimum KL divergence, training the model using the obtained source domain sample weight and the corresponding loss function, and making the source domain feature distribution and the target domain pseudo-label feature form a class-by-class alignment; based on the updated model, reselecting high-confidence target samples to generate more accurate pseudo-labels, forming a closed-loop iteration mechanism of "model optimization-pseudo-label update", and terminating the optimization process when the maximum number of iterations is reached, finally obtaining a domain-invariant feature representation and a target domain strongly adapted domain adaptation image classification model;
[0012] Step 4, application: the target domain obtains an estimation result through the currently trained domain adaptation image classification model, i.e., an image classification result.
[0013] Further, the step 1 is specifically:
[0014] Firstly, the source domain and target domain images are standardized and preprocessed;
[0015] Subsequently, the image feature library of the source domain and the target domain is constructed respectively, which is used to depict the structure distribution and local feature association relationship in each domain, for any image, its local manifold neighborhood is retrieved from the feature library of its corresponding domain based on the Euclidean distance, the top k neighbors are selected, and the neighborhood structure features are formed through weighted aggregation;
[0016] Then, the original features of the image and its neighborhood structure features are jointly input into the cross-attention module, in which the original features of the image are taken as Query, the neighborhood structure features are taken as Key and Value, the attention weight is calculated through the similarity of Query-Key, and the Value is weighted and aggregated based on the attention weight;
[0017] Finally, an extended source and target dataset is formed containing the original features and their manifold neighborhood-attention enhanced versions.
[0018] Further, the training process of the domain adaptation image classification model in step 3 adopts a loss function based on the variance of the source domain, and the parameters of the model are updated through back propagation to make the feature distributions of the source and target domains consistent, specifically:
[0019] 1) An initial weight is randomly assigned to the source domain data, and the calculation formula is as follows:
[0020] ;
[0021] wherein, is the number of samples, is the weight vector of the sample, and the sum of the weights of all samples is 1;
[0022] 2) Calculate the mean vector and covariance matrix of the weighted source domain:
[0023] Using the obtained initial weight, calculate the mean vector and covariance matrix of each class of weighted source domain, and the calculation formula is as follows:
[0024] ;
[0025] ;
[0026] wherein, is the probability distribution of the th class of the weighted source domain, , is the sample matrix of the th class in the source domain, is the dimension of the data, and denotes element-wise multiplication, is the indicator vector of the th class, indicates that the sample belongs to the th class, when belongs to the th class, the th value is 1, otherwise, the value is 0, is a diagonal matrix constructed by element-wise squaring the weight ;
[0027] 3) Using the mean vector and covariance matrix of the weighted source domain, a small number of labeled samples are used for each class of the target domain, and the weighted source domain and the target domain KL divergence value of each class, indirectly control the overall KL divergence by minimizing the maximum KL divergence value in all classes, so as to obtain the optimized source domain sample weight, the calculation formula is as follows:
[0028] ;
[0029] ;
[0030] Wherein, represents the optimized source domain sample weight, represents the number of categories, is the target domain The probability distribution of the first category, is the covariance matrix of the target domain The first category, is the mean vector of the target domain The first category, represents the inverse of
[0031] ; 4) using the optimized source domain sample weight , the loss of the source domain sample and the covariance
[0032] of the loss are calculated, the calculation formula is as follows:
[0033] ;
[0034] ; Wherein,
[0035] represents that the sample obeys the weighted source domain distribution , and the expectation is denoted as , is the hinge loss of multi-classification, represents the true label of the sample , represents the domain adaptation image classification model, represents the predicted label of the model after the sample inputs the model;
[0036] 5) train the model by using the total loss function , and update the parameters of the model by back propagation, and the total loss function is as follows:
[0037] .
[0038] Compared with the prior art, the present application has the following advantages:
[0039] (1) Compared with MMD, Gaussian Mixture Model (GMM) can more finely depict the belonging relationship of samples to each class due to its unique probability modeling advantage, which not only improves the accuracy of distribution modeling, but also significantly enhances the interpretability of the model. By representing the data distribution as a weighted combination of multiple Gaussian components, GMM can more naturally describe complex data distribution patterns and provide more reliable basis for cross-domain classification.
[0040] (2) The application adopts a mixed Gaussian model based on source domain variance to model the probability of source domain data, which can finely depict the belonging relationship of samples to each class and significantly enhance the interpretability of data distribution modeling.
[0041] (3) The application first theoretically proposes an upper bound of the target domain loss generalization error containing KL divergence term, source domain loss and source domain loss variance, innovatively introduces the loss fluctuation of source domain samples into the cross-domain optimization target, fully excavates the information value of source domain samples, guides the model to have stronger robustness and generalization ability while adapting to the target domain distribution, and thus significantly enhances the image classification performance in complex cross-domain scenarios. BRIEF DESCRIPTION OF DRAWINGS
[0042] Figure 1 The figure is a domain adaptation image classification process;
[0043] Figure 2 The figure is an architecture diagram of a domain adaptation image classification model;
[0044] Figure 3 After the weights are optimized, the distribution diagram of the weighted source domain and the unweighted source domain and the target domain ;
[0045] Figure 4 The figure is a visualization diagram of the used dataset. DETAILED DESCRIPTION
[0046] In order to further illustrate the technical scheme of the application, the application will be further described through examples.
[0047] Example 1
[0048] As shown in Figure 1 , a sample weighted domain adaptation image classification method based on source domain variance and mixed Gaussian distribution modeling of the present embodiment comprises the following steps:
[0049] Step 1, data augmentation stage: the present application first proposes an image feature enhancement strategy that fuses manifold structure perception and cross-attention mechanism to improve cross-domain feature modeling capability. After obtaining source domain and target domain images, structure-enhanced source domain features are generated on the source domain side, and structure-enhanced target domain features are generated on the target domain side, thereby forming an extended source domain and target domain dataset containing original features and their enhanced versions;
[0050] More specifically, after obtaining source domain and target domain images, first, the source domain and target domain images are standardized for pretreatment; then, the source domain and target domain image feature libraries are respectively constructed to depict the structure distribution and local feature association relationship in each domain. For any image, its local manifold neighborhood is retrieved from the feature library of its corresponding domain based on Euclidean distance, the top k neighbors are selected, and the neighborhood structure features are formed through weighted aggregation. Next, the original features of the image and its neighborhood structure features are jointly input into the cross-attention module. In the module, the original features of the image are taken as Query, and the neighborhood structure features are taken as Key and Value. The attention weight is calculated through the similarity of Query-Key, and the Value is weighted and aggregated based on the attention weight, so as to realize the selective integration and enhancement of the image features with respect to the neighborhood structure information, and better approximate the manifold space where the image and its neighborhood features are located. Finally, an extended source domain and target domain dataset containing original features and their manifold neighborhood-attention enhanced versions is formed.
[0051] Step 2, pseudo-label pre-training stage: a domain adaptation image classification model (such as Figure 2 indicated) composed of a ResNet50-based feature extractor and a multi-layer perceptron classifier (including multiple groups of Linear and Relu) is constructed. The model is supervised trained using the extended source domain dataset obtained in step 1 to learn the basic classification ability, and the model is tested using the extended target domain dataset to preliminarily establish the target domain pseudo-label set to support subsequent adaptive training.
[0052] Step 3, formal training stage: the target domain pseudo-label set and the source domain are jointly trained, the source domain sample weight is dynamically calculated by minimizing the KL divergence, and the obtained source domain sample weight and the corresponding loss function are used to train the model, so that the source domain feature distribution and the target domain pseudo-label feature form a class-by-class alignment. Based on the updated model, high-confidence target samples are re-screened to generate more accurate pseudo-labels, forming a closed-loop iteration mechanism of "model optimization-pseudo-label update". Through dynamically adjusting the source domain sample weight and the target domain pseudo-label confidence threshold, the gradual alignment of cross-domain feature distribution is realized. When the maximum number of iterations is reached, the optimization process is terminated, and finally the domain-invariant feature representation and the target domain strongly adapted domain adaptation image classification model are obtained.
[0053] In this embodiment, the training process of the domain-adaptive image classification model employs a loss function based on the source domain variance and updates the model parameters through backpropagation to make the feature distributions of the source and target domains tend to be consistent. Specifically, this includes:
[0054] 1) Randomly assign an initial weight to the source domain data, calculated using the following formula:
[0055] (2);
[0056] in, It is the number of samples. It is the weight vector of the samples, and the sum of the weights of all samples is 1;
[0057] 2) Calculate the mean vector and covariance matrix of the source domain after weighting:
[0058] Using the obtained initial weights, calculate the mean vector of the source domain after weighting for each class. Covariance Matrix The calculation formula is as follows:
[0059] (3);
[0060] (4);
[0061] in, It is a source domain with authority The Class probability distribution, , It is the first in the source domain Class sample matrix, It represents the dimension of the data; ⊙ indicates element-wise multiplication. It is the first Class indicator vector, Indicates that the sample belongs to the first Class, when Belongs to the When class, then the first The value is 1 if it is a given condition, and 0 otherwise. For weights A diagonal matrix constructed by squaring each element;
[0062] 3) Using the weighted mean vector of the source domain Covariance Matrix For each class in the target domain, a small number of labeled samples are used to calculate the weighted source domain for each class separately. and target domain The KL divergence value is used to indirectly control the overall KL divergence by minimizing the maximum KL divergence value among all classes, thus obtaining the optimized source domain sample weights. The calculation formula is as follows:
[0063] (5);
[0064] (6);
[0065] wherein, denotes the optimized source domain sample weight, denotes the number of categories, is the target domain class probability distribution, is the target domain class covariance matrix, is the target domain class mean vector, denotes the inverse of ;
[0066] 4) using the optimized source domain sample weight , the loss of the source domain sample and the loss covariance are calculated, and the calculation formula is as follows:
[0067] (7);
[0068] (8);
[0069] (9);
[0070] wherein, denotes that the sample obeys the weighted source domain distribution , and the expectation is denoted as , is the hinge loss of multi-classification, denotes the real label of the sample , denotes the domain adaptation image classification model, denotes the predicted label of the model after the sample inputs the model;
[0071] 5) the model is trained by using the total loss function , and the parameters of the model are updated by back propagation, and the total loss function is as follows:
[0072] (10).
[0073] The target domain is given pseudo labels by using the currently trained model, and step 3 is executed again, and after step 3 is iteratively executed 5 times, the model training is completed.
[0074] Step 4, application: the target domain obtains an estimation result by using the domain-adapted image classification model trained at present, that is, an image classification result.
[0075] To further reveal the theoretical basis of the method of the present application, the following upper bound of generalization error of target domain loss is proposed:
[0076] (1);
[0077] wherein, denotes the target domain distribution, denotes the source domain distribution, denotes the loss measurement function. The inequality reveals for the first time that the upper bound of the target domain loss is composed of three parts: (1) the domain difference term, which measures the distribution difference between the source domain and the target domain, which is characterized by the KL divergence; (2) the source domain loss, which reflects the average performance of the model on the source domain; (3) the variance of the source domain loss, which represents the volatility within the source domain samples.
[0078] Based on the above theoretical basis, a training strategy based on a weighting mechanism is designed in step 3, aiming to optimize the three parts in formula (1) respectively:
[0079] Steps 1) to 3) aim to minimize the domain difference term in the upper bound, by minimizing the class-level KL divergence, to estimate and optimize the weight of the source domain sample;
[0080] Steps 4) and 5) further weight the loss expectation and variance of the source domain based on the above weight, to control the source domain loss and the variance of the source domain loss in the upper bound, thereby improving the cross-domain generalization ability and robustness of the model.
[0081] In summary, the present application innovatively uses Gaussian mixture model to probabilistically model the feature space, which can intuitively represent the data distribution. At the same time, by introducing the source domain loss variance optimization adaptation limit, the source domain sample weight is dynamically adjusted, effectively reducing the feature distribution difference between the source domain and the target domain, thereby improving the accuracy and stability of cross-domain image classification. Compared with traditional methods, the present method not only alleviates the problem of mismatch between the source domain and the target domain distribution, but also more accurately models the variability within the source domain, ensuring excellent recognition performance of the classification model in complex cross-domain environment.
[0082] Example 2
[0083] To verify the effectiveness of this method, this embodiment conducts experiments on the Office Caltech 10 dataset, which consists of four domains (Amazon(A), Caltech(C), Dslr(D), Webcam(W)), paired to form 12 groups: source domain → target domain. The experiment is conducted for 30 epochs and evaluated using two metrics: Precision and Accuracy. The evaluation results are shown in Table 1. The data in Table 1 is the average of the 30 epochs. L and LV in Table 1 correspond to the distribution alignment and model training using equations (7) and (10), respectively. It can be observed that aligning the distribution through equation (10) can more effectively bring the distributions of the source domain and the target domain closer together, improving the domain adaptation effect.
[0084] Table 1 Performance Evaluation Results
[0085]
[0086] As can be seen from Table 1, the method (LV) described in this invention has the best performance in most cross-domain tasks, especially in tasks with large distribution differences (such as D→A and D→C), which significantly improves its image classification effectiveness.
[0087] Figure 2 In the model, the outputs of the source and target domains after passing through the domain-adapted image classification model represent the model's predicted labels for the source and target domains, respectively.
[0088] from Figure 3 It can be seen that the weighted source domain Distribution and target domain distribution The distance is significantly closer, indicating that the distribution of the two domains is effectively aligned by the method described in this invention, thereby verifying the effectiveness of image classification.
[0089] Figure 4 This paper presents a comparison of four vision domains—Amazon, Caltech, DSLR, and Webcam—in the Office Caltech 10 dataset, highlighting the domain differences that need to be overcome in cross-domain adaptation tasks.
[0090] The foregoing has shown and described the main features and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within the present invention.
[0091] Furthermore, it should be understood that although the specification is described in terms of embodiments, not every embodiment includes every feature described. The specification can include implicit combinations of explicitly mentioned features and / or explicit combinations of implicitely mentioned features. Each embodiment depends on the explicit combinations of features and / or the implicit combinations of features made specifically within that embodiment, and each such embodiment can be combined with every other such embodiment to create further embodiments.
Claims
1. A sample weighted domain adaptation image classification method based on source domain variance and Gaussian mixture distribution modeling, characterized in that, The method comprises the following steps: Step 1, data enhancement stage: after obtaining source domain and target domain images, structure-enhanced source domain features are generated on the source domain side, and structure-enhanced target domain features are generated on the target domain side, thereby forming an extended source domain and target domain dataset containing original features and their enhanced versions; Step 2, pseudo-label pre-training stage: a domain adaptation image classification model composed of a ResNet50-based feature extractor and a multi-layer perceptron classifier is constructed, the extended source domain dataset obtained in step 1 is used to supervise the training of the model, the model is used to test the extended target domain dataset, and a target domain pseudo-label set is preliminarily established; Step 3, formal training stage: the target domain pseudo-label set and the source domain are jointly trained, the source domain sample weight is dynamically calculated by minimizing the KL divergence, the source domain sample weight and the corresponding loss function are used to train the model, so that the source domain feature distribution and the target domain pseudo-label feature form a class-by-class alignment; based on the updated model, high-confidence target samples are re-screened to generate more accurate pseudo-labels, forming a closed-loop iteration mechanism of "model optimization-pseudo-label update", and the optimization process is terminated when the maximum number of iterations is reached, and finally a domain-invariant feature representation and a target domain strongly adapted domain adaptation image classification model are obtained; Step 4, application: the target domain obtains an estimation result by using the current trained domain adaptation image classification model, that is, an image classification result is obtained.
2. The sample weighted domain adaptation image classification method based on source domain variance and Gaussian mixture distribution modeling according to claim 1, characterized in that, The step 1 is specifically: Firstly, the source domain and target domain images are standardized and pretreated; Subsequently, the image feature libraries of the source domain and the target domain are respectively constructed to depict the structure distribution and the local feature association relationship in each domain, for any image, its local manifold neighborhood is retrieved from the feature library of its corresponding domain based on the Euclidean distance, the first k nearest neighbors are selected, and the neighborhood structure features are formed by weighted aggregation; Then, the original features of the image and the neighborhood structure features are jointly input into a cross-attention module, in which the original features of the image are taken as Query, the neighborhood structure features are taken as Key and Value, the attention weight is calculated through the similarity of Query-Key, and the Value is weighted and aggregated according to the attention weight; Finally, an extended source domain and target domain dataset containing original features and their manifold neighborhood-attention enhanced versions is formed. 3.The sample weighted domain adaptation image classification method based on source domain variance and Gaussian mixture distribution modeling of claim 1, wherein, The training process of the domain adaptation image classification model in step 3 adopts a loss function based on the variance of the source domain, and updates the parameters of the model through back propagation, so that the feature distributions of the source and target domains tend to be consistent, specifically: 1) an initial weight is randomly assigned to the source domain data, and the calculation formula is as follows: ； wherein, is the number of samples, is the weight vector of the samples, the sum of the weights of all samples being 1; 2) the mean vector and the covariance matrix of the weighted source domain are calculated: Using the initial weights obtained, the mean vector of each weighted source domain is calculated and the covariance matrix , the calculation formula is as follows: ； ； wherein, is a weighted source domain of the th class probability distribution, , is a sample matrix of the th class in the source domain, is the dimension of the data, and is an indicator vector of the th class, indicates that the sample belongs to the th class, and when belongs to the th class, the th value is 1, otherwise, the value is 0, is a diagonal matrix constructed by element-wise squaring the weight . 3) using the weighted source domain mean vector and covariance matrix , for each class of the target domain, a small number of labeled samples are used to calculate the KL divergence value of each class between the weighted source domain and the target domain , by minimizing the maximum KL divergence value in all categories to indirectly control the overall KL divergence, thereby obtaining the optimized source domain sample weight, the calculation formula is as follows: ； ； wherein, denotes the optimized source domain sample weight, denotes the number of classes, is the target domain probability distribution of the th class, is the covariance matrix of the target domain of the th class, is the mean vector of the target domain of the th class, denotes the inverse of ; 4) using the optimized source domain sample weights , the loss of the source domain samples and the covariance of the loss , the calculation formula is as follows: ； ； ； wherein, representative samples subject to a weighted source domain distribution , denoted as , is a hinge loss for multi-classification, denotes a true label of a sample , denotes a domain-adapted image classification model, denotes a predicted label of a sample after inputting the sample into the model; 5) training the model using a total loss function and updating the parameters of the model by backpropagation, the total loss function is: 。