An unsupervised multi-source partial domain adaptation image classification method

Abstract

The application discloses a kind of unsupervised multi-source partial domain adaptation image classification methods, comprising: the extraction of domain invariant feature and domain private feature is carried out to each sample of source domain and target domain;Using maximum average difference measure and conditional distribution difference measure, the alignment degree of source domain and target domain sample distribution is calculated, to reduce the distribution difference of both, and by the measurement result of the Euclidean distance between the obtained samples, after normalization processing, the balance factor ω is calculated;Source domain sample is imported into the obtained classifier of training, and the source domain sample is weighted according to probability label to reduce the negative transfer caused by irrelevant class sample;Domain classifier is constructed, and the domain private feature extractor is further trained, so that the obtained domain private feature is more discriminative;Calculate overall loss, and iteratively update optimization model parameters.The application can solve the optimization inconsistency problem between domain alignment task and classification task itself and the negative transfer problem caused by irrelevant class sample in partial domain adaptation.

Description

Technical Field

[0001] The present invention belongs to the technical field of domain adaptation in transfer learning, and in particular relates to an unsupervised multi-source partial domain adaptation image classification method based on diversity feature learning and contribution dynamic weighted learning. Background Art

[0002] Domain adaptation primarily addresses the problem of having source and target domains with the same features and categories, but with different data feature distributions. By transferring knowledge from the source domain to the target domain, the target domain task can be solved. Traditional machine learning often assumes that training set samples and test set samples have the same distribution, thereby improving the accuracy of the prediction model when predicting labels. However, in many current learning scenarios, these two often have different distributions, which are reflected in the marginal distribution and conditional distribution. In real-world scenarios, there are often large training datasets. Training with large amounts of data can significantly improve prediction accuracy. However, if a model is trained on multiple datasets, the distribution differences between the data in different datasets may have a greater impact on the model's test results. Ignoring distribution differences often leads to overfitting of the model, resulting in prediction results that do not achieve ideal accuracy.

[0003] While existing domain adaptation models have been successfully applied in various fields, they primarily rely on the assumption of a fully shared label space by reducing domain differences. However, in real-world applications, the target domain label space is often a subset of the source domain label space. Domain adaptation in this scenario is called partial domain adaptation (PDA). When discussing domain adaptation from a big data perspective, we can assume that the diversity of large-scale datasets is sufficient to cover all categories in small-scale datasets, which fits the definition of the partial domain adaptation problem. With the widespread use of big data, the partial domain adaptation problem will receive more extensive attention and research. Compared to standard domain adaptation problems, the partial domain adaptation problem is more universal and challenging.

[0004] In addition, although current research in the field of domain adaptation has studied both unsupervised multi-source domain scenarios and partial domain adaptation problems, there is little research on partial domain adaptation problems in unsupervised multi-source domain scenarios. Summary of the Invention

[0005] Technical problems solved: In response to the shortcomings of existing technologies and the lack of relevant research, the present invention proposes an unsupervised multi-source partial domain adaptation image classification method based on diversity feature learning and contribution dynamic weighted learning to solve the optimization inconsistency problem between the domain alignment task and the classification task itself in the existing unsupervised multi-source domain partial domain adaptation method, as well as the negative transfer problem caused by irrelevant class samples in partial domain adaptation.

[0006] Technical solution:.

[0007] An unsupervised multi-source partial domain adaptation image classification method, the classification method comprising the following steps:

[0008] S1, Diversity Feature Extraction: Extract domain-invariant features and domain-private features for each sample in the source and target domains;

[0009] S2, constructing a dynamic weight factor: using the maximum average difference metric and the conditional distribution difference metric to calculate the degree of alignment of the source domain and target domain sample distributions to narrow the distribution difference between the two, and calculating the balance factor ω after normalizing the Euclidean distance measurement results between the obtained samples;

[0010] S3, dynamic weighting of source domain samples: The source domain samples are imported into the trained classifier and weighted according to the probability labels to reduce the negative transfer caused by irrelevant class samples;

[0011] S4, build a domain classifier and further train the domain-private feature extractor to make the obtained domain-private features more discriminative;

[0012] S5, calculate the overall loss and iteratively update the optimized model parameters.

[0013] Furthermore, in step S1, the process of extracting diversity features includes the following steps:

[0014] Construct two feature extractors and use them to obtain domain-invariant features f c and domain-private feature f d The domain-invariant feature representation is shown in formula (1):

[0015]

[0016] The domain-private feature representation is shown in formula (2):

[0017]

[0018] in, j=1,2,…,n t ,k=1,2,…,K; represents the number of samples in the k-th source domain, n t represents the number of samples in the target domain, represents the features of the i-th sample of the k-th source domain extracted by the k-th domain-invariant feature extractor, represents the feature of the jth sample in the target domain extracted by the kth domain-invariant feature extractor, represents the features of the i-th sample of the k-th source domain extracted by the k-th domain-private feature extractor, represents the features of the jth sample of the kth source domain extracted by the kth domain-private feature extractor, represents the feature extractor of the k-th source domain, are the relevant parameters, They represent the features in the shared feature space of the i-th sample in the k-th source domain and the features in the shared feature space of the i-th sample in the target domain, respectively, and K represents the number of source domains.

[0019] Furthermore, in step S2, the process of constructing the dynamic weight factor includes the following steps:

[0020] S21, calculate the data distribution alignment between each sample in the source domain according to formula (3) to minimize the difference in domain invariant features of the source domain

[0021]

[0022] in, are samples from the k1th and k2th source domains, respectively, by minimizing the loss To align the distribution of source domain samples;

[0023] S22, calculate the data distribution alignment between each sample in the source domain according to formula (4) to maximize the difference of domain-private features in the source domain

[0024]

[0025] According to formula (5), the source domain distribution alignment target is calculated

[0026]

[0027] S23, calculate the discriminability of the source domain and the target domain according to formula (3)

[0028]

[0029] in, X t are samples from the source domain and target domain respectively;

[0030] S24: Conditionally align the source and target domain samples to reduce the intra-class distance and expand the inter-class distance.

[0031]

[0032] Among them, C is the number of source domain classes, r1, r2 are the corresponding categories, represents the samples of the same r class in the k-th source domain and target domain, Represents samples of different categories in each source domain, Represents samples of different categories in the target domain, and λ is a hyperparameter;

[0033] S25, combined with the above formulas (6) and (7) to form the target domain adaptation loss

[0034]

[0035] S26, construct a balancing factor w based on the distance measurement between source domain and target domain samples:

[0036]

[0037] in, represents the initial weight of the k-th source domain, Represents the corresponding weight after normalization.

[0038] Furthermore, in step S3, the process of dynamically weighting source domain samples includes the following steps:

[0039] Construct a classifier to obtain probability labels, use the obtained probability labels to calculate the weight γ, and eliminate the negative transfer caused by irrelevant class samples in the source domain; update the network parameters of the domain invariant feature extractor and classifier according to formula (10):

[0040]

[0041] Among them, G c ,G y are domain-invariant feature extractors and classifiers, θ c ,θ y denote the domain-invariant feature extractor and classifier parameters respectively, y is the true label of the source domain, and L y (θ c ,θ y ) represents the classifier training loss, n s represents the number of source domain samples;

[0042] The probability label y obtained by the target domain sample through the classifier is defined as the target domain sample pseudo label, and the weight γ is calculated according to formula (11):

[0043]

[0044] Among them, n t Indicates the number of target domain samples.

[0045] Furthermore, in step S4, the network parameters of the domain-specific feature extractor and domain classifier are updated according to formula (12):

[0046]

[0047] Among them, G p ,G d are domain-specific feature extractors and domain classifiers, θ p ,θ d denote the domain-specific feature extractor and domain classifier parameters, d is the domain label, and n s Indicates the number of source domain samples.

[0048] Furthermore, in step S5, the overall loss is calculated according to formula (13):

[0049]

[0050] Among them, α and β are hyperparameters.

[0051] Furthermore, the classification method further comprises the following steps:

[0052] According to formula (14), using the obtained weight Predict the label:

[0053]

[0054] in, is a linear combination of domain-invariant features and domain-private features; Y t is the predicted target domain label.

[0055] Beneficial effects:

[0056] This paper proposes an unsupervised multi-source partial domain adaptation image classification method based on diversity feature learning and dynamic weighted contribution learning. First, a domain-invariant feature extractor and a domain-private feature extractor are constructed to extract features from source and target domain samples. Then, the marginal distributions and conditional distributions of the source and target domains are aligned using the maximum average difference metric and the conditional distribution difference metric to eliminate their distribution differences. Secondly, a distance metric is used to dynamically weight the contribution balance factor w of different source domains to the domain adaptation process. Then, a classifier and a domain classifier are constructed to further train the domain-invariant feature extractor and the domain-private feature extractor. Finally, the overall target loss is calculated through continuous iteration, and the model parameters are optimized and updated. In this way, both domain alignment learning and classification learning can be kept in an effective learning state, and the negative transfer problem in the partial domain adaptation process can be effectively solved. The present invention combines domain alignment learning and weighted learning in unsupervised domain adaptation. In multi-source domain adaptation problems and partial domain adaptation problems, domain alignment loss and classification loss are calculated and corresponding weights are obtained. The present invention updates the network parameter θ based on the calculated domain alignment loss and uses θ to calculate the classification loss, thereby enhancing the consistency of the optimized network parameters. The present invention makes the model more generalizable and has higher classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a flow chart of the unsupervised multi-source partial domain adaptation image classification method according to an embodiment of the present invention;

[0058] Figure 2 This is a network model diagram of the unsupervised multi-source partial domain adaptation image classification method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The following examples may enable those skilled in the art to more fully understand the present invention, but are not intended to limit the present invention in any way.

[0060] The present invention proposes an unsupervised multi-source partial domain adaptation image classification method based on diversity feature learning and contribution dynamic weighted learning, which includes the following steps. Figure 1 As shown:

[0061] Step 1: Diversity feature extraction.

[0062] In the unsupervised multi-source domain adaptation problem, the knowledge of a single source domain may not be sufficient to predict the target task. Therefore, it is considered to utilize multiple source domains with richer transferable knowledge to complete the target task. However, different source domains have different knowledge, and ignoring these differences often fails to achieve ideal transfer results, resulting in negative transfer during the domain adaptation process.

[0063] In order to more effectively utilize the rich knowledge in multiple source domains, the present invention constructs domain-invariant feature and domain-private feature extractors to extract features from multiple source domains to promote model migration and generalization capabilities. Specifically, the domain-invariant feature and domain-private feature extraction method for each sample in the source domain and the target domain is as follows: Two feature extractors are constructed, whose representations are shown in Equations (1) and (2), and the domain-invariant feature f is obtained. c and domain-private feature f p .

[0064] The domain-invariant feature representation is shown in formula (1):

[0065]

[0066] The domain-private feature representation is shown in formula (2):

[0067]

[0068] in, j=1,2,…,n t ,k=1,2,…,K, represents the feature extractor of the k-th source domain, are the relevant parameters, f t Represents features in a shared feature space.

[0069] Step 2: Construct dynamic weight factors.

[0070] In unsupervised multi-source domain adaptation, different source domains contribute differently to the target domain migration task. Therefore, properly measuring the similarity between different source and target domains is key to solving the multi-source domain adaptation problem. This paper uses the maximum average difference metric and the conditional distribution difference metric to calculate the degree of alignment between the source and target domain sample distributions, narrowing the distribution gap to a certain extent. The resulting Euclidean distance between samples is then normalized to calculate the balancing factor ω.

[0071] Specifically, the degree of data distribution alignment between samples in the source domain is calculated to minimize the difference in domain-invariant features of the source domain:

[0072]

[0073] in, are the k1th and k2th source domain samples respectively, and the distribution between source domain samples is aligned by minimizing the loss.

[0074] According to formula (4), the degree of data distribution alignment between samples in the source domain is calculated to maximize the difference of domain-private features in the source domain:

[0075]

[0076] Therefore, combining the above steps, we can derive the source domain distribution alignment target, as shown in formula (5):

[0077]

[0078] The discriminability of the source domain and the target domain is calculated according to formula (3):

[0079]

[0080] in, X t are samples from the source domain and target domain respectively.

[0081] Then, conditional distribution alignment is performed on the source domain and target domain samples, which reduces the intra-class distance and expands the inter-class distance:

[0082]

[0083] Among them, C is the number of source domain classes, r1 and r2 are the corresponding categories.

[0084] Combining the above formulas (6) and (7) to form the target domain adaptation loss:

[0085]

[0086] Construct a balancing factor ω based on the distance measurement between source domain and target domain samples:

[0087]

[0088] Step 3: Dynamic weighting of source domain samples.

[0089] In the partial domain adaptation problem, irrelevant source domain samples can cause significant negative transfer during the migration process. To effectively eliminate this negative transfer, the present invention weights source domain samples accordingly using their probabilistic labels, thereby reducing the importance of irrelevant source domain samples in the migration process. Specifically, the present invention constructs a classifier to obtain probabilistic labels, and then uses these probabilistic labels to calculate the weight γ, thereby eliminating the negative transfer caused by irrelevant source domain samples.

[0090] The classification loss is calculated and the network parameters θ are updated according to formula (10):

[0091]

[0092] Among them, G c ,G y are domain-invariant feature extractors and classifiers, θ c ,θy denote the domain-invariant feature extractor and classifier parameters respectively, and y is the true label of the source domain.

[0093] The probability label y obtained by the target domain sample through the classifier is used as the target domain sample pseudo label, and the weight γ is obtained from it:

[0094]

[0095] Step 4: Further train the feature extractor.

[0096] To promote positive transfer during domain adaptation, i.e., to promote the alignment of marginal distributions and conditional distributions in the proposed method, the present invention further trains the constructed domain-private feature extractor to make the extracted domain-private features more discriminative. Specifically, by constructing a domain classifier, the domain-private feature extractor is further trained to make the extracted domain-private features more discriminative. The classification loss is calculated, and the network parameters θ are updated according to Equation (12):

[0097]

[0098] Among them, G p ,G d are domain-specific feature extractors and domain classifiers, θ p ,θ d denote the domain-specific feature extractor and domain classifier parameters, respectively, and d is the domain label.

[0099] Step 5: Calculate the overall loss and iteratively update and optimize the model parameters.

[0100] Combined with the target loss proposed in the above process, the overall target loss of the model constructed by the present invention is constructed, and its form is as shown in formula (13):

[0101]

[0102] Among them, α and β are hyperparameters.

[0103] For the final prediction of the target domain sample label, considering that different source domains have different contributions to the target domain task, the present invention uses the obtained weights The main form of label prediction is as follows:

[0104]

[0105] in, That is, the linear combination of domain-invariant features and domain-private features, Y t is the predicted target domain label.

[0106] The present invention provides an unsupervised multi-source partial domain adaptation image classification method based on diversity feature learning and dynamic contribution weighted learning. There are many methods and approaches to implement this technical solution. The above is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention. Such improvements and modifications should also be considered within the scope of protection of the present invention. Any components not specified in this embodiment may be implemented using existing technologies.

Claims

1. An unsupervised multi-source partial domain adaptation image classification method, characterized by: The classification method comprises the following steps: S1, Diversity Feature Extraction: Extract domain-invariant features and domain-private features for each sample in the source and target domains; S2, constructing a dynamic weight factor: using the maximum average difference metric and the conditional distribution difference metric to calculate the degree of alignment of the source and target domain sample distributions, narrowing the distribution difference between the source and target domain samples, and calculating the balance factor ω after normalizing the Euclidean distance measurement results between the obtained samples; S3, dynamic weighting of source domain samples: The source domain samples are imported into the trained classifier and weighted according to the probability labels to reduce the negative transfer caused by irrelevant class samples; S4, build a domain classifier and further train the domain-private feature extractor to make the obtained domain-private features more discriminative; S5, calculate the overall loss and iteratively update the optimized model parameters; In step S1, the process of extracting diversity features includes the following steps: Construct two feature extractors and use them to obtain domain-invariant features f c and domain-private feature f d ; The domain-invariant feature representation is shown in formula (1): The domain-private feature representation is shown in formula (2): in, j=1,2,…,n t ,k=1,2,…,K; represents the number of samples in the k-th source domain, n t represents the number of samples in the target domain, represents the features of the i-th sample of the k-th source domain extracted by the k-th domain-invariant feature extractor, represents the feature of the jth sample in the target domain extracted by the kth domain-invariant feature extractor, represents the features of the i-th sample of the k-th source domain extracted by the k-th domain-private feature extractor, represents the features of the jth sample of the kth source domain extracted by the kth domain-private feature extractor, represents the feature extractor of the k-th source domain, are the relevant parameters, They represent the features in the shared feature space of the i-th sample in the k-th source domain and the features in the shared feature space of the i-th sample in the target domain, respectively, and K represents the number of source domains; In step S2, the process of constructing the dynamic weight factor includes the following steps: S21, calculate the data distribution alignment between each sample in the source domain according to formula (3) to minimize the difference in domain invariant features of the source domain in, are samples from the k1th and k2th source domains, respectively, by minimizing the loss To align the distribution of source domain samples; S22, calculate the data distribution alignment between each sample in the source domain according to formula (4) to maximize the difference of domain-private features in the source domain According to formula (5), the source domain distribution alignment target is calculated S23, calculate the discriminability of the source domain and the target domain according to formula (3) in, X t are samples from the source domain and target domain respectively; S24: Conditionally align the source and target domain samples to reduce the intra-class distance and expand the inter-class distance. Among them, C is the number of source domain classes, r1, r2 are the corresponding categories, represents the samples of the same r class in the k-th source domain and target domain, Represents samples of different categories in each source domain, Represents samples of different categories in the target domain, and λ is a hyperparameter; S25, combined with the above formulas (6) and (7) to form the target domain adaptation loss S26, construct a balancing factor w based on the distance measurement between source domain and target domain samples: in, represents the initial weight of the k-th source domain, Represents the corresponding weight after normalization; In step S3, the process of dynamically weighting source domain samples includes the following steps: Construct a classifier to obtain probability labels, use the obtained probability labels to calculate the weight γ, and eliminate the negative transfer caused by irrelevant class samples in the source domain; update the network parameters of the domain invariant feature extractor and classifier according to formula (10): Among them, G c ,G y are domain-invariant feature extractors and classifiers, θ c ,θ y denote the domain-invariant feature extractor and classifier parameters respectively, y is the true label of the source domain, and L y (θ c ,θ y ) represents the classifier training loss, n s represents the number of source domain samples; Pass the target domain sample through the classifier to obtain the probability label It is defined as the pseudo label of the target domain sample, and the weight γ is calculated according to formula (11): Among them, n t Indicates the number of samples in the target domain; In step S4, the network parameters of the domain-specific feature extractor and domain classifier are updated according to formula (12): Among them, G p ,G d are domain-specific feature extractors and domain classifiers, θ p ,θ d denote the domain-specific feature extractor and domain classifier parameters, d is the domain label, and n s represents the number of source domain samples; In step S5, the overall loss is calculated according to formula (13): Among them, α and β are hyperparameters.

2. The unsupervised multi-source partial domain adaptation image classification method according to claim 1, characterized in that The classification method further comprises the following steps: According to formula (14), using the obtained weight Predict the label: in, is a linear combination of domain-invariant features and domain-private features; Y t is the predicted target domain label.

Images