General domain adaptation method based on unified optimal transport framework

Abstract

The application provides a universal domain adaptation method based on a unified optimal transport framework, relates to the fields of deep learning and computer vision, and comprises the following steps: feeding source domain and target domain image samples into a feature extractor to obtain regularized features; performing similarity calculation on the target domain features and source domain prototypes; feeding a similarity matrix into an unbalanced optimal transport solver to obtain an allocation matrix; performing adaptive padding and then solving an optimization problem; using statistical mean to screen out high-confidence samples as common categories, and giving pseudo-labels to calculate a domain alignment loss function; wherein, the target domain samples are spliced with the nearest neighbor features in the memory queue, similarity calculation is performed on the spliced features and target domain prototypes, a similarity matrix is fed into an optimal transport solver to obtain an allocation matrix, the obtained allocation matrix is used as a pseudo-label to calculate a target domain feature discovery loss function, and network training is performed to solve the universal domain adaptation problem.

Description

Technical Field

[0001] This invention relates to a general-purpose adaptive method, belonging to the fields of deep learning and computer vision. Background Technology

[0002] Deep neural networks have improved the performance of many computer vision tasks, but they lack good generalization performance for cross-domain tasks. In cross-domain tasks, the source and target domain data come from different data distributions (illumination, pose, image quality, etc.), causing classifiers trained on labeled source domain data to fail to generalize well to unlabeled target domain data. Unsupervised domain adaptation aims to solve the cross-domain knowledge transfer problem under the closed-set assumption, i.e., the label class sets of the source and target domains are completely identical. Therefore, this closed-set assumption relies heavily on strong prior knowledge about the target domain. However, for unsupervised information in the target domain, the acquisition of strong prior knowledge depends on human annotation, which limits the application of unsupervised domain adaptation in more general and real-world scenarios.

[0003] General domain adaptation relaxes the closed-set assumption, making it a problem more suitable for general application scenarios. It allows both the source and target domains to have their own private categories, thus eliminating the constraint of strong prior knowledge. In other words, general domain adaptation does not require prior knowledge of the corresponding common categories, the total number of categories in the target domain, or other such prior information.

[0004] Existing general-purpose domain adaptation methods use manually defined and tuned hyperparameters as threshold parameters to distinguish common categories. Therefore, these methods are not universally applicable to different common category proportion distributions. In addition, most general-purpose domain adaptation methods treat the target domain's private categories as a uniform class, ignoring their underlying structure. Summary of the Invention

[0005] The technical problem to be solved by this invention is that, to date, the main difficulties faced by general domain adaptation are: most existing methods rely on specific, non-general hyperparameters to distinguish common categories; most existing methods crudely treat the private categories of the target domain as a unified class, resulting in low quality of extracted features.

[0006] To address the aforementioned technical problems, the present invention provides a general domain adaptive method based on a unified optimal transportation framework, characterized by comprising the following steps:

[0007] Step 1: Based on the source domain sample image x s and target domain sample image x t Obtain the source domain feature set corresponding to the training batch. and target domain feature set Where: B is the batch size; These are the B-th source domain feature and the target domain feature, respectively.

[0008] Step 2: Source domain features go through the classifier to get the prediction vector, and the cross-entropy classification loss function L is calculated using the label of the source domain and the classifier prediction vector cls , where each neuron parameter of the classifier is regarded as a prototype of the source domain class in the feature space is the class set of the source domain is the number of classes of the source domain is the th source domain prototype

[0009] Step 3: Similarity calculation is performed between the target domain features and the source domain prototypes to obtain a similarity matrix S The similarity matrix S st is input into the non-equilibrium optimal transport solver to solve the optimized optimal allocation matrix Q st

[0010] Step 4: Normalization is performed on the obtained optimal allocation matrix Q st to obtain the normalized optimal allocation matrix

[0011] Step 5: According to the normalized optimal allocation matrix , the weight of the target domain sample perspective is calculated and the weight of the source domain prototype perspective is calculated , where: is the weight of the i th target domain sample perspective, the superscript t represents the target domain identifier, and the subscript i represents the i th target domain sample feature is the i th row, n th column element of the normalized optimal transport matrix, the superscript st represents the source and target domain identifier, the subscript i represents the i th row, and the subscript n represents the n th column is the weight of the j th source domain prototype for, the superscript s represents the source domain identifier, and the subscript j represents the j th source domain prototype

[0012] Step 6: According to the weight of the target domain sample perspective and the weight of the source domain prototype perspective , high-confidence target domain samples are screened out as public class samples, identified by the pseudo label δ i = 1, where δ i = 1 only when and are both true; otherwise δ i = 0, where is the pseudo label of the i th target domain sample, the superscript t represents the target domain identifier, the subscript i represents the i th target domain sample feature, and the maximum value of the classifier prediction vector is obtained

[0013] ​Step 7: Similarity matrix S st The row maximum operation is used to calculate the percentage of the maximum value of each row being greater than or equal to a rough boundary γ. Based on the percentage, it is determined whether the target domain has more potential public class samples or more potential target domain private class samples.

[0014] Step 8: Adaptive padding mechanism is used to balance the positive and negative sample ratio of the target domain samples. When the public class samples are more, false sample vectors are synthesized as private class samples using a mixed feature vector. When the private class samples are more, the high-confidence target domain samples obtained in step 6 are used for padding.

[0015] Step 9: After padding the samples, steps 3 to 6 are repeated to obtain high-confidence target domain samples in step 6 and assign pseudo labels. A transfer loss function based on the pseudo labels is calculated where L CE (·,·) is the cross-entropy loss function.

[0016] Step 10: The target domain feature Z t is used as a query object to find its nearest neighbor feature in the memory queue as a neighbor feature. The target domain feature Z t and its neighbor feature are vertically concatenated and labeled as

[0017] Step 11: The concatenated target domain feature is compared with a set of learnable target domain prototypes C t to obtain a similarity matrix S The similarity matrix S tt is input into a balanced optimal transport solver to obtain an optimized optimal allocation matrix Q tt .

[0018] Step 12: The obtained optimal allocation matrix Q tt is operated to obtain a soft pseudo label matrix K is the pre-defined number of target domain prototypes.

[0019] Step 13: To pull each target domain sample closer to the matched target domain prototype C t , a loss function based on global clustering is calculated where is the i-th row of the pseudo label matrix obtained by multiplying the optimal transport matrix by a coefficient, and the cross-entropy loss function is defined as q i,k is the k-th element of the label vector of the i-th sample target, p i,k is the k-th element of the obtained prediction probability vector of the i-th sample target, c kis the kth prototype feature vector, z i is the feature vector of the ith sample target;

[0020] Step 14: To make each target domain sample have neighborhood consistency, a local clustering-based loss function L wherein, is the ith target domain feature vector is the corresponding nearest neighbor sample feature vector;

[0021] Step 15: Summing and averaging the global clustering-based loss function L global and the local clustering-based loss function L local , to obtain the target domain representation learning loss function L PCD ;

[0022] Step 16: Weighting the source domain supervised classification loss function L cls , the pseudo-label-based transfer loss function L CCD and the target domain representation learning loss function L PCD , to obtain the final total training loss function L overall ;

[0023] Step 17: Based on the total training loss function L overall , back propagation is performed to calculate the gradient, and the feature extractor parameters and the target domain prototype parameters are updated;

[0024] Step 18: The feature vectors in the memory queue are updated with the target domain sample features of the current training batch;

[0025] Step 19: The weight vector β of the non-uniform optimal transport is adaptively updated, wherein, is the column sum of the normalized optimal allocation matrix , and μ is a parameter for sliding momentum update, controlling the degree of vector update;

[0026] Step 20: Repeat steps 1 to 19 until the specified minimum training iteration number is reached, and finally achieve model convergence, realize inter-domain common class alignment and intra-domain representation learning, and finally realize universal domain adaptation.

[0027] Preferably, in step 1, the source domain sample picture x s and the target domain sample picture x t are input into the feature extractor f to obtain the source domain feature set and the target domain feature set.

[0028] Preferably, in step 3, the optimal allocation matrix Q st is calculated according to the following formula:

[0029]

[0030] wherein: Q is the optimal transport matrix to be solved; Tr(·) is the trace calculation of a matrix; epsilon is the coefficient to control the regularization entropy term; kappa is the coefficient to control the soft penalty term; 1 B is a vector with all elements being 1 with dimension B; B is the training batch size; is a vector with all elements being 1 with dimension , wherein is the number of source domain classes; D KL (·) is the KL divergence; H(·) is the entropy.

[0031] Preferably, in step 7, if the obtained percentage is less than 50%, it means that there are more potential public class samples in the target domain; otherwise, there are more potential target domain private class samples.

[0032] Preferably, in step 11, the optimized optimal distribution matrix Q is solved by using the following formula tt :

[0033]

[0034] wherein, K is the pre-defined number of target domain prototypes, 1 K is a vector with all elements being 1 with dimension K.

[0035] Preferably, in step 15, the target domain representation learning loss function is

[0036] Preferably, in step 16, the total training loss function L overall = L cls + lambda (L CCD + L PCD ), wherein lambda is the weight parameter to control the multi-loss function training.

[0037] Firstly, the present application proposes a partial alignment method based on optimal transport and an adaptive padding mechanism, aiming to generally solve the public class detection problem under different public class ratio distribution conditions, and does not need to rely on specific adjustment parameters, and is more suitable for the problem of domain adaptation in real scenarios. Secondly, the present application proposes a representation learning method based on optimal transport, aiming to help explore the internal structure of the target domain, so as to learn features with strong discrimination and improve the performance of the model in the target domain.

[0038] Therefore, compared with the prior art, the present application has the following beneficial effects:

[0039] (1) The application proposes a unified optimal transport framework for simultaneously solving the common class detection in general domain adaptation and the target domain private class exploration problem.

[0040] (2) The application proposes an optimal transport-based partial alignment method and an adaptive padding mechanism, aiming to generally solve the common class detection problem under different common class ratio distribution conditions, and does not need to rely on specific adjustment parameters, and is more suitable for the domain adaptation problem in real scenarios.

[0041] (3) The application proposes an optimal transport-based representation learning method, aiming to help explore the internal structure of the target domain, so as to learn discriminative features and improve the performance of the model in the target domain. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 The unified optimal transport framework for general domain adaptation is illustrated;

[0043] Figure 2 The common class detection flowchart is shown;

[0044] Figure 3 The adaptive padding mechanism flowchart is shown. DETAILED DESCRIPTION

[0045] The application will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the application and not to limit the scope of the application. In addition, it should be understood that those skilled in the art can make various modifications or changes to the application after reading the content taught by the application, and these equivalent forms also fall within the scope defined by the appended claims.

[0046] In conjunction with Figure 1 , the general domain adaptation method based on the unified optimal transport framework disclosed in this embodiment specifically includes the following steps:

[0047] Step 1: The source domain sample picture x s and the target domain sample picture x t are input into the feature extractor f based on the convolutional neural network (CNN), so as to obtain a source domain feature set and a target domain feature set corresponding to a training batch, wherein B is the batch size; and x are the Bth source domain feature and the target domain feature, respectively, and the feature is a feature vector with a modulus of 1 after L2 regularization.

[0048] In this embodiment, the feature extractor f includes ResNet50 and two layers of full connection mapping layers with the same latitude mapping, and finally maps the 2048-dimensional high-dimensional features to a 256-dimensional feature space through the feature extractor f.

[0049] Step 2: Source domain feature set Z s The source domain features in Z are input into a classifier (without bias) composed of a fully connected layer to obtain a prediction vector. The cross-entropy classification loss function L can be calculated using the labels of the source domain and the prediction vector of the classifier. cls It is worth noting that each neuron parameter of the classifier can be regarded as a prototype of the source domain class in the feature space. Wherein: is the class set of the source domain; is the number of classes of the source domain; is the i-th source domain prototype of the source domain.

[0050] Step 3: Target domain feature set Z t The target domain features in Z are input into the source domain prototype in the class set to perform similarity calculation to obtain a similarity matrix The similarity matrix S st is input into an unbalanced optimal transport solver to solve an optimized optimal allocation matrix Wherein: Q is the optimal transport matrix to be solved; Tr(·) is the trace calculation of the matrix; ε is a coefficient for controlling the regularization entropy term; κ is a coefficient for controlling the soft penalty term; 1 B is a vector of dimension B with all elements being 1; B is a vector of dimension B with all elements being 1; is a vector of dimension with all elements being 1, wherein is the number of source domain classes; D KL (·) is the KL divergence; H(·) is the entropy.

[0051] This step realizes the matching of the target domain features to the source domain prototypes. In this embodiment, the unbalanced optimal transport solver adopts a generalized Sinkhorn algorithm. The two penalty terms based on the KL divergence constrain the row sum and the column sum of Q. During the solving process, ε = 0.01 and κ = 0.05.

[0052] Step 4: Normalization operation is performed on the obtained optimal allocation matrix Q st

[0053] Step 5: According to the normalized optimal allocation matrix obtained in step 4, the weight of the target domain sample perspective and the angle weight of the source domain prototype are calculated respectively. is the weight of the i-th target domain sample perspective, wherein the superscript t represents the target domain identifier, and the subscript i represents the i-th target domain sample feature.​​ is the i-th row, n-th column element of the normalized optimal transport matrix, the superscript st represents the source domain and target domain identity, the subscript i represents the i-th row, and the subscript n represents the n-th column; is the weight of the j-th source domain prototype for, the superscript s represents the source domain identity, and the subscript j represents the j-th source domain prototype.

[0054] Step 6: According to the weight of the target domain sample view and the source domain prototype angle weight Filter out high-confidence target domain samples as public class samples, and use pseudo-label δ i = 1 to identify, where only when and are both true, δ i = 1; otherwise δ i = 0, where, is the pseudo-label of the i-th target domain sample, the superscript t represents the target domain identity, the subscript i represents the i-th target domain sample feature, and is obtained by taking the maximum value of the classifier prediction vector.

[0055] Step 7: Take the row maximum value operation on the similarity matrix S st , and count the percentage of rows whose maximum value is greater than or equal to a rough boundary γ = 0.7. If the obtained percentage is less than 50%, it means that there are more potential public class samples in the target domain; otherwise, there are more potential private class samples in the target domain.

[0056] Step 8: Perform an adaptive padding mechanism to balance the positive and negative sample ratios of the target domain samples, i.e., by padding to keep the number of potential public class samples and private class samples in the target domain constant. When there are more public class samples, synthetic false sample vectors are used as private class samples in the form of mixed feature vectors, i.e., where, is the feature vector of the synthetic false sample vector as the private class sample, is the i-th target domain feature vector, where the superscript t is the target domain identity, and the subscript i is the i-th target domain feature vector; when there are more private class samples, the high-confidence target domain samples obtained in step 6 are used for padding.

[0057] The approximate process of the adaptive padding mechanism is shown in Figure 3 .

[0058] Step 9: After padding the samples, repeat steps 3 to 6 to reacquire the high-confidence target domain samples in step 6 and assign pseudo-labels, and calculate the transfer loss function based on the pseudo-labels where L CE (·,·) is the cross-entropy loss function.

[0059] The general process of public category detection is shown in Figure 2 .

[0060] Step 10: Use the target domain feature Z t as the query object, find its nearest neighbor feature in the memory queue as the neighbor feature, longitudinally splice the target domain feature Z t and the neighbor feature, marked as

[0061] Step 11: Spliced target domain feature and a set of learnable target domain prototypes C t to perform similarity calculation to obtain a similarity matrix S Input the similarity matrix S tt into a balanced optimal transport solver to solve an optimized optimal allocation matrix Q tt = argmax Q∈U Tr(Q T S tt )+ εH(Q), where, K is a pre-defined number of target domain prototypes, 1 K is a vector with K dimensions, and all elements are 1.

[0062] This step realizes matching of the target domain feature to the target domain prototype, wherein the optimal transport solver adopts a Sinkhorn algorithm.

[0063] Step 12: Perform operation on the obtained optimal allocation matrix Q tt to obtain a soft pseudo-label matrix

[0064] Step 13: To pull each target domain sample to the target domain prototype C t to which it is matched, calculate a global clustering-based loss function wherein, is the i-th row of the pseudo-label matrix obtained by multiplying the optimal transport matrix by a coefficient, and the cross-entropy loss function is defined as q i.k is the k-th element of the label vector of the i-th sample target, p i.k is the k-th element of the obtained prediction probability vector of the i-th sample target, c k is the k-th prototype feature vector, z i is the feature vector of the i-th sample.

[0065] Step 14: To make each target domain sample have neighborhood consistency, calculate a local clustering-based loss function wherein, is the i-th target domain feature vector corresponding nearest neighbor sample feature vector.

[0066] Step 15: Add and average the global clustering based loss function L global and the local clustering based loss function L local to obtain the target domain representation learning loss function L

[0067] Step 16: Add the above source domain supervised classification loss function L cls , the pseudo label based transfer loss function L CCD and the target domain representation learning loss function L PCD to obtain the final total training loss function L overall = L cls + λ (L CCD + L PCD ), where λ is a weight parameter controlling the multi-loss function training, in the embodiment, λ = 0.1.

[0068] Step 17: Based on the total training loss function L overall , the gradient is calculated by back propagation, and the feature extractor parameters and the target domain prototype parameters are updated.

[0069] Step 18: Update the feature vectors in the memory queue with the target domain sample features of the current training batch.

[0070] Step 19: Self-adaptively update the weight vector β of the unbalanced optimal transport, wherein, the column sum of the normalized optimal allocation matrix , and μ is a parameter of sliding momentum update, controlling the degree of vector update.

[0071] Step 20: Repeat steps 1 to 19 until the specified minimum training iteration number is reached to achieve model convergence, realize inter-domain common class alignment and intra-domain representation learning, and finally realize universal domain adaptation.

[0072] In the testing phase, only the feature extractor and the source domain prototype based classifier are retained, and after the target domain test sample is obtained by the feature extractor, a splicing operation is performed. Repeat steps 3 to 8 to calculate for each sample , and assign the nearest source domain prototype to the test sample as the predicted label, and determine other samples as unknown classes, i.e. target domain private classes, wherein n is the total number of features after adaptive padding.

[0073] The method is suitable for open domain pattern recognition problems, especially in the case that there is a large feature distribution difference between the training set and the test set, and a large number of unknown labels and no prior samples, such as automatic driving target classification, industrial object detection, face recognition and other industrial recognition tasks.

Claims

1. A general domain adaptive method based on a unified optimal transportation framework, characterized in that, Includes the following steps: Step 1: Based on the source domain sample image x s and target domain sample image x t Obtain the source domain feature set corresponding to the training batch. and target domain feature set Where: B is the batch size; These are the B-th source domain feature and the target domain feature, respectively. Step 2: The source domain features are processed by a classifier to obtain a predicted vector. The cross-entropy classification loss function L is then calculated using the source domain labels and the classifier's predicted vector. cls In this approach, each neuron parameter of the classifier is considered as a prototype of the source domain class in the feature space. in: It is the set of categories of the source domain; The number of categories in the source domain; It is the first in the source domain One source domain prototype; Step 3: Calculate the similarity matrix by comparing the target domain features with the source domain prototype. The similarity matrix S st The input is fed into the unbalanced optimal transport solver to solve for the optimized allocation matrix Q. st ; Step 4: Apply the obtained optimal allocation matrix Q st Perform a normalization operation to obtain the normalized optimal allocation matrix. Step 5: Based on the normalized optimal allocation matrix Calculate the weights of the target domain sample perspectives respectively. and source domain prototype angle weights in: It represents the weight from the perspective of the i-th target domain sample, where the superscript t indicates the target domain identifier and the subscript i indicates the feature of the i-th target domain sample; It is the element in the i-th row and n-th column of the normalized optimal transport matrix. The superscript st indicates the source and destination domain identifiers, the subscript i indicates the i-th row, and the subscript n indicates the n-th column. It is the weight of the j-th source domain prototype, where the superscript s represents the source domain identifier and the subscript j represents the j-th source domain prototype; Step 6: Weights based on the target domain sample perspective Source domain prototype angle weights High-confidence target domain samples were selected as common category samples, and pseudo-labels δ were used. i =1 identifier, where only when and When they are established simultaneously, δ i =1; otherwise δ i =0, where, It is the pseudo-label of the i-th target domain sample, where the superscript t represents the target domain identifier and the subscript i represents the feature of the i-th target domain sample, which is obtained by taking the maximum value of the classifier prediction vector; Step 7: For the similarity matrix S st The maximum value operation is performed in each row. The percentage of each row whose maximum value is greater than or equal to a rough limit γ is counted. Based on the percentage, it is determined whether there are more potential common category samples or potential private category samples in the target domain. Step 8: Implement an adaptive filling mechanism to balance the ratio of positive to negative samples in the target domain: when there are more common category samples, use a hybrid feature vector approach to synthesize spurious sample vectors as private category samples; when there are more private category samples, use the high-confidence target domain samples obtained in Step 6 as filling. Step 9: After filling the samples, repeat steps 3 to 6, re-obtain the high-confidence target domain samples from step 6, assign pseudo-labels, and calculate the migration loss function based on the pseudo-labels. Among them, L CE (·, ·) is the cross-entropy loss function; Step 10: Use target domain features Z t As the query object, find its nearest neighbor features in the memory queue and use them as neighbor features, then vertically concatenate the target domain features Z. t and its neighboring features, marked as Step 11: Concatenating target domain features With a set of learnable target domain prototypes C t Perform similarity calculations to obtain the similarity matrix. The similarity matrix S tt Input into the balanced optimal transport solver to solve for the optimized allocation matrix Q. tt ; Step 12: Apply the obtained optimal allocation matrix Q tt Perform operations to obtain a soft pseudo-label matrix. K is the number of predefined target domain prototypes; Step 13: To ensure that each target domain sample is aligned with the matched target domain prototype C t Zoom in and calculate the loss function based on global clustering. in, It is the i-th row of the pseudo-label matrix obtained by multiplying the optimal transport matrix by the coefficients. The cross-entropy loss function is defined as... q i，k p is the k-th element of the label vector of the i-th sample target. i，k It is the k-th element of the predicted probability vector obtained for the i-th sample target. c k It is the k-th prototype feature vector, z i It is the feature vector of the i-th sample target; Step 14: To ensure neighborhood consistency for each target domain sample, calculate the loss function based on local clustering. In the formula, It is the feature vector of the i-th target domain. The corresponding nearest neighbor sample feature vector; Step 15: Summarize and average the loss function L based on global clustering global and loss function L based on local clustering local Obtain the target domain representation learning loss function L. PCD ; Step 16: Weighted source domain supervised classification loss function L cls 1. Pseudo-label-based migration loss function L CCD and the target domain representation learning loss function L PCD The final total training loss function L is obtained. overall ; Step 17: Based on the total training loss function L overall Backpropagation is used to calculate the gradient and update the feature extractor parameters and the target domain prototype parameters. Step 18: Update the feature vector in the memory queue with the features of the target domain samples in this training batch; Step 19: Adaptively update the weight vector β of the unbalanced optimal transport. in, The vector is a normalized optimal allocation matrix. The column sum, μ is the parameter for sliding momentum update, which controls the degree of vector update; Step 20: Repeat steps 1 to 19 until the minimum number of training iterations is reached, eventually achieving model convergence, realizing cross-domain common category alignment and intra-domain representation learning, and ultimately achieving general domain adaptation.

2. The general domain adaptive method based on a unified optimal transportation framework as described in claim 1, characterized in that, In step 1, the source domain sample image x s and the target domain sample image x t The source domain feature set and the target domain feature set are obtained through feature extractor f.

3. The general domain adaptive method based on a unified optimal transportation framework as described in claim 1, characterized in that, In step 3, the optimal allocation matrix Q st The calculation formula is as follows: Where: Q is the optimal transfer matrix to be solved; Tr(·) is the trace of the matrix; ε is the coefficient controlling the regularization entropy term; κ is the coefficient controlling the soft penalty term; 1 B It is a vector of dimension B, with all elements being 1; B is the training batch size; It is a dimension of A vector consisting entirely of 1s, where It is the number of source domain classes; D KL (·) is the KL divergence; H(·) is the entropy.

4. The general domain adaptive method based on a unified optimal transportation framework as described in claim 1, characterized in that, In step 7, if the obtained percentage is less than 50%, it means that there are more potential public category samples in the target domain; otherwise, there are more potential private category samples in the target domain.

5. The general domain adaptive method based on a unified optimal transportation framework as described in claim 1, characterized in that, In step 11, the optimized allocation matrix Q is solved using the following formula. tt : in, K is the number of predefined target domain prototypes, 1 K It is a vector of dimension K with all elements equal to 1.

6. The general domain adaptive method based on a unified optimal transportation framework as described in claim 1, characterized in that, In step 15, the target domain representation learning loss function 7. The general domain adaptive method based on a unified optimal transportation framework as described in claim 1, characterized in that, In step 16, the total training loss function L overall =L cls +λ(L CCD +L PCD ), where λ is the weight parameter that controls the training of multiple loss functions.

Images