Domain adaptation method and device based on contrastive adversarial learning

Abstract

Domain adaptation method and device based on contrastive adversarial learning, using loss function L on source domain data cls (x s ,y s ) Train the entire network model, fix the parameters in the feature extractor, only update the classifiers C1 and C2, minimize the classifier classification loss and maximize the classifier's discrimination difference on the target domain samples, fix the parameters in the classifiers C1 and C2, and use L dis Update the parameters in the feature extractor, retaining the adaptive loss term in this step. Classifiers C1 and C2 use features from different data augmentation methods, thus ensuring classifier diversity. This allows the dual classifier to more efficiently identify samples at the classification boundary, ensuring that the learned features contain more effective information, thereby effectively solving the problem of unsupervised domain adaptation. Based on the traditional dual-classifier adversarial method, this method not only considers the decision boundary of the classifier in the target domain, but also further focuses on the differences between domains.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a domain adaptation method based on contrastive adversarial learning, and a domain adaptation device based on contrastive adversarial learning. Background Art

[0002] The present invention focuses on solving the image classification problem of unsupervised domain adaptation, combining deep networks and domain adaptation problems. By processing the features of the samples in the feature space, the difference in distribution between the source domain and the target domain is reduced, so that the knowledge learned in the source domain can also be applied to the target domain. Deep unsupervised domain adaptation is a research field that has attracted much attention, and a large number of scholars have participated in research work in this field. At present, deep unsupervised domain adaptation methods can be mainly divided into three categories: methods based on distribution distance metrics, methods based on domain adversarial methods, and methods based on reconstruction errors.

[0003] (1) Method based on distribution distance measurement:

[0004] The core idea of this type of method is to align data from different domains by minimizing the distribution distance between them. Currently, three distribution distance metrics—maximum mean discrepancy (MMD), correlation alignment (CORAL), and the Wasserstein metric—are widely used in this type of algorithm. The maximum mean discrepancy (MMD) was originally used to test whether two distributions are identical, but is now generally used to measure the similarity between them. This metric finds a continuous function in the sample space and calculates the mean of the function values of samples from different distributions on this function to determine the mean difference between the two distributions corresponding to this function. If the means differ, the samples are likely not from the same distribution, as the MMD distance between them is zero if and only if the two distributions are equal. When applying this idea to deep adaptive methods, the MMD distance between feature distributions can be reduced by constraining the deep network to minimize the MMD distance between them, thereby minimizing the distributional differences between the output features of corresponding layers in the deep network. Considering the multi-layered structure of a deep network, each layer outputs corresponding features. To explore the compatibility between MMD distance constraints and outputs from different layers, Long et al. proposed the polynomial kernel MMD (MK-MMD) algorithm. Furthermore, Bousmalis et al. explored the effectiveness of MMD-based domain adaptation methods. Adaptive methods based on correlation alignment (CORAL), similar to MK-MMD, consider the spatial structure of the entire domain and align inter-domain distributions from the perspective of data volatility. These methods reduce the distance between the second-order statistics (covariance) of source and target features to make the data volatility similar. This approach makes the structures of different distributions in the two domains more consistent, thereby reducing the difference between the two distributions. The Wasserstein distance, also known as the earthmoving distance (EMD), defined by the optimal transportation problem, is used to measure the distance between different distributions. These methods minimize the difference in domain distributions by solving the optimal transportation planning problem.

[0005] (2) Domain adversarial-based methods:

[0006] Scholars have incorporated adversarial thinking into DA, resulting in domain adaptation methods based on domain adversarial methods. These adversarial domain adaptation methods introduce a domain discriminator, successfully combining the domain adaptation problem with adversarial networks. The domain discriminator maximizes the difference in domain distributions, while the feature extractor minimizes the difference. Through adversarial learning between the feature extractor and the domain discriminator, these methods project data from the source and target domains into a common space, obtaining domain-invariant representations of the data in this space. These domain-invariant representations are then used to eliminate inter-domain data differences. This domain adversarial approach typically requires searching for the optimal solution through alternating iterations. To reduce the model's time complexity, a gradient reversal layer is introduced, making the network an end-to-end model and thus reducing model complexity. Furthermore, Shen et al. used a Wasserstein distance-constrained domain discriminator for domain adversarial learning, reducing the difference in data distributions and achieving promising results.

[0007] (3) Error reconstruction-based methods

[0008] Rather than reducing the differences between data distributions, this method assumes that features that provide sample classification information can be obtained and used to reconstruct the original data. This type of method requires an encoder and a decoder. The encoder encodes the source domain data, and the classifier then classifies the features. This ensures that the features generated by the encoder can distinguish source domain samples (i.e., they are relatively good features). The decoder decodes the target domain features and tries to restore the target domain samples as closely as possible. The feature space of the features thus generated is similar for source and target domain samples.

[0009] The above methods all have their own advantages, but they also commonly face challenges that need to be addressed, namely how to fully exploit the effective information of source domain samples during the adaptation process, effectively prevent mismatches, and how to adapt the underlying features during the adaptation process. Summary of the Invention

[0010] In order to overcome the defects of the existing technology, the technical problem to be solved by the present invention is to provide a domain adaptation method based on contrastive adversarial learning, which ensures the diversity of classifiers, enables the dual classifier to more efficiently find samples at the classification boundary, and makes the features learned by the model contain more effective information, thereby better solving the unsupervised domain adaptation problem. On the basis of the traditional dual classifier adversarial method, it not only considers the decision boundary of the classifier in the target domain, but also further pays attention to the differences between domains.

[0011] The technical solution of the present invention is: this domain adaptation method based on contrastive adversarial learning, the method comprises the following steps:

[0012] (1) Using the loss function L on the source domain data cls (xs ,y s ) trains the entire network model, and the optimization process is defined as formula (1):

[0013]

[0014] Among them, L ce (·,·) is the cross entropy loss, θ g ,θ c1 θ c2 are the parameters in the feature networks G, C1, and C2 respectively;

[0015] (2) Fix the parameters in the feature extractor and only update the classifiers C1 and C2, minimize the classification loss of the classifier and maximize the classifier's discrimination difference on the target domain samples. The loss function is: (2):

[0017]

[0018] Among them, L dis (·,·) means that the dual classifier only updates the parameters in the classifier for the target domain sample discrimination difference. At the same time, the model adds distribution alignment loss and minimizes feature similarity, which is defined as formula (7):

[0019]

[0020] Among them, θ c1 ,θ c2 Represent the parameters in classifiers C1 and C2 respectively, and λ and η represent the balance parameters in the loss function respectively;

[0021] (3) Fix the parameters of classifiers C1 and C2 and use L dis Update the parameters in the feature extractor. In this step, the adaptive loss term is retained, which is defined as formula (8):

[0022]

[0023] The present invention uses the loss function L on the source domain data cls (x s ,y s ) Train the entire network model, fix the parameters in the feature extractor, only update the classifiers C1 and C2, minimize the classifier classification loss and maximize the classifier's discrimination difference on the target domain samples, fix the parameters in the classifiers C1 and C2, and use L disThe parameters in the feature extractor are updated. The adaptive loss term is retained in this step, thereby ensuring the diversity of the classifier. This enables the dual classifier to more efficiently find samples at the classification boundary, and makes the features learned by the model contain more effective information, thereby better solving the unsupervised domain adaptation problem. On the basis of the traditional dual-classifier adversarial method, it not only considers the decision boundary of the classifier in the target domain, but also further pays attention to the differences between domains.

[0024] A domain adaptation device based on contrastive adversarial learning is also provided, comprising:

[0025] The training module is configured to use the loss function L on the source domain data cls (x s ,y s ) trains the entire network model, and the optimization process is defined as formula (1):

[0026]

[0027] Among them, L ce (·,·) is the cross entropy loss, θ g ,θ c1 θ c2 are the parameters in the feature network G, C1, and C2 respectively; the classifier update module is configured to fix the parameters in the feature extractor and only update the classifiers C1 and C2, minimizing the classifier classification loss and maximizing the classifier's discrimination difference on the target domain samples. The loss function is formula (2):

[0028]

[0029] Among them, L dis (·,·) means that the dual classifier only updates the parameters in the classifier for the target domain sample discrimination difference. At the same time, the model adds distribution alignment loss and minimizes feature similarity, which is defined as formula (7):

[0030]

[0031] Among them, θ c1 ,θ c2 Represent the parameters in classifiers C1 and C2 respectively, and λ and η represent the balance parameters in the loss function respectively;

[0032] The feature extractor update module is configured to fix the parameters in classifiers C1 and C2 using L dis Update the parameters in the feature extractor. In this step, the adaptive loss term is retained, which is defined as formula (8):

[0033] BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Steps two and three of the traditional dual-classifier adversarial training method are shown.

[0035] Figure 2 It is a flow chart of the model structure of the present invention.

[0036] Figure 3 This is the data distribution structure diagram of the four methods.

[0037] Figure 4 is the classifier difference map.

[0038] Figure 5 4 is a flow chart of a domain adaptation method based on contrastive adversarial learning according to the present invention. DETAILED DESCRIPTION

[0039] like Figure 5 As shown in FIG, this domain adaptation method based on contrastive adversarial learning includes the following steps:

[0040] (1) Using the loss function L on the source domain data cls (x s ,y s ) trains the entire network model, and the optimization process is defined as formula (1):

[0041]

[0042] Among them, L ce (·,·) is the cross entropy loss, θ g ,θ c1 θ c2 are the parameters in the feature networks G, C1, and C2 respectively;

[0043] (2) Fix the parameters in the feature extractor and only update the classifiers C1 and C2, minimize the classification loss of the classifier and maximize the difference in the classifier's discrimination of the target domain samples. The loss function is formula (2):

[0044]

[0045] Among them, L dis (·,·) means that the dual classifier only updates the parameters in the classifier for the target domain sample discrimination difference. At the same time, the model adds distribution alignment loss and minimizes feature similarity, which is defined as formula (7):

[0046]

[0047] Among them, θ c1 ,θ c2Represent the parameters in classifiers C1 and C2 respectively, and λ and η represent the balance parameters in the loss function respectively;

[0048] (3) Fix the parameters of classifiers C1 and C2 and use L dis Update the parameters in the feature extractor. In this step, the adaptive loss term is retained, which is defined as formula (8):

[0049]

[0050] The present invention uses the loss function L on the source domain data cls (x s ,y s ) Train the entire network model, fix the parameters in the feature extractor, only update the classifiers C1 and C2, minimize the classifier classification loss and maximize the classifier's discrimination difference on the target domain samples, fix the parameters in the classifiers C1 and C2, and use L dis The parameters in the feature extractor are updated. The adaptive loss term is retained in this step, thereby ensuring the diversity of the classifier. This enables the dual classifier to more efficiently find samples at the classification boundary, and makes the features learned by the model contain more effective information, thereby better solving the unsupervised domain adaptation problem. On the basis of the traditional dual-classifier adversarial method, it not only considers the decision boundary of the classifier in the target domain, but also further pays attention to the differences between domains.

[0051] Preferably, in step (1), firstly, random data enhancement is used to i Enhance twice to obtain and Then the data from the two perspectives are simultaneously fed into the feature extractor, and the features of different perspectives are obtained after being mapped by the hidden layer in the classifier. and

[0052] Preferably, in step (2), the cosine similarity is used to measure the difference between features of different viewing angles, which is formula (4):

[0053]

[0054] in, express and The cosine similarity between the two classifiers is minimized and The cosine similarity between them is calculated, and the feature extractor is expected to maximize the similarity between the two.

[0055] Preferably, in step (2), MLP and gradient stopping techniques are used to prevent mode collapse, and the difference loss of the classifier with symmetric relationship is formula (5):

[0056]

[0057] Among them, the input and output dimensions of module M are consistent to meet the needs of vector cosine similarity calculation.

[0058] Preferably, in step (3), the Sliced Wasserstein distance is used to reduce the difference in label distribution between the source domain and the target domain data, so that the target domain data moves in the correct direction, and the adaptive loss is formula (6):

[0059]

[0060] Among them, f s and f t represent the source domain and target domain features respectively.

[0061] Preferably, the method is applied to the Image-CLEF-DA dataset, which consists of three subdomains: Caltech-256 (C), ImageNet ILSVRC 2012 (I), and Pascalvoc 2012 (P). The entire dataset contains 1,800 image samples, each subdomain contains 600 image samples and includes 12 categories.

[0062] A domain adaptation device based on contrastive adversarial learning is also provided, comprising:

[0063] A domain adaptation device based on contrastive adversarial learning is also provided, comprising:

[0064] The training module is configured to use the loss function L on the source domain data cls (x s ,y s ) trains the entire network model, and the optimization process is defined as formula (1):

[0065]

[0066] Among them, L ce (·,·) is the cross entropy loss, θ g ,θ c1 θ c2 are the parameters in the feature network G, C1, and C2 respectively; the classifier update module is configured to fix the parameters in the feature extractor and only update the classifiers C1 and C2, minimizing the classifier classification loss and maximizing the classifier's discrimination difference on the target domain samples. The loss function is formula (2):

[0067]

[0068] Among them, L dis (·,·) means that the dual classifier only updates the parameters in the classifier for the target domain sample discrimination difference. At the same time, the model adds distribution alignment loss and minimizes feature similarity, which is defined as formula (7):

[0069]

[0070] Among them, θ c1 ,θ c2 Represent the parameters in classifiers C1 and C2 respectively, and λ and η represent the balance parameters in the loss function respectively;

[0071] The feature extractor update module is configured to fix the parameters in classifiers C1 and C2 using L dis

[0072] Update the parameters in the feature extractor. In this step, the adaptive loss term is retained, which is defined as formula (8):

[0073]

[0074] The present invention will be described in more detail below.

[0075] This paper mainly studies the problem of unsupervised domain adaptive classification. and target domain data x s and x t are the source domain and data, y s is the label of the source domain data, which has a total of C categories. The target domain data categories are consistent with the source domain but lack true sample labels. For ease of understanding, this section first introduces the traditional dual-classifier adversarial method and then introduces the innovations and improvements of the present invention.

[0076] 1 Traditional dual classifier adversarial learning method

[0077] The traditional dual-classifier adversarial model mainly consists of two parts: feature extractor G and two classifiers C1 and C2. First, the source domain and target domain data are fed into the feature extractor to obtain the features f s and f t , and then feed the features into different classifiers at the same time. The classifier outputs the probability distribution p1(y s |x s ), p2(y s |x s ), p1(y t |x t ), p2(y t |x t). The adversarial process occurs between the feature extractor and the two classifiers, and the classifier maximizes p1(y t |x t ) and p2(y t |x t ), and the feature extractor expects to minimize the output difference between the two classifiers by extracting common features. The purpose of this model is to find the target domain samples at the decision boundary of the classifier and to improve the confidence of the boundary points by limiting the classifier output to be consistent. The overall structure diagram of the model processing target domain data is shown in the figure below. Figure 1 shown.

[0078] The three steps of the traditional dual classifier adversarial training method are as follows:

[0079] Step 1: Use the loss function L on the source domain data cls (x s ,y s ) Train the entire network model, and the optimization process is defined as follows:

[0080]

[0081] In the above formula, L ce (·,·) is the cross entropy loss, θ g ,θ c1 θ c2 are the parameters in the feature networks G, C1, and C2 respectively.

[0082] Step 2: Fix the parameters in the feature extractor and only update the classifiers C1 and C2. Minimize the classification loss of the classifier and maximize the classifier's discrimination difference for target domain samples. The loss function is defined as follows:

[0083]

[0084] In the above formula, L dis (·,·) represents the difference in the discrimination of target domain samples by the two classifiers.

[0085] Step 3: Fix the parameters of classifiers C1 and C2 and minimize L dis (·,·) updates the parameters in the feature extractor, and the loss function is defined as follows:

[0086]

[0087] Repeat the above three steps until the model converges. The model can effectively find the decision boundary sample points of the target domain data and improve the model classification performance by utilizing the diversity of the dual classifier.

[0088] 2 Dual Classifier Method Based on Contrastive Adversarial Learning

[0089] As mentioned above, the dual classifier adversarial method can find samples at the classification boundary based on the differences between different classifiers. However, classifiers C1 and C2 share features in the feature extractor, and relying solely on the parameters in the classifier is not enough to ensure the diversity of the classifier. In order to improve the diversity of different classifiers, the present invention refers to the Simsiam model and uses different data enhancement methods to obtain features from different perspectives. At the same time, the dual classifier adversarial model is closely combined with the dual classifier idea. First, random data enhancement is used to i Enhance twice to obtain and Then the data from the two perspectives are simultaneously fed into the feature extractor, and the features of different perspectives are obtained after being mapped by the hidden layer in the classifier. and Since the features input into different classifiers are different, the diversity of classifier outputs is guaranteed from the perspective of input data.

[0090] Traditional dual-classifier adversarial models typically use the L1 norm to measure the difference in classifier output. However, considering only the difference in predicted probabilities prevents the model from focusing on the effective information in the features. Therefore, the dual-classifier model incorporates contrastive learning, using the feature differences between different classifiers to define the difference between the classifiers. Based on the contrastive learning concept, it is known that by reducing the difference in features from different viewpoints, the representation ability of the data can be enhanced, thereby improving the performance of the model. This paper uses cosine similarity to measure the difference between features from different viewpoints, and the formula is defined as follows:

[0091]

[0092] The above formula express and The cosine similarity between the two classifiers is expected to minimize and The cosine similarity between the two is calculated, and the feature extractor aims to maximize the similarity between the two. In contrastive learning, optimizing only the above formula can cause problems such as mode collapse, so MLP and gradient stopping techniques are used to prevent this problem. In summary, the difference loss of the classifier with symmetric relationship is defined as follows:

[0093]

[0094] In the above formula, the input and output dimensions of module M are consistent to meet the needs of vector cosine similarity calculation.

[0095] Although dual-classifier adversarial models can effectively identify samples on the boundary of classification, they cannot guarantee that the model will converge in the correct direction. For example, in a three-category classification task, the classifiers predict the probabilities of the current target sample as [0.98, 0.01, 0.01] and [0.97, 0.01, 0.02], respectively, but the true label of the sample may be [0, 1, 0]. Although the difference in the output of the two classifiers is small, the output of both classifiers is still incorrect. This is because during training, the model only focuses on the difference in the predictions of the two classifiers on the target domain data, but minimizing this difference does not effectively guide the boundary points to converge to the corresponding category center. Existing domain adaptation theory shows that the classification error of the model in the target domain is mainly determined by the discrimination error of the source domain data and the difference between the two domains. Ignoring domain distribution matching will significantly limit model performance.

[0096] To solve the above problem, the Sliced Wasserstein distance is used to reduce the difference in the label distribution of the source and target domain data, so that the target domain data moves in the right direction. The final adaptive loss is defined as follows:

[0097]

[0098] In the above formula, f s and f t Represent the source domain and target domain features respectively. The overall optimization training process of the model is as follows:

[0099] In step 1, the model, like the traditional dual-classifier adversarial model, uses the cross-entropy loss of the source domain data to update the parameters of the entire model.

[0100] In step 2, the parameters in the feature extractor are fixed, and only the parameters in the classifier are updated. At the same time, the model adds distribution alignment loss and minimizes feature similarity. The formula is defined as follows:

[0101]

[0102] In the above formula, θ c1 ,θ c2 They represent the parameters in classifiers C1 and C2 respectively, and λ and η represent the balance parameters in the loss function respectively.

[0103] In step 3, the model fixes the parameters of the two classifiers and uses L dis Update the parameters in the feature extractor. In this step, the adaptive loss term is retained, and the formula is defined as follows:

[0104]

[0105] Figure 2 Medium x sand x t Represent the original source domain and target domain data respectively, and It is an image sample obtained by randomly enhancing the source domain data twice. and It is an image sample obtained by random data augmentation twice on the target domain data. v1 and f v2 They are the features of different perspectives after being mapped by the feature extractor G and the fully connected layer FC. and is the predicted probability of the source domain and the target domain by the classifier C1, and is the predicted probability of the source and target domains by classifier C2. The MLP module in the figure above includes a batch normalization layer in each layer. It is mainly composed of three fully connected layers, with an output layer dimension of 1024 and a hidden layer node count of 512. This makes the MLP structure similar to a bottleneck. MLP module mapping can prevent feature mode collapse.

[0106] The present invention experimentally verifies the above method and uses the average classification accuracy to measure the performance of the model, and achieves significant results. The samples used in the experimental part are labeled source domain samples and unlabeled target domain samples.

[0107] This experiment used Image-CLEF to validate the model's performance. The Image-CLEF-DA dataset consists of three subdomains: Caltech-256 (C), ImageNet ILSVRC 2012 (I), and Pascalvoc 2012 (P). The entire dataset contains 1,800 image samples, with each subdomain containing 600 images across 12 categories. To visually demonstrate the model's performance, data visualization experiments were conducted. Ablation experiments were also conducted on this dataset to demonstrate the effectiveness of each module in the model. All code was written in Python and PyTorch, using an RTX 3090 graphics card.

[0108] In order to highlight the performance of the model, the experiment selected three mainstream comparison methods:

[0109] Source-only: This method uses ResNet-50 as the model's backbone. This method trains a classifier using source domain data and then uses the model to directly classify target domain data. This method serves as the baseline for all comparison methods.

[0110] Classic Methods: To demonstrate the performance advantages of dual-classifier adversarial models, this paper selected the classic MMD distance metric-based method and the domain discriminator adversarial model method. The Deep Adaptation Network (DAN) method uses MMD distance to reduce the difference between distributions. The Domain Adversarial Neural Network (DANN) uses a discriminator to distinguish whether the current data comes from the source domain or the target domain. Based on DANN, the Conditional Domain Adversarial Network (CDAN) uses multiple linear mappings for feature fusion, making the distribution structure of the source and target domains clearer, thereby achieving domain distribution structure matching and improving model performance.

[0111] To demonstrate the performance advantages of this model, we also compare it with a dual-classifier adversarial approach. MCD is the first model to utilize dual-classifier adversarial training. This approach uses the L1 norm to measure the difference between the two classifiers. Building on MCD, SWD uses the Wasserstein distance to measure the difference between the outputs of two classifiers. The recent JADA model combines the concepts of discriminator adversarial methods with MCD, successfully using the discriminator to align two data distributions. Comparing these approaches demonstrates the effectiveness of this model.

[0112] This section discusses the experimental results of the model on the Image-CLEF dataset. All experimental data are the output after 30 iterations of the model. All experimental results are recorded in Table 1, where the recognition task is denoted as S→T, where S represents the source domain with labeled data and T represents the target domain with unlabeled data. S→T represents the classification task of using labeled data from the source domain to solve the target domain data:

[0113] Method I→P P→I I→C C→I C→P P→C Avg Source-only 74.8 83.9 91.5 78.0 65.5 91.2 80.7 DAN 74.5 82.2 92.8 86.3 69.2 89.8 82.5 DANN 75.0 86.0 96.2 87.0 74.3 91.5 85.0 CDAN 76.7 90.6 97.0 90.5 74.5 93.5 87.1 MCD 77.3 89.2 92.7 88.2 71.0 92.3 85.1 SWD 76.9 90.7 93.8 88.3 74.2 93.8 86.3 JADA 78.2 90.1 95.9 90.8 76.8 94.1 87.7 Ours 79.9 92.5 95.4 92.7 78.8 94.2 88.8

[0114] Table 1

[0115] As shown in Table 1, the model selected in this paper achieves the best average precision. This model achieves the most significant improvement, particularly for the more challenging task of C→P transfer. Experimental results demonstrate that contrastive learning can fully capture the valuable information in complex samples.

[0116] To more intuitively demonstrate the advantages of the model, this paper uses T-SNE to visualize the data dimension reduction of the migration task P→C. Source-only, MCD, and SWD are also selected as comparison methods. The data visualization results are shown in the figure below. Figure 3 As shown in the figure, “ο” represents source domain data and “×” represents target domain data. Figure 3As shown in the figure, due to domain differences, source-only methods cannot effectively improve the model's classification accuracy in the target domain. Compared to source-only methods, MCD and SWD can reduce samples at the classification boundary in the target domain. However, when the target domain data distribution structure is complex, the distances between samples of different categories remain unclear. As can be seen from the figure, the data distribution structure is clearer in this model, and the distances between different categories are larger. The same categories in different domains are also more closely aligned, demonstrating the effectiveness of label distribution matching.

[0117] Depend on Figure 4 As can be seen from the figure, the classifier output accuracy differs significantly at the beginning of model training, and both classifiers perform inferior to the combined performance of the dual classifier in the initial stages of the model. This suggests that the data from the two perspectives can provide complementary information in the early stages of training, thereby improving the combined classifier output performance. As the model is trained more times, the model performance gradually converges, with the three curves converging. This indicates that the output differences between the dual classifiers are small, demonstrating the influence of feature similarity on classifier output. These experimental results demonstrate that combining domain alignment and contrastive learning can effectively improve model performance.

[0118] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of protection of the technical solution of the present invention.

Claims

1. A domain adaptation method based on contrastive adversarial learning, characterized by: The method comprises the following steps: (1) Using the loss function L on the source domain data cls (x s ,y s ) trains the entire network model, and the optimization process is defined as formula (1): Among them, L ce (·,·) is the cross entropy loss, θ g ,θ c1 θ c2 are the parameters in the feature networks G, C1, and C2 respectively; (2) Fix the parameters in the feature extractor and only update the classifiers C1 and C2, minimize the classification loss of the classifier and maximize the difference in the classifier's discrimination of the target domain samples. The loss function is formula (2): Among them, L dis (·,·) means that the dual classifier only updates the parameters in the classifier for the target domain sample discrimination difference. At the same time, the model adds distribution alignment loss and minimizes feature similarity, which is defined as formula (7): Among them, θ c1 ,θ c2 Represent the parameters in classifiers C1 and C2 respectively, and λ and η represent the balance parameters in the loss function respectively; (3) Fix the parameters of classifiers C1 and C2 and use L dis Update the parameters in the feature extractor. In this step, the adaptive loss term is retained, which is defined as formula (8): In the step (1), firstly, random data enhancement is used to i Enhance twice to obtain and Then the data from the two perspectives are simultaneously fed into the feature extractor, and the features of different perspectives are obtained after being mapped by the hidden layer in the classifier. and In step (2), the cosine similarity is used to measure the difference between features of different viewpoints, which is formula (4): in, express and The cosine similarity between the two classifiers is minimized and The cosine similarity between them is calculated, and the feature extractor is expected to maximize the similarity between the two; In step (2), MLP and gradient stopping techniques are used to prevent mode collapse, and the difference loss of the classifier with symmetric relationship is formula (5): Among them, the input and output dimensions of module M are consistent to meet the needs of vector cosine similarity calculation; In step (3), the Sliced Wasserstein distance is used to reduce the difference in the label distribution of the source domain and the target domain data, so that the target domain data moves in the correct direction. The adaptive loss is formula (6): Among them, f s and f t Represent the source domain and target domain features respectively; This method is applied to the Image-CLEF-DA dataset, which consists of three subdomains: Caltech-256 (C), ImageNet ILSVRC 2012 (I), and Pascalvoc2012 (P). The entire dataset contains 1,800 image samples, each subdomain contains 600 image samples and includes 12 categories.

2. The device for the domain adaptation method based on contrastive adversarial learning according to claim 1, characterized in that: The device includes: The training module is configured to use the loss function L on the source domain data cls (x s ,y s ) trains the entire network model, and the optimization process is defined as formula (1): Among them, L ce (·,·) is the cross entropy loss, θ g ,θ c1 θ c2 are the parameters in the feature network G, C1, and C2 respectively; the classifier update module is configured to fix the parameters in the feature extractor and only update the classifiers C1 and C2, minimizing the classifier classification loss and maximizing the classifier's discrimination difference on the target domain samples. The loss function is formula (2): Among them, L dis (·,·) means that the dual classifier only updates the parameters in the classifier for the target domain sample discrimination difference. At the same time, the model adds distribution alignment loss and minimizes feature similarity, which is defined as formula (7): Among them, θ c1 ,θ c2 Represent the parameters in classifiers C1 and C2 respectively, and λ and η represent the balance parameters in the loss function respectively; The feature extractor update module is configured to fix the parameters in classifiers C1 and C2 using L dis Update the parameters in the feature extractor. In this step, the adaptive loss term is retained, which is defined as formula (8):

Images