Cross-domain image classification method based on double contrast constraints

Abstract

The invention discloses a cross-domain image classification method based on double contrast constraints, and relates to the technical field of cross-domain image classification. Comprising the following steps: acquiring at least two source domain image data sets and a target domain image data set; constructing a cross-domain image classification model; performing nearest neighbor matching on the target domain data set based on the category center of each real label sample, screening target samples in the target domain data set and endowing the target samples with pseudo labels to obtain a pseudo label sample set; and respectively applying class-level contrast constraint and domain-level contrast constraint in the fusion feature space, taking minimization of total target loss as a target, continuously updating parameters of a feature extractor and a classifier through back propagation, and obtaining a trained cross-domain image classification model. Coarse-grained domain alignment and fine-grained semantic alignment are organically unified through double comparison, class-level-oriented semantic alignment and domain-level-oriented distribution alignment are provided under multi-source setting, and a more robust adaptation mechanism is provided for cross-domain image classification.

Description

Technical Field

[0001] This invention relates to the field of cross-domain image classification technology, and in particular to a cross-domain image classification method based on dual contrast constraints. Background Technology

[0002] Deep learning has made significant progress driven by large-scale labeled data, but the sensitivity of models to data distribution leads to a sharp performance drop in cross-domain applications. Unsupervised Domain Adaptation (UDA), a key research direction in deep learning, aims to enable models trained on source domain data to be effectively applied to target domains without labeled data, thereby overcoming the limitations of data domains and improving the transferability of models. However, real-world scenarios often involve multiple source domains with different origins, which has spurred the more challenging but application-oriented research of Multi-source Unsupervised Domain Adaptation (MUDA).

[0003] MUDA aims to leverage multiple labeled source domains to adapt the model to an unlabeled target domain with a different distribution than the source domains. In the real world, well-trained image classification models often perform poorly in new domains; this is the domain shift problem. For example, a model trained on images of objects taken in natural light will show a significant drop in classification accuracy on images of the same objects taken at night. This is because there is a significant distribution difference between the training data (source domain) and the test data (target domain).

[0004] In recent years, contrast-quantification methods have been introduced into MUDA to enhance semantic alignment of cross-domain images. Contrast-quantification methods enhance the discriminativeness of representations and promote semantic convergence by constructing cross-domain positive and negative sample pairs or class prototypes. The core of contrast-quantification methods is to bring positive sample features (features of the same class in the source and target domains) closer together and negative sample features (features of different classes in the source and target domains) further apart in the feature space, thereby simultaneously ensuring the compactness of cross-domain positive samples and the separability of negative samples.

[0005] However, existing contrast measurement methods mainly focus on class-level contrast between source and target, emphasizing the semantic constraint of "bringing similar samples closer and dissimilar samples further apart," but lack domain-level structural contrast constraints across source and target domains. They often ignore the role of domain differences between the source and target domains in the contrast constraints, which can easily lead to excessively bringing similar samples closer in source-target pairs, weakening the stability of the overall alignment and inducing negative transfer. Summary of the Invention

[0006] Therefore, it is necessary to provide a cross-domain image classification method based on dual contrast constraints to address the aforementioned technical problems.

[0007] This invention provides a cross-domain image classification method based on dual contrast constraints, comprising: Obtain at least two source domain image datasets and one target domain image dataset. Each dataset should have inter-domain differences in visual style and contain samples from multiple categories. A cross-domain image classification model is constructed, which includes a feature extractor and a classifier connected sequentially; features are extracted from the source domain dataset and the target domain dataset by the feature extractor to obtain multi-layer image features; and a feature fusion function is used to fuse the multi-layer image features to obtain a fused feature space. Based on the class centers of each real label sample in the source domain dataset, nearest neighbor matching is performed on the target domain dataset. Target samples in the target domain dataset are filtered through multiple filtering mechanisms and assigned pseudo labels to obtain a pseudo label sample set. Based on real label samples and pseudo-label samples, class-level contrast constraints and domain-level contrast constraints are applied in the fused feature space to obtain class-level contrast constraint loss and domain-level contrast constraint loss. Class-level contrast constraints are used to minimize the feature differences of label samples of the same class and maximize the feature differences of label samples of different classes, while domain-level contrast constraints are used to minimize the feature differences of pseudo-domain label samples of the same class and maximize the feature differences of pseudo-domain label samples of different classes. With the goal of minimizing the total target loss obtained by weighted summation of class-level contrast constraint loss, domain-level contrast constraint loss and classification loss, the parameters of the feature extractor and classifier are continuously updated through backpropagation to obtain the trained cross-domain image classification model; The trained cross-domain image classification model is used to classify and predict the target domain dataset to obtain the cross-domain image classification results.

[0008] Optionally, class-level contrast constraints are applied in the fused feature space, specifically including: The following formula is used to construct the distance matrix between categories in the source domain, the distance matrix between categories in the target domain, and the distance matrix between category pairs between categories in the source domain and categories in the target domain: ; ; ; The baseline bandwidth and kernel function bandwidth between category pairs in the source and target domains are determined based on the following formula: ; ; ; The multi-kernel Gaussian kernel function is determined based on the bandwidth of multiple kernel functions using the following formula: ; Based on the distance matrix and the multi-kernel Gaussian kernel function, the kernel matrices for the source domain, target domain, and cross-domain are calculated using the following formula: ; ; ; The MMD values ​​of class pairs between the source and target domains are determined based on the following formula, and a class contrast constraint matrix is ​​formed: ; Class-level comparison constraints are determined based on the following formula: ; in, m For the first m One source domain-target domain pair, x For input, c For categories within the source domain, c' For categories within the target domain, ( c , c' () represents a category pair between categories in the source domain and categories in the target domain. The set of samples representing categories within the source domain. The set of samples belonging to the categories within the target domain. The number of samples of each category within the source domain. The number of samples for each category within the target domain. α l For the network l The nuclear factor of the layer, m l For the network l Number of kernels in a layer For the network l Distance matrix of categories within the source domain, For the network l Distance matrix of categories within the target domain of the layer. For the network l The distance matrix between categories in the source domain and categories in the target domain. For the network l The baseline bandwidth of category pairs between categories in the source domain and categories in the target domain. For the network l The kernel function bandwidth of category pairs between categories in the source domain and categories in the target domain. For the network l Multi-kernel Gaussian kernel function The kernel matrix represents the categories within the source domain. The kernel matrix of categories within the target domain. For cross-domain category kernel matrix, This represents the MMD value of the category pairs between categories in the source domain and categories in the target domain. This serves as a class-level comparison constraint between the source and target domains. C The total number of categories, The MMD values ​​are for labeled samples of the same type in both the source and target domains. The MMD values ​​are the heterogeneous label samples from the source and target domains.

[0009] Optionally, domain-level contrast constraints are applied in the fused feature space, specifically including: Will C The categories were reorganized into D Each pseudo-domain contains [number] pseudo-domains. K The pseudo-domain sample sets of the source and target domains are determined based on the following formula: ; ; The number of samples in each pseudo-domain is determined by the following formula: the sum of the number of samples in all categories contained in that pseudo-domain. ; ; The following formula is used to construct the domain distance matrix of the pseudo-domain within the source domain, the domain distance matrix of the pseudo-domain within the target domain, and the domain distance matrix of the pseudo-domain pairs between the source and target domains: ; ; ; The baseline bandwidth of the pseudo-neighborhood pair between the source and target domains and the bandwidth of multiple kernel functions are determined based on the following formula: ; ; ; The multi-kernel Gaussian function that determines the pseudo-neighborhood pair between the source and target domains is based on the following formula: ; Based on the distance matrix and the multi-kernel Gaussian kernel function, the pseudo-neighborhood kernel matrices for the source domain, target domain, and pseudo-neighborhood pairs between the source and target domains are determined according to the following formula: ; ; ; The MMD value of the pseudo-neighborhood pair between the source and target domains is determined based on the following formula, and the neighborhood contrast constraint matrix is ​​obtained: ; The domain-level comparison constraint between the source and target domains is obtained based on the following formula: ; in, m For the first m One source domain-target domain pair, x For input, d For pseudo-domains within the source domain, d' For the pseudo-domain within the target domain, ( d , d' () represents a pseudo-domain pair between the source and target domains. The set of samples within the pseudo-domain of the source domain. The set of samples within the pseudo-domain of the target domain. The number of samples in the pseudo-domain within the source domain. The number of samples in the pseudo-domain within the target domain. α l For the network l The nuclear factor of the layer, m l For the network l Number of cores in the layer For the network l The square of the Euclidean distance between layer features, For the network l The distance matrix of the pseudo-neighborhood within the source domain. For the network l The distance matrix of the pseudo-neighborhood within the target domain of the layer. For the network l The distance matrix between the pseudo-neighborhood in the source domain and the pseudo-neighborhood in the target domain. For the network l The baseline bandwidth of pseudo-neighborhood pairs between the source and target domains. For the network l The kernel function bandwidth of pseudo-neighborhood pairs between the source and target domains. For the network l Multi-kernel Gaussian kernel function The kernel matrix of the pseudo-domain within the source domain. The kernel matrix of the pseudo-neighborhood within the target domain. For cross-domain pseudo-domain kernel matrix, This represents the maximum mean difference between pseudo-neighborhood pairs between the source and target domains. This serves as a domain-level comparison constraint between the source and target domains. D For the total number of pseudo-domains, The MMD values ​​of samples with pseudo-domain labels in both the source and target domains are given. This represents the MMD value of pseudo-domain label samples with different source and target domains.

[0010] Optionally, based on the class centers of each real labeled sample in the source domain dataset, nearest neighbor matching is performed on the target domain dataset. Target samples in the target domain dataset are then filtered through a multi-layered filtering mechanism and assigned pseudo-labels, resulting in a pseudo-label sample set, specifically including: The multiple filtering mechanism includes: sequentially executed multi-source consistency filtering, confidence threshold filtering, and minimum class sample number filtering; Based on the cosine distance from each sample in the target domain dataset to the center of each class in the source domain dataset, we obtain the data from... M Predicted labels and corresponding confidence levels for each source domain; The number of times each target sample is predicted to each category by different source domains is obtained. The category with the most predictions is taken as the consensus candidate category. When the number of predictions of the consensus candidate category reaches the preset majority threshold, the target sample is determined to pass the multi-source consensus filtering and the initial filtered sample is obtained. For each initial filtered sample, obtain the confidence scores corresponding to all samples that predict the initial filtered sample as a consensus category; if the confidence score of each source domain is not lower than the preset confidence score threshold, then the initial filtered sample is determined to have passed the confidence score threshold filtering and is used as an intermediate filtered sample. Obtain the number of samples contained in each pseudo-label category in the intermediate filtered samples, and retain only the categories whose sample number is not less than the preset minimum category sample number threshold and their corresponding intermediate filtered samples to obtain the final pseudo-label sample set.

[0011] Optionally, the total target loss, obtained by weighted summation of class-level contrastive constraint loss, domain-level contrastive constraint loss, and classification loss, is determined based on the following formula: ; ; ; ; in, For classifying losses, For domain-level contrast constraint loss, For class-level contrast constraint loss, β For the weights of the domain-level contrastive constraint loss and the class-level contrastive constraint loss, For the overall target loss, ∈{1,2,…, C} is a sample The true label, p This represents the predicted probability given the input.

[0012] The cross-domain image classification method based on dual contrast constraints provided in this invention has the following advantages compared with the prior art: This invention jointly applies a local class-level contrast constraint (CCC) based on category construction and a global domain-level contrast constraint (DCC) based on pseudo-domain construction between each source and target domain. DCC plays a crucial role in suppressing domain shift caused by multi-source heterogeneity and complements CCC, mitigating class shift while suppressing global domain shift. This technical process organically unifies coarse-grained domain alignment and fine-grained semantic alignment through dual contrast, providing class-level semantic alignment and domain-level distribution alignment in multi-source scenarios, offering a more robust adaptation mechanism for cross-domain image classification. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating a cross-domain image classification method based on dual contrast constraints provided in one embodiment; Figure 2 This is a general schematic diagram of a cross-domain image classification method based on dual contrast constraints provided in one embodiment; Figure 3 This is an example dataset diagram of a cross-domain image classification method based on dual contrast constraints provided in one embodiment; Figure 4 The image classification method based on dual contrast constraints provided in one embodiment is shown in the accuracy curves of different methods on the A, P, R→C task and the A, P, C→R task. Figure 4 (a) in the figure is a comparison diagram of the DUCC method with the Baseline method and the DCC method on the A, P, R → C task. Figure 4 (b) in the figure is a comparison of the DUCC method with the Baseline method and the CCC method on the A, P, R → C task; Figure 4 (c) in the figure is a comparison diagram of the DUCC method with the Baseline method and the DCC method on the A, P, C→R task; Figure 4 (d) in the figure is a comparison of the DUCC method with the Baseline method and the CCC method on the A, P, C→R task.

[0014] Figure 5 This is a visualization of different methods of a cross-domain image classification method based on dual contrast constraints provided in one embodiment on the A, P, R→R tasks. Figure 5 (a) in the figure is a visualization of the Baseline method on the A, P, R→R task. Figure 5 (b) in the figure is a visualization of the DCC method on the A, P, R→R task. Figure 5(c) in the figure is a visualization of the CCC method on the A, P, R→R tasks. Figure 5 (d) in the figure is a visualization of the DUCC method on the A, P, R→R task. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0016] Existing MUDA research often employs domain-level distribution alignment strategies, such as aligning multiple source and target domains in the global feature space using adversarial or statistical distributions to learn domain-invariant representations. While these methods can reduce distributional discrepancies to some extent, they generally neglect the more refined class-conditional distribution structure, leading to severe negative transfer. To alleviate these problems, subsequent research has shifted to a class-level alignment paradigm, introducing mechanisms such as pseudo-labels to achieve cross-domain semantic consistency at a finer granularity. Class-level methods have become the mainstream approach in MUDA, but key limitations remain: for example, previous methods often process class samples independently, lacking the ability to mine potential relationships between class samples, and failing to fully utilize the contextual information in class data to improve domain adaptability.

[0017] This invention provides a cross-domain image classification method based on dual contrast constraints, such as... Figure 2 As shown, the method includes: Obtain at least two source domain image datasets and one target domain image dataset. Each dataset should have inter-domain differences in visual style and contain samples from multiple categories.

[0018] A cross-domain image classification model is constructed, consisting of a feature extractor and a classifier connected sequentially. The feature extractor extracts features from both the source and target domain datasets, obtaining multi-layer image features. A feature fusion function is then used to fuse these multi-layer image features, resulting in a fused feature space.

[0019] Based on the class centers of each real label sample in the source domain dataset, nearest neighbor matching is performed on the target domain dataset. Target samples in the target domain dataset are then filtered through multiple filtering mechanisms and assigned pseudo-labels to obtain a pseudo-label sample set.

[0020] Based on real and pseudo-label samples, class-level contrast constraints and domain-level contrast constraints are applied in the fused feature space to obtain class-level contrast constraint loss and domain-level contrast constraint loss. Class-level contrast constraints are used to minimize the feature differences of similar label samples and maximize the feature differences of dissimilar label samples, while domain-level contrast constraints are used to minimize the feature differences of similar pseudo-domain label samples and maximize the feature differences of dissimilar pseudo-domain label samples.

[0021] With the goal of minimizing the total target loss obtained by weighted summation of class-level contrast constraint loss, domain-level contrast constraint loss and classification loss, the parameters of the feature extractor and classifier are continuously updated through backpropagation to obtain the trained cross-domain image classification model.

[0022] The trained cross-domain image classification model is used to classify and predict the target domain dataset to obtain the cross-domain image classification results.

[0023] Preferably, based on the class centers of each real label sample in the source domain dataset, nearest neighbor matching is performed on the target domain dataset. Target samples in the target domain dataset are then filtered through a multi-layered filtering mechanism and assigned pseudo-labels, resulting in a pseudo-label sample set. Specifically, this includes: The multi-filtering mechanism includes: sequentially executed multi-source consistency filtering, confidence threshold filtering, and minimum class sample number filtering.

[0024] Based on the cosine distance from each sample in the target domain dataset to the center of each class in the source domain dataset, we obtain the data from... M The predicted labels and corresponding confidence levels for each source domain.

[0025] The number of times each target sample is predicted to each category by different source domains is obtained. The category with the most predictions is taken as the consensus candidate category. When the number of predictions of the consensus candidate category reaches the preset majority threshold, the target sample is determined to pass the multi-source consensus filtering and the initial filtered sample is obtained.

[0026] For each initial filtered sample, obtain the confidence scores corresponding to all samples that predict the initial filtered sample as a consensus category. If the confidence score of each source domain is not lower than a preset confidence threshold, then the initial filtered sample is determined to have passed the confidence threshold filtering, and the initial filtered sample is used as an intermediate filtered sample.

[0027] Obtain the number of samples in each pseudo-label category in the intermediate filtered samples, and retain only the categories with a sample number not less than the preset minimum category sample number threshold and their corresponding intermediate filtered samples to obtain the final pseudo-label sample set.

[0028] like Figure 1As shown, a specific embodiment of the present invention is provided: three source domains and one target domain from the Office-Home dataset are selected as experimental data, and ResNet50 is used as the basic network structure. First, source and target domain image datasets with inter-domain visual style differences and containing samples of multiple categories are acquired, and the image samples are subjected to random cropping, random horizontal flipping, and brightness adjustment data enhancement processing. Second, ResNet50 is used to extract the average pooling layer features and Softmax processing features of the images, and the two layers of features are fused through a feature fusion function to obtain a fused feature representation. Then, based on the class centers of each category in the source domain, nearest neighbor matching is performed on the target domain samples, and a high-quality target domain pseudo-label sample set is generated through a multi-filtering mechanism consisting of multi-source consistency filtering, confidence threshold filtering, and minimum class sample number filtering. Further, during training, a classification loss is established using source domain samples, and class-level contrast constraint loss and domain-level contrast constraint loss are established based on the fused feature representation. Then, the classification loss, class-level contrast constraint loss, and domain-level contrast constraint loss are weighted and summed to form the total objective loss. The network parameters are continuously updated through backpropagation with the total objective loss as the optimization objective until training converges. Finally, the trained cross-domain image classification model is used to classify the target domain samples. The specific process of this invention is as follows: a dual contrast constraint is constructed in the network model, namely class-level contrast constraint and domain-level contrast constraint, so as to achieve a significant improvement in cross-domain image classification performance through synergistic effect.

[0029] Specifically as follows: 1. Symbols.

[0030] In MUDA, yes M ( M The joint distribution of ≥2) source domains, It is the joint distribution of the target domain. Given M Group source domain samples and a set of target domain samples Furthermore, the source domain label is known, while the target domain label is unknown. Therefore, in MUDA, labeled source domain data is used. and unlabeled target data T Train the network to T To make accurate predictions.

[0031] 2. Problem Analysis.

[0032] 2.1 MUDA Objectives.

[0033] In multi-source unsupervised domain adaptive MUDA scenarios, the goal is to learn a feature extractor. f (∙) and classifier g (∙), forming a discriminant modelh ( x )= g ( f ( x This makes its prediction risk in the target domain... Minimize the size of the target domain label when it is unknown.

[0034] 2.2 Shortcomings of existing technologies.

[0035] Representative methods in MUDA based on contrast metrics generally focus only on class-level semantic alignment, improving local semantic discriminativeness by bringing similar samples closer together and distancing dissimilar samples across domains. However, these methods lack explicit domain-level structural constraints, leading to two core problems in environments with significant multi-source heterogeneity: first, global geometric misalignment—although local clusters tend to be consistent in the feature space, the overall distribution of different domains may still intersect; second, insufficient stability—when there are significant style differences or class imbalances between the source and target domains, class-level constraints are prone to erroneous convergence, accumulating noise and amplifying the risk of negative transfer. Furthermore, these methods are highly dependent on pseudo-labels; once noise enters the alignment process, it will rapidly spread under multi-source conditions and disrupt the stability of cross-domain features.

[0036] 2.3 Research Motivation.

[0037] Based on the above observations, it is believed that MUDA needs to simultaneously achieve both class-level and domain-level contrast alignment within a unified feature space: class-level contrast alignment ensures fine-grained semantic consistency, making cross-domain samples of the same type compact and separable in local structure; domain-level contrast alignment ensures the geometric stability of the macroscopic distribution, keeping different domains distinguishable and without offset in overall structure. This local-global collaborative constraint can effectively alleviate the global structural confusion problem of single class-level contrast methods, while reducing semantic alignment errors of single domain-level methods when class offset exists. Especially in multi-source scenarios, the correlation between different source domains and the target domain varies greatly; relying solely on local class alignment cannot characterize this difference, nor can it achieve truly robust transfer.

[0038] 3. Class-level contrast constraints (used to minimize the feature differences of similar labeled samples and maximize the feature differences of dissimilar labeled samples, so as to bring the feature distance of similar labeled samples closer and widen the feature distance of dissimilar labeled samples, thereby achieving fine-grained semantic alignment of cross-domain images).

[0039] Class-level contrastive constraints focus on category information in the source and target domains and measure intra-class and inter-class differences across domains. Minimizing intra-class differences reduces feature variations within the same category due to domain differences. Maximizing inter-class differences ensures that samples from different classes are as far apart as possible in the learned feature space, maintaining or enhancing their discriminability. CCC improves domain adaptation performance by jointly optimizing intra-class and inter-class differences.

[0040] Deep Networks l Layer to input x The features are represented as For a certain layer l Calculate the squared Euclidean distance between features: (1) in, For the network l The square of the Euclidean distance between layer features.

[0041] Category within the source domain c Distance matrix, category in target domain c' The distance matrix and the class pairs between categories in the source domain and categories in the target domain ( c , c' The distance matrices of ) are expressed as equations (2) to (4).

[0042] (2) (3) (4) here The set of samples representing categories within the source domain. It is a sample set of categories within the target domain. For the network l Distance matrix of categories within the source domain, For the network l Distance matrix of categories within the target domain of the layer. For the network l The distance matrix between categories in the source domain and categories in the target domain.

[0043] For the l Category pairs between layers and source and target domains ( c , c' ), calculate the baseline bandwidth: (5) here ; in, For the network lThe baseline bandwidth of category pairs between categories in the source domain and categories in the target domain. The number of samples of each category within the source domain. The number of samples of each category within the target domain.

[0044] For the l The layer uses multiple Gaussian kernel functions, each with a different bandwidth parameter, for a reference bandwidth. ,calculate m l Bandwidth per kernel function: (6) in, For the network l The kernel function bandwidth of category pairs between categories in the source domain and categories in the target domain. α l For the network l The nuclear factor of the layer, m l Network No. l The number of kernels in a layer. A multi-kernel Gaussian function is defined as:

[0045] (7) in, For the network l Multi-kernel Gaussian kernel function.

[0046] For each pair of categories ( c , c' By fusing the features processed by the average pooling layer and softmax, three kernel matrices are obtained, namely: (8) (9) (10) in, The kernel matrix of the source domain, The kernel matrix of the target domain, It is a cross-domain kernel matrix.

[0047] Because it combines features from the average pooling layer and the softmax processing, therefore L =2. For category pairs between the source and target domains ( c , c' ), calculate the maximum mean discrepancy (MMD):

[0048] (11) in, This represents the MMD value of the category pair between categories in the source domain and categories in the target domain.

[0049] This forms a comparison constraint matrix. ,in Therefore, the final CCC can be represented as:

[0050] (12) in, This serves as a class-level comparison constraint between the source and target domains. C The total number of categories, The MMD values ​​are for labeled samples of the same type in both the source and target domains. The MMD values ​​are the heterogeneous label samples from the source and target domains.

[0051] 4. Domain-level contrast constraints (used to minimize the feature differences of pseudo-domain label samples with the same domain and maximize the feature differences of pseudo-domain label samples with different domains, so as to align the consistency of source domain data and target domain data in the overall feature distribution structure and achieve coarse-grained domain distribution alignment of cross-domain images).

[0052] Domain-level contrastive constraints focus on domain information in both the source and target domains, measuring intra-domain and inter-domain differences. Minimizing intra-domain differences reduces feature variations within the same pseudo-domain due to domain differences; maximizing inter-domain differences ensures that different pseudo-domain samples are as far apart as possible in the learned feature space, maintaining or enhancing their discriminability. DCC improves domain adaptation performance by jointly optimizing intra-domain and inter-domain differences.

[0053] Will C The categories were reorganized into D Each pseudo-domain contains [number] pseudo-domains. K Categories: (13) (14) in, The set of samples within the pseudo-domain of the source domain. This is the sample set of the pseudo-domain within the target domain.

[0054] The number of samples in each pseudo-domain is the sum of the number of samples in all categories contained in that pseudo-domain: (15) (16) in, d For pseudo-domains within the source domain, d' For the pseudo-domain within the target domain, ( d , d'() represents a pseudo-domain pair between the source and target domains. The number of samples in the pseudo-domain within the source domain. This represents the number of samples in the pseudo-domain within the target domain.

[0055] source domain m Internal pseudo-domain d The neighborhood distance matrix and the pseudo-neighborhood within the target domain d' The neighborhood distance matrix and the source domain m Internal pseudo-domain d pseudo-domain within the target domain d' The neighborhood distance matrices between them are as follows: (17) (18) (19) in, For the network l The distance matrix of the pseudo-neighborhood within the source domain. For the network l The distance matrix of the pseudo-neighborhood within the target domain of the layer. For the network l The distance matrix between the pseudo-domain within the source domain and the pseudo-domain within the target domain.

[0056] For the l Layers and each pair of pseudo-domains ( d , d' The bandwidth parameter is calculated as follows: (20) here, ,in, For the network l The baseline bandwidth of pseudo-domain pairs between the source and target domains.

[0057] For the l Layers and pseudo-domains ( d , d' ),generate m l One bandwidth parameter: (twenty one) in, For the network l Kernel function bandwidth of pseudo-neighborhood pairs between the source and target domains α l It is a nuclear factor. m l It refers to the number of kernels. The multi-kernel Gaussian function is defined as:

[0058] (twenty two) in, For the network l Multi-kernel Gaussian kernel function.

[0059] For each pair of pseudo-domains ( d , d' By fusing the features processed by the average pooling layer and softmax, three kernel matrices are obtained, namely: (twenty three) (twenty four) (25) in, The kernel matrix of the pseudo-domain within the source domain. The kernel matrix of the pseudo-neighborhood within the target domain. It is a kernel matrix for cross-domain pseudo-domains.

[0060] Because it combines features from the average pooling layer and the softmax processing, therefore L =2. For each pair of categories ( d , d' ), Calculate the MMD value:

[0061] (26) in, This represents the maximum mean difference between pseudo-neighborhood pairs between the source and target domains.

[0062] This constructs the domain contrast constraint matrix. ,in Therefore, the final DCC is expressed as:

[0063] (27) in, This serves as a domain-level comparison constraint between the source and target domains. D For the total number of pseudo-domains, The MMD values ​​of samples with pseudo-domain labels in both the source and target domains are given. This represents the MMD value of pseudo-domain label samples with different source and target domains.

[0064] 5. Multiple filtering mechanisms.

[0065] Pseudo-labels are not only necessary but also crucial for implementing DUC (Dual Encoding Computation). The quality of pseudo-labels directly affects the effectiveness of DUC. Inaccurate pseudo-labels will mislead the representation learning process, leading to a decrease in the alignment of features between the source and target domains, and ultimately weakening the model's adaptive performance. In this invention, a multi-filtering mechanism is proposed to obtain high-quality pseudo-labels.

[0066] For each source domain S m Calculate each category c class center : (28) The source domain S m Medium category c sample set , It is a category c In the source domain S m The number of samples in It is a sample x j Features after global average pooling in ResNet50. Source domain. S m Class center matrix:

[0067] (29) For each sample in the target domain Extract its features after global average pooling: Therefore, the feature matrix of the target domain is represented as: (30) For the source domain S m Target sample Distance vectors to all class centers: .

[0068] here d cos This refers to the cosine distance. d cos ∈[0, 1]. The predicted label based on the nearest neighbor principle is represented as:

[0069] (31) The minimum distance value is expressed as: (32) The distance is converted into a confidence score, which is defined as follows: (33) Equation (33) shows that the smaller the distance, the higher the confidence level. For the target sample It collects predictions from all source domains, and its definition is as follows:

[0070] (34) For target samples The number of times each category is predicted is counted, defined as follows: (35) Here, 1[∙] is an indicator function. Let represent the set of all categories. Find the category that receives the most predictions, defined as follows:

[0071] (36) The consensus category set is represented as: (37) 1) Condition 1: Multi-source consistency filtering.

[0072] Define majority threshold If and only if: (38) sample Through multi-source consistency. If condition 1 is met, the consensus category is represented as:

[0073] (39) 2) Condition 2: Confidence threshold filtering.

[0074] For samples that meet condition 1 Collect all predictions as consensus categories y i The source domain confidence is defined as follows: (40) Set confidence threshold τ conf If and only if:

[0075] (41) sample Filtering based on confidence threshold. Sample sets based on conditions 1 and 2: Generate pseudo-labels for these samples: .

[0076] 3) Condition 3: Minimum number of samples for filtering.

[0077] For high-confidence pseudo-label sets The number of samples in each category is expressed as: (42) Set a minimum sample size threshold for each category. The set of valid categories that meets the minimum sample size requirement is represented as: (43) Only samples belonging to the valid categories are retained, resulting in the final sample set: (44) Based on the above analysis, the final pseudo-label sample set used for training is represented as follows: (45) Sample size Final sample size for each category: .

[0078] 6. Overall target loss.

[0079] In DUCC, a fully connected layer is set up, and the neurons in this layer are related to the task of the dataset. The network is trained using labeled source domain data, and the classification loss is minimized to achieve superior classification performance.

[0080] (46) here, For classifying losses, ∈{1,2,…, C} is a sample The true label, p This represents the predicted probability given the input.

[0081] In MUDA, the DCC loss and CCC loss are ultimately expressed as Equation (47) and Equation (48) for multi-source scenarios.

[0082] (47) (48) in, For domain-level contrast constraint loss, For class-level comparison constraint loss.

[0083] The target domain pseudo-label estimation and feature representation are jointly optimized using alternating updates. Specifically, after training for a certain number of iterations, the current feature representation is fixed, and the target domain pseudo-label is updated through a multi-filtering mechanism. Then, the CCC loss and DCC loss are minimized based on the updated target domain pseudo-label, i.e., the network parameters are updated through backpropagation. Therefore, the total target loss can be expressed as:

[0084] (49) in, β For the weights of the domain-level contrastive constraint loss and the class-level contrastive constraint loss, The overall objective is to minimize the loss. and This achieves precise alignment of data of the same category and data of the same pseudo-domain between the source and target domains, while maximizing the difference between data of different categories and data of different pseudo-domains between the source and target domains.

[0085] 7. Experiment.

[0086] The Office-Home dataset contains four domains, each encompassing 65 categories, making it undoubtedly a complex dataset. Given the large number of domains, four tasks were designed for this dataset: C, P, R → A, A, P, R → C, A, C, R → P, and A, C, P → R. This invention uses the Office-Home dataset for experiments, with all experiments focusing on classification tasks. Example images corresponding to the dataset are shown below. Figure 3 As shown.

[0087] 7.1 Parameter settings.

[0088] In all experiments, ResNet50 was used as the base network architecture. The original fully connected classification layer (FC) was replaced with a new fully connected layer designed for a specific recognition task. During model fine-tuning, labeled samples from the source domain and unlabeled data from the target domain were used together for training. In the data preprocessing stage, data augmentation strategies such as random cropping, random horizontal flipping, and brightness adjustment were employed to enhance the diversity of the dataset. In the testing stage, the input image was center-cropped to obtain a 224×224 pixel image for classification evaluation. The network training used a stochastic gradient descent optimizer with a momentum coefficient set to 0.9 to accelerate convergence. The learning rate update strategy was based on the formula... Perform calculations, where p The initial learning rate of the convolutional layer increases linearly from 0 to 1 until training ends. The initial learning rate is set to 0.001, while the initial learning rate of the fully connected layer is set to 0.01.

[0089] 7.2 Experimental Results.

[0090] To verify the effectiveness of class-level contrast constraints (CCC) and domain-level contrast constraints (DCC) in cross-domain classification, the performance of Baseline, CCC, DCC, and DUCC (a combination of both) was compared. Baseline is a baseline method without any MUDA. CCC adds class-level contrast constraints to Baseline, DCC adds domain-level contrast constraints to Baseline, and DUCC combines the two to achieve local-global dual-scale constraint optimization.

[0091] 7.2.1 Comparison and analysis of accuracy.

[0092] Figure 4The convergence processes of the Baseline, CCC, DCC, and DUCC methods on the A, P, R → C and A, P, C → R tasks are shown. Figure 4 (a) in the figure is a comparison diagram of the DUCC method with the Baseline method and the DCC method on the A, P, R → C task. Figure 4 (b) in the figure is a comparison of the DUCC method with the Baseline method and the CCC method on the A, P, R → C task; Figure 4 (c) in the figure is a comparison diagram of the DUCC method with the Baseline method and the DCC method on the A, P, C→R task; Figure 4 Figure (d) shows a comparison of the DUCC method with the Baseline and CCC methods on the A, P, C→R tasks. Overall, DUCC exhibits higher initial convergence speed and better performance across all tasks. Both CCC and DCC methods individually outperform the Baseline in convergence, reflecting the independent contributions of class-level semantic constraints and domain-level distribution constraints in MUDA. However, the final performance of both individually is slightly lower than DUCC, indicating that feature alignment at only local or global scales still has bottlenecks. DUCC combines both methods, effectively mitigating semantic or overall structural shifts that may result from single-scale optimization, thus maintaining a leading advantage during convergence.

[0093] 7.2.2 Comparison and analysis of t-SNE visualization.

[0094] To gain a deeper understanding of the feature alignment effects of different methods in multi-source domain tasks, the feature distributions of the Baseline, DCC, CCC, and DUCC methods were visualized for the A, P, R→R tasks, such as... Figure 5 As shown.

[0095] In the Baseline method ( Figure 5 In (a) of the model, the distribution of similar samples from different domains in the feature space is significantly discrete. Clusters of the same category lack aggregation across domains, and some clusters overlap significantly, indicating a deficiency in domain-invariant feature extraction. In the DCC method (… Figure 5 In (b) of the model, the distance between samples of the same class across domains is shortened, and the overall distribution balance is improved, reflecting the role of domain-level structural constraints in global geometric space alignment. However, the internal compactness of some categories remains limited, indicating that relying solely on global alignment is insufficient to completely eliminate local semantic differences. The CCC method ( Figure 5Compared to the Baseline method, (c) of the DUCC method significantly enhances the aggregation of similar samples, resulting in clearer cluster boundaries and demonstrating the advantage of class-level semantic constraints in local feature alignment. However, its global distribution still exhibits slight misalignment, with some overlap between cross-domain clusters. In contrast, the DUCC method ( Figure 5 (d) in the model exhibits highly compact and well-defined clusters in the feature space, with a high degree of overlap between similar samples from different domains, and clear boundaries between clusters without significant structural shifts. This effect is attributed to the synergistic effect of class-level and domain-level contrast constraints: at the local scale, the CCC method strengthens the aggregation of similar samples across domains, improving intra-class consistency; at the global scale, the DCC method ensures the consistency of multi-domain distribution structures, thereby reducing inter-domain distribution misalignment. The combination of these two approaches forms a local-global dual-scale optimization mechanism, achieving the simultaneous maintenance of high aggregation and balance in the feature spaces of different domains.

[0096] Experimental results show that the DUCC method significantly improves cross-domain classification accuracy on public benchmark datasets and achieves state-of-the-art performance. Theoretically, the DUCC method achieves a balance between semantic discriminativity and structural stability in a multi-layered fusion space. Methodologically, it is the first to unify class-level and domain-level alignment constraints into a fusion representation space and establishes a multi-layered filtering mechanism for label noise. This lays a solid foundation for future extensions to more complex multi-source adaptation tasks (such as open sets).

[0097] 8. Effect analysis.

[0098] This invention addresses the problem that existing cross-domain image classification methods generally focus on class-level semantic alignment while neglecting domain-level structural alignment. It proposes a Dual Contrast Constraint (DUCC) method that achieves local-global collaborative optimization through multi-layer feature fusion. This method jointly applies local class-level contrast constraints (CCC) and global domain-level contrast constraints (DCC) between each source and target domain, mitigating class shift while suppressing global domain shift.

[0099] To adapt to the heterogeneity and high noise levels of pseudo-labels in multi-source scenarios, DUCC designed a multi-filtering mechanism to effectively obtain high-quality pseudo-labels and provide stable and reliable supervision signals for dual contrast constraints.

[0100] Experimental results on benchmark datasets show that DCC plays a key role in suppressing domain shifts caused by multi-source heterogeneity and complements CCC, significantly improving the network model's ability to classify images across domains. This fully verifies the effectiveness and universality of local-global dual contrast constraints in multi-source domain adaptation tasks.

[0101] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A cross-domain image classification method based on dual contrast constraints, characterized in that, include: Obtain at least two source domain image datasets and one target domain image dataset. Each dataset should have inter-domain differences in visual style and contain samples from multiple categories. A cross-domain image classification model is constructed, comprising: a feature extractor and a classifier connected in sequence; feature extraction is performed on the source domain dataset and the target domain dataset by the feature extractor to obtain multi-layer image features; and a feature fusion function is used to fuse the multi-layer image features to obtain a fused feature space. Based on the class centers of each real label sample in the source domain dataset, nearest neighbor matching is performed on the target domain dataset. Target samples in the target domain dataset are filtered through multiple filtering mechanisms and assigned pseudo labels to obtain a pseudo label sample set. Based on real label samples and pseudo label samples, class-level contrast constraints and domain-level contrast constraints are applied in the fused feature space to obtain class-level contrast constraint loss and domain-level contrast constraint loss. The class-level contrast constraints are used to minimize the feature differences of label samples of the same class and maximize the feature differences of label samples of different classes, and the domain-level contrast constraints are used to minimize the feature differences of pseudo-domain label samples of the same class and maximize the feature differences of pseudo-domain label samples of different classes. The optimization objective is to minimize the total target loss obtained by weighted summation of class-level contrast constraint loss, domain-level contrast constraint loss and classification loss. The parameters of the feature extractor and classifier are continuously updated through backpropagation to obtain the trained cross-domain image classification model. The trained cross-domain image classification model is used to classify and predict the target domain dataset to obtain the cross-domain image classification results.

2. The cross-domain image classification method based on dual contrast constraints as described in claim 1, characterized in that, The application of class-level contrast constraints in the fused feature space specifically includes: The following formula is used to construct the distance matrix between categories in the source domain, the distance matrix between categories in the target domain, and the distance matrix between category pairs between categories in the source domain and categories in the target domain: ； ； ； The baseline bandwidth and kernel function bandwidth between category pairs in the source and target domains are determined based on the following formula: ； ； ； The multi-kernel Gaussian kernel function is determined based on the bandwidth of multiple kernel functions using the following formula: ； Based on the distance matrix and the multi-kernel Gaussian kernel function, the kernel matrices for the source domain, target domain, and cross-domain are calculated using the following formula: ； ； ； The MMD values ​​of class pairs between the source and target domains are determined based on the following formula, and a class contrast constraint matrix is ​​formed: ； Class-level comparison constraints are determined based on the following formula: ； in, m For the first m One source domain-target domain pair, x For input, c For categories within the source domain, c' For categories within the target domain, ( c , c' () represents a category pair between categories in the source domain and categories in the target domain. The set of samples representing categories within the source domain. The set of samples belonging to the categories within the target domain. The number of samples of each category within the source domain. The number of samples for each category within the target domain. α l For the network l The nuclear factor of the layer, m l For the network l Number of cores in the layer For the network l The square of the Euclidean distance between layer features, For the network l Distance matrix of categories within the source domain, For the network l Distance matrix of categories within the target domain of the layer. For the network l The distance matrix between categories in the source domain and categories in the target domain. For the network l The baseline bandwidth of category pairs between categories in the source domain and categories in the target domain. For the network l The kernel function bandwidth of category pairs between categories in the source domain and categories in the target domain. For the network l Multi-kernel Gaussian kernel function The kernel matrix represents the categories within the source domain. The kernel matrix of categories within the target domain. For cross-domain category kernel matrix, This represents the MMD value of the category pairs between categories in the source domain and categories in the target domain. This serves as a class-level comparison constraint between the source and target domains. C The total number of categories, The MMD values ​​are for labeled samples of the same type in both the source and target domains. The MMD values ​​are the heterogeneous label samples from the source and target domains.

3. The cross-domain image classification method based on dual contrast constraints as described in claim 1, characterized in that, The application of domain-level contrast constraints in the fused feature space specifically includes: Will C The categories were reorganized into D Each pseudo-domain contains [number] pseudo-domains. K The pseudo-domain sample sets of the source and target domains are determined based on the following formula: ； ； The number of samples in each pseudo-domain is determined by the following formula: the sum of the number of samples in all categories contained in that pseudo-domain. ； ； The following formula is used to construct the domain distance matrix of the pseudo-domain within the source domain, the domain distance matrix of the pseudo-domain within the target domain, and the domain distance matrix of the pseudo-domain pairs between the source and target domains: ； ； ； The baseline bandwidth of the pseudo-neighborhood pair between the source and target domains and the bandwidth of multiple kernel functions are determined based on the following formula: ； ； ； The multi-kernel Gaussian function that determines the pseudo-neighborhood pair between the source and target domains is based on the following formula: ； Based on the distance matrix and the multi-kernel Gaussian kernel function, the pseudo-neighborhood kernel matrices of the source domain, target domain, and pseudo-neighborhood pairs between the source and target domains are determined according to the following formula: ； ； ； The MMD value of the pseudo-neighborhood pair between the source and target domains is determined based on the following formula, and the neighborhood contrast constraint matrix is ​​obtained: ； The domain-level comparison constraint between the source and target domains is obtained based on the following formula: ； in, m For the first m One source domain-target domain pair, x For input, d For pseudo-domains within the source domain, d' For the pseudo-domain within the target domain, ( d , d' () represents a pseudo-domain pair between the source and target domains. The set of samples within the pseudo-domain of the source domain. The set of samples within the pseudo-domain of the target domain. The number of samples in the pseudo-domain within the source domain. The number of samples in the pseudo-domain within the target domain. α l For the network l The nuclear factor of the layer, m l For the network l Number of cores in the layer For the network l The square of the Euclidean distance between layer features, For the network l The distance matrix of the pseudo-neighborhood within the source domain. For the network l The distance matrix of the pseudo-neighborhood within the target domain of the layer. For the network l The distance matrix between the pseudo-neighborhood in the source domain and the pseudo-neighborhood in the target domain. For the network l The baseline bandwidth of pseudo-neighborhood pairs between the source and target domains. For the network l The kernel function bandwidth of pseudo-neighborhood pairs between the source and target domains. For the network l Multi-kernel Gaussian kernel function The kernel matrix of the pseudo-domain within the source domain. The kernel matrix of the pseudo-neighborhood within the target domain. For cross-domain pseudo-domain kernel matrix, This represents the maximum mean difference between pseudo-neighborhood pairs between the source and target domains. This serves as a domain-level comparison constraint between the source and target domains. D For the total number of pseudo-domains, The MMD values ​​of samples with pseudo-domain labels in both the source and target domains are given. This represents the MMD value of pseudo-domain label samples with different source and target domains.

4. The cross-domain image classification method based on dual contrast constraints as described in claim 1, characterized in that, The process involves performing nearest neighbor matching on the target domain dataset based on the class centers of each real labeled sample in the source domain dataset, filtering target samples in the target domain dataset through a multi-stage filtering mechanism, and assigning pseudo-labels to them to obtain a pseudo-label sample set. Specifically, this includes: The multi-filtering mechanism includes: sequentially executed multi-source consistency filtering, confidence threshold filtering, and minimum class sample number filtering; Based on the cosine distance from each sample in the target domain dataset to the center of each class in the source domain dataset, we obtain the data from... M Predicted labels and corresponding confidence levels for each source domain; The number of times each target sample is predicted to each category by different source domains is obtained. The category with the most predictions is taken as the consensus candidate category. When the number of predictions of the consensus candidate category reaches the preset majority threshold, the target sample is determined to pass the multi-source consensus filtering and the initial filtered sample is obtained. For each initial filtered sample, obtain the confidence scores corresponding to all samples that predict the initial filtered sample as a consensus category; if the confidence score of each source domain is not lower than the preset confidence score threshold, then the initial filtered sample is determined to have passed the confidence score threshold filtering and is used as an intermediate filtered sample. Obtain the number of samples in each pseudo-label category in the intermediate filtered samples, and retain only the categories with a sample number not less than the preset minimum category sample number threshold and their corresponding intermediate filtered samples to obtain the final pseudo-label sample set.

5. The cross-domain image classification method based on dual contrast constraints as described in claim 1, characterized in that, The total target loss, obtained by weighted summation of class-level contrastive constraint loss, domain-level contrastive constraint loss, and classification loss, is determined based on the following formula: ； ； ； ； in, For classifying losses, For domain-level contrast constraint loss, For class-level contrast constraint loss, β For the weights of the domain-level contrastive constraint loss and the class-level contrastive constraint loss, For the overall target loss, ∈{1,2,…, C } is a sample The true label, p This represents the predicted probability given the input.

Images