Unsupervised domain adaptation method based on progressive prototype alignment and entropy regularization

Abstract

This invention discloses an unsupervised domain adaptation method based on progressive prototype alignment and entropy regularization, comprising the following steps: Step 1, initializing a deep neural network and constructing a teacher network, a student network, and a discriminator; Step 2, the student network learns from the source domain through supervised loss; Step 3, adversarial training, in which the student network attempts to deceive the discriminator, which distinguishes between the source and target domains. This invention utilizes spatial context and employs various enhancement masking techniques to generate multi-view domain-invariant features, ensuring predictive consistency between the randomly masked target image and its pseudo-labels, and constructing a high-order graph; low-rank constraints are applied along the horizontal and vertical dimensions to better reveal inter-view and inter-class correlations among multi-view features.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to an unsupervised domain adaptive method based on progressive prototype alignment and entropy regularization. Background Technology

[0002] Deep neural networks (DNNs) have had a profound impact on the field of computer vision, driving significant advancements in areas such as image recognition, object detection, and data-driven sustainability. These advancements have facilitated transformative applications across various domains, including face recognition, medical imaging, and autonomous driving. Despite their impressive performance, DNNs typically require large amounts of labeled datasets and often face challenges when deployed across different domains.

[0003] Unsupervised domain adaptation (UDA) aims to overcome this limitation by achieving knowledge transfer from a labeled source domain to an unlabeled target domain, improving the performance of target data by aligning feature distributions without requiring labeling in the target domain. UDA methods can be broadly categorized into three types: discrepancy-based methods, adversarial methods, and self-supervised learning techniques. Discrepancy-based methods focus on minimizing the distributional differences between the feature representations of the source and target domains by using specific metrics. While effective in aligning marginal distributions, these methods often perform poorly near task-specific decision boundaries, resulting in weak feature discrimination at class boundaries. Adversarial methods aim to learn domain-invariant features by training a feature extractor to deceive the domain discriminator. However, these methods may suffer from instability during training, often leading to a bias towards the source domain at decision boundaries. Self-supervised learning methods utilize pseudo-labels or pre-tasks to facilitate domain adaptation (Chen, Lin et al., 2022). Despite these contributions, significant challenges remain, particularly in distinguishing categories with similar visual features due to the lack of true supervision for accurate feature distribution alignment. Summary of the Invention

[0004] To address the problems existing in the prior art, the present invention aims to provide an unsupervised domain adaptation method based on progressive prototype alignment and entropy regularization. This method solves the problems of existing unsupervised domain adaptation methods, such as weak feature discrimination ability near task-specific decision boundaries, unstable training process, bias towards source domain decision boundaries, and lack of accurate feature distribution alignment and real supervision when distinguishing similar visual feature categories.

[0005] To solve the above problems, the present invention adopts the following technical solution; An unsupervised domain adaptive method based on progressive prototype alignment and entropy regularization includes the following steps: Step 1: Initialize the deep neural network and construct the teacher network. Student Network With discriminator ; Step Two: Student Network By monitoring losses From the source domain The learning formula is as follows: ; in Source domain monitoring loss, For source domain samples, For source domain tags, The total number of samples in the source domain. The total number of categories, Let j be the true label of the j-th source domain sample in the k-th class. Student network right Predicted as the first The probability of a class; Step 3: Competitive training, student network Attempting to deceive the discriminator Discriminator Distinguish between source and target domains to combat loss. The formula is as follows: ; in To counter the loss of the domain, For the target domain sample, ( The discriminator is input with the log probability from the source domain. To determine the log probability of assuming the input comes from the target domain; Step 4: Apply various masking enhancements to the target domain samples. The teacher network generates pseudo-labels for the original image, and the student network generates predictions for the masked image. By comparing weighted similarity between categories and adding consistency between the teacher and student networks, the class separability is enhanced. ; in, For mask enhancement functions, This is a learnable inter-class weight matrix. For the first Class and the Weighted similarity between class predictions For the first The sum of weighted similarities between this class and all classes. The probability distribution of the teacher network's predictions for the masked image. The distribution of student networks' predictions for the same masked image; Step 5: After training the model using various enhancement masking techniques, extract features from the source and target domains. and( , where (i The resulting joint feature space is denoted as ( ; Then learn a unified higher-order graph representation ( Used to integrate multi-view features ( .

[0006] Step 6: For each view i, construct a similarity matrix. , The number of samples in the target domain; Calculate the Laplace matrix: ; in, Let be the normalized Laplace matrix of the i-th view. Let be the similarity matrix of the i-th view. Let be the degree matrix of the i-th view; Step 7: For each view i, solve for the embedding matrix. Preserve the popular structure, let (F= , of which each ( It is a column vector; Step 8: Embed each view into the graph Stacked as third-order tensors, =I is the orthogonality of the embedding matrix: ; in( ; Step Nine: Learning Higher-Order Graphs Jointly optimize the following objectives: ; in, Let be the spectral embedding matrix of the i-th view. For the total number of views, and All of these are regularization parameters. The trace of the matrix, For stacked tensors, To minimize the learning graph The difference from the original stacked tensor Let be the nuclear norm of the i-th view slice. To fix the first One sample, the number of images across all views; The formula is optimized using an alternating optimization method; Step 10: Utilize the learned higher-order graphs Construct an affinity matrix to map multi-view features to a joint subspace, thereby enhancing domain invariance; Step 11: For the target sample, calculate the distance from its features in the unified subspace to each class prototype, and average the results of multiple views to obtain the final prediction.

[0007] As a further description of the above technical solution: Teacher network in step one With student network Each has a feature extractor and a classifier.

[0008] As a further description of the above technical solution: The mask enhancement methods in step four include Gaussian blurring, random erasure, and grayscale conversion.

[0009] As a further description of the above technical solution: Steps three and four employ a joint optimization framework of maxima and minima, and the optimization formula is as follows: ; in, To optimize student network parameters Teacher network parameters This minimizes the overall loss. To optimize discriminator parameters To maximize its resistance and losses Weighting coefficients to counteract losses, These are the weighting coefficients for the TCACL loss.

[0010] As a further description of the above technical solution: The optimization term in step nine is divided into three terms. The first term is the spectral embedding matrix defined by the graph structure. The total variance captured; the second term minimizes the stacked tensor using the Frobenius norm ( and the learned graphs ( The differences between them; the third item on the graph ( It imposed a joint low-rank and sparse structure.

[0011] As a further description of the above technical solution: The alternating optimization method in step nine involves fixing the formula first. ,optimization Then fix ,optimization .

[0012] Compared with the prior art, the advantages of this invention are: (1) This scheme utilizes spatial context and employs various enhancement masking techniques to generate multi-view domain invariant features, ensuring the predictive consistency between the random masked target image and its pseudo-label. By combining Laplacian graphs to propagate these multi-view features, a higher-order graph is constructed to propagate multi-view features and explore the consistency and complementary information in different contextual relationships.

[0013] (2) In this scheme, low-rank constraints are applied along the horizontal and vertical dimensions to better reveal the inter-view and inter-class correlations among multi-view features. The high-order graph is used to create an affinity matrix that maps multi-view features into a unified subspace. Subsequently, prototype vectors and unsupervised clustering are used to calculate the conditional probability of the UDA task. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the unsupervised domain adaptive method. Detailed Implementation

[0015] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0016] Please see Figure 1 The following examples are provided: An unsupervised domain adaptive method based on progressive prototype alignment and entropy regularization includes the following steps: Step 1: Initialize the deep neural network and construct the teacher network. Student Network With discriminator Three clearly defined functional modules are established: a teacher network provides stable labels, a student network is used for feature extraction and classification, and a discriminator assists in adversarial training, forcing the student network to learn domain-invariant features. With student network Each has a feature extractor and a classifier.

[0017] Step Two: Student Network By monitoring losses From the source domain Learning involves the model first learning to correctly classify the source domain, providing a reliable initial decision boundary for subsequent adaptation. The learning formula is as follows: ; in Source domain monitoring loss, For source domain samples, For source domain tags, The total number of samples in the source domain. The total number of categories, Let j be the true label of the j-th source domain sample in the k-th class. Student network right Predicted as the first The probability of a class.

[0018] Step 3: Competitive training, student network Attempting to deceive the discriminator Discriminator Distinguish between source and target domains to combat loss. The formula is as follows: ; in To counter the loss of the domain, For the target domain sample, ( The discriminator is input with the log probability from the source domain. To determine the log probability of assuming the input comes from the target domain; To mitigate the performance degradation caused by the distribution shift between the source and target domains, we employed adversarial training, which included a discriminator. and combined feature extractors and classifiers The adversarial objective is formulated as a mini-maximum problem, in which student networks... The aim is to minimize inter-domain differences, while the discriminator The goal is to maximize its ability to distinguish between different domains; thus forcing the features extracted by the student network to be indistinguishable from the source domain or the target domain by the discriminator, thereby learning domain-invariant features.

[0019] Step 4: Apply various masking enhancement methods to the target domain samples, including Gaussian blur, random erasure, and grayscale conversion. The teacher network generates pseudo-labels for the original image, and the student network generates predictions for the masked image. By comparing weighted similarity between categories and adding consistency between the teacher and student networks, the class separability is enhanced. ; in, For mask enhancement functions, This is a learnable inter-class weight matrix. For the first Class and the Weighted similarity between class predictions For the first The sum of weighted similarities between this class and all classes. The probability distribution of the teacher network's predictions for the masked image. This represents the predicted distribution of the student network for the same masked image.

[0020] This loss function aims to make class boundaries in the target feature space clearer and more distinguishable by utilizing the contextual relationships between target samples; masking function. During training, target domain samples are augmented by applying transformations such as Gaussian blur, random erasure, and grayscale conversion to selectively mask regions of the input data. These augmentations introduce controlled distortions, forcing the model to learn invariant and robust features. For example, Gaussian blur reduces detail, prompting the model to focus on key structures; random erasure occludes parts of the image, enhancing the model's ability to infer missing information; and grayscale conversion removes color, making the model rely more on texture and shape features. To enhance the consistency of class distribution, the weights of each output are adjusted during computation, involving different categories. and A normalized comparison of the weighted classifier outputs between classes promotes clearer class distinctions. By calculating and averaging the absolute differences across all classes, the model is encouraged to reduce misclassification and improve the separability of class-specific features. Furthermore, the teacher network... It generates pseudo-labels for the target samples, and compares these pseudo-labels with the prediction results of the mask image.

[0021] Step 5: After training the model using various enhancement masking techniques, extract features from the source and target domains. and( , where (i The resulting joint feature space is denoted as ( ; Then learn a unified higher-order graph representation ( Used to integrate multi-view features ( ; A pre-trained student network is used to apply v different enhancement masks to each sample, extracting v sets of feature vectors. These features represent the behavior of the same object under different "interferences", forming multi-view data.

[0022] Step 6: For each view i, construct a similarity matrix. , The number of samples in the target domain; Calculate the Laplace matrix: ; in, Let be the normalized Laplace matrix of the i-th view. Let be the similarity matrix of the i-th view. Let be the degree matrix of the i-th view; The goal of this higher-order graph learning module is to construct higher-order graphs. This effectively captures relationships within and across views, thereby holistically integrating multi-view features. First, a view-specific similarity map is constructed. To capture the local relationships and structure within each view (v), this ensures an accurate representation of the inherent similarities and differences between data samples. The similarity map is used to capture local structural relationships, while the Laplace matrix provides a global perspective of the data by integrating normalized pairwise relationships between points.

[0023] Step 7: For each view i, solve for the embedding matrix. Preserve the popular structure, let (F= , of which each ( It is a column vector; Step 8: Embed each view into the graph Stacked as third-order tensors, =I is the orthogonality of the embedding matrix: ; in( ; Embedded graphs for each view As a slice of a tensor, it forms a third-order tensor.

[0024] Step Nine: Learning Higher-Order Graphs Jointly optimize the following objectives: ; in, Let be the spectral embedding matrix of the i-th view. For the total number of views, and All of these are regularization parameters. The trace of the matrix, For stacked tensors, To minimize the learning graph The difference from the original stacked tensor Let be the nuclear norm of the i-th view slice. To fix the first One sample, the number of images across all views; The formula is optimized using an alternating optimization method.

[0025] Step 10: Utilize the learned higher-order graphs Construct an affinity matrix to map multi-view features to a joint subspace, thereby enhancing domain invariance.

[0026] Step 11: For the target sample, calculate the distance from its features in the unified subspace to each class prototype, and average the results of multiple views to obtain the final prediction; Steps three and four employ a joint optimization framework of maxima and minima, with the optimization formula as follows: ; in, To optimize student network parameters Teacher network parameters This minimizes the overall loss. To optimize discriminator parameters To maximize its resistance and losses Weighting coefficients to counteract losses, These are the weighting coefficients for the TCACL loss; parameter and , representing the weights of the teacher network and student network respectively, are optimized to minimize the overall loss, while the parameters of the discriminator are... It is then adjusted to maximize the ability to mitigate losses. and These are used as weighting factors to adjust the contributions of adversarial loss and target class alignment and consistency loss, respectively. Model training is complete, and the student network... Features will be used to extract enhanced context from the source and target domains.

[0027] The optimization term in step nine is divided into three terms. The first term is the spectral embedding matrix defined by the graph structure. The total variance captured; the second term minimizes the stacked tensor using the Frobenius norm ( and the learned graphs ( The differences between them; the third item on the graph ( It imposed a joint low-rank and sparse structure.

[0028] The first term maximizes the spectral embedding matrix defined by the graph structure. The total variance captured. By maximizing the trace, we effectively enhance the representation of the data points to ensure they are well separated according to the graph structure. Constraints ( This ensures that the feature transformations are orthogonal, preserves the distance, and provides an effective orthogonal basis for the transformed features.

[0029] The second term uses the Frobenius norm to minimize the stacked tensor. and the learned diagrams The differences between them, among which As a regularization parameter, it controls | By minimizing this difference, we promote consistency across different views while preserving the discriminative structure of the data. The third term enhances the discriminative performance of multi-view clustering. Here, This represents the (v)th dimension element in the higher-order graph, which, along with elements from other views, is used to maintain consistency across views.

[0030] The last item is related to the graph. A joint low-rank and sparse structure is imposed to ensure the relevance of feature selection. We apply a nuclear norm to ensure the low-rank constraint, while using... Norms promote group sparsity, thereby selecting the most relevant features.

[0031] By alternately optimizing these terms, we are able to generate a high-order graph representation that integrates multi-view features by capturing the local and global structure of each view. The imposed graph constraints enhance the model's cross-domain generalization ability, ensuring that the graph captures key relationships while maintaining low rank and sparsity.

[0032] The alternating optimization method in step nine involves fixing the formula first. ,optimization Then fix ,optimization For each view, we compute its spectral embedding matrix and use it to optimize the final higher-order graph representation.

[0033] The above description is merely a preferred embodiment of the present invention; however, the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and its improved concepts, should be covered within the scope of protection of the present invention.

Claims

1. An unsupervised domain adaptation method based on progressive prototype alignment and entropy regularization, characterized in that, Includes the following steps: Step one, initialize the deep neural network, build the teacher network , student network and discriminator ; Step Two: Student Network By monitoring losses Learning from the source domain, the learning formula is as follows: ； in Source domain monitoring loss, For source domain samples, For source domain tags, The total number of samples in the source domain. The total number of categories, Let j be the true label of the j-th source domain sample in the k-th class. Student network Predicted as the first The probability of a class; Step three, adversarial training, student network Trying to fool the discriminator , the discriminator Discriminating between source and target domains, adversarial loss The formula is as follows: ； wherein is an adversarial domain discrimination loss, is a target domain sample, is a discriminator artificial input of a log probability from the source domain, is a log probability from the target domain that the discriminator thinks the input is from.​ Step 4: Apply various masking enhancements to the target domain samples. The teacher network generates pseudo-labels for the original image, and the student network generates predictions for the masked image. By comparing weighted similarity between categories and adding consistency between the teacher and student networks, the class separability is enhanced. ； in, For mask enhancement functions, This is a learnable inter-class weight matrix. For the first Class and the Weighted similarity between class predictions For the first The sum of weighted similarities between this class and all classes. The probability distribution of the teacher network's predictions for the masked image. The distribution of student networks' predictions for the same masked image; Step 5: After training the model using various enhancement masking techniques, extract features from the source and target domains. and( Where (i, the resulting joint feature space is denoted as ( ; Then learn a unified high-order graph representation , for integrating multi-view features ; Step six, for each view i, construct a similarity matrix , is the number of target domain samples; Calculate the Laplace matrix: ； in, Let be the normalized Laplace matrix of the i-th view. Let be the similarity matrix of the i-th view. Let be the degree matrix of the i-th view; Step seven, for each view i, solve for the embedding matrix , preserving the popular structure, let (F where each (X is a column vector; Step eight, embed each view into a graph Stacking into a third-order tensor, =I is the embedding matrix orthogonal: ； wherein ; Step nine, learn high order graph jointly optimize the following objectives: ； in, Let be the spectral embedding matrix of the i-th view. For the total number of views, and All of these are regularization parameters. The trace of the matrix, For stacked tensors, minimize the learning graph. The difference from the original stacked tensor is the nuclear norm of the i-th view slice. To fix the first One sample, the number of images across all views; The formula is optimized using an alternating optimization method; Step ten, utilizing the learned high order graph Constructing affinity matrix, mapping multi-view features to joint subspace, enhancing domain invariance; Step 11: For the target sample, calculate the distance from its features in the unified subspace to each class prototype, and average the results of multiple views to obtain the final prediction.

2. The unsupervised domain adaptive method based on progressive prototype alignment and entropy regularization according to claim 1, characterized in that: The teacher network in step one has a feature extractor and a classifier. The student network also has a feature extractor and a classifier. 3.The unsupervised domain adaptation method based on progressive prototypical alignment and entropy regularization of claim 1, wherein: The mask enhancement methods in step four include Gaussian blurring, random erasure, and grayscale conversion.

4. The unsupervised domain adaptation method based on progressive prototypical alignment and entropy regularization of claim 1, wherein: Steps three and four employ a joint optimization framework of maxima and minima, and the optimization formula is as follows: ； in, To optimize student network parameters Teacher network parameters To minimize the overall loss To optimize discriminator parameters To maximize its resistance and losses Weighting coefficients to counteract losses, These are the weighting coefficients for the TCACL loss.

5. The unsupervised domain adaptation method based on progressive prototypical alignment and entropy regularization of claim 1, wherein: The optimization term in step nine is divided into three terms. The first term is the spectral embedding matrix defined by the graph structure. The total variance captured; the second term minimizes the stacked tensor using the Frobenius norm ( and the learned graphs ( The differences between them; the third item on the graph ( It imposed a joint low-rank and sparse structure.

6. The unsupervised domain adaptation method based on progressive prototypical alignment and entropy regularization of claim 1, wherein: The alternating optimization method in step nine involves fixing the formula first. ,optimization Then fix ,optimization .

Images