Cluster-based Adversarial Partial Domain Adaptation Method for Cross-Subject EEG Emotion Recognition

Abstract

The present invention discloses a clustering-based adversarial partial domain adaptation cross-subject EEG emotion recognition method, which calculates the cluster centers using the features of source domain samples, takes the true labels of the source domain as cluster labels, introduces a consistency matching algorithm and a cross-domain clustering consensus metric, uses Kmeans clustering to obtain the corresponding cluster labels and cluster centers of unlabeled target domain samples, performs consistency matching between the source domain cluster centers and the target domain cluster centers, for two successfully matched clusters, assigns the source domain labels to the target domain clusters with common semantics, and at the same time calculates the cross-domain clustering consensus metric to search for the optimal number of target domain clusters, ultimately realizing the association of common categories and the separation of private categories between the source domain and the target domain. This method fully considers the feature space distribution structure of unlabeled data, has high universality, can greatly improve the model training efficiency, and provides technical support for clinical applications.

Description

Technical Field

[0001] The present invention relates to the field of electroencephalogram (EEG) emotion recognition, and proposes an adversarial partial domain adaptation algorithm based on clustering, which is applicable to the scenario where the target sample category is a subset of the source sample category, and solves the problem of cross-subject EEG individual differences and the partial domain adaptation problem of cross-domain class imbalance. Background Art

[0002] How to effectively solve the problem of poor model generalization performance when deep neural networks perform EEG emotion recognition across subjects is a hot topic widely concerned in the current fields of machine learning and brain-computer interfaces. The traditional method is to manually design and extract effective EEG emotion features, and use machine learning models such as support vector machines for emotion classification, which requires relying on expert knowledge and is time-consuming and laborious. With the development of deep learning, it has been widely used in various fields due to its advantages of not requiring manual feature extraction, being able to automatically learn features, and having powerful data representation capabilities. In the field of EEG emotion recognition, the most commonly used deep neural networks at present are convolutional neural networks and long short-term memory networks. However, due to the characteristics of low signal-to-noise ratio and non-stationarity of EEG emotion signals, and the fact that each subject expresses emotions in different ways, there are differences in the emotional characteristics of EEG collected by different individuals and single individuals, which easily leads to uneven sample feature distributions and poor generalization performance of cross-subject EEG emotion datasets, making the practical application of EEG emotion recognition limited. Therefore, an EEG emotion recognition method that is applicable to cross-subjects and can adaptively adjust the sample feature distribution to improve the generalization performance of the EEG emotion recognition model is needed.

[0003] Domain adaptation in transfer learning is a machine learning algorithm for solving the distribution shift between the source domain and the target domain. The current main focus of domain adaptation methods is how to minimize the distribution difference, aiming to directly apply the classifier learned from the source domain to the target domain by learning the domain-invariant features of the source domain and the target domain in the case where there are no or few labels in the target domain. At the same time, in the field of cognitive neuroscience, some scholars have also begun to apply transfer learning to the analysis of neurophysiological signals, but there is still a large room for improvement in its accuracy and universality. Currently, the biggest problem in EEG emotion recognition is how to avoid the influence of individual differences, that is, there are huge differences in the EEG signals of each subject under the same cognitive state.

[0004] From the perspective of the cross-domain conditional class distribution, existing unsupervised domain adaptation methods are all for tasks where the source domain and the target domain share classes. That is to say, the class information of source samples and target samples is consistent. However, in more realistic and challenging scenarios, there may be various challenges such as uneven sample distribution and class imbalance in the sample data. How to further transfer knowledge in the case of class imbalance is a more challenging problem in current domain adaptation. At present, some scholars have begun to study the cross-domain class imbalance in computer vision, such as partial domain adaptation. When performing EEG emotion recognition, if one wants to learn the emotion recognition patterns of subjects in the existing source domain and transfer them to a new target domain, and the emotion recognition states of the target domain are not completely consistent with those of the source domain, it is called partial domain adaptation of emotion recognition. In real scenarios, we often want to learn common knowledge from datasets with class imbalance. Therefore, the present invention will construct a partial domain adaptation model applicable to emotion recognition algorithms in cases of large individual differences and cross-domain class imbalance for practical clinical applications. Summary of the Invention

[0005] Most existing methods directly use the classifier trained on source domain data to provide pseudo-labels for unlabeled data in the target domain. For target tasks with relatively fuzzy decision boundaries, and in the context of partial domain adaptation, the reliability of pseudo-labels cannot be guaranteed, which is likely to have a greater negative impact on the model. Most of the current relevant research results are applied in image recognition, object detection, etc., and no relevant research has been carried out in neurophysiology. Therefore, the present invention provides a clustering-based adversarial partial domain adaptation cross-subject EEG emotion recognition method, and proposes a clustering-based adversarial partial domain adaptation algorithm applicable to the partial domain adaptation problem of cross-subject emotion recognition. For scenarios where EEG data may have individual differences and class imbalance between datasets, and the target sample classes are subsets of the source sample classes, it fully learns the structural features of labeled samples, aligns the feature distributions of the target domain and the source domain, so as to achieve positive transfer of features between domains. The proposed method mainly has two aspects:

[0006] On the one hand, feature distribution alignment based on the domain adversarial method: First, we use the domain adversarial method to align the distributions between the source domain and the target domain, and construct an EEG emotion recognition model. This model includes a discriminator, a feature extractor, and a classifier. The domain discriminator is used to judge whether the output features of the feature extractor come from the source domain or the target domain, and the task of the feature extractor is to extract similar features between the two domains so that the discriminator cannot distinguish them. Through such a game process, the purpose of aligning the marginal distributions of the source domain and the target domain features is achieved.

[0007] On the other hand, partial domain transfer based on unsupervised Kmeans clustering: In this paper, a clustering algorithm based on Kmeans is used to perform unsupervised clustering on the target domain, and the class cluster centers of the source domain specific categories are aligned with those of the target domain to achieve the separation of common categories and private categories across domains. First, the source domain features are obtained through a feature extractor, and the L2 regularization is performed on the source domain features. The feature centers of each specific category sample in the source domain are calculated, and the class cluster labels are assigned to them using the true labels of the source domain. Then, the feature centers of the source domain samples are used as the initial class cluster centers for the target domain clustering, and the unsupervised Kmeans clustering method is used to perform initial clustering on the features of the target domain data. However, since the labels of the target domain are unknown and are a subset of the source domain labels, the actual number of class clusters in the target domain cannot be determined. Therefore, we search for the optimal number of class clusters, perform clustering on the target domain data with multiple different numbers of class clusters, and use a cross-domain clustering consensus evaluation index to evaluate the quality of the cross-domain clustering effect. Finally, the optimal class cluster centers are selected, and the consistency matching method is used to associate the common class clusters with the same semantic categories in the source domain and the target domain, naturally separating the private categories. Finally, the true labels of the source domain that match the semantics of the target domain class clusters are used as the pseudo-labels of the target domain, which have higher credibility. Essentially, this algorithm can solve the situation where the label spaces of the target domain and the source domain are inconsistent from the perspective of structural features, and achieve the association of common categories and the separation of private categories between the source domain and the target domain.

[0008] In summary, the present invention aims at emotion recognition based on cross-subject EEG, takes individual differences as the key starting point, and recognizes the emotional states of the subjects. Its core technology is mainly to construct an adversarial partial domain adaptation method based on the Kmeans clustering algorithm, which solves to a certain extent the problems of individual differences in EEG data and the imbalance of label categories between source samples and target samples, and realizes the association of common categories and the separation of private categories across domains. The present invention forms the existing trained subject data into a source domain, and the new test subject data as the target domain. The class cluster centers are calculated using the high-dimensional domain-invariant features obtained by the neural network from the source domain samples, and the true labels of the source domain are used as the class cluster labels. The consistency matching algorithm and the cross-domain clustering consensus index are introduced, and the corresponding class cluster labels and class cluster centers of the unlabeled target domain samples are obtained by Kmeans clustering. The source domain class cluster centers and the target domain class cluster centers are matched for consistency, and the two successfully matched class clusters are regarded as class clusters of common categories with consistent semantics, and the source domain labels are assigned to the target domain class clusters with common semantics. At the same time, the cross-domain clustering consensus index is calculated to search for the optimal number of target domain class clusters, and finally the association of common categories and the separation of private categories between the source domain and the target domain are realized. This method fully considers the feature space distribution structure of unlabeled data, has high universality, can greatly improve the model training efficiency, and provides technical support for clinical applications.

[0009] The technical solution adopted to overcome the deficiencies of the existing methods is as follows:

[0010] A cross-subject EEG emotion recognition method based on clustering for adversarial partial domain adaptation proposed by the present invention is applicable to the scenario of partial domain adaptation. By analyzing the features of EEG emotion data, the emotional cognitive states of the subjects are classified.

[0011] The present invention is premised on the SEED emotion dataset:

[0012] Step 1: Data preprocessing;

[0013] The publicly available SEED dataset is used as the training dataset; the electroencephalogram signal data needs to be preprocessed before being input into the model. The differential entropy (DE) features are extracted from 5 frequency bands of the SEED dataset per second: δ: 1 - 3 Hz, θ: 4 - 7 Hz, α: 8 - 13 Hz, β: 14 - 30 Hz, γ: 31 - 50 Hz. The feature dimension is 310 (62 channels × 5 frequency bands).

[0014] Step 2: Data definition

[0015] Given N EEG data with individual differences, all subject individuals are used as the source domain and their cognitive state labels are known. This source domain is denoted as X s , the source domain label is denoted as Y s , the number of classes is C, and the new unlabeled subject individuals are used as the target domain X t , the number of classes K is unknown.

[0016] Step 3: Construct and train the EEG emotion recognition model;

[0017] The described EEG emotion recognition model includes a shared feature extractor G, a classifier F, and a discriminator D.

[0018] Furthermore, the specific method of Step 3 is as follows:

[0019] Input: The labeled source domain data {X s ,Y s}, this source domain is denoted as X s , the source domain label is denoted as Y s , the number of classes is C, the unlabeled target domain data X t (the number of classes K is unknown), and the maximum number of iterations T;

[0020] 3 - 1. Use the shared feature extractor G to extract the potential common features of the source domain and target domain sample data, and map the extracted common features to a common feature space;

[0021] 3 - 2. Using the source domain sample data X sThe specific class label is the constrained optimization classifier F, and the objective function is the supervised classification loss L ce ;

[0022] 3-3. Using the domain adversarial method, the feature extractor G can learn invariant features from both the source domain and the target domain to confuse the discriminator D into thinking that the features come from the same domain, aligning the marginal distributions of the two domains in the feature space. The adversarial objective function is L d ;

[0023] 3-4. The source domain data X s and the target domain data X t can extract domain-invariant feature vectors V s and V t through the feature extractor G. The source domain data of a specific class with the true label c is represented as The feature vector extracted from is Finally, the cluster center of the samples of class c in the source domain is calculated through the specific class feature vector The cluster labels of the source domain samples are assigned based on the true labels, and finally the cluster centers of all classes in the source domain and the cluster labels {1, …, C} are obtained;

[0024] 3-5. Use the Kmeans clustering method to cluster the target domain sample features V t to obtain the corresponding target domain sample cluster centers and the cluster labels {1, …, K};

[0025] 3-6. Since the cluster labels obtained by Kmeans clustering do not have a corresponding relationship with the true labels of the source domain, use consistency matching to associate the cluster centers of the common classes with consistent semantics in the source domain with the target domain cluster centers Given a pair of cluster features of the source domain and the target domain as and where the cluster from one domain searches for the closest cluster center μ in the other domain, and then determines whether both are the closest cluster centers to each other. If both are the closest cluster centers to each other, such a pair of matching clusters is considered a common cluster, and the cluster labels of this pair of clusters are {c, k}. Then, the pseudo-label of the k-th cluster target domain sample corresponding to it is assigned the matching source domain label c;

[0026] 3-7. Use cosine similarity to calculate the distance from the samples of a certain domain to all the cluster centers of the other domain. For the i-th sample in the source domain of a pair of clusters, calculate its distance to all the target domain cluster centers Cosine similarity Similarly, calculate the cosine similarity between the i-th sample in the target domain and all source domain cluster centers Cosine similarity

[0027] 3-8. Calculate the source domain clustering consensus score using the results obtained in 3-7 and the target domain clustering consensus score

[0028] 3-9. Then, take the average of the source domain clustering consensus score obtained in 3-8 and the target domain clustering consensus score to obtain the cross-domain clustering consensus score S for this pair of clusters, and finally calculate the average value S of the consensus scores of all cluster pairs (c,k) ; total ;

[0029] 3-10. To determine the number of clusters K in the target domain clustering, set different K values for the Kmeans algorithm, repeat steps 3-4 to 3-9 for multiple clusterings, and determine the optimal number of clusters according to the cross-domain clustering consensus score S total Finally, perform clustering with the optimal number of clusters to obtain the optimal target domain cluster centers;

[0030] 3.11. To improve the discriminability of the target clustering, reconstruct the target domain dataset with pseudo-labels and apply a prototype regularization term to the neural network using the target data with pseudo-labels to promote the optimization of clustering and align the features at the class level.

[0031] 3-12. The overall objective optimization function is:

[0032]

[0033] where λ1 and λ2 are hyperparameters of the model.

[0034] 3-13. Before optimizing the neural network, perform an initial clustering to obtain the target domain dataset with pseudo-labels To avoid the accumulation of inaccurate labels, update the clustering alternately while optimizing the model. For every five optimizations of the neural network, re-perform the clustering, reconstruct the pseudo-label data, until the iteration reaches T times.

[0035] Output: The cluster label corresponding to the target domain sample that is closest to the cluster prototype center.

[0036] The beneficial effects of the present invention are as follows:

[0037] First, a new problem in this field is discovered and solved, namely the partial domain adaptation problem in unsupervised domain adaptation. The present invention can be applied generally to the domain adaptation problem with inconsistent cross-domain feature distributions, and to a certain extent, greatly reduces the model complexity and significantly improves the time efficiency. Secondly, the present invention proposes a clustering-based adversarial partial domain adaptation algorithm applicable to the state of class imbalance between the source domain and the target domain; research shows that a large number of models directly use the source domain classifier to provide pseudo-labels for unlabeled target domain samples, but in the state of label class imbalance, the reliability of the pseudo-labels cannot be guaranteed, which is likely to bring a greater negative impact to the model; therefore, starting from the structural features of the samples themselves, the present invention proposes a clustering algorithm that fully considers the structural distribution characteristics of samples between classes, and realizes the separation of public classes and private classes through cross-domain class cluster consistency matching and cross-domain clustering consensus evaluation. Finally, the present invention effectively solves the problem of individual differences in electroencephalogram signals in the field of brain cognitive computing, can be applied to the recognition of cognitive states based on EEG under any task, has strong generalization ability, and can be well applied to clinical diagnosis and practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is the structural diagram of the implementation model of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] The present invention will be further described below with reference to the drawings.

[0040] As Figure 1 shown, it is the structural diagram of the model of the clustering-based adversarial partial domain adaptation cross-subject EEG emotion recognition method, which mainly includes the following steps:

[0041] Step 1: Data preprocessing

[0042] The electroencephalogram signals of the data set are preprocessed before being input into the framework. The differential entropy (DE) features are extracted from 5 frequency bands of the SEED data set per second: δ: 1 - 3 Hz, θ: 4 - 7 Hz, α: 8 - 13 Hz, β: 14 - 30 Hz, γ: 31 - 50 Hz. The feature dimension is 310 (62 channels × 5 frequency bands).

[0043] For a specific length of EEG signal approximately following a Gaussian distribution its differential entropy is:

[0044]

[0045] equal to the logarithm of the energy spectrum on a specific frequency band.

[0046] The SEED dataset is a publicly available dataset from the BCMI Laboratory of Shanghai Jiao Tong University, consisting of 15 participants. Each person was asked to watch 15 emotional movie clips to elicit three emotions: positive, negative, and neutral. The electroencephalogram (EEG) signals were recorded using a 62-channel ESI neuroscan system at a sampling rate of 1000 Hz.

[0047] Step 2: Data Definition

[0048] Given N EEG data with individual differences, all subject individuals are used as the source domain and their cognitive state labels are known. This source domain is denoted as X s , and the source domain labels are denoted as Y s , with the number of classes C, while the new unlabeled subject individuals are used as the target domain X t , and the number of classes K is unknown.

[0049] Step 3: A Clustering-Based Adversarial Partial Domain Adaptation Method for Cross-Subject EEG Emotion Recognition

[0050] Input: Source domain sample data {X s , Y s} with emotion classification labels, the number of classes C, target domain sample data X without emotion state labels t (the number of classes K is unknown), and the maximum number of iterations T;

[0051] 3-1. Use the shared feature extractor G to extract the potential common features of the source domain and target domain sample data, and map the extracted common features to a common feature space. The common feature extractor uses a neural network with 3 hidden layers, and each hidden layer of the network has 512 nodes, and the ReLU activation function is used. The output of the network has the same 310 dimensions as the input data;

[0052] 3-2. Optimize the classifier F with the unique class labels of the source domain sample data X s . The network layer of the classifier F is set to 3 layers. Each hidden layer of the network has 64 nodes, and the ReLU activation function is used. The input of the network is 310 dimensions, and the output is 3 dimensions. The objective function is the supervised classification loss loss:

[0053]

[0054] ​​3-3. Using the domain adversarial method, the feature extractor G can learn invariant features from both the source domain and the target domain to confuse the discriminator D into thinking that the features come from the same domain. The discriminator adopts the same network structure as the classifier. Finally, the feature extractor G can extract domain-invariant features from the source domain and target domain data, promoting the alignment of the marginal distributions of the features in the feature space, and the adversarial objective function is L d as follows:

[0055]

[0056] L d = L adv_g (X s , X t , G) + L adv_d (X s , X t , D)

[0057] 3-4. The source domain data X s and the target domain data X t can extract domain-invariant feature vectors V s and V t through the feature extractor G. Represent the source domain data of a specific class with the true label C as The feature vector extracted from is Finally, we calculate the cluster center of the samples of class C in the source domain through the specific class feature vector The cluster labels of the source domain samples are assigned based on the true labels, and finally the cluster centers of all classes in the source domain and the cluster labels {1,..., C} are obtained.

[0058] The calculation of the source domain cluster center is as follows:

[0059] G(X s ) = V s

[0060]

[0061] where represents the number of source domain samples of the c-th class.

[0062] 3-5. Use the Kmeans clustering method to cluster the target domain sample features V t to obtain the corresponding target domain sample cluster centers and the cluster labels {1,..., K};

[0063] 3-6. Since the cluster labels obtained by Kmeans clustering do not correspond to the true labels of the source domain, consistency matching is used to associate the source domain cluster centers of the common categories with consistent semantics with the target domain cluster centers Given a pair of cluster features of the source domain and the target domain as and where the cluster from one domain searches for the closest cluster center μ to this cluster in the other domain, and then determines whether both are the closest cluster centers to each other. If both are the closest cluster centers to each other, such a pair of matching clusters is considered a common cluster, and the cluster labels of this pair of clusters are {c, k}. Then, the pseudo-label of the k-th cluster target domain sample corresponding to it is assigned as the matching source domain label c;

[0064] 3-7. Use cosine similarity to calculate the distance from a sample from a certain domain to all cluster centers of another domain. For the i-th sample in the source domain of a pair of clusters, calculate its cosine similarity with all target domain cluster centers The calculation formula is as follows:

[0065]

[0066] Similarly, the cosine similarity between the i-th sample in the target domain and all source domain cluster centers can be calculated The formula is as follows:

[0067]

[0068] 3-8. Use the results obtained in 3-7 to calculate the source domain clustering consensus score and the target domain clustering consensus score

[0069] The source domain cross-domain clustering consensus score can be expressed as the proportion of samples that reach a consensus:

[0070]

[0071] where is an index to judge whether the i-th source domain feature vector holds the corresponding cluster index k. Similarly, the target domain is obtained

[0072] 3-9. Then, take the average of the source domain clustering consensus score obtained in 3-8 and the target domain clustering consensus score to obtain the cross-domain clustering consensus score S of this pair of clusters (c,k), finally calculate the average value \(S\) of the consensus scores of all cluster pairs total , \(m\) represents the number of matched cluster pairs, and the calculation formula is as follows:

[0073]

[0074] 3-10. To determine the number of clusters \(K\) in the target domain clustering, set different \(K\) for the Kmeans algorithm, repeat steps 3-4 to 3-9 for multiple clusterings, and according to the cross-domain clustering consensus score \(S\) total determine the optimal number of clusters, and finally perform clustering with the optimal number of clusters to obtain the optimal target domain cluster center;

[0075] 3-11. To improve the discriminability of the target clustering, reconstruct the target domain dataset with pseudo-labels and apply a prototype regularization term to the network using the target data with pseudo-labels to promote the optimization of clustering and make the features align at the category level.

[0076]

[0077] Here is the one-hot encoded target domain cluster label, and:

[0078]

[0079] \(v\) i is the \(i\)-th target domain sample feature vector, \(\tau\) is a temperature parameter that controls the density of this distribution, and it is set to 0.1 according to experience.

[0080] 3-12. The overall objective optimization function is:

[0081]

[0082] where \(\lambda_1\), \(\lambda_2\) are hyperparameters of the model.

[0083] 3-13. Before optimizing the neural network, perform an initial clustering to obtain the target domain dataset with pseudo-labels To avoid the accumulation of inaccurate labels, update the clustering alternately while optimizing the model. For every five optimizations of the neural network, re-perform the clustering and reconstruct the pseudo-label data until the neural network iterates \(T\) times.

[0084] Output: The target domain sample label corresponds to the category domain label closest to the cluster prototype center, which can be formally defined as:

[0085]

Claims

1. A method for cross-subject EEG emotion recognition based on clustering-based adversarial partial domain adaptation, characterized in that The steps are as follows: Step 1: Data preprocessing; Use the publicly available SEED dataset as the training dataset; the electroencephalogram (EEG) signal data needs to be preprocessed before inputting into the model; differential entropy features are extracted from 5 frequency bands of the SEED dataset per second: δ: 1 - 3 Hz, θ: 4 - 7 Hz, α: 8 - 13 Hz, β: 14 - 30 Hz, γ: 31 - 50 Hz; the feature dimension is 310; Step 2: Data definition Given N EEG data with individual differences, all subjects are regarded as the source domain and their cognitive state labels are known. This source domain is denoted as X s , and the source domain labels are denoted as Y s , the number of classes is C, and the new unlabeled subjects are regarded as the target domain X t , and the number of classes K is unknown; Step 3: Construct and train the EEG emotion recognition model; The described EEG emotion recognition model includes a shared feature extractor G, a classifier F, and a discriminator D; The specific method of Step 3 is as follows: Input: Source domain data with labels {X s , Y s}, where the source domain is represented as X s , and the source domain labels are represented as Y s , the number of classes C, target domain data X without labels t , the number of classes K is unknown, and the maximum number of iterations T; 3 - 1. Use the shared feature extractor G to extract the potential common features of the source domain and target domain sample data, and map the extracted common features to a common feature space; 3-2. Using the source domain sample data X s constrain and optimize the classifier F with the unique class label, and the objective function is the supervised classification loss L ce ; 3-3. Using the domain adversarial method, the feature extractor G can learn invariant features from both the source domain and the target domain to confuse the discriminator D into believing that the features come from the same domain, aligning the marginal distributions of the two domains in the feature space. The adversarial objective function is L d ; 3-4. Source domain data X s and target domain data X t Through the feature extractor G, domain-invariant feature vectors V s and V t can be extracted. The specific-class source domain data with the true label c is represented as The feature vector extracted from is Finally, through the specific-class feature vector the cluster center of the samples of class c in the source domain is calculated The cluster labels of the source domain samples are assigned based on the true labels, and finally the source domain cluster centers of all classes and the cluster labels {1, …, C} are obtained; 3-5. Use the Kmeans clustering method to cluster the target domain sample features V t to obtain the corresponding target domain sample cluster centers and cluster labels {1,…,K}; 3-6. Since the cluster labels obtained by Kmeans clustering do not correspond to the true labels of the source domain, consistency matching is used to associate the source domain cluster centers of the common categories with consistent semantics with the target domain cluster centers Given a pair of cluster features of the source domain and the target domain as and where the cluster from one domain searches for the nearest cluster center μ to this cluster in the other domain, and then determines whether both are the nearest cluster centers to each other. If both are the nearest cluster centers to each other, then such a pair of matching clusters is considered a common cluster. The cluster labels of this pair of matching clusters are {c, k}, and then the pseudo-label of the k-th cluster target domain sample is assigned the matching source domain label c; 3-7. Use cosine similarity to calculate the distances from samples in one domain to the cluster centers of all classes in another domain; for the $i$-th sample in the source domain within a pair of clusters, calculate its cosine similarity with all the cluster centers in the target domain Cosine similarity Similarly, calculate the cosine similarity of the $i$-th sample in the target domain with all the cluster centers in the source domain Cosine similarity 3 - 8. Calculate the source domain clustering consensus score using the result obtained in 3 - 7 and the target domain clustering consensus score 3 - 9. Then, for the source domain clustering consensus score obtained from 3 - 8 and the target domain clustering consensus score take the average to obtain the cross - domain clustering consensus score S of this pair of clusters (c,k) , and finally calculate the average value S of the consensus scores of all cluster pairs total ; 3-10. To determine the number of clusters K for the target domain clustering, different K values are set for the Kmeans algorithm, and steps 3-4 to 3-9 are repeated for multiple clusterings. According to the cross-domain clustering consensus score S total the optimal number of clusters is determined. Finally, clustering is performed with the optimal number of clusters to obtain the optimal cluster centers for the target domain; 3.

11. To improve the discriminability of target clustering, reconstruct the target domain dataset with pseudo-labels And apply a prototype regularization term to the neural network using the target data with pseudo-labels Promote the optimization of clustering, making the features aligned at the category level; 3 - 12. The overall objective optimization function is: where λ1 and λ2 are hyperparameters of the model; 3-13. Perform an initial clustering before optimizing the neural network to obtain a target domain dataset with pseudo-labels To avoid the accumulation of inaccurate labels, update the clustering alternately while optimizing the model; for every five optimizations of the neural network, perform clustering again, reconstruct the pseudo-label data, until the iteration reaches T times; Output: The cluster label of the target domain sample corresponding to the closest distance to the cluster prototype center; The specific implementation of Step 3 - 1 is as follows: Use the shared feature extractor G to extract the potential common features of the source domain and target domain sample data, and map the extracted common features to a common feature space. The common feature extractor uses a neural network with 3 hidden layers, and each hidden layer of the network has 512 nodes, and the ReLU activation function is used. The output of the network has the same 310 dimensions as the input data; The specific implementation of Step 3 - 2 is as follows: Using the source domain sample data X s Optimize the classifier F with the unique class label as the constraint. The number of network layers of the classifier F is set to 3 layers. Each hidden layer of the network has 64 nodes and the ReLU activation function is used. The input of the network is 310-dimensional and the output is 3-dimensional. The objective function is the supervised classification loss loss: The adversarial objective function is L d as follows: L d = L adv_g (X s , X t , G) + L adv_d (X s , X t , D) The calculation of the source domain cluster center is as follows: G(X s ) = V s Among them represents the number of source domain samples of the c-th category; The specific implementation of 3 - 7 is as follows: Use cosine similarity to calculate the distances from samples in one domain to the cluster centers of all classes in another domain; for the $i$-th sample in the source domain of a pair of clusters, calculate its cosine similarity with all the cluster centers in the target domain as follows The calculation formula is as follows: Similarly, the cosine similarity between the i-th sample in the target domain and all source domain cluster centers can be calculated is as follows The formula is as follows: The source domain cross - domain clustering consensus score can be expressed as the proportion of samples that reach consensus: Among them is an index for judging whether the $i$-th source domain feature vector holds the corresponding cluster index $k$. Similarly, the target domain can be obtained Prototype regularization term As follows: Among them, is the one-hot encoded target domain cluster label, and: v i is the feature vector of the i-th target domain sample, and τ is a temperature parameter that controls the distribution density, which is set to 0.1 according to experience.

Images