Unsupervised domain adaptation method based on template learning pointing to target domain

Abstract

The invention discloses an unsupervised domain adaptation method based on template learning pointing to a target domain, and the method comprises the steps: obtaining labeled source domain data and unlabeled target domain data, obtaining a source domain feature and a target domain feature through a feature extractor in a domain adaptation classification model, and constructing a category template shared by the source domain and the target domain; calculating cross entropy loss according to the source domain features and the category template, calculating minimum class confusion loss based on Riemannian distance according to the target domain features and the category template, and taking the sum of the cross entropy loss and the minimum class confusion loss as a total loss function; and training the domain adaptation classification model through the source domain data and the total loss function, and applying the trained domain adaptation classification model to the target domain data to realize unsupervised domain adaptation. According to the method, the class confusion degree of the target domain data is reduced through the class confusion minimum loss of the learnable class template based on the Riemannian distance, and the prediction precision of the model on the target domain data is improved.

Description

Technical Field

[0001] The present invention relates to the fields of transfer learning and template learning, and in particular to an unsupervised domain adaptation method based on template learning pointing to a target domain. Background Art

[0002] Methods based on deep learning have been widely used in various data processing scenarios. However, since training deep neural networks usually requires a large amount of data, this limits the application of deep learning in some scenarios where data acquisition is difficult. In order to solve the problem of model training under small sample sizes, there are currently two solutions:

[0003] Transfer learning: It is specifically used to deal with small sample problems in specific scenarios. In unsupervised transfer learning, data usually does not come from the same distribution, but from two or more similar but not identical distributions, such as data obtained through different implementation objects. According to the source of the data, the data can be divided into source domain and target domain. The data in the source domain contains corresponding labels, while the target domain may have only a small amount of data and no labels at all. Combining the data from the source domain and the target domain to train a model so that the model can achieve optimal performance on the unlabeled target domain data is usually called an unsupervised domain adaptation problem.

[0004] Template learning: usually involves learning a specific template for each type of data. By using methods such as Euclidean distance and cosine similarity to measure the similarity or distance between data points and each template, the data points are classified into the category corresponding to the best matching template. In the case of small sample sizes, compared with the classifier in the conventional network model, applying template learning directly in the feature space for classification can show better generalization performance.

[0005] Reference 1 (Prototypical networks for few-shot learning) discloses a method for training a classifier in combination with template learning under a deep learning framework, including: after the feature extraction part of the neural network encodes the original data into latent space variables of the final feature layer, the center point of the data features of different categories is determined by Euclidean distance, and the center point is used as the "feature template" of the category. After the template is determined, the Euclidean distance between each sample and each category template is calculated, and the negative of the distance is converted into probability through softmax, that is, the distance between the sample and each category template is converted into the probability that the sample belongs to the category, and then the network weight can be updated with cross entropy loss. Studies have shown that cross entropy loss based on template learning can be more efficient than conventional cross entropy loss in the case of small samples.

[0006] Reference 2 (Minimum class confusion for versatiledomain adaptation) discloses that the class confusion matrix is ​​a sensitive indicator for distinguishing target domain data from source domain data. Because the model performs labeled supervised classification training on the source domain data, it tends to give more certain classification judgments for the source domain data. However, because the target domain data lacks labels, the model cannot perform labeled supervised training during the training process, which will cause the classification results of the target domain data to be more dispersed. Therefore, in order to allow the model to learn more features related to the target domain, a feasible method is to "penalize" the degree of confusion of the classification results of the target domain data, so that the model's prediction results for the target are more concentrated. Reference 2 proposes a method for measuring the degree of class confusion after standardization (i.e., Minimum Class Confusion, MCC), and adds the MCC of the target domain as part of the loss to the total loss. It is found that it can achieve better classification performance on the target domain data.

[0007] Unsupervised domain adaptation faces two major challenges in many application fields: one is the lack of labels in the target domain, and the other is the limited amount of available data. For example, in medical data, a model that predicts the unlabeled data of the target domain subjects is needed through limited source domain subject data and labels. Template learning can learn efficient category discrimination information in small sample conditions, but usually such information is concentrated around the center of the template corresponding to the source domain data, and the distribution difference between the source domain data and the target domain data will result in the information being unable to effectively cover the target domain data. Summary of the invention

[0008] In order to solve the problem of how to make the category template closer to the target domain data rather than the source domain data, the category information obtained by template learning is aggregated into the target domain data to achieve better domain adaptation migration performance. The present invention provides an unsupervised domain adaptation method based on template learning pointing to the target domain, which aims to combine the advantages of template learning and transfer learning, and better reflect the data distribution of the target domain rather than the source domain by adjusting the category template to be close to the data characteristics of the target domain, thereby improving the effect of domain adaptation migration and enabling the model to achieve good classification accuracy even with only a small amount of unlabeled target domain data.

[0009] To achieve the above-mentioned object of the invention, an unsupervised domain adaptation method based on template learning pointing to a target domain is provided in an embodiment, comprising:

[0010] Step 1: Obtain labeled source domain data and unlabeled target domain data, obtain source domain features and target domain features respectively through the feature extractor in the domain adaptation classification model, and construct a learnable category template shared by the source domain and the target domain;

[0011] Step 2: Calculate the cross entropy loss based on the distance between the source domain features and the category template, calculate the class correlation matrix based on the target domain features, and use the unit matrix or the ideal class confusion matrix determined based on the category template as the class confusion matrix. Calculate the minimum class confusion loss by the Riemann distance between the base class correlation matrix and the class confusion matrix, and take the sum of the cross entropy loss and the minimum class confusion loss as the total loss function;

[0012] Step 3: Use the total loss function to train the domain adaptation classification model, and apply the trained domain adaptation classification model to the target domain data to achieve unsupervised domain adaptation classification.

[0013] Preferably, in step 1, constructing a learnable category template common to the source domain and the target domain includes:

[0014] With the number of data categories as the number of rows and the dimension of the feature extractor output data as the number of columns, a matrix is ​​established and randomly initialized as a learnable category template.

[0015] Preferably, in step 2, calculating the cross entropy loss according to the distance between the source domain feature and the category template includes:

[0016] The distance between the source domain feature and the category template is calculated, and the negative of the distance is converted into a predicted probability value through the softmax function, and the cross entropy loss is calculated based on the predicted probability value and the label of the source domain data.

[0017] Preferably, in step 2, calculating the class correlation matrix according to the target domain features includes:

[0018] Based on the target domain features, the predicted probabilities of all samples in the target domain data belonging to each category are calculated to form a predicted probability matrix of the target domain data; the entropy of the predicted probability is calculated for each sample in the target domain data to evaluate the degree of uncertainty in the predicted probability; weights are assigned to each sample based on the calculated entropy to construct a weight matrix; the class correlation matrix is ​​calculated based on the weight matrix and the predicted probability matrix.

[0019] Preferably, when the class confusion matrix is ​​a unit matrix, the minimum class confusion loss L based on the Riemann distance is rMCC It is expressed as:

[0020] L rIMCC =||logS t || F

[0021] Among them, S t represents the class correlation matrix calculated based on the unlabeled target domain data, ||·|| F represents the F-norm.

[0022] Preferably, determining an ideal class confusion matrix based on the class template includes:

[0023] When all samples in the target domain data are assigned a category template based on the category template, the predicted probability of each category in the category template is calculated to form a predicted probability matrix of the category template, and the ideal class confusion matrix is ​​constructed based on the predicted probability matrix of the category template itself and its transposed matrix.

[0024] Preferably, when the class confusion matrix is ​​an ideal class confusion matrix, the minimum class confusion loss L based on the Riemann distance is rMCC It is expressed as:

[0025]

[0026] Among them, it represents S t represents the class correlation matrix calculated based on the unlabeled target domain data, S p represents the ideal class confusion matrix, ||·|| F represents the F-norm.

[0027] Preferably, in step 2, the total loss function is expressed as:

[0028] L=L cls +λL rMCC

[0029] Among them, Table L cls represents the cross entropy loss, L rMCC represents the minimum class confusion loss based on Riemann distance, and λ represents the weight parameter.

[0030] Preferably, the domain adaptation classification model is composed of a feature extractor and a classifier, and in the process of training the domain adaptation classification model, the category template is updated and the model parameters are optimized simultaneously;

[0031] In each iteration, the category template is updated through the total loss function, and the updated category template is used to calculate the total loss function in the next iteration until the set number of iterations is reached;

[0032] At the same time, the domain adaptation classification model is trained using labeled source domain data, and the model parameters are optimized according to the total loss function of each iteration.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention adopts a learnable category template, calculates the template-based cross entropy loss for the source domain data and the template-based minimum class confusion loss for the target domain data, and updates the category template to make it better reflect the data distribution of the target domain; and proposes a class confusion degree calculation method based on the positive definite matrix Riemann distance, so as to reduce the class confusion degree on the target domain data and improve the prediction accuracy of the domain adaptation classification model for the target domain data. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 Flow chart of an unsupervised domain adaptation method based on template learning in an embodiment of the present invention;

[0037] Figure 2 This is a schematic diagram of the domain adaptation classification model structure in an embodiment of the present invention;

[0038] Figure 3 : is the feature distribution based on EDPNet in the embodiment of the present invention, where (a) and (b) represent the feature distribution obtained before the loss of the present invention is added, where (a) uses the category as the color, and (b) uses whether the data source is the source domain or the target domain as the color; (c) and (d) represent the feature distribution obtained after the loss of the present invention is added, where (c) uses the category as the color, and (d) uses whether the data source is the source domain or the target domain as the color. DETAILED DESCRIPTION

[0039] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0040] The inventive concept of the present invention is: an unsupervised domain adaptation method based on template learning pointing to the target domain. For labeled source domain data and unlabeled target domain data, the data is first mapped to the latent variable space through a feature extraction network. In the latent variable space, the target domain data and the source domain data have a set of common category template vectors. By calculating the Euclidean distance between the data point and each category template in the latent variable space, the degree of similarity between the data and each category can be measured, and then the negative of the distance is converted into the corresponding probability through the softmax layer. For the source domain data, after obtaining the category probability predicted by the model, the model can be optimized for supervised learning under labels through cross entropy loss. For the target domain data, the class confusion degree of the prediction result will be calculated, that is, the Riemann distance between the class correlation matrix and the unit matrix or the template class correlation matrix is ​​measured, which can shorten the distance between the target domain data and the category template, so that the information around the category template is more effective for the target domain data. Hereinafter, we refer to it as Riemannian Distance-based Minimum Class Confusion (rMCC).

[0041] like Figure 1 As shown, an unsupervised domain adaptation method based on template learning pointing to a target domain is provided in an embodiment, comprising:

[0042] S1: Obtain labeled source domain data and unlabeled target domain data, obtain source domain features and target domain features respectively through the feature extractor in the domain adaptation classification model, and construct a learnable category template shared by the source domain and the target domain.

[0043] From the source domain D s N s data, and obtain a labeled source domain dataset From the target domain D t N t data, and obtain an unlabeled target domain dataset in, represents a vector space of dimension d1, Represents source domain data samples The corresponding label, Represents the target domain data sample. The source domain dataset and the target domain dataset have the same feature dimension d1.

[0044] The source domain data and target domain data are mapped to the latent variable space through the feature extractor f(·) in the domain adaptation classification model, and the source domain features are obtained respectively. and target domain features in, Represents a vector of dimension d2. All source domain features constitute the source domain feature set, and all target domain features constitute the target domain feature set. The source domain feature set and the target domain feature set have the same feature dimension d2.

[0045] In the latent variable space, each category of data has a learnable variable as a category template. With the number of data categories K as the number of rows and the dimension d2 of the feature extractor output data as the number of columns, a matrix is ​​established and randomly initialized to obtain a learnable category template shared by the source domain and the target domain.

[0046] S2: The cross entropy loss is calculated based on the distance between the source domain features and the category template, and the class correlation matrix is ​​calculated based on the target domain features. The unit matrix or the ideal class confusion matrix determined based on the category template is used as the class confusion matrix. The Riemann distance between the base class correlation matrix and the class confusion matrix is ​​used to calculate the minimum class confusion loss. The sum of the cross entropy loss and the minimum class confusion loss is used as the total loss function.

[0047] In the embodiment, the cross entropy loss is calculated according to the distance between the source domain feature and the category template, specifically including:

[0048] For source domain features Calculate its class template The k-th category vector in The Euclidean distance is used as the corresponding feature distance

[0049]

[0050] Then, the Softmax layer in the classifier converts the negative values ​​of all feature distances into predicted probability values, expressed as:

[0051]

[0052] in, Represents source domain samples The predicted probability value predicted as the kth category, y represents the predicted value, and j represents the category index;

[0053] For labeled source domain data, the corresponding labels can be used to calculate the cross entropy loss:

[0054] L cls =-∑ky k log(p(y=k|x)) (3)

[0055] The role of the cross entropy loss function is to provide the category information corresponding to the real label for model training, so that the model can learn the features that distinguish different categories during the training process. By adopting a learnable category template, the cross entropy loss function will not only guide the data to move closer to their corresponding categories during the update process, thereby reducing the Euclidean distance between the data and the corresponding category, but also increase the distance between different categories, so that the distribution of data of different categories is far away from each other, enhancing the distinguishability between categories.

[0056] For target domain data, due to the lack of supervised training with the above cross entropy loss during training, the model may become more confused when predicting categories. Calculate its class template The k-th category vector in The Euclidean distance is used as the corresponding feature distance

[0057]

[0058] Then pass all the Softmax layer in the classifier The negative number is converted into a predicted probability value And form a matrix representing the predicted probability express:

[0059]

[0060] in, Represents the target domain sample The predicted probability value predicted as the kth category, y represents the predicted value, j represents the category index, Represented in the predicted probability matrix The i-th target domain sample in The predicted probability value belonging to the jth category is different from the previous formula (2) in that the distance is divided by a constant T. This is because the model obtained by formula (2) usually gives an overly confident result for category prediction. Dividing it by a constant T greater than 1 can better reflect the model's probability prediction of the sample.

[0061] Target domain data Calculate predicted probability values Entropy That is, the degree of confidence in the predicted probability is weighted, expressed as:

[0062]

[0063] Entropy reflects the uncertainty of the predicted probability value of a sample, which can be converted into a weight with a value near 1. Re-weighting the data in the same batch according to the confidence level of different prediction results is conducive to making the model pay more attention to samples with high confidence levels, thereby providing more accurate estimates. The corresponding weight of sample i is:

[0064]

[0065] Construct a diagonal matrix W∈R using weights B×B , then the normalized class correlation matrix can be calculated

[0066]

[0067] Among them, |C K | indicates the total number of categories.

[0068] Assume that N t In the data, the frequency of each category is approximately equal, and the model's prediction for each sample is close to a one-hot vector, that is, the probability that the sample belongs to a certain category is close to 1, and the probability that it belongs to other categories is close to 0. It can be inferred that S t It should be close to the identity matrix I. However, since the target domain data has not been supervised with labels, the prediction results are usually more divergent than the source domain data, resulting in the actual class correlation matrix S t Deviates from the ideal unit matrix I. Therefore, in order to enable the model to obtain more concentrated prediction results in the target domain data, a feasible method is to measure the matrix S t The difference between the class correlation matrix S and the identity matrix I is used as a new loss to push the prediction result of the model on the target domain data to the unique hot vector. Different from the prior art, the present invention adopts a new, simpler and more effective method to measure the class correlation matrix S t The degree of difference between it and the identity matrix I.

[0069] Note that the class correlation matrix S t is always a symmetric positive definite matrix, so the difference between two positive definite matrices can be measured by the Riemann distance on the positive definite manifold. In the positive definite manifold, two symmetric positive definite matrices S A With S B The Riemann distance formula is:

[0070]

[0071] Therefore, based on the class correlation matrix S t and the minimum class confusion loss L of the Riemann distance between the identity matrix I rIMCC It can be expressed as:

[0072] L rIMCC =||logS t || F (10)

[0073] The actual effect of this loss in the template learning framework can be divided into two parts: 1. When the class template vector is assumed to be fixed, in order to reduce this loss, it is necessary to improve the concentration of the prediction results of the target domain data. Considering that the prediction result is directly related to the distance from the sample to the class template vector, this loss will push the target domain data as close to the class template vector as possible, and assuming that the probability of each class is consistent, this loss will also make the number of target domain data corresponding to each template sample as consistent as possible. 2. When the target domain data is assumed to be fixed, this loss will widen the distance between the class template vectors of different categories as much as possible without significantly widening the distance between the class template and the target domain data.

[0074] Ideally, the loss converges to 0 when each category template vector is pulled to infinity and the target domain data is evenly distributed near different template vectors.

[0075] In the embodiment, another type of confusion matrix is ​​also proposed. Considering that the above loss requires different template vectors to be infinitely far apart to converge, and the distribution of the target domain data is used to determine the direction in which the template vector is pushed outward, but when the distribution of the target domain data is quite different from that of the source domain data, one possible situation is that the distribution of the target domain data is not well coupled with the category corresponding to the template vector, that is, pushing the template vector out according to the distribution of the target domain data may cause the model classification performance to collapse. A more conservative approach is to provide the thrust for separating different templates only through the classification loss of the labeled source domain data. For the target domain data, it is hoped that only the distance between the target domain data and the category template vector is shortened.

[0076] Now assume that N t The ideal distribution of target domain data is that each data is evenly assigned to a certain category template vector. In this case, we can obtain the normalized class confusion matrix which is actually completely determined by each category template vector. First, calculate the predicted probability distribution of each category template vector in formula (5): The prediction probability matrix of the available category template itself

[0077]

[0078] in, represents the category template of the i-th category, Represents the Euclidean distance between the category template of the i-th category and the category template of the j-th category.

[0079] Since the distance between the category template and itself is 0, when i=j,

[0080] Through the prediction probability matrix Ideally, we can get the normalized class confusion matrix when the target domain data is evenly distributed to a certain category template:

[0081]

[0082] Based on this, based on the class correlation matrix S t And the class confusion matrix S P The minimum class confusion loss L between the Riemann distance rIMCC It can be expressed as:

[0083]

[0084] Formula (13) represents L rPMCC Compared with L expressed by formula (10) rIMCC The difference is that when each target domain data is evenly assigned to a certain category template, the loss converges to 0. That is, the loss will only shorten the distance between each target domain data and the category template, and will not increase the distance between category templates according to the distribution of the target domain data, thus avoiding the possibility of increasing the template distance in the wrong direction due to the uncertainty of the target domain data label.

[0085] The above two forms of loss L rPMCC Both can bring the target domain data and the category template closer. In template learning, the usual distribution near the category template usually means a more confident classification judgment. Therefore, by combining the above two losses with template learning, we can train a domain adaptation classification model with classification information concentrated on the target domain data, and achieve better unsupervised domain adaptation classification performance.

[0086] Specifically, the sum of the above cross entropy loss and the minimum class confusion loss is used as the total loss function, expressed as:

[0087] L=L cls +λL rMCC

[0088] S3: Train the domain adaptation classification model through the source domain data and the total loss function, and apply the trained domain adaptation classification model to the target domain data to achieve unsupervised domain adaptation.

[0089] like Figure 2 As shown in , the domain adaptation classification model consists of a feature extractor and a classifier. In the process of training the domain adaptation classification model, the category template is updated and the model parameters are optimized at the same time, as shown in Figure 1As shown in the figure, in each round of iteration, the category template is updated through the total loss function L, and the updated category template is used to calculate the total loss function L in the next round of iteration until the set number of iterations is reached; at the same time, the domain adaptation classification model is trained using labeled source domain data, and the model parameters are optimized according to the total loss function of each round of iteration.

[0090] The unsupervised domain adaptation method based on template learning pointing to the target domain provided by the present invention can be applied to the unsupervised transfer learning problem in the case of small samples. Compared with the conventional unsupervised transfer learning method, the method of the present invention effectively combines the strategy of template learning, which can make the target domain data distributed around the category template, thereby achieving smaller intra-class variance and larger inter-class variance on the target domain data, effectively improving the generalization performance, so as to achieve better classification performance on the target domain data even when there is less training data. The actual effect of the present invention is tested in the following specific application scenario of motor imagery EEG cross-subject decoding.

[0091] In a non-invasive brain-computer interface, the EEG data of the subject is decoded by a model, so that the subject's behavioral intention, psychological state and other brain activities can be identified. For example, in the motor imagery paradigm, the subject's EEG is usually collected while the subject is asked to perform different types of motor imagery according to instructions. Common motor imagery includes imagining the movements of different parts of the body such as the left and right hands, feet, and tongue. Through this paradigm, different types of motor imagery EEG fragments can be collected for each subject. In an unsupervised cross-subject test method, the training data usually includes labeled motor imagery EEG data of multiple subjects and unlabeled motor imagery EEG data of the target subject. It is necessary to combine the two types of data to train the model to achieve a higher accuracy on the target subject. Here are the specific experimental performances of the present invention on two classic motor imagery EEG public data sets, BCI competition IV 2a and BCIcompetition IV 2b.

[0092] BCI competition IV 2a and BCI competition IV 2b are 4-category motor imagery and 2-category motor imagery datasets, respectively. BCI competition IV 2a collected EEG data of 9 subjects when they were performing 4 types of motor imagery: left hand, right hand, foot and tongue, and BCI competition IV 2b collected EEG data of 9 subjects when they were performing 2 types of motor imagery: left hand and right hand. According to the dataset division method officially given by BCI competition IV, the data of each subject in the two datasets can be divided into two data subsets, a training set and a test set. In the 2a dataset, the training set data of each subject includes 4 categories, 72 EEG samples in each category, and a total of 288 samples. The test set also has 4 categories, 72 EEG samples in each category, and a total of 288 samples. In the 2b dataset, the training data set of each subject includes 2 categories, 200 in each category, and a total of 400 samples. The test data has 2 categories, 160 in each category, and a total of 320 samples. For each sample in the two datasets, we selected the EEG data corresponding to each 4-second motor imagery task.

[0093] In the cross-subject unsupervised domain adaptation task of the present invention, one of the subjects is selected as the target domain, and the remaining 8 subjects are selected as the source domain. The model is trained using labeled source domain data and unlabeled target domain data, and then tested on the target domain subjects to obtain the final model performance. The embodiment does not perform any preprocessing on the original data, but directly inputs the original data into the neural network model for training.

[0094] The method of the present invention is mainly aimed at improving the loss function during the training process, and has no special requirements for the network structure. Three publicly available network structures for processing EEG motor imagery data are used here: EEGNet, EEGConformer, and EDPNet. EEGNet is one of the earliest networks to use deep learning methods for motor imagery EEG processing. It uses multiple convolutional layers to extract the time-frequency information of EEG signals, and shows an accuracy rate that is better than traditional machine learning methods. EEGConformer is a new generation of EEG motor imagery processing network based on the transformer architecture. It combines convolutional layers as feature preprocessing layers, and then inputs the obtained time-frequency information into the transformer for further deep integration. EDPNet is one of the latest EEG motor imagery processing networks, showing very excellent generalization performance, and is therefore one of the best networks currently used for cross-subject EEG processing.

[0095] By inputting data into the network model, the characteristics of the latent variable space corresponding to the data can be obtained. After mapping the data to the latent variable space, the distance between each sample and the different category templates can be calculated according to the category template vectors corresponding to different categories. On this basis, the cross entropy classification loss on the labeled source domain data can be obtained, and the class confusion loss based on the Riemann distance on the target domain corresponding to the present invention is calculated to obtain the total loss. During the test, the weight parameter λ of the total loss is 1. According to the calculated total loss, conventional methods such as gradient descent can be used to update the model parameters. The above process can be implemented through the conventional deep learning implementation framework pytorch or other frameworks.

[0096] exist Figure 3 In the figure, a more intuitive result is given to show the practical effect of the present invention: the results of the experiment without using L are respectively plotted. rPMCC and using L rPMCC The feature distribution is obtained under the condition of L, and different colors are added using the data category or whether the data is the source domain, the target domain or the template as the label. rPMCC The results obtained are as follows Figure 3 As shown in (a) and (b), we can see that the data category distribution is very mixed, especially in the target domain data, there is no good distinction. rPMCC After that, the data distribution shows that the middle category is more mixed, but the categories of the areas closer to the category template are more consistent, such as Figure 3 As shown in (c), by combining the labels of the target domain and the source domain, we can further find that compared with the source domain data, the target domain data is basically concentrated in the area where the category information is relatively consistent, that is, around the category template, such as Figure 3 As shown in (d), the desired goal of the present invention is achieved: the target domain data is gathered near the template, so that the category information learned in the network is more concentrated on the target domain, and better category distinction is achieved on the target domain.

[0097] Table 1 shows the results of these methods when 9 subjects in the BCI 2a dataset are used as target subjects and the rest of the subjects are used as source domains, plus the L rPMCC Changes in classification accuracy before and after the loss (random level is 25%):

[0098] Table 1 Different network structures of BCI 2a dataset plus L rPMCC Changes in accuracy before and after

[0099]

[0100]

[0101] Table 1 shows the EEGNet, EEGConformer, EDPNet and L rPMCC The change in accuracy before and after, the relative improvement represents the relative percentage increase in accuracy after adding the loss compared to when not adding the loss. The last row of p-values ​​represents the paired signed rank test of the two groups of data before and after adding the loss.

[0102] We also used EDPNet as the network to further compare the improvements brought by different loss functions on the BCI 2a dataset:

[0103] Table 2 Accuracy changes of EDPNet before and after adding different losses on BCI 2a dataset

[0104]

[0105] Among them, EDPNet can be used as the baseline method. MCC and L MCC +L dis As two comparison methods. MCC This is the method used in Reference 2. By calculating the class correlation matrix on the target domain data as the empirical class confusion matrix, the value of its non-diagonal elements is used as the loss, so that it can reduce the degree of confusion of the target domain class, making the model more suitable for the target domain data. MCC +L dis On this basis, the strategy of adversarial learning is further introduced, and the classifier nuclear norm that can reflect class confusion is used as the distribution difference indicator of adversarial learning, so as to dynamically balance the class confusion degree of the source domain and the target domain during the optimization process, and finally achieve a better effect of reducing the class confusion degree of the target domain. The embodiment gives the two as comparative methods, and the last two columns are the performance of the corresponding methods of the present invention. It can be seen from the above table that L MCC , L MCC +L dis Participants 6 and L rIMCC The accuracy of test subject 5 in the above method is much lower than that of other methods. This is actually related to the potential risks of class confusion loss given in the technical solution of the present invention. rPMCC This avoids widening the template distance in the wrong direction due to uncertainty in the target domain data labels, and may be more robust in unsupervised domain adaptation.

[0106] The cross-subject classification performance of the method of the present invention in BCI 2a and 2b datasets using EDPNet as the network is further given compared with the methods given in other studies:

[0107] Table 3 Cross-subject performance of different methods in BCI 2a and 2b datasets

[0108]

[0109] The above experimental results reflect that the present invention has reached an advanced level in the cross-subject motor imagery EEG classification. The effectiveness of the present invention has been proved. The accuracy of the cross-subject motor imagery classification obtained by the present invention has reached a relatively high level, thus promoting the application of brain-computer interfaces based on motor imagery and greatly shortening the learning time of new users using motor imagery brain-computer interfaces. In addition, the present invention can also be applied to other unsupervised transfer learning application scenarios, and may have further application value that can be explored.

[0110] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An unsupervised domain adaptation method based on template learning pointing to the target domain, characterized in that: include: Step 1: Obtain labeled source domain data and unlabeled target domain data, obtain source domain features and target domain features respectively through the feature extractor in the domain adaptation classification model, and construct a learnable category template shared by the source domain and the target domain; Step 2: Calculate the cross entropy loss based on the distance between the source domain features and the category template, calculate the class correlation matrix based on the target domain features, and use the unit matrix or the ideal class confusion matrix determined based on the category template as the class confusion matrix. Calculate the minimum class confusion loss by the Riemann distance between the base class correlation matrix and the class confusion matrix, and take the sum of the cross entropy loss and the minimum class confusion loss as the total loss function; Step 3: Use the total loss function to train the domain adaptation classification model, and apply the trained domain adaptation classification model to the target domain data to achieve unsupervised domain adaptation classification.

2. The unsupervised domain adaptation method based on template learning pointing to the target domain according to claim 1, characterized in that: In step 1, a learnable category template common to the source domain and the target domain is constructed, including: With the number of data categories as the number of rows and the dimension of the feature extractor output data as the number of columns, a matrix is ​​established and randomly initialized as a learnable category template.

3. The unsupervised domain adaptation method based on template learning pointing to the target domain according to claim 1, characterized in that: In step 2, the cross entropy loss is calculated based on the distance between the source domain feature and the category template, including: The distance between the source domain feature and the category template is calculated, and the negative of the distance is converted into a predicted probability value through the softmax function, and the cross entropy loss is calculated based on the predicted probability value and the label of the source domain data.

4. The unsupervised domain adaptation method based on template learning pointing to the target domain according to claim 1, characterized in that: In step 2, the class correlation matrix is ​​calculated according to the target domain features, including: Based on the target domain features, the predicted probabilities of all samples in the target domain data belonging to each category are calculated to form a predicted probability matrix of the target domain data; the entropy of the predicted probability is calculated for each sample in the target domain data to evaluate the degree of uncertainty in the predicted probability; weights are assigned to each sample based on the calculated entropy to construct a weight matrix; the class correlation matrix is ​​calculated based on the weight matrix and the predicted probability matrix.

5. The unsupervised domain adaptation method based on template learning pointing to the target domain according to claim 1, characterized in that: When the class confusion matrix is ​​the identity matrix, the minimum class confusion loss L based on the Riemann distance is rMCC It is expressed as: L rIMCC =||logS t || F Among them, S t represents the class correlation matrix calculated based on the unlabeled target domain data, ||·|| F represents the F-norm.

6. The unsupervised domain adaptation method based on template learning pointing to the target domain according to claim 1, characterized in that: Determine the ideal class confusion matrix based on the class template, including: When all samples in the target domain data are assigned a category template based on the category template, the predicted probability of each category in the category template is calculated to form a predicted probability matrix of the category template, and the ideal class confusion matrix is ​​constructed based on the predicted probability matrix of the category template itself and its transposed matrix.

7. The unsupervised domain adaptation method based on template learning pointing to the target domain according to claim 1 or 6, characterized in that: When the class confusion matrix is ​​an ideal class confusion matrix, the minimum class confusion loss L based on the Riemann distance is rMCC It is expressed as: Among them, it represents S t represents the class correlation matrix calculated based on the unlabeled target domain data, S p represents the ideal class confusion matrix, ||·|| F represents the F-norm.

8. The unsupervised domain adaptation method based on template learning pointing to the target domain according to claim 1, characterized in that: In step 2, the total loss function is expressed as: L=L cls +λL rMCC Among them, Table L cls represents the cross entropy loss, L rMCC represents the minimum class confusion loss based on Riemann distance, and λ represents the weight parameter.

9. The unsupervised domain adaptation method based on template learning pointing to the target domain according to claim 1, characterized in that: The domain adaptation classification model is composed of a feature extractor and a classifier. In the process of training the domain adaptation classification model, the category template is updated and the model parameters are optimized at the same time; In each iteration, the category template is updated through the total loss function, and the updated category template is used to calculate the total loss function in the next iteration until the set number of iterations is reached; At the same time, the domain adaptation classification model is trained using labeled source domain data, and the model parameters are optimized according to the total loss function of each iteration.

Images