An unsupervised domain adaptation method based on template learning pointing to the target domain

Abstract

The present invention discloses an unsupervised domain adaptation method based on template learning pointing to a target domain, comprising: obtaining labeled source domain data and unlabeled target domain data, obtaining source domain features and target domain features respectively through a feature extractor in a domain adaptation classification model, and constructing a category template shared by the source and target domains; calculating a cross-entropy loss based on the source domain features and the category template, calculating a minimum class confusion loss based on the Riemann distance based on the target domain features and the category template, and using the sum of the cross-entropy loss and the minimum class confusion loss as a total loss function; training a domain adaptation classification model using the source domain data and the total loss function, and applying the trained domain adaptation classification model to the target domain data to achieve unsupervised domain adaptation. The present invention reduces the degree of class confusion on the target domain data and improves the model's prediction accuracy for the target domain data by learning the minimum class confusion loss based on the Riemann distance of the category template.

Description

Technical Field

[0001] The present invention relates to the fields of transfer learning and template learning, and in particular to an unsupervised domain adaptation method based on template learning pointing to a target domain. Background Art

[0002] Deep learning-based methods have been widely used in various data processing scenarios. However, since training deep neural networks generally requires a large amount of data, this limits the application of deep learning in some scenarios where data acquisition is difficult. To solve the problem of model training with small sample sizes, there are currently two solutions:

[0003] Transfer learning: It is specifically used to deal with small sample problems in specific scenarios. In unsupervised transfer learning, data does not usually come from the same distribution, but from two or more similar but not identical distributions, such as data obtained through different implementation objects. Based on the source of the data, the data can be divided into source domain and target domain. The data in the source domain contains corresponding labels, while the target domain may have only a small amount of data and no labels at all. Combining data from the source and target domains to train a model so that the model can achieve optimal performance on unlabeled target domain data is usually called an unsupervised domain adaptation problem.

[0004] Template learning typically involves learning a specific template for each data type. Methods such as Euclidean distance and cosine similarity are used to measure the similarity or distance between a data point and each template, thereby classifying the data point into the category corresponding to the most matching template. When using small sample sizes, applying template learning directly in the feature space can demonstrate better generalization performance than classifiers in conventional network models.

[0005] Reference 1 (Prototypical networks for few-shot learning) discloses a method for training a classifier in combination with template learning within a deep learning framework. The method includes the following steps: After encoding the raw data into latent space variables in the final feature layer, the feature extraction portion of the neural network uses Euclidean distance to determine the center point of features for different categories of data. This center point is used as the "feature template" for that category. After determining the template, the Euclidean distance between each sample and each category template is calculated, and the negative of the distance is converted to a probability using softmax. This converts the distance between the sample and each category template into the probability that the sample belongs to that category. Cross-entropy loss can then be used to update the network weights. Research has shown that template-based cross-entropy loss can be more efficient than conventional cross-entropy loss in small sample sizes.

[0006] Reference 2 (Minimum class confusion for versatile domain adaptation) discloses that the class confusion matrix is ​​a sensitive indicator for distinguishing target domain data from source domain data. Because the model performs labeled supervised classification training on the source domain data, it tends to make more certain classification judgments for the source domain data. However, because the target domain data lacks labels, the model cannot perform labeled supervised training during training, which makes the classification results of the target domain data more dispersed. Therefore, in order to allow the model to learn more features related to the target domain, a feasible method is to "penalize" the degree of confusion of the classification results of the target domain data, so that the model's prediction results for the target are more concentrated. Reference 2 proposes a method to measure the degree of class confusion after standardization (i.e., Minimum Class Confusion, MCC), and adds the MCC of the target domain as part of the loss to the total loss. It is found that this can achieve better classification performance on the target domain data.

[0007] Unsupervised domain adaptation faces two major challenges in various application areas: a lack of labels in the target domain and limited available data. For example, in medical data, a model that predicts unlabeled data from target subjects is required using limited source domain subject data and labels. Template learning can effectively learn class discrimination information with a small sample size, but this information is often concentrated around the template center corresponding to the source domain data. The difference in the distribution of source and target domain data can prevent this information from effectively covering the target domain data. Summary of the Invention

[0008] To solve the problem of how to make the category template closer to the target domain data rather than the source domain data, and thus aggregate the category information obtained by template learning into the target domain data, and achieve better domain adaptation transfer performance. The present invention provides an unsupervised domain adaptation method based on template learning pointing to the target domain, aiming to combine the advantages of template learning and transfer learning. By adjusting the category template to be closer to the data characteristics of the target domain, it better reflects the data distribution of the target domain rather than the source domain, thereby improving the effect of domain adaptation transfer and enabling the model to achieve good classification accuracy even with only a small amount of unlabeled target domain data.

[0009] To achieve the above-mentioned object of the invention, an embodiment provides an unsupervised domain adaptation method based on template learning pointing to a target domain, comprising:

[0010] Step 1: Obtain labeled source domain data and unlabeled target domain data, obtain source domain features and target domain features through the feature extractor in the domain adaptation classification model, and construct a learnable category template shared by the source and target domains;

[0011] Step 2: Calculate the cross entropy loss based on the distance between the source domain features and the category template, calculate the class correlation matrix based on the target domain features, and use the identity matrix or the ideal class confusion matrix determined based on the category template as the class confusion matrix. Calculate the minimum class confusion loss by the Riemann distance between the base class correlation matrix and the class confusion matrix. The sum of the cross entropy loss and the minimum class confusion loss is used as the total loss function.

[0012] Step 3: Use the total loss function to train the domain adaptation classification model, and apply the trained domain adaptation classification model to the target domain data to achieve unsupervised domain adaptation classification.

[0013] Preferably, in step 1, constructing a learnable category template shared by the source domain and the target domain includes:

[0014] With the number of data categories as the number of rows and the dimension of the feature extractor output data as the number of columns, a matrix is ​​established and randomly initialized as a learnable category template.

[0015] Preferably, in step 2, calculating the cross entropy loss based on the distance between the source domain feature and the category template includes:

[0016] The distance between the source domain feature and the category template is calculated, and the negative of the distance is converted into a predicted probability value through the softmax function. The cross entropy loss is calculated based on the predicted probability value and the label of the source domain data.

[0017] Preferably, in step 2, calculating the class correlation matrix according to the target domain features includes:

[0018] Based on the target domain features, the predicted probabilities of all samples in the target domain data belonging to each category are calculated to form a predicted probability matrix of the target domain data; the entropy of the predicted probability is calculated for each sample in the target domain data to evaluate the degree of uncertainty of the predicted probability; a weight is assigned to each sample based on the calculated entropy to construct a weight matrix; the class correlation matrix is ​​calculated based on the weight matrix and the predicted probability matrix.

[0019] Preferably, when the class confusion matrix is ​​a unit matrix, the minimum class confusion loss L based on the Riemann distance is rMCC Expressed as:

[0020] L rIMCC =||logS t || F

[0021] Among them, S t represents the class correlation matrix calculated based on the unlabeled target domain data, ||·|| F represents the F-norm.

[0022] Preferably, determining an ideal class confusion matrix based on the class template includes:

[0023] When all samples in the target domain data are assigned a category template based on the category template, the predicted probability of each category in the category template is calculated to form the predicted probability matrix of the category template. The ideal class confusion matrix is ​​constructed based on the predicted probability matrix of the category template itself and its transposed matrix.

[0024] Preferably, when the class confusion matrix is ​​an ideal class confusion matrix, the minimum class confusion loss L based on the Riemann distance is rMCC Expressed as:

[0025]

[0026] Where, represents S t represents the class correlation matrix calculated based on the unlabeled target domain data, S p represents the ideal class confusion matrix, ||·|| F represents the F-norm.

[0027] Preferably, in step 2, the total loss function is expressed as:

[0028] L=L cls +λL rMCC

[0029] Among them, Table L cls represents the cross entropy loss, L rMCC represents the minimum class confusion loss based on Riemann distance, and λ represents the weight parameter.

[0030] Preferably, the domain adaptation classification model is composed of a feature extractor and a classifier, and in the process of training the domain adaptation classification model, the category template is updated and the model parameters are optimized simultaneously;

[0031] In each iteration, the category template is updated through the total loss function, and the updated category template is used to calculate the total loss function in the next iteration until the set number of iterations is reached;

[0032] At the same time, the domain adaptation classification model is trained using labeled source domain data, and the model parameters are optimized according to the total loss function of each iteration.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention adopts a learnable category template, calculates the template-based cross entropy loss for the source domain data and the template-based minimum class confusion loss for the target domain data, and updates the category template to make it better reflect the data distribution of the target domain; and proposes a class confusion degree calculation method based on the positive definite matrix Riemann distance, so as to reduce the class confusion degree of the target domain data and improve the prediction accuracy of the domain adaptation classification model for the target domain data. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 Flowchart of an unsupervised domain adaptation method based on template learning in an embodiment of the present invention;

[0037] Figure 2 This is a schematic diagram of the domain adaptation classification model structure in an embodiment of the present invention;

[0038] Figure 3 : The feature distribution based on EDPNet in the embodiment of the present invention, where (a) and (b) represent the feature distribution obtained before the loss of the present invention is added, where (a) uses the category as the color, and (b) uses whether the data source is the source domain or the target domain as the color; (c) and (d) represent the feature distribution obtained after the loss of the present invention is added, where (c) uses the category as the color, and (d) uses whether the data source is the source domain or the target domain as the color. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0040] The inventive concept of the present invention is: an unsupervised domain adaptation method based on template learning pointing to the target domain. For labeled source domain data and unlabeled target domain data, the data is first mapped to a latent variable space through a feature extraction network. In the latent variable space, the target domain data and the source domain data have a set of common category template vectors. By calculating the Euclidean distance between the data point and each category template in the latent variable space, the degree of similarity between the data and each category can be measured, and then the negative of the distance is converted into the corresponding probability through the softmax layer. For the source domain data, after obtaining the category probability predicted by the model, the model can be optimized for supervised learning under labels using the cross entropy loss. For the target domain data, the class confusion degree of the prediction result is calculated, that is, the Riemannian distance between the class correlation matrix and the identity matrix or the template class correlation matrix is ​​measured. This can narrow the distance between the target domain data and the category template, thereby making the information around the category template more effective for the target domain data. We will refer to it as Riemannian distance-based minimum class confusion (rMCC) below.

[0041] like Figure 1 As shown, the embodiment provides an unsupervised domain adaptation method based on template learning pointing to the target domain, including:

[0042] S1: Obtain labeled source domain data and unlabeled target domain data, obtain source domain features and target domain features respectively through the feature extractor in the domain adaptation classification model, and construct a learnable category template shared by the source and target domains.

[0043] From the source domain D s N s data, and obtain a labeled source domain dataset From the target domain D t N t data, and obtain an unlabeled target domain dataset in, represents the vector space of dimension d1, Represents source domain data samples The corresponding label, Represents the target domain data sample. The source domain dataset and the target domain dataset have the same feature dimension d1.

[0044] The source domain data and target domain data are mapped to the latent variable space through the feature extractor f(·) in the domain adaptation classification model to obtain the source domain features and target domain features in, Represents a vector of dimension d2. All source domain features constitute the source domain feature set, and all target domain features constitute the target domain feature set. The source domain feature set and the target domain feature set have the same feature dimension d2.

[0045] In the latent variable space, each category of data has a learnable variable as a category template. With the number of data categories K as the number of rows and the dimension d2 of the feature extractor output data as the number of columns, a matrix is ​​established and randomly initialized to obtain a learnable category template shared by the source domain and the target domain.

[0046] S2: Calculate the cross entropy loss based on the distance between the source domain features and the category template, calculate the class correlation matrix based on the target domain features, and use the unit matrix or the ideal class confusion matrix determined based on the category template as the class confusion matrix. Calculate the minimum class confusion loss by the Riemann distance between the base class correlation matrix and the class confusion matrix, and take the sum of the cross entropy loss and the minimum class confusion loss as the total loss function.

[0047] In the embodiment, the cross entropy loss is calculated based on the distance between the source domain feature and the category template, specifically including:

[0048] For source domain features Calculate its difference with the class template The k-th category vector in The Euclidean distance is the corresponding feature distance

[0049]

[0050] Then, the Softmax layer in the classifier converts the negative values ​​of all feature distances into predicted probability values, which can be expressed as:

[0051]

[0052] in, Represents source domain samples The predicted probability value of the k-th category, y represents the predicted value, and j represents the category index;

[0053] For labeled source domain data, the corresponding labels can be used to calculate the cross entropy loss:

[0054] L cls =-∑ky k log(p(y=k|x)) (3)

[0055] The purpose of this cross-entropy loss function is to provide model training with category information corresponding to the true labels, allowing the model to learn the features that distinguish different categories during training. By adopting a learnable category template, the cross-entropy loss function not only guides the data to move closer to its corresponding category during the update process, thereby reducing the Euclidean distance between the data and the corresponding category, but also increases the distance between different categories, thereby distributing data of different categories away from each other and enhancing the distinguishability between categories.

[0056] For target domain data, due to the lack of supervised training with the above cross entropy loss during training, the model may become more confused when predicting categories. Calculate its difference with the class template The k-th category vector in The Euclidean distance is the corresponding feature distance

[0057]

[0058] Then pass all the The negative number is converted into a predicted probability value And form a matrix representing the predicted probability express:

[0059]

[0060] in, Represents the target domain sample The predicted probability value of the k-th category, y represents the predicted value, j represents the category index, Represented in the predicted probability matrix The i-th target domain sample in The predicted probability value belonging to the jth category is different from the previous formula (2) in that the distance is divided by a constant T. This is because the model obtained by formula (2) usually gives an overly confident result for category prediction. Dividing it by a constant T greater than 1 can better reflect the model's probability prediction of the sample.

[0061] Target domain data Calculate predicted probability values Entropy That is, the degree of confidence in the predicted probability is weighted, expressed as:

[0062]

[0063] Entropy reflects the uncertainty of the sample prediction probability value, which can be converted into a weight with a value near 1. Reweighting the data in the same batch according to the confidence level of different prediction results helps the model pay more attention to samples with high confidence levels, thereby providing more accurate estimates. The corresponding weight of sample i is:

[0064]

[0065] Use weights to construct a diagonal matrix W∈R B×B , then the normalized class correlation matrix can be calculated

[0066]

[0067] Among them, |C K |Indicates the total number of categories.

[0068] Assume that N t In the data, the frequency of each category is approximately equal, and the model's prediction of each sample is close to a one-hot vector, that is, the probability that the sample belongs to a certain category is close to 1, and the probability of belonging to other categories is close to 0. It can be inferred that S t It should be close to the identity matrix I. However, since the target domain data has not been supervised by a label, the prediction results are usually more divergent than the source domain data, resulting in the actual class correlation matrix S t Deviate from the ideal unit matrix I. Therefore, in order to enable the model to obtain more concentrated prediction results in the target domain data, a feasible method is to measure the matrix S t The difference between the class correlation matrix S and the identity matrix I is used as a new loss to push the model's prediction results for the target domain data to a unique hot vector. Different from the existing technology, this paper adopts a new, simpler and more effective method to measure the class correlation matrix S t The degree of difference from the identity matrix I.

[0069] Note that the class correlation matrix S t is always a symmetric positive definite matrix, so the difference between two positive definite matrices can be measured by the Riemann distance on the positive definite manifold. A With S B The Riemann distance formula is:

[0070]

[0071] Therefore, based on the class correlation matrix S t and the minimum class confusion loss L of the Riemann distance between the identity matrix I rIMCC It can be expressed as:

[0072] L rIMCC =||logS t || F (10)

[0073] The actual effect of this loss in the template learning framework can be divided into two parts: 1. When the class template vector is assumed to be fixed, in order to reduce this loss, it is necessary to improve the centrality of the prediction results of the target domain data. Considering that the prediction result is directly related to the distance between the sample and the class template vector, this loss will push the target domain data as close to the class template vector as possible. Assuming that the probability of each class is consistent, this loss will also make the number of target domain data corresponding to each template sample as consistent as possible. 2. When the target domain data is assumed to be fixed, this loss will maximize the distance between class template vectors of different categories without significantly widening the distance between the class template and the target domain data.

[0074] Ideally, the loss converges to 0 when each category template vector is pulled to infinity and the target domain data is evenly distributed around different template vectors.

[0075] In the embodiments, another type of confusion matrix is ​​proposed. Considering that the aforementioned loss requires different template vectors to be infinitely far apart to converge, and that the distribution of the target domain data is used to determine the direction in which the template vector is pushed outward, when the distribution of the target domain data differs significantly from that of the source domain data, one possible scenario is that the distribution of the target domain data is not well coupled with the category corresponding to the template vector. In other words, pushing the template vector outward according to the distribution of the target domain data may cause the model classification performance to collapse. A more conservative approach is to use only the classification loss of the labeled source domain data to provide the thrust to separate different templates. For the target domain data, it is desirable to only close the distance between the target domain data and the category template vector.

[0076] Now assume that N t The ideal distribution of target domain data is that each data is evenly assigned to a certain category template vector. In this case, we can obtain the normalized class confusion matrix which is actually completely determined by each category template vector. First, calculate the predicted probability distribution of each category template vector in formula (5) The predicted probability matrix of the available category template itself

[0077]

[0078] in, represents the category template of the i-th category, It represents the Euclidean distance between the category template of the i-th category and the category template of the j-th category.

[0079] Since the distance between the category template and itself is 0, when i=j,

[0080] Through the prediction probability matrix Ideally, we can get the normalized class confusion matrix when the target domain data is evenly distributed to a certain category template:

[0081]

[0082] Based on this, based on the class correlation matrix S t and class confusion matrix S P The minimum class confusion loss L between the Riemann distance rIMCC It can be expressed as:

[0083]

[0084] L expressed by formula (13) rPMCC L expressed by formula (10) rIMCC The difference is that the loss converges to 0 when each target domain data is evenly assigned to a category template. In other words, the loss only shortens the distance between each target domain data and the category template, and does not increase the distance between category templates based on the distribution of the target domain data. This avoids widening the template distance in the wrong direction due to uncertainty in the target domain data labels.

[0085] The above two forms of loss L rPMCC Both can bring the target domain data and the category template closer together. In template learning, the common distribution near the category template usually means more confident classification judgment. Therefore, by combining the above two losses with template learning, we can train a domain adaptation classification model with classification information concentrated on the target domain data, and achieve better unsupervised domain adaptation classification performance.

[0086] Specifically, the sum of the above cross entropy loss and the minimum class confusion loss is used as the total loss function, expressed as:

[0087] L=L cls +λL rMCC

[0088] S3: Train the domain adaptation classification model using source domain data and the total loss function, and apply the trained domain adaptation classification model to the target domain data to achieve unsupervised domain adaptation.

[0089] like Figure 2 As shown in , the domain adaptation classification model consists of a feature extractor and a classifier. During the training of the domain adaptation classification model, the category template is updated and the model parameters are optimized at the same time, as shown in Figure 1As shown in the figure, each round of iteration updates the category template through the total loss function L, and the updated category template is used to calculate the total loss function L in the next round of iteration until the set number of iterations is reached; at the same time, the domain adaptation classification model is trained using labeled source domain data, and the model parameters are optimized according to the total loss function of each round of iteration.

[0090] The unsupervised domain adaptation method based on template learning pointing to the target domain provided by the present invention can be applied to the unsupervised transfer learning problem in the case of small samples. Compared with the conventional unsupervised transfer learning method, the method of the present invention effectively combines the strategy of template learning, which can make the target domain data distributed around the category template, thereby achieving smaller intra-class variance and larger inter-class variance on the target domain data, effectively improving the generalization performance, and thus achieving better classification performance on the target domain data even when there is less training data. The actual effect of the present invention is tested in the following specific application scenario of motor imagery EEG cross-subject decoding.

[0091] In a non-invasive brain-computer interface, the EEG data of the subject is decoded by a model, which can realize the recognition of brain activities such as the subject's behavioral intention and psychological state. For example, in the motor imagery paradigm, the subject's EEG is usually collected while the subject is asked to perform different types of motor imagery according to instructions. Common motor imagery includes imagining the movements of different parts of the body such as the left and right hands, feet, and tongue. Through this paradigm, different types of motor imagery EEG fragments can be collected for each subject. In an unsupervised cross-subject test method, the training data usually includes labeled motor imagery EEG data of multiple subjects and unlabeled motor imagery EEG data of the target subject. It is necessary to combine the two types of data to train the model to achieve a higher accuracy on the target subject. Here are the specific experimental performances of the present invention on two classic motor imagery EEG public datasets: BCI competition IV 2a and BCIcompetition IV 2b.

[0092] BCI competition IV 2a and BCI competition IV 2b are four-category and two-category motor imagery datasets, respectively. BCI competition IV 2a collected EEG data from nine subjects while they performed four types of motor imagery: left hand, right hand, foot, and tongue. BCI competition IV 2b collected EEG data from nine subjects while they performed two types of motor imagery: left hand and right hand. According to the official BCI competition IV dataset partitioning method, each subject's data in both datasets can be divided into two subsets: a training set and a test set. In the 2a dataset, each subject's training set consists of four categories, with 72 EEG samples per category, for a total of 288 samples. The test set also consists of four categories, with 72 EEG samples per category, for a total of 288 samples. In the 2b dataset, each subject's training set consists of two categories, with 200 samples per category, for a total of 400 samples. The test set consists of two categories, with 160 samples per category, for a total of 320 samples. For each sample in the two datasets, we selected the EEG data of the corresponding 4-second motor imagery task.

[0093] In the cross-subject unsupervised domain adaptation task of the present invention, one subject is selected as the target domain, and the remaining eight subjects are selected as the source domain. The model is trained using labeled source domain data and unlabeled target domain data, and then tested on the target domain subjects to obtain the final model performance. This embodiment does not perform any preprocessing on the raw data; instead, the raw data is directly input into the neural network model for training.

[0094] The method of the present invention is mainly aimed at improving the loss function during the training process, and has no special requirements for the network structure. Three publicly available network structures for processing EEG motor imagery data are used here: EEGNet, EEGConformer, and EDPNet. EEGNet is one of the earliest networks to use deep learning methods for motor imagery EEG processing. It uses multiple convolutional layers to extract the time-frequency information of EEG signals, and shows an accuracy rate that is better than traditional machine learning methods. EEGConformer is a new generation of EEG motor imagery processing network based on the transformer architecture. It combines convolutional layers as feature preprocessing layers, and then inputs the obtained time-frequency information into the transformer for further deep integration. EDPNet is one of the latest EEG motor imagery processing networks, which shows very excellent generalization performance and is therefore one of the best networks currently used for cross-subject EEG processing.

[0095] By inputting data into the network model, the characteristics of the latent variable space corresponding to the data can be obtained. After mapping the data to the latent variable space, the distance between each sample and the different category templates can be calculated based on the category template vectors corresponding to different categories. On this basis, the cross entropy classification loss on the labeled source domain data can be obtained, and the class confusion loss based on the Riemann distance on the target domain corresponding to the present invention is calculated to obtain the total loss. During the test, the weight parameter λ of the total loss is 1. According to the calculated total loss, conventional methods such as gradient descent can be used to update the model parameters. The above process can be implemented using the conventional deep learning implementation framework pytorch or other frameworks.

[0096] exist Figure 3 In the paper, a more intuitive result is given to show the actual effect of the present invention: the results of the experiment without L are plotted. rPMCC and adopt L rPMCC The feature distribution is obtained under the condition of L, and different colors are added using the category of the data or whether the data is the source domain, the target domain or the template as the label. rPMCC The results obtained, such as Figure 3 As shown in (a) and (b), we can see that the data category distribution is very mixed, especially in the target domain data, there is no good distinction. rPMCC After that, the data distribution shows that the middle category is more mixed, but the categories of the areas closer to the category template are more consistent, such as Figure 3 As shown in (c), combining the labels of the target domain and the source domain, we can further find that compared with the source domain data, the target domain data is basically concentrated in the area with relatively consistent category information, that is, around the category template, such as Figure 3 As shown in (d), the goal expected by the present invention is achieved: the target domain data is aggregated near the template, thereby converging more category information learned in the network on the target domain, achieving better category distinction in the target domain.

[0097] Table 1 shows the results of these methods when 9 subjects in the BCI 2a dataset are used as target subjects and the rest of the subjects are used as source domains, plus the L rPMCC Changes in classification accuracy before and after the loss (random level is 25%):

[0098] Table 1 Different network structures of BCI 2a dataset plus L rPMCC Changes in accuracy before and after

[0099]

[0100]

[0101] Table 1 shows the EEGNet, EEGConformer, EDPNet and L rPMCC The change in accuracy before and after the addition of the loss. The relative improvement represents the percentage increase in accuracy after adding the loss compared to without adding the loss. The p-value in the last row represents the paired signed-rank test between the two groups of data before and after the addition of the loss.

[0102] We also used EDPNet as the network to further compare the improvements brought by different loss functions on the BCI 2a dataset:

[0103] Table 2 Accuracy changes of BCI 2a dataset EDPNet before and after adding different losses

[0104]

[0105] Among them, EDPNet can be used as the baseline method. MCC and L MCC +L dis As two comparison methods. MCC This is the method used in Reference 2. By calculating the class correlation matrix on the target domain data as the empirical class confusion matrix, the value of its non-diagonal elements is used as the loss, so that it can reduce the degree of confusion of the target domain class, making the model more suitable for the target domain data. MCC +L dis On this basis, we further introduce the strategy of adversarial learning, and use the classifier nuclear norm that can reflect class confusion as the distribution difference indicator of adversarial learning, so as to dynamically balance the class confusion degree of the source domain and the target domain during the optimization process, and finally achieve a better effect of reducing the class confusion degree of the target domain. The embodiment gives the two as comparative methods, and the last two columns are the performance of the corresponding methods of the present invention. As can be seen in the above table, L MCC , L MCC +L dis Participants 6, L rIMCC The accuracy of test subject 5 was much lower than that of other methods. This is actually related to the potential risks of class confusion loss given in the technical solution of the present invention. rPMCC This avoids widening the template distance in the wrong direction due to uncertainty in the target domain data labels, and thus may be more robust in unsupervised domain adaptation.

[0106] The cross-subject classification performance of the method of the present invention using EDPNet as the network in the BCI 2a and 2b datasets is further given compared with the methods given in other studies:

[0107] Table 3 Cross-subject performance of different methods in BCI 2a and 2b datasets

[0108]

[0109] The above experimental results reflect that the present invention has reached an advanced level in cross-subject motor imagery EEG classification. This proves the effectiveness of the present invention. The cross-subject motor imagery classification accuracy obtained by the present invention has reached a high level, thus promoting the application of brain-computer interfaces based on motor imagery and greatly shortening the learning time for new users to use motor imagery brain-computer interfaces. In addition, the present invention can also be applied to other unsupervised transfer learning application scenarios and may have further application value that can be explored.

[0110] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An unsupervised domain adaptation method based on template learning pointing to the target domain, characterized in that include: Step 1: Obtain labeled source domain motor imagery EEG data and unlabeled target domain motor imagery EEG data. Use the feature extractor in the domain adaptation classification model to obtain source domain features and target domain features, respectively, and construct a learnable category template shared by the source and target domains. Step 2: Calculate the cross entropy loss based on the distance between the source domain features and the class template, calculate the class correlation matrix based on the target domain features, and use the identity matrix or the ideal class confusion matrix determined based on the class template as the class confusion matrix. Calculate the minimum class confusion loss based on the Riemann distance between the class correlation matrix and the class confusion matrix, and take the sum of the cross entropy loss and the minimum class confusion loss as the total loss function; Determining an ideal class confusion matrix based on the class template includes: calculating, based on the class template, a predicted probability of each class in the class template when all samples in the target domain data are assigned to each class template, thereby forming a predicted probability matrix of the class template; and constructing an ideal class confusion matrix based on the predicted probability matrix of the class template itself and its transposed matrix; Step 3: Use the total loss function to train the domain adaptation classification model, and apply the trained domain adaptation classification model to the target domain motor imagery EEG data to achieve unsupervised domain adaptation classification.

2. The unsupervised domain adaptation method based on template learning pointing to the target domain according to claim 1 is characterized in that In step 1, a learnable category template shared by the source and target domains is constructed, including: With the number of data categories as the number of rows and the dimension of the feature extractor output data as the number of columns, a matrix is ​​established and randomly initialized as a learnable category template.

3. The unsupervised domain adaptation method based on template learning pointing to the target domain according to claim 1, characterized in that In step 2, the cross entropy loss is calculated based on the distance between the source domain features and the category template, including: The distance between the source domain feature and the category template is calculated, and the negative of the distance is converted into a predicted probability value through the softmax function. The cross entropy loss is calculated based on the predicted probability value and the label of the source domain data.

4. The unsupervised domain adaptation method based on template learning pointing to the target domain according to claim 1, characterized in that In step 2, the class correlation matrix is ​​calculated based on the target domain features, including: Based on the target domain features, the predicted probabilities of all samples in the target domain data belonging to each category are calculated to form a predicted probability matrix of the target domain data; the entropy of the predicted probability is calculated for each sample in the target domain data to evaluate the degree of uncertainty of the predicted probability; a weight is assigned to each sample based on the calculated entropy to construct a weight matrix; the class correlation matrix is ​​calculated based on the weight matrix and the predicted probability matrix.

5. The unsupervised domain adaptation method based on template learning pointing to the target domain according to claim 1, characterized in that When the class confusion matrix is ​​the identity matrix, the minimum class confusion loss based on Riemann distance L rMCC Expressed as: ； in, represents the class correlation matrix calculated based on the unlabeled target domain data, represents the F-norm.

6. The unsupervised domain adaptation method based on template learning pointing to the target domain according to claim 1, characterized in that When the class confusion matrix is ​​an ideal class confusion matrix, the minimum class confusion loss based on Riemann distance is L rMCC Expressed as: ； in, represents the class correlation matrix calculated based on the unlabeled target domain data, represents the ideal class confusion matrix, represents the F-norm.

7. The unsupervised domain adaptation method based on template learning pointing to the target domain according to claim 1, characterized in that In step 2, the total loss function is expressed as: ； in, represents the cross entropy loss, represents the minimum class confusion loss based on Riemann distance, Represents the weight parameter.

8. The unsupervised domain adaptation method based on template learning pointing to the target domain according to claim 1, characterized in that The domain adaptation classification model is composed of a feature extractor and a classifier. During the training of the domain adaptation classification model, the category template is updated and the model parameters are optimized simultaneously. In each iteration, the category template is updated through the total loss function, and the updated category template is used to calculate the total loss function in the next iteration until the set number of iterations is reached; At the same time, the domain adaptation classification model is trained using labeled source domain data, and the model parameters are optimized according to the total loss function of each iteration.

Images