Source Domain Selection Method for Multisource EEG Migration

Abstract

The present invention discloses a source domain selection method for multi-source EEG migration. First, the tangent space features and Grassmann manifold features are extracted to minimize the difference in the marginal probability distributions between the source domain and the target domain. After obtaining the manifold features, the structural risk minimization and the minimization of the conditional probability distribution difference between the source domain and the target domain are used as the objective function to train the classification model for each source domain. Each classifier predicts the target domain respectively, and the prediction results of different source domains are integrated by voting. After the first iteration, a classifier is trained for each source domain respectively, and finally a multi-source classifier is generated by voting, thus meeting the conditions of LSA. At this time, an LSA is performed to obtain the transferability estimation values of different source domains, and k source domains are selected. In subsequent iterations, it is only necessary to repeat the iteration of training the classifier for the remaining source domains, improving the operation efficiency.

Description

Technical Field

[0001] The present invention belongs to the research field of the movement control mechanism of the nervous system, EEG signal preprocessing, EEG feature extraction, manifold feature alignment and extraction, and multi-source migration framework design, thereby performing multi-source EEG migration learning. Background Art

[0002] The brain, as the center for controlling human thoughts, behaviors, emotions, and other activities, analyzes and processes information obtained from the external environment and communicates with the outside world through neuromuscular pathways. However, diseases such as spinal cord injury, amyotrophic lateral sclerosis, stroke, Parkinson's disease, and brain trauma often damage or weaken the function of the nerve center, leading to varying degrees of perception, sensation, speech, and movement disorders. On the one hand, breakthroughs in brain-computer interface (BCI) technology are expected to achieve functional compensation and functional reconstruction by directly establishing high-precision information interaction and control between the brain and external devices; on the other hand, active rehabilitation training based on BCI technology can enhance neural remodeling, promote the recovery of patients' limb motor function, and improve patients' quality of life and happiness index, which is of great significance to patients, families, and society.

[0003] A core issue in BCI systems is that traditional supervised classifier model training requires a large amount of labeled EEG data. This excessively long training time places a significant psychological and physical burden on patients, hindering the development and application of BCI systems. Therefore, designing and implementing EEG signal parsing models with strong adaptability, high recognition rates, and minimal user training time has become a common, key basic scientific issue that needs to be addressed in the practical application of BCI systems, and further research and methodological development are needed.

[0004] In response to the problems of training time and recognition performance in EEG signal decoding, this paper starts from the theory of transfer learning and conducts research on multi-source manifold feature transfer learning in brain-computer interface, enhances the generalization ability of the transfer model and the robustness of the classifier, realizes the transfer between healthy subjects or patients, and improves the performance of the brain-computer interface system.

[0005] In recent years, the research and application of transfer learning in BCI have emerged in an endless stream. Among them, the application of Riemannian geometry in BCI has become popular. As the covariance matrix is a symmetric positive definite (SPD) matrix, the covariance matrix of each electroencephalogram (EEG) trial can be regarded as a point on the SPD manifold. Zanini et al. proposed a Riemannian alignment (RA) framework for calibrating EEG covariance matrices from different source domains. However, these Riemannian space-based methods are computationally intensive and incompatible with machine learning methods in Euclidean space. Traditional transfer learning methods aim to solve the single-to-single (STS) transfer problem. In experiments, we can find that even with the simplest transfer learning algorithm, a good source domain can help achieve very high classification accuracy, so the quality of the source domain is very important. However, in reality, we are likely to have multiple source domains, just like BCI devices often have a lot of previously labeled data. Therefore, when there are multiple source domains, good source domains are more likely to be included. In multi-source transfer, due to the expansion of data, good source domains can reduce the impact of negative transfer caused by bad source domains, and multi-source transfer learning can usually obtain more stable and higher classification accuracy than STS transfer learning. In recent years, multi-source unsupervised domain adaptation transfer learning has received increasing attention, such as (Yao, & Doretto, 2010; Lin, An, & Zhang, 2013; Li et al., 2019; Zhu, Zhung, & Wang, 2019; Zhang, & Wu, 2020). On the other hand, when there are many source domains, while good source domains are included, there will also be bad source domains. Therefore, a domain selection method is needed to determine the performance of source domains on the target domain, select good source domains for transfer, and discard bad source domains that may bring negative transfer. Existing research has proposed some domain selection methods, such as methods based on similarity metrics, including Rank Of Domains (ROD) (Gong et al., 2010), Domain Transferability Estimation (DTE) (Zhang & Wu, 2020), and methods based on performance testing, such as (Yao, & Doretto, 2010). Summary of the Invention

[0006] The purpose of the present invention is to provide a source domain selection method for multi-source EEG transfer, named Label Similarity Analysis (LSA).

[0007] To achieve the above object, the method of the present invention mainly includes the following steps:

[0008] Step (1), extraction of electroencephalogram shape features;

[0009] Specifically: By calculating the covariance matrix of each sample electroencephalogram signal, aligning different source domains and target domains on the SPD manifold, and extracting tangent space features; reconstructing the extracted tangent space features back to the Grassmann manifold to extract Grassmann manifold features, so as to minimize the marginal probability distributions of the source domain and the target domain;

[0010] Step (2), manifold feature migration;

[0011] According to the manifold features obtained in step (1) that minimize the marginal probability distributions of the source domain and the target domain, migrate the manifold features to minimize the conditional probability distributions of the source domain and the target domain;

[0012] Step (3) source domain selection;

[0013] After the first iteration, perform an LSA to obtain the transferability estimation values of different source domains; according to the transferability estimation values, remove k source domains, and in subsequent iterations, repeat the remaining source domains to train the classifier iteratively;

[0014] The specific LSA is as follows:

[0015] Obtain the single-source classifier and multi-source classifier of each source domain through the multi-source transfer framework, compare the prediction results of the single-source classifier trained for each source domain with the prediction results of the multi-source classifier, and calculate the transferability estimation values of different source domains.

[0016] Preferably, obtain the single-source classifier and multi-source classifier of each source domain through the multi-source transfer framework, compare the prediction results of the single-source classifier trained for each source domain with the prediction results of the multi-source classifier, and calculate the transferability estimation values of different source domains;

[0017] Specifically: The true label of the target domain is represented as The single-source classifier f trained for each domain s And the multi-source classifier f generated by the final voting m The predicted labels are respectively represented as And Use sim(·) to represent the similarity between the two labels, such as y real And y ps The similarity calculation method between them is

[0018]

[0019]

[0020] n t represents the number of samples in the target domain

[0021] Estimate the transferability of a source domain through the STS transfer classification accuracy

[0022] Accuracy = sim(y real , y ps ) / n t (26)

[0023] A higher STS transfer classification accuracy means higher transferability; based on the assumptions of the three distribution types of the samples, we get

[0024]

[0025] The so-called respectively represent the predicted labels of the single-source classifier for the first and third types of samples respectively represent the predicted labels of the multi-source classifier for the first and third types of samples respectively represent the true labels of the first and third types of samples

[0026] And

[0027]

[0028] It can be seen from equations (27) and (28) that Higher means that the source domain has high transferability; the target domain label y rea l is unknown. Is it possible to find a reliable analysis method to represent the transferability of the source domain? In multi-source transfer learning, the presence of multiple source domains significantly improves the classification performance and stability of single-source transfer, mainly because multi-source transfer learning has higher accuracy when predicting the second type of samples If a single-source classifier can accurately predict the label of the second type of samples as accurately as a multi-source classifier, that is Higher, then the probability that the source domain has high transferability is higher

[0029] Based on the above analysis and assumptions, use the predicted label of the multi-source classifier, label y pm to identify the transferability of different source domains

[0030]

[0031] Substitute equation (27) into equation (29). As long as sim(y ps , y pm ) is high, it means is high, that is, the transferability of a source domain is high

[0032] Therefore, the transferability between the j-th source domain and the target domain is calculated as:

[0033]

[0034] where f is a classification algorithm.

[0035] Preferably, f is linear discriminant analysis or support vector machine.

[0036] Preferably, the extraction of the electroencephalogram waveform features is specifically as follows:

[0037] Denote the covariance matrix of the EEG signals of an experiment as P, P = XX T , and P is a SPD matrix; use and to represent the covariance matrices of all samples in the source domain and the target domain , the one-dimensional distribution of the distances on the SPD manifold, M s and M t are the domain distribution means, and represent the variances of the covariance matrix distributions. Select invertible matrices A and B as linear transformations to align the domain distribution means; after the linear transformation, the samples of the source domain and the target domain are and According to the congruence invariance property of the Riemannian distance, the covariance matrices of all features only change in the reference position in space. Therefore, after the transformation, the and of the source domain and the target domain remain unchanged, and the transformed source domain distribution is The target domain distribution is

[0038] Use the KL divergence to measure the distribution difference between the source domain and the target domain. The objective function for minimizing the marginal probability distribution is:

[0039]

[0040] where KL(·) is the calculation of the KL divergence, using the probability density of the standard normal distribution

[0041]

[0042] x represents the covariance matrix of an experiment on the SPD manifold;

[0043] The KL divergence is calculated to obtain

[0044]

[0045] and

[0046]

[0047] Substituting equations (2) and (3) into equation (1), the objective function is simplified to

[0048]

[0049] When A T M s A = B T M t B, the optimal solution of the objective function can be obtained, such as

[0050]

[0051] and

[0052]

[0053] where E is the identity matrix. The alignment method in equation (6) aligns each domain to its own distribution mean, while equation (7) aligns all samples in the source domain to the target domain. This method adopts the alignment method of equation (6) because after such alignment, the multi-covariance matrices of the source domain and target domain samples are whitened;

[0054] After aligning the distribution means, the covariance matrices of all samples in the source domain and target domain are respectively and n s ,n t are the number of samples in the source domain and target domain respectively. Through equation (8)

[0055]

[0056] it is calculated that represents the covariance matrix of the i-th experiment in the source domain, represents the covariance matrix of the j-th experiment in the target domain;

[0057] Through the aligned covariance matrix, project it onto the tangent space to obtain the tangent space features, convert the original two-dimensional covariance matrix features into one-dimensional vector form, and the calculation method is shown in equation (9);

[0058]

[0059] where upper(·) is the operation of taking the upper triangular part of the c×c SPD matrix to form a vector The tangent space features of the source domain and target domain obtained are respectively and

[0060] Finally, the obtained one-dimensional tangent space features are reconstructed back to the Grassmann manifold space

[0061] z = g(x) = Φ(t) T x (10)

[0062] Calculate the feature mapping G through Equation (11)

[0063]

[0064] Finally, the Grassmann manifold features are obtained through Equation (12)

[0065]

[0066] The finally obtained Grassmann manifold features z minimize the variance originating from the target domain distribution as much as possible and the differences.

[0067] Preferably, the manifold feature transfer is specifically as follows:

[0068] In step one, the manifold features that minimize the marginal probability distribution are obtained. In step two, the manifold features need to be transferred to minimize the conditional probability distributions of the source domain and the target domain; the objective function of the required classifier f is shown in Equation (13), and the SRM classifier is used to minimize the conditional probability distributions of the source domain and the target domain

[0069]

[0070] where the first two terms are the SRM classifier, and the third term represents the source domain and the target domain the difference in the conditional probability distribution between them;

[0071] where the SRM classifier is expressed as

[0072]

[0073] where E is a diagonal matrix used to record labels, and when the sample classes are imbalanced, the samples belonging to the class with fewer samples can obtain greater weights;

[0074]

[0075] where n s,(c＝1) and n s,(c＝2) respectively represent the numbers of samples belonging to the first and second classes in the source domain;

[0076] The third term can be expressed as

[0077]

[0078] where represents the alignment of the conditional probability distribution of the c-th class of samples;

[0079] Using the representation theory, a span of the classifier f is

[0080]

[0081] where K is a kernel that maps the original feature vectors to the Hilbert space and is the corresponding coefficient vector;

[0082] Therefore, Equation (16) can be written as

[0083]

[0084] where represents the Frobenious norm, is the kernel matrix where K ij = K(z i , z j ), Y = [y1,..., y n are the source domain labels and the pseudo-labels of the target domain, n = n s + n t , and tr(·) is the trace of the matrix;

[0085] Equation (17) is written as

[0086]

[0087] where M c is the MMD matrix

[0088]

[0089] where and represent the samples belonging to the c-th class in the source domain and the target domain, respectively;

[0090] Substituting Equation (18) and Equation (19) into Equation (13), the objective function of the classifier is

[0091]

[0092] By taking the derivative with respect to to minimize the objective function, the optimal solution is

[0093] α = ((E + λM c )K + σI) -1EY T (22)

[0094] Substituting Equation (22) into Equation (17), the prediction information of the classifier can be obtained.

[0095] Compared with the traditional myoelectric coupling analysis method, the present invention has the following advantages:

[0096] 1. This method is simple and intuitive, and is more accurate and effective than the existing unsupervised source domain selection methods, and can cooperate with the SRM classifier to mine the label information of the source domain.

[0097] 2. This method can be well combined with the proposed multi-source migration framework, and the source domain selection can be completed during the intermediate process of multi-source migration, consuming extremely little computing time. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1 are the distributions of three types of samples in the target domain and their true labels;

[0099] Figure 2 is the flowchart of the present invention patent;

[0100] Figure 3 is the experimental result diagram of the present invention patent on five data sets. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0101] The following will describe in detail the embodiments of the present invention in conjunction with the drawings: These embodiments are implemented on the premise of the technical solution of the present invention, and detailed implementation schemes and specific operation processes are given.

[0102] The present invention proposes a source domain selection method for multi-source EEG migration. On the framework of multi-source migration, the labels generated in the middle are analyzed to perform source domain selection. As Figure 2 shown, the implementation of the present invention mainly includes three steps: (1) multi-source migration framework; (2) label similarity analysis; (3) source domain selection.

[0103] The following will describe each step in detail.

[0104] Step 1: Multi-source migration framework

[0105] Denote the covariance matrix of the EEG signals in one experiment as P, P = XX T , and P is a SPD matrix. Use and to represent the covariance matrices of all samples in the source domain and the target domain, the one-dimensional distribution of the distance on the SPD manifold, M s and M t are the domain distribution means (the Riemannian mean of all samples in one domain), and It represents the variance of the covariance matrix distribution. The larger the variance value, the more dispersed the distribution of samples in this domain on the SPD manifold. In order to reduce the marginal probability distributions of the source domain and the target domain by changing the reference position on the Riemannian manifold, invertible matrices A and B are selected as linear transformations to align the distribution means of the domains. After the linear transformation, the samples of the source domain and the target domain are and According to the congruence invariance property of the Riemannian distance, the covariance matrices of all features only change in the reference position of the space. Therefore, after the transformation, the and remain unchanged, and the transformed distributions are and

[0106] Using the KL divergence to measure the distribution difference between the source domain and the target domain, the objective function for minimizing the marginal probability distribution is:

[0107]

[0108] where KL(·) is the calculation of the KL divergence. Using the probability density of the standard normal distribution

[0109]

[0110] The KL divergence can be calculated as

[0111]

[0112] and

[0113]

[0114] Substituting equations (2) and (3) into equation (1), the objective function is simplified to

[0115]

[0116] When A T M s A = B T M t B, the objective function can obtain the optimal solution, such as:

[0117]

[0118] and

[0119]

[0120] where E is the identity matrix, such as Figure 2As shown in Figure 2, different solutions correspond to different alignment methods. The alignment method in formula (6) aligns each domain to its own distribution mean, while formula (7) aligns all samples in the source domain to the target domain. This method adopts the alignment method of formula (6) because after alignment, the multi-covariance matrix of the samples from the source and target domains can be whitened.

[0121] After aligning the distribution means, the covariance matrices of all samples in the source and target domains are and n s , n t are the number of samples in the source domain and the target domain respectively, which can be obtained by formula (8)

[0122]

[0123] Calculated.

[0124] The aligned covariance matrix can be projected onto the tangent space to obtain the tangent space features, and the original two-dimensional covariance matrix features can be converted into a one-dimensional vector form. The calculation method is shown in formula (9).

[0125]

[0126] Where upper(·) is the vector composed of the upper triangular part of the c×c SPD matrix The tangent space features of the source domain and the target domain are obtained as follows: and

[0127] Finally, the obtained one-dimensional tangent space features are reconstructed back to the Grassmann manifold space

[0128] z=g(x)=Φ(t) T x (10)

[0129] The feature map G is calculated by formula (11)

[0130]

[0131] Finally, the Grassmann manifold characteristics are obtained through formula (12):

[0132]

[0133] The final Grassmann manifold feature z can eliminate the distribution variance from the target domain as much as possible. and difference.

[0134] Step 2: Manifold feature transfer

[0135] In Step 1, the manifold features with minimized marginal probability distribution are obtained. In Step 2, the manifold features need to be migrated to minimize the conditional probability distributions of the source domain and the target domain. The objective function of the classifier f to be obtained is shown in Equation (13). This method uses the SRM classifier to minimize the conditional probability distributions of the source domain and the target domain as well as the label similarity.

[0136]

[0137] Among them, the first two terms are the SRM classifier, and the third term represents the source domain and the target domain The difference in the conditional probability distribution between them.

[0138] Among them, the SRM classifier can be expressed as

[0139]

[0140] Among them, E is a diagonal matrix used to record labels. And when the sample classes are imbalanced, the samples belonging to the class with fewer samples can obtain greater weights.

[0141]

[0142] Among them, n s,(c＝1) and n s,(c＝2) respectively represent the numbers of samples belonging to the first class and the second class in the source domain.

[0143] The third term can be expressed as

[0144]

[0145] Among them represents the alignment of the conditional probability distribution of the samples of the c-th class.

[0146] Using the representational theory ( Herbrich, & Smola, 2001), a span of the classifier f is

[0147]

[0148] Among them, K is a kernel that maps the original feature vector to the Hilbert space and is the corresponding coefficient vector.

[0149] Therefore, Equation (16) can be written as

[0150]

[0151] Among them represents the Frobenious norm, is the kernel matrix where K ij = K(z i , z j ), Y = [y1, …, y n are the source domain labels and the pseudo labels of the target domain, n = n s + n t , and tr(·) is to calculate the trace of a matrix.

[0152] Equation (17) can be written as

[0153]

[0154] where M c is the MMD (maximum mean discrepancy) matrix

[0155]

[0156] where and respectively represent the samples belonging to the c-th class in the source domain and the target domain.

[0157] Substituting Equation (18) and Equation (19) into Equation (13), the objective function of the classifier is

[0158]

[0159] By taking the derivative with respect to to minimize the objective function, the optimal solution can be obtained as

[0160] α = ((E + λM c )K + σI) -1 EY T (22)

[0161] Substituting Equation (22) into Equation (17), the prediction information of the classifier can be obtained.

[0162] After obtaining the training method of the transfer classifier, as Figure 1 shown, for z source domains After aligning the distribution means, there are still differences in the conditional probability distributions of all source domains. When their MMD matrices are calculated together, some conditional probability distribution information may be lost. Therefore, the result of aligning the joint probability distribution may not be better than that of only aligning the marginal distributions. Compared with traditional multi-source transfer, the multi-source transfer framework of this method does not simply put together multiple aligned source domains to train a classifier. Instead, through the classifier design method in step two, a classifier is trained separately for each aligned source domain. Finally, the quantization prediction values made by each source domain for the target domain are accumulated to obtain the final classification result. This approach maximally preserves the conditional probability distribution information of each source domain, and the classifier adopted by this method is a quantization classifier, which can better describe the probability that each sample belongs to a certain class in the case of multi-source voting. Specifically:

[0163] Obtain the covariance matrices of the z source domains after alignment through Equation (8) and the covariance matrix of the target domain after alignment Calculate the tangent space features through Equation (9) Finally, perform Grassmann manifold feature learning through Equations (11) and (12) to obtain the manifold features that minimize the marginal probability distribution Then, use the obtained manifold features to train a classifier f for each source domain through Equation (22) i , i = 1, 2, … z,, and perform quantization voting

[0164]

[0165] Obtain the final multi-source classifier f.

[0166] Step two: Label similarity analysis

[0167] After obtaining the multi-source transfer framework, as Figure 1 shown, during the first iteration, label similarity analysis needs to be performed. The ideas and methods of label similarity analysis are introduced in detail below.

[0168] When the learning tasks of two domains are not relevant / similar or the data distributions of the source domain and the target domain are different, different subjects may show strong individual differences in BCI training. This may lead to negative transfer (NT) (Zhang, Deng, Zhang, & Wu, 2020). Generally, if the knowledge contained in the source domain can help the target domain obtain higher classification accuracy, then the source domain has higher transferability. On the contrary, some source domains contain data / features that are very different from the target domain and are considered to have low transferability. To avoid this negative operation, this study proposes a test-based domain selection method - Label Similarity Analysis (LSA) to avoid the effect of bad source domains in multi-source transfer. By comparing the labels generated by the multi-source classifier and the STS classifier, the quality of a source domain is determined. Its essence is to use the reliability of multi-source transfer to help identify the transferability of each source domain.

[0169] To explore the similarities and differences between STS and multi-source transfer, this study hypothesizes that there are three types of distributions of the EEG signal features in the target domain, as Figure 2 shown.

[0170] Type 1: Light gray represents some samples that are easily classified correctly.

[0171] Type 2: Dark gray represents samples with low discriminability. When different source domains are transferred to help classify the target domain, the difference in transfer effect mainly lies in whether these samples can be classified correctly.

[0172] Type 3: Black represents samples with large differences in the distribution of the same type of samples and are difficult to classify correctly.

[0173] The true labels of the target domain are represented as The single-source classifier f trained for each domain s and the multi-source classifier f generated by the final voting m The predicted labels are respectively represented as and where k = 1, 2, 3 represents Figure 2 three types of the three distributions, and sim(·) is used to represent the similarity between two labels, such as y real and y ps The similarity calculation method between them is

[0174]

[0175]

[0176] The transferability of a source domain can be estimated by the classification accuracy of STS transfer

[0177] Accuracy=sim(y real ,y ps ) / n t (26)

[0178] Higher accuracy means higher transferability. According to the previous assumptions about the three distribution types of the samples, we can obtain

[0179]

[0180] and

[0181]

[0182] It can be seen from equations (27) and (28) that higher means higher transferability of the source domain. However, the target domain label y real is unknown. Is it possible to find a reliable analysis method to represent the transferability of the source domain? In multi-source transfer learning, the existence of multiple source domains significantly improves the classification performance and stability of single-source transfer. This is mainly because multi-source transfer learning has higher accuracy when predicting the second type of samples If a source domain can accurately predict the label of the second type of samples like a multi-source classifier, that is higher, then the probability that this source domain has higher transferability is higher.

[0183] Based on the above analysis and assumptions, this study uses the predicted label of the multi-source classifier, label y pm to identify the transferability of different source domains.

[0184]

[0185] Substituting equation (27) into equation (29), it can be found that as long as sim(y ps ,y pm ) is high, it means is high, that is, the transferability of a source domain is high.

[0186] Therefore, the transferability between the j-th source domain and the target domain is calculated as:

[0187]

[0188] where f can be a classification algorithm, such as linear discriminant analysis (LDA) or support vector machine (SVM). However, to estimate the portability of the source domain, training an additional classifier to perform STS transfer and prediction is time-consuming. At this point, LSA is easier to integrate into the multi-source transfer framework proposed in Step 1 because the framework exactly trains classifiers for each source domain.

[0189] Step 3: Source domain selection

[0190] As Figure 1 shown, after the first iteration, a classifier is trained for each source domain respectively, and finally a multi-source classifier is generated by voting, which meets the conditions of LSA. At this time, perform an LSA to obtain the portability estimation values of different source domains, and select k source domains, and this value can be set by yourself. In subsequent iterations, only need to repeat the iteration of training classifiers for the remaining source domains, which can improve the operation efficiency. Experiment:

[0191] 1. Dataset:

[0192] To verify the performance of the method of the present invention, experiments were carried out on 5 publicly available EEG datasets in the experimental part. The specific descriptions are as follows:

[0193] (1) MI1 (BCI CompetitionⅢ DatasetⅣa): This dataset contains EEG signals of 5 subjects. Each subject (numbered A1 - A5) performs two types of motor imagery tasks, and is required to imagine the movement of the right hand or foot after a visual cue. Each experiment lasts for 3.5 s. Each group of EEG signals is recorded using 118 electrodes, the sampling frequency is 100 Hz, and the electrode positions adopt the international 10 / 20 system. Each subject conducts 200 experiments. In this experiment, only the EEG signals of the left hand and right hand are selected for testing. Among the 200 experiments, the left hand and right hand motor imagery are both performed 100 times.

[0194] (2) MI2 (BCI CompetitionⅣ DatasetⅡa): This dataset contains EEG signals of 9 subjects (numbered C1 - C9). Each subject performs four types of motor imagery tasks: left hand, right hand, right foot, and tongue. Each experiment lasts for 4 s. All experiments are recorded using 22 electrodes, the sampling frequency is 250 Hz, and the electrode positions adopt the international 10 / 20 system. Each subject conducts 144 experiments. In this experiment, only the EEG signals of the left hand and right hand are selected for testing. Among the 144 experiments, the left hand and right hand motor imagery are both performed 72 times.

[0195] (3) MI3 and MI4 (Cho, 2017): This dataset contains EEG and EMG signals of 52 subjects (numbered S1 - S52). All experiments were recorded using 64 electrodes with a sampling frequency of 512 Hz. Each subject conducted 200 experiments. In this experiment, only the EEG signals of the left and right hands were selected for testing. Among the 200 experiments, the left and right hand motor imagery were each performed 100 times. Considering the large number of subjects in the dataset and that nearly half of the subjects had low transferability, for the rationality of the experiment, 52 subjects were divided into groups of 10 for a simple transfer experiment. Subjects with an average classification accuracy lower than 60% were not selected. Finally, the data of 20 subjects were selected. The first 10 (S1, S3, S4, S5, S9, S10, S14, S19, S20, S23) formed MI3, and the last 10 (S24, S25, S28, S31, S33, S36, S43, S47, S48, S49) formed MI4.

[0196] (4) RSVP (Matran-Fernandez, & Polo, 2017): The RSVP dataset contains 8-channel electroencephalogram recordings of 11 healthy subjects in a Rapid Serial Visual Presentation (RSVP) experiment. In three different experiments, images were presented at different rates (5, 6, and 10 Hz). Only the 5 Hz version was used in this experiment. The goal was to classify whether the target image or non-target image seen by the subject from the electroencephalogram, for example, images with or without an airplane. The number of images for different subjects ranged from 368 - 565, and the ratio of target to non-target was approximately 1:10. The sampling rate of the EEG signals in the RSVP dataset was 2048 hz, and the band-pass filter was set to 0.15 - 28 hz.

[0197] (5) ERN (Margaux et al., 2012): The ERN dataset is for a feedback error-related negativity (ERN) experiment, which was used for a two-class classification experiment in the Kaggle competition. It was collected from 26 subjects and divided into a training set (16 subjects) and a test set (10 subjects). Since the complete data of the test set could not be accessed, only the training set was used in this experiment. The average ratio of target to non-target was approximately 1:4. The 56-channel electroencephalogram data was sampled at a frequency of 200 Hz.

[0198] 2. Experimental Procedures and Evaluation Metrics:

[0199] Among the 5 EEG datasets, the ERN and RSVP datasets have classification imbalance. Therefore, we use balanced classification accuracy (BCA) to measure the classification performance.

[0200]

[0201] where \(n\) pk and \(n\) k are the number of true positive samples of the \(k\)-th class and the number of samples of the actual \(k\)-th class, respectively. When the sample classes are balanced, BCA is equivalent to the ordinary classification accuracy.

[0202] Suppose there are \(z + 1\) subjects in a dataset. In multi-source migration, each subject is taken as the target domain in turn, and the remaining subjects are taken as the source domain. Therefore, there will be \(z + 1\) different migration tasks in total. The mean value of BCA for the \(z + 1\) migration tasks is used as the final measure of the classification performance of a method on this dataset.

[0203] To evaluate whether each source domain selection method can accurately distinguish the transferability of the source domain, in this experiment, the proposed LSA method is compared with two other domain selection methods (DTE and ROD). Since different classification algorithms combined with domain selection methods will produce different effects, the classifier is the one in Step 1. In each dataset, each subject is selected as the target domain in turn, and the rest are regarded as the source domain. First, three domain selection methods are used to calculate the similarity between the source domain and the target domain; then, the source domain with the lowest similarity to the target domain is removed one by one until the number of source domains is reduced to only one, and the mean value of the classification BCA is recorded. For example, in MI1, there are 7 subjects. In each migration task, except for 1 subject as the target domain, there are 6 source domains left. The same is true for other datasets. In each migration task of MI2, MI3, MI4, RSVP, and ERN datasets, there are 6, 8, 9, 9, 10, and 15 source domains, respectively.

[0204] 3. Experimental comparison methods:

[0205] (1) Rank Of Domains (ROD) (Gong et al., 2010),

[0206] (2) Domain Transferability Estimation (DTE) (Zhang & Wu, 2020).

[0207] 4. Experimental results:

[0208] The experimental results are as Figure 3 shown, Figure 3The change curves of the average BCA of 5 datasets with the decrease in the number of source domains are given. During the process of gradually reducing the number of source domains to 1, the average BCA of LSA in all datasets can maintain a more stable level compared with the other two comparison methods. Especially in MI3, with the decrease in the number of source domains, the average BCA increases, which is because the domain selection method removes some bad source domains that cause negative transfer. On the contrary, when the number of sources is reduced by more than half, the average BCA values of ROD and DTE decrease significantly. The results show that compared with the two comparison methods, the proposed LSA source domain selection method in this method can more accurately identify the transferability of source domains.

[0209] The embodiments described above are only descriptions of the preferred example modes of the present invention, and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A source domain selection method for multi-source EEG transfer, characterized by: The method comprises the following steps: Step (1), electroencephalographic feature extraction; Specifically, the covariance matrix of each sample EEG signal is calculated, and different source and target domains are aligned on the SPD manifold to extract tangent space features. The extracted tangent space features are reconstructed back into the Grassmann manifold to extract Grassmann manifold features, thereby minimizing the marginal probability distribution between the source and target domains. Step (2), manifold feature migration; According to the manifold features that minimize the marginal probability distribution of the source domain and the target domain obtained in step (1), the popular features are migrated to minimize the conditional probability distribution of the source domain and the target domain; Step (3) source domain selection; After the first iteration, LSA is performed to obtain the transferability estimates of different source domains. Based on the transferability estimates, k source domains are removed, and in subsequent iterations, the classifier training iteration is repeated for the remaining source domains. The LSA is specifically: Through the multi-source transfer framework, we obtain single-source classifiers and multi-source classifiers for each source domain. We compare the prediction results of the single-source classifier trained for each source domain with the prediction results of the multi-source classifier, and calculate the estimated value of the transferability of different source domains. The multi-source transfer framework is used to obtain a single-source classifier and a multi-source classifier for each source domain, and the prediction results of the single-source classifier trained for each source domain are compared with the prediction results of the multi-source classifier to calculate the estimated value of the transferability of different source domains; Specifically: The real label of the target domain is represented as A single-source classifier f is trained for each domain s And the final voting generated multi-source classifier f m The predicted labels are represented as and Use sim(·) to represent the similarity between two labels, such as y real and y ps The similarity calculation method between n t Indicates the number of samples in the target domain; The transferability of a source domain is estimated by the STS transfer classification accuracy. Accuracy=sim(y real ,y ps ) / n t (3)。 2. The source domain selection method for multi-source EEG transfer according to claim 1, characterized in that: f s 、f m is linear discriminant analysis or support vector machine.

3. The source domain selection method for multi-source EEG transfer according to claim 1, characterized in that: The electroencephalographic feature extraction is specifically as follows: Let the covariance matrix of the EEG signal X of an experiment be P, P = XX T , and P is the SPD matrix; use and Indicates the source domain and target and domain The covariance matrix of all samples, the one-dimensional distribution of distance on the SPD manifold, M s and M t is the domain distribution mean, and Representing the variance of the covariance matrix distribution, we select the reversible matrices A and B as linear transformations to align the distribution means of the domain; After linear transformation, the samples of source domain and target domain are and According to the congruence invariance property of Riemann distance, the covariance matrix of all features only changes in the reference position of the space. Therefore, the covariance matrix of the source domain and the target domain after the transformation is and The transformed source domain distribution is unchanged. Target domain distribution for Using the KL divergence measure to determine the distribution difference between the source and target domains, the objective function to minimize the marginal probability distribution is: Where KL(·) is the calculation of KL divergence, using the probability density of the standard normal distribution x represents the covariance matrix of the last experiment on the SPD manifold; KL divergence is calculated and Substituting equations (5) and (6) into equation (4), the objective function is simplified to When A T M s A=B T M t When B, the objective function can obtain the optimal solution, such as: and Where E is the identity matrix. The alignment method in formula (9) aligns each domain to its own distribution mean, while formula (10) aligns all samples in the source domain to the target domain. This method adopts the alignment method of formula (9) because the multi-covariance matrix of the source and target domain samples is whitened after alignment. After aligning the distribution means, the covariance matrices of all samples in the source and target domains are and n s , n t are the number of samples in the source domain and the target domain respectively, and through formula (11) Calculated, represents the covariance matrix of the i-th experiment in the source domain, represents the covariance matrix of the jth experiment in the target domain; The aligned covariance matrix is projected onto the tangent space to obtain the tangent space features, and the original two-dimensional covariance matrix features are converted into a one-dimensional vector form. The calculation method is shown in formula (12); Where upper(·) is the vector composed of the upper triangular part of the c×c SPD matrix The tangent space features of the source domain and the target domain are obtained as follows: and Finally, the obtained one-dimensional tangent space features are reconstructed back to the Grassmann manifold space z=g(x)=Φ(t) T x (13) The feature map G is calculated by formula (14) Finally, the Grassmann manifold characteristics are obtained through formula (15): The final Grassmann manifold feature z eliminates the variance of the distribution from and to the target domain as much as possible. and difference.

4. The source domain selection method for multi-source EEG transfer according to claim 1, characterized in that: The manifold feature migration is specifically as follows: In step 1, the manifold features that minimize the marginal probability distribution are obtained. In step 2, the manifold features are migrated to minimize the conditional probability distribution of the source domain and the target domain. The objective function of the classifier f is shown in formula (13). The SRM classifier is used to minimize the conditional probability distribution of the source domain and the target domain. The first two items are SRM classifiers, and the third item represents the source domain. and target domain The difference in conditional probability distribution between ; Among them, the SRM classifier is expressed as Where E is a diagonal matrix used to record labels, and when the sample class is unbalanced, it belongs to the sample The sample type with fewer samples can get a larger weight; where n s,(c＝1) and n s,(c＝2) Respectively represent the number of samples belonging to category 1 and category 2 in the source domain; The third term can be expressed as in Indicates that the conditional probability distribution of the c-th class sample is aligned; Using representation theory, a tensor of the classifier f becomes Where K is the mapping from the original eigenvector to the Hilbert space A core of is the corresponding coefficient vector; Therefore, formula (19) can be written as in represents the Frobenious norm, is the kernel matrix where K ij =K(z i ,z j ),Y=[y1,…,y n ] are the source domain labels and the target domain pseudo labels, n=n s +n t , tr(·) is the trace of the matrix; Formula (20) is written as Among them, M c is the MMD matrix in and Represents the samples belonging to the cth category in the source domain and the target domain respectively; n s , n t respectively is the number of samples in the source domain and the target domain; Substituting equations (21) and (22) into equation (23), the objective function of the classifier is By derivation, To minimize the objective function, the optimal solution is α=((E+λM c )K+σI) -1 EY T (25) Substituting formula (25) into formula (20), we can obtain the prediction information of the classifier.

Images