EEG signal decoding method based on Riemann transfer learning

Abstract

The invention relates to an EEG signal decoding method based on Riemann transfer learning. According to the method, EEG signals are represented as covariance matrix features and are converted into a symmetric positive definite Riemann space, and a Riemann transfer learning method is utilized to reduce data distribution differences of EEG signals collected by the same subject in different time periods or EEG signals collected by different subjects under the same experimental normal form, so that the decoding precision of target EEG signals to be recognized is improved. According to the method, a source data set corresponding to EEG signals of other time periods or other subjects and a target data set corresponding to EEG signals to be identified are translated and converted into uniform distribution by utilizing Riemannian geometric transformation, and then the target EEG signals are identified by utilizing a probability learning vector quantization method which is trained on the source data set and is based on logarithm Euclidean metric learning. And the recognition precision of the target EEG signal can be greatly improved. Meanwhile, logarithmic Euclidean measurement is adopted, the calculation efficiency is higher than that of affine invariant measurement, and the method has important theoretical significance and practical value.

Description

Technical Field

[0001] The present invention belongs to the technical field of biological signal recognition, and specifically is an EEG signal decoding method based on Riemann transfer learning. Background Art

[0002] Brain-computer interfaces (BCI) technology is to establish a direct connection between the brain and external devices for information exchange. It has broad application prospects in many fields such as neurorehabilitation and intelligent human-computer integration. This technology can use brain activity to directly control external devices. There are many ways to capture brain activity, among which the most popular is electroencephalogram (EEG). The EEG signal acquisition method is non-invasive, the acquisition equipment is lightweight, the acquisition method is simple and flexible, the user recognition is higher, and the possibility of promotion and application is greater. However, due to the low spatiotemporal resolution and low signal-to-noise ratio of EEG signals, decoding EEG signals is extremely challenging.

[0003] In the field of brain-computer interfaces, the number of training samples that can be obtained is relatively small, and usually each subject only has data from dozens or hundreds of trials each time. The main reason is that modulating or training the brain-computer interface system is relatively boring, and long-term EEG signal acquisition will put a heavy burden on the brain, thereby causing negative emotions to the subjects. Therefore, usually for each subject, EEG signals are collected multiple times at different times. However, when the same subject puts on the EEG cap to collect EEG signals again at different times, the distribution of the EEG signals collected twice may be different due to slight differences in physical condition or the position of the EEG cap electrodes. This difference will affect the performance of the brain-computer interface.

[0004] At the same time, in the field of brain-computer interface, although each subject obtains less training data, EEG signals of multiple subjects are usually collected. However, due to individual differences among different subjects, the distribution of collected EEG signal data is also different. Therefore, an EEG signal decoding model is trained for each subject. However, under the same EEG signal acquisition paradigm, the EEG signals collected by different subjects will also have some commonalities. Therefore, using EEG data from other subjects can assist in the training of the EEG signal decoding model of the current subject.

[0005] There are currently some EEG signal decoding methods. Among them, the EEG decoding method based on Riemann space has achieved good performance in EEG signal decoding due to its good robustness, good versatility, small sample learning and other characteristics. However, the current EEG decoding method based on Riemann space completely ignores the problems of data distribution changes and individual differences. Although this type of method has good robustness, if the differences in data distribution can be fully considered, it may be possible to further improve the accuracy of EEG signal decoding. Summary of the invention

[0006] The purpose of the present invention is to provide an EEG signal decoding method based on Riemann transfer learning to overcome the above-mentioned defects.

[0007] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0008] A method for decoding EEG signals based on Riemann transfer learning comprises the following steps:

[0009] 1) Obtain source EEG signals and target EEG signals;

[0010] 2) extracting the covariance matrix of the source EEG signal and the target EEG signal respectively to form a source data set and a target data set;

[0011] 3) Using Riemannian geometry transformation to form a unified distribution of the source data set and the target data set;

[0012] 4) Using the source data set after Riemann geometry transformation to train a probabilistic learning vector quantization model based on logarithmic Euclidean metric learning;

[0013] 5) Use the probabilistic learning vector quantization model to identify the EEG signals in the target data set after Riemannian geometry transformation.

[0014] The covariance matrix X i for:

[0015]

[0016] in, represents the EEG signal E collected in the i-th experiment i The normalized signal after bandpass filtering is n is the number of electrodes and l is the number of sampling points.

[0017] The step 3) comprises the following steps:

[0018] 3.1) Re-center the source data set and the target data set respectively, and translate the geometric centers of the two data sets to the unit matrix;

[0019] 3.2) Perform a stretching operation on the source dataset and rescale the distributions of the two datasets so that their dispersion around the geometric mean is the same.

[0020] The redefinition center operation is specifically as follows:

[0021]

[0022]

[0023] in, and Respectively represent the source data and target data after the geometric center is translated, Represents the sample input in the source data set, i=1,…,m s ,m s Indicates the number of samples in the source data set; Represents the sample input in the target data set, i=1,…,m t ,m t Represents the number of samples in the target data set, M s and M t Respectively represent the geometric center of samples in the source and target datasets under the logarithmic Euclidean metric:

[0024]

[0025]

[0026] The stretching operation is specifically as follows:

[0027]

[0028] in, is the source data after stretching transformation, γ is the intermediate variable, d s With d t Represent the discreteness of the source data set and the target data set respectively:

[0029]

[0030]

[0031] The step 4) is specifically as follows: using the source data after geometric transformation as training samples Its category label is y, and several labeled prototypes are learned in the training samples, denoted as Among them, W j represents the features of the jth prototype, c j Represents the category label of the prototype, M represents the number of learned prototypes, all prototypes are updated each time, and the update degree of each prototype is determined according to the posterior probability. The learning rule of the prototype for:

[0032]

[0033] Among them, α represents the learning rate, σ 2 represents the variance, Representation sample By the lth prototype W l The posterior probability generated by the Gaussian component of the determined correct class is, Representation sample By W l The posterior probability generated by the determined Gaussian components is:

[0034]

[0035]

[0036] Among them, the function Tr represents the trace of the computation matrix.

[0037] The step 5) comprises the following steps:

[0038] 5.1) The target data after Riemann geometry transformation Calculate the logarithmic Euclidean distance between the sample to be identified and the prototype learned in the source data set

[0039]

[0040] 5.2) Find the sample to be identified The prototype with the smallest distance predicts the category label corresponding to the prototype as the category of the sample to be identified

[0041]

[0042] An EEG signal decoding system based on Riemann transfer learning, comprising:

[0043] A data acquisition module, used for acquiring source EEG signals and target EEG signals;

[0044] A covariance matrix extraction module is used to extract covariance matrices from the source EEG signal and the target EEG signal to form a source data set and a target data set;

[0045] A data set processing module, used to form a uniform distribution of the source data set and the target data set by using Riemann geometry transformation;

[0046] A probability learning vector quantization model learning module is used to train a probability learning vector quantization model based on logarithmic Euclidean metric learning using a source data set after Riemann geometry transformation;

[0047] The signal recognition module is used to recognize the EEG signal in the target data set after Riemann geometry transformation by using the probability learning vector quantization model.

[0048] The present invention has the following beneficial effects and advantages:

[0049] 1. The present invention adopts Riemann transfer learning to reduce the impact of the distribution difference of EEG signal data collected by the same subject at different times on the recognition accuracy of the EEG decoding model.

[0050] 2. The present invention adopts Riemann transfer learning to reduce the differences in EEG signal distribution between different subjects, and can reuse the EEG signal data of other subjects to improve the recognition accuracy of the EEG decoding model of the current subject.

[0051] 3. The present invention adopts logarithmic Euclidean metric to improve the computational complexity caused by using affine invariant metric, thereby improving algorithm efficiency and classification performance.

[0052] 4. The present invention is a universal EEG signal decoding method, which is applicable to various EEG signal extraction paradigms, such as motor imagery, SSVEP, etc., except that the way of representing EEG as a covariance matrix is ​​different. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 The framework diagram of the present invention;

[0054] Figure 2 Spatial distribution diagram of EEG data collected at different time periods before and after migration;

[0055] Figure 3 Spatial distribution of EEG signals of different subjects before and after migration;

[0056] Figure 4 Comparison of training time between the proposed method RPA-LEM and the RPA method;

[0057] Figure 5 Comparison of recognition accuracy before and after cross-session migration of EEG data collected at different time periods;

[0058] Figure 6 Comparison chart of recognition accuracy between the proposed method and the top three in the competition;

[0059] Figure 7 Comparison of recognition accuracy before and after migration of EEG data of different subjects (cross subject). DETAILED DESCRIPTION

[0060] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0061] In order to enable those skilled in the art to better understand the scheme of the present invention, the EEG signal collected by the motor imagery brain signal extraction paradigm is used as an implementation case and is described in detail below.

[0062] like Figure 1 As shown, the specific implementation of the present invention includes the following steps:

[0063] First, EEG signals were collected according to the commonly used motor imagery paradigm. A commonly used wet electrode EEG cap was placed on the subject's head, and the subject was asked to perform a motor imagery task according to the instructions on the display. The actions of motor imagery include but are not limited to: imagining left hand movement, imagining right hand movement, imagining foot movement, and imagining tongue movement. Each motor imagery task lasted for 4 seconds. After completing a motor imagery task, rest for 1 to 2 seconds. The EEG signal data set collected during this process is recorded as Where E i represents the EEG signal collected during the i-th motor imagery task, y i It represents the action category corresponding to the motor imagery task, and m represents the total number of times the motor imagery task is performed in this process.

[0064] For the same subject, the previously collected EEG signals are used to assist the recognition of the currently collected EEG signals, which is called cross-session migration. The previously collected EEG signals are called source EEG signal datasets and are recorded as The currently collected EEG signal becomes the target EEG signal dataset and is recorded as

[0065] For different subjects, the EEG signals of the subjects to be identified are called the target EEG signal dataset D t , and the EEG signals of other subjects used to assist the current subject in identifying the EEG signal are called the source EEG signal dataset D s .

[0066] Secondly, for each collected EEG signal E i , i = 1, ..., m, use a bandpass filter to filter it to the 10-30 Hz frequency band, and calculate its covariance matrix X i , converting its EEG signal to a symmetric positive definite Riemann space. The bandpass filter mentioned here can be but is not limited to a 5th-order Butterworth bandpass filter. The filtered EEG signal is recorded as is the normalized signal, so the covariance matrix X i Calculate according to the following formula:

[0067]

[0068] in, Represents the EEG signal collected in the i-th experiment Normalized signal after bandpass filtering, where n is the number of electrodes and l is the number of sampling points.

[0069] Co-defense difference matrix X i is an n×n symmetric positive definite matrix located in a curved symmetric positive definite Riemann space, where n is the number of channels from which the EEG signal is collected. Will form the source data set in is the category label of the i-th sample of the source EEG signal. And the target EEG signal dataset The target data set will be formed

[0070] Next, the source data set χ s and the target dataset χ t A Riemannian geometric transformation is performed to reduce the distribution difference between the source data set and the target data set. The Riemannian geometric transformation includes re-centering and stretching operations.

[0071] Recentering is to translate the data set to the origin and normalize it. In a symmetric positive definite Riemann space, the identity matrix plays the role of the origin. Therefore, recentering is to translate the data in the data set so that their geometric mean is the identity matrix I n Based on the logarithmic Euclidean metric, the geometric center M of the samples of the source dataset and the target dataset are calculated according to the following formulas: s and M t :

[0072]

[0073]

[0074] Where exp is the matrix exponential and log is the matrix logarithm. Recentering is to transform the samples of the source and target datasets as follows:

[0075]

[0076]

[0077] in, Represents the sample input in the source data set, i=1,…,m s ,m s Indicates the number of samples in the source data set; Represents the sample input in the target data set, i=1,…,m t ,m t Indicates the number of samples in the target data set; M s and M t They represent the geometric centers of samples in the source and target datasets under the log-Euclidean metric, respectively.

[0078] The stretching operation can further adjust the distribution of the source and target datasets so that the discreteness of the source and target datasets is the same. s With d t The calculation formula is as follows:

[0079]

[0080]

[0081] The source dataset is stretched according to the following formula to match the discreteness of the target dataset:

[0082]

[0083] Among them, d s With d t Represent the discreteness of the source data set and the target data set respectively, i = 1, ..., m t .

[0084] like Figure 2 As shown in Figure 3, the distribution of EEG signals collected by the same subject at different time periods has large differences before migration. After Riemann migration, the distribution differences are significantly reduced.

[0085] like Figure 3 As shown in Figure 3, the EEG distributions of different subjects were quite different before the transfer, but after the Riemann transfer, the distribution differences were significantly reduced.

[0086] Then, the decoding model of the EEG signal is trained on the source data set after geometric transformation. Here, the probability learning vector quantization method based on logarithmic Euclidean metric learning (PLVQ-LEML) is adopted. The principle of the PLVQ-LEML learning method is to learn several labeled prototypes on the training data set, denoted as Where W j Represents the features of the jth prototype, which is in the same space as the training sample, c jis the category label of the prototype, M represents the number of prototypes learned, which is much smaller than the number of samples. This method updates all prototypes each time and determines the update degree of each prototype according to the posterior probability. While learning the prototype, this method also learns the distance metric matrix between prototypes, and obtains a more separable distance metric matrix in Riemann space through logarithmic Euclidean metric learning. For training samples Its category label is y, and the learning rule of the prototype is as follows:

[0087]

[0088] Where α is a constant, greater than 0 and less than 1, representing the learning rate, σ 2 It is also a constant, similar to the variance in the Gaussian function; Representation sample By W l Determine the correct class (c l =y), Representation sample By W l The posterior probability generated by the determined Gaussian components is:

[0089]

[0090] Among them, the function Tr represents the trace of the computation matrix.

[0091] Finally, the sample to be identified after Riemann geometry transformation Compute the logarithmic Euclidean distance with the prototype learned on the source dataset:

[0092]

[0093] Find the sample to be identified The prototype with the smallest distance predicts the category label corresponding to the prototype as the category of the sample to be identified:

[0094]

[0095] In order to test the effectiveness of the EEG signal recognition method, the recognition accuracy without Riemannian migration was compared on the public dataset IV2a of the Brain-Computer Interface Competition. The recognition accuracy is measured by the kappa value.

[0096] K=(p a -p c ) / (1-p c )

[0097] where p a represents the ratio of all correct identifications, p cRepresents the proportion of correct random classifications.

[0098] like Figure 4 As shown, in the two cases of cross-session migration of EEG signal data collected from the same subject in different time periods, and cross-subject migration of brain signals collected from different subjects, the proposed Riemann transfer RPA-LEM based on logarithmic Euclidean metric is faster than the current Riemann transfer RPA method based on affine invariant metric.

[0099] like Figure 5 As shown in the figure, the recognition accuracy of EEG data collected in different time periods (crosssession) is greatly improved after Riemann migration operation. Figure 6 A comparison of the recognition accuracy of the proposed method with the top three in the competition shows that the proposed method achieves significantly higher average recognition accuracy.

[0100] Figure 7 The results of cross-subject transfer are given. Five datasets, S1, S3, S7, S8 and S9, are used as test sets, and the remaining eight datasets are used as training sets. The experiment is repeated 10 times, and the average recognition accuracy is given, with the standard deviation in brackets. Similarly, Riemann transfer greatly improves the recognition accuracy.

Claims

1. An EEG signal decoding method based on Riemann transfer learning, characterized in that: The following steps are involved: 1) Obtain source EEG signals and target EEG signals; 2) extracting the covariance matrix of the source EEG signal and the target EEG signal respectively to form a source data set and a target data set; 3) Using Riemannian geometry transformation to form a unified distribution of the source data set and the target data set; 4) Using the source data set after Riemann geometry transformation to train a probabilistic learning vector quantization model based on logarithmic Euclidean metric learning; 5) Use the probabilistic learning vector quantization model to identify the EEG signals in the target data set after Riemannian geometry transformation.

2. The EEG signal decoding method based on Riemann transfer learning according to claim 1, characterized in that: The covariance matrix X i for: in, represents the EEG signal E collected in the i-th experiment i The normalized signal after bandpass filtering is n is the number of electrodes and l is the number of sampling points.

3. The EEG signal decoding method based on Riemann transfer learning according to claim 1, characterized in that: The step 3) comprises the following steps: 3.1) Re-center the source data set and the target data set respectively, and translate the geometric centers of the two data sets to the unit matrix; 3.2) Perform a stretching operation on the source dataset and rescale the distributions of the two datasets so that their dispersion around the geometric mean is the same.

4. The EEG signal decoding method based on Riemann transfer learning according to claim 3, characterized in that: The redefinition center operation is specifically as follows: in, and Respectively represent the source data and target data after the geometric center is translated, Represents the sample input in the source data set, i=1,…,m s ,m s Indicates the number of samples in the source data set; Represents the sample input in the target data set, i=1,…,m t ,m t Represents the number of samples in the target data set, M s and M t Respectively represent the geometric center of samples in the source and target datasets under the logarithmic Euclidean metric:

5. The EEG signal decoding method based on Riemann transfer learning according to claim 3, characterized in that: The stretching operation is specifically as follows: in, is the source data after stretching transformation, γ is the intermediate variable, d s With d t Represent the discreteness of the source data set and the target data set respectively:

6. The EEG signal decoding method based on Riemann transfer learning according to claim 1, characterized in that: The step 4) is specifically as follows: using the source data after geometric transformation as training samples Its category label is y, and several labeled prototypes are learned in the training samples, denoted as Among them, W j represents the features of the jth prototype, c j Represents the category label of the prototype, M represents the number of learned prototypes, all prototypes are updated each time, and the update degree of each prototype is determined according to the posterior probability. The learning rule of the prototype is W l new for: Among them, α represents the learning rate, σ 2 represents the variance, Representation sample By the lth prototype W l The posterior probability generated by the Gaussian component of the determined correct class is, Representation sample By W l The posterior probability generated by the determined Gaussian components is: Among them, the function Tr represents the trace of the computation matrix.

7. The EEG signal decoding method based on Riemann transfer learning according to claim 1, characterized in that: The step 5) comprises the following steps: 5.1) The target data after Riemann geometry transformation Calculate the logarithmic Euclidean distance between the sample to be identified and the prototype learned in the source data set 5.2) Find the sample to be identified The prototype with the smallest distance predicts the category label corresponding to the prototype as the category of the sample to be identified 8. An EEG signal decoding system based on Riemann transfer learning, characterized in that: include: A data acquisition module, used for acquiring source EEG signals and target EEG signals; A covariance matrix extraction module is used to extract covariance matrices from the source EEG signal and the target EEG signal to form a source data set and a target data set; A data set processing module, used to form a uniform distribution of the source data set and the target data set by using Riemann geometry transformation; A probability learning vector quantization model learning module is used to train a probability learning vector quantization model based on logarithmic Euclidean metric learning using a source data set after Riemann geometry transformation; The signal recognition module is used to recognize the EEG signal in the target data set after Riemann geometry transformation by using the probability learning vector quantization model.

Images