A Method for EEG Data Migration of Natural Hand Movements Based on Riemannian Space

Abstract

The present invention discloses a method for migrating EEG data of natural hand movements based on Riemannian space, including: respectively extracting the spatial covariance matrices of EEG signals of natural hand movements of new users and historical users as the target dataset and the historical dataset; selecting the source dataset from the historical dataset based on the sum of distances between Riemannian centers of different categories, and aligning the Riemannian centers of the source dataset and the target dataset; screening out source dataset samples to be added to the training set based on the distance relationship between source dataset samples and target dataset samples; performing geometric transformation on the aligned target dataset in Riemannian space; finally, merging the transformed target dataset samples with annotations and the screened source dataset samples into a training set for establishing a model for new users. The present invention can assist in establishing a model for new users by leveraging the EEG data of natural hand movements of historical users, shortening the time for model calibration.

Description

Technical Field

[0001] The present invention belongs to the field of electroencephalogram (EEG) signal processing, and particularly relates to a method for migrating EEG data of natural hand movements based on Riemannian space. Background Art

[0002] Brain-computer interface technology can decode brain activities by collecting and processing EEG signals, and generate control instructions to directly control external devices such as neuroprosthetics and robotic arms. When a user performs natural hand movements such as grasping, pinching, and turning, movement-related cortical potentials (MRCPs) will be induced in the brain, which contain rich movement behavior information. Using this information, more precise control of external devices such as neuroprosthetics can be achieved. However, due to differences in physiological structures and movement habits among users, the EEG data of natural hand movements collected from historical users cannot be directly used for training new user models. Therefore, when establishing a new user model, it takes a long time to re-collect EEG data and perform calibration.

[0003] Transfer learning methods can use existing data to assist in the establishment of new user models. Its main principle is to reduce the distribution differences between user datasets, so that the EEG data of historical users can be used for training new user models, thereby establishing a good model without or with little annotation of new user data, and reducing the calibration time of new classifiers. Previous transfer learning methods were generally based on traditional Euclidean space. With the gradual emergence of unique advantages of Riemannian geometry methods in EEG signal processing, it is of great practical significance to study the method for migrating EEG data of natural hand movements based on Riemannian space. Summary of the Invention

[0004] To solve the above problems, the present invention discloses a method for migrating EEG data of natural hand movements based on Riemannian space, so as to solve the problem of long model calibration time caused by the difficulty in using historical data due to different distributions of EEG data among users when establishing a new user model.

[0005] To achieve the above object, the technical solution of the present invention is as follows:

[0006] A method for migrating EEG data of natural hand movements based on Riemannian space, comprising the following steps:

[0007] (1) Suppose there are EEG data of m historical users and a new user when performing k natural hand movement tasks. After preprocessing the EEG data, extract the spatial covariance matrix of the user EEG signal samples X ∈ R N×T as a statistical feature, and combine it with the corresponding class label as a dataset. Define the dataset from the a-th historical user and the dataset of the new user as the historical dataset and the target dataset Among them, the target data set The labeled sample set in is denoted as The unlabeled sample set is denoted as

[0008] (2) For each historical data set, in the Riemannian space, according to the Riemannian distance metric, find the Riemannian centers corresponding to the data points of each category in this historical data set and calculate the sum of the pairwise Riemannian distances between the Riemannian centers of each category Select the historical data set with the largest sum of the pairwise Riemannian distances between the Riemannian centers of each category as the source data set to assist in the establishment of the new model;

[0009] (3) In the Riemannian manifold, according to the Riemannian distance metric, find the Riemannian center M of the source data set and the Riemannian center of the target data set and use M and as reference matrices to transform the covariance matrix C i and and to obtain new data sets and

[0010] (4) Denote the set of unlabeled samples in the data set as First, calculate the Riemannian distances between all samples in the data set and all samples in the data set Then, select several samples in the data set that are closest to the samples in the data set and add them to the training set;

[0011] (5) Denote the set of labeled samples in the data set as Calculate the distributions d of the data set and with respect to their respective Riemannian centers and Take d and as parameters to scale-transform the matrix in the data set to obtain the data set

[0012] (6) Calculate the Riemannian centers M of the data points corresponding to each category in the data set and the data set k and On this basis, find an orthogonal matrix U to perform a rotation transformation on the matrix in the data set to obtain the data set

[0013] (7) Denote the dataset The subset with annotations in it is The subset without annotations is When training the model for a new user, the above dataset and are merged as the training set, and is used as the test set.

[0014] Furthermore, step (2) includes:

[0015] (2.1) For each historical dataset, calculate the Riemannian center corresponding to the data of each category as follows:

[0016] Define the Riemannian distance between any two covariance matrices C i , C j as:

[0017]

[0018] where λ k is the eigenvalue of the matrix .

[0019] Furthermore, define the Riemannian center point of K data points as:

[0020]

[0021] where is a symmetric positive definite matrix, which is defined as the point with the minimum sum of Riemannian distances to other data points. This formula has no analytical solution and instead requires using the gradient descent algorithm to iteratively solve.

[0022] (2.2) For each historical dataset, calculate the sum of pairwise Riemannian distances between each Riemannian center , and select the historical dataset with the largest such sum as the source dataset The calculation method is as follows:

[0023]

[0024] where represent the Riemannian centers of the i-th and j-th categories respectively.

[0025] Furthermore, step (3) specifically includes:

[0026] (3.1) Calculate the Riemannian center M of and the Riemannian center of in the target dataset in the same way as step (2).

[0027] (3.2) Use M and as the reference matrix to perform the following transformation on the covariance matrix C i and :

[0028]

[0029] where is the transformed covariance matrix.

[0030] Furthermore, step (4) specifically includes:

[0031] (4.1) For each unlabeled dataset sample, calculate the Riemannian distance between it and all samples in the dataset and select several samples with the closest Riemannian distances from them.

[0032] (4.2) Combine all the selected samples together, remove the duplicates, and obtain a new dataset

[0033] Furthermore, step (5) specifically includes:

[0034] (5.1) Calculate the distributions d and and of the dataset n with respect to the identity matrix I

[0035]

[0036] (5.2) Calculate the scaling parameter s:

[0037]

[0038] Furthermore, perform the following scaling transformation on the covariance matrix in the dataset :

[0039]

[0040] where represents the covariance matrix after the scaling transformation, forming the dataset

[0041] Furthermore, step (6) specifically includes:

[0042] (6.1) Calculate the Riemannian centers M of the data points corresponding to each category in the dataset and the dataset ​​k and The method is as follows:

[0043]

[0044]

[0045] Among them, y i and represent the class label, and k represents the class.

[0046] (6.2) By solving the calculation the transformation matrix U is obtained,

[0047] where L is the number of classes, and ω k ∈ [0, 1] is a coefficient that allows balancing optimization according to the Riemannian mean of each class,

[0048] Furthermore, the definition of G k and is as follows:

[0049]

[0050] where M and have the same meaning as described in (3).

[0051] (6.3) Use the transformation matrix U to perform the following rotation transformation on the matrix in the data set :

[0052] where represents the covariance matrix after the rotation transformation, and constitutes the data set

[0053] The beneficial effects of the present invention are:

[0054] (1) In terms of the selection of the source data set, the method proposed by the present invention can select the data set that is most beneficial to the new user model effect from numerous historical user data sets as the source data set, better improving the classification performance of the new model.

[0055] (2) The method for migrating EEG data of natural hand movements based on the Riemannian space proposed by the present invention can match the data distributions in the source data set and the target data set collected when the user performs natural hand movements, and transfer the existing experience to the new user model, reducing the time required for establishing the new user model.

[0056] (3) The present invention effectively utilizes the label information contained in the target user data set, improving the data migration effect compared with the method that completely does not rely on label information.

[0057] (4) Using the method proposed by the present invention for data migration will not affect the selection of the classifier. That is, in the subsequent classification process, a classifier based on the Riemannian space can be used, or the data can be projected onto the tangent plane and then a traditional classifier based on the Euclidean space can be used, which has good applicability. Brief Description of the Drawings

[0058] Figure 1 It is a flowchart of a method for migrating EEG data of natural hand movements based on the Riemannian space proposed by the present invention. Detailed Embodiment

[0059] The present invention will be further clarified below in conjunction with the drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and not to limit the scope of the present invention.

[0060] As Figure 1 shown, the present invention provides a method for migrating EEG data of natural hand movements based on the Riemannian space, which uses the EEG data of historical users to assist in the establishment of a new user model, thereby realizing the migration of knowledge. The specific steps are as follows:

[0061] (1) Assume that there are m historical users and the EEG data of a new user performing k natural hand movement tasks. After preprocessing the EEG data, extract the spatial covariance matrix of the user's EEG signal samples X ∈ R N×T as the statistical feature, and combine it with the corresponding class label as the data set. The data sets from the a-th historical user and the new user are respectively defined as the historical data set and the target data set and are expressed as:

[0062]

[0063] wherein is the covariance matrix, which has the matrix form of symmetric positive definite and can be regarded as a point located on the symmetric positive definite Riemannian manifold, is the label of the corresponding class, K a and K T are the numbers of samples of this historical user and the target user respectively.

[0064] The present invention considers a semi-supervised transfer learning method, that is, the samples in the historical data set all have corresponding labels, while only a small part of the samples in the target data set are labeled. Therefore, the target data set is further divided into a labeled subset and an unlabeled subset

[0065] (2) For each historical dataset, calculate the Riemannian center corresponding to the data of each category in the Riemannian space according to the Riemannian distance metric, and calculate the sum of the pairwise Riemannian distances between these Riemannian centers. Select the historical dataset with the largest sum of pairwise Riemannian distances between the Riemannian centers of each category as the source dataset to assist in the establishment of the new model. Then the source dataset can be expressed as: where $C \in \mathbb{R}$ is the covariance matrix, $y$ is the label of the corresponding category, and $K$ represents the number of samples of the source user. Specifically, step (2) includes: (2.1) For each historical dataset, calculate the Riemannian center corresponding to the data of each category as follows: Define the Riemannian distance between any two covariance matrices $C$, $C$ as:

[0066]

[0067] where $\lambda$ are the eigenvalues of the matrix. i ∈R n×n For each historical dataset, calculate the Riemannian center corresponding to the data of each category as follows: i where $C \in \mathbb{R}$ is the covariance matrix, $y$ is the label of the corresponding category, and $K$ represents the number of samples of the source user. s Specifically, step (2) includes:

[0068] (2.1) For each historical dataset, calculate the Riemannian center corresponding to the data of each category as follows:

[0069] (2.1) For each historical dataset, calculate the Riemannian center corresponding to the data of each category as follows:

[0070] Define the Riemannian distance between any two covariance matrices $C$, $C$ as: i ,C j as:

[0071]

[0072] where $\lambda$ k is the eigenvalue of the matrix .

[0073] Furthermore, define the Riemannian center point of $K$ data points as:

[0074]

[0075] where is a symmetric positive definite matrix, defined as the point with the minimum sum of Riemannian distances to other data points. This formula has no analytical solution and needs to be solved iteratively using the gradient descent algorithm.

[0076] (2.2) For each historical dataset, calculate the sum of the pairwise Riemannian distances between these Riemannian centers, and select the historical dataset with the largest sum as the source dataset. The calculation method is as follows: where $C_i$, $C_j$ represent the Riemannian centers of the $i$-th and $j$-th categories respectively. The calculation method is as follows:

[0077]

[0078] where, $C_i$, $C_j$ represent the Riemannian centers of the $i$-th and $j$-th categories respectively.

[0079] (3) In a Riemannian manifold, the Riemannian center M of the source dataset and the Riemannian center of the target dataset are respectively calculated according to the Riemannian distance metric. of the source dataset and the Riemannian center of the target dataset are used, and M and are used as reference matrices to transform the covariance matrix C i and so that the Riemannian center of the transformed covariance matrix is the identity matrix I n .

[0080] Specifically, step (3) includes:

[0081] (3.1) Calculate the Riemannian center M of and the Riemannian center of the target dataset in the same way as step (2). The method is the same as that in step (2).

[0082] (3.2) Use M and as reference matrices to transform the covariance matrix C i and The transformation method is as follows:

[0083]

[0084] where is the transformed covariance matrix.

[0085] Denote the transformed source dataset and target dataset as and respectively, and their Riemannian center points are both the identity matrix I n .

[0086] (4) Denote the set of unlabeled samples in the dataset as First, calculate the Riemannian distances between all samples in the dataset and all samples in the dataset , and then select several samples in the dataset that are closest to the samples in the dataset as the training set.

[0087] Specifically, step (4) includes:

[0088] (4.1) For each unlabeled dataset sample, calculate the Riemannian distances between it and all samples in the dataset , and select several samples with the closest Riemannian distances from them. ​

[0089] (4.2) Combine all the selected samples together, remove the repeatedly selected parts, and obtain a new data set.

[0090] (5) Denote the data set The set of labeled samples in it is Calculate the distributions d of the data set and with respect to their respective Riemannian centers, and Use d and as parameters to perform a scaling transformation on the matrices in the data set to obtain the data set

[0091] Specifically, step (5) includes:

[0092] (5.1) Calculate the distributions d of the data set and with respect to the identity matrix I n and The method is as follows:

[0093]

[0094] (5.2) Calculate the scaling parameter s:

[0095]

[0096] Furthermore, perform the following scaling on the covariance matrix in the data set :

[0097]

[0098] where s is the scaling parameter, represents the covariance matrix after the scaling transformation, and constitutes the data set

[0099] (6) Calculate the Riemannian centers M of the data points corresponding to each category in the data set and k and On this basis, find an orthogonal matrix U to perform a rotation transformation on the matrices in the data set to obtain the data set

[0100] Specifically, step (6) includes:

[0101] (6.1) Calculate respectively the data sets and the data set The Riemannian center M of the data points corresponding to each category k and The method is as follows:

[0102]

[0103]

[0104] where y i and represent the class label, and k represents the class.

[0105] (6.2) By solving the calculation obtain the transformation matrix I,

[0106] where L is the number of classes, and ω k ∈ [0, 1] is a coefficient that allows balancing optimization according to the Riemannian mean of each class,

[0107] Furthermore, the definition of G k and is as follows:

[0108]

[0109] where M and have the same meaning as described in (2).

[0110] (6.3) Use the transformation matrix I to perform the following rotation transformation on the matrices in the dataset :

[0111]

[0112] where represents the covariance matrix after the rotation transformation, and constitutes the dataset

[0113] (7) Denote the subset with annotations in the dataset as and the subset without annotations as When training a new user model, use the above datasets and as the training set, and use as the test set.

[0114] It should be noted that the above content only illustrates the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. For those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and retouches can be made, and these improvements and retouches all fall within the protection scope of the claims of the present invention.

Claims

1. A method for migrating EEG data of natural hand movements based on Riemannian space, characterized in that: including the following steps, (1) Collect the electroencephalogram (EEG) data of m historical users and a new user when performing k natural hand movement tasks. After preprocessing the EEG data, extract the spatial covariance matrix of the user's EEG signal samples \(X\in\mathbb{R}\) N×T as statistical features, and combine the corresponding class labels as the dataset; Define the dataset from the \(a\) -th historical user and the dataset of the new user as the historical dataset and the target dataset where, the labeled sample set in the target dataset is denoted as and the unlabeled sample set is denoted as (2) For each historical dataset, calculate the Riemannian center corresponding to the data points of each category in the Riemannian space according to the Riemannian distance metric, and calculate the sum of the pairwise Riemannian distances between the Riemannian centers of each category; select the historical dataset with the largest sum of the pairwise Riemannian distances between the Riemannian centers of each category as the source dataset to assist in the establishment of the new model. in the data points of each category, and calculate the sum of the pairwise Riemannian distances between the Riemannian centers of each category The historical dataset with the largest sum of the pairwise Riemannian distances between the Riemannian centers of each category is selected as the source dataset to assist in the establishment of the new model; (2.1) For each historical data set, calculate the Riemannian center corresponding to the data of each category as follows: Define the Riemannian distance between any two covariance matrices as follows: where λ k is the eigenvalue of the matrix ; Define the Riemannian center point of K data points as: where is a symmetric positive definite matrix, which is defined as the point with the minimum sum of Riemannian distances to other data points. This equation needs to be iteratively solved using the gradient descent algorithm; (2.2) For each historical data set, calculate each Riemannian center the sum of pairwise Riemannian distances between them, and select the historical data set with the largest sum as the source data set The calculation method is as follows: Among them, respectively represent the Riemann centers of the i-th and j-th categories; (3) In a Riemannian manifold, respectively find the Riemannian center M of the source data set and the Riemannian center of the target data set and use M and as reference matrices to transform the covariance matrices C i and to obtain new data sets and (4) Denote the set of unlabeled samples in the dataset as First, calculate the Riemannian distances between all samples in the dataset and all samples in the dataset . Then, select several samples in the dataset that are closest to the samples in the dataset and add them to the training set. (5) Denote the set of labeled samples in the dataset as Calculate the distributions d and of the dataset with respect to their respective Riemannian centers Take d and as parameters to scale the matrices in the dataset to obtain the dataset (6) Calculate the dataset and the dataset the Riemannian center M of the data points corresponding to each category in k and On this basis, find an orthogonal matrix U to perform a rotation transformation on the matrix in the dataset to obtain the dataset (6.1) Calculate the Riemann centers M of the data points corresponding to each category in the dataset and the dataset respectively, as follows: k and The method is as follows: Among them, y i and represent the class label, and k represents the class; (6.2) By solving the calculation the transformation matrix U is obtained. where L is the number of categories, and ω k ∈ [0, 1] is a coefficient that allows balancing the optimization according to the Riemannian mean of each category. G k is defined as follows in relation to : where M and have the same meanings as described in (3); (6.3) Use the transformation matrix U to perform the following rotation transformation on the matrix in the dataset : Among them represents the covariance matrix after rotation transformation, forming a data set (7) Denote the dataset The subset with annotations in The subset without annotations is When training the model for a new user, the above dataset and are merged as the training set, and is used as the test set.

2. The method for migrating EEG data of natural hand movements based on Riemann space according to claim 1, characterized in that: Step (3) specifically includes: (3.1) Calculate the Riemann center M of and the Riemann center of in the target dataset, the method is the same as in step (2); (3.2) Use M and as the reference matrix to perform the following transformation on the covariance matrix C i and as follows: Among them, is the transformed covariance matrix.

3. A method for migrating EEG data of natural hand movements based on Riemann space according to claim 1, characterized in that: Step (4) specifically includes: (4.1) For each unlabeled dataset sample, calculate the Riemannian distance between it and all samples in the dataset respectively, and select several samples with the closest Riemannian distances from them; ​ (4.2) Combine all the selected samples together, excluding the repeatedly selected parts, to obtain a new dataset 4. A method for migrating EEG data of natural hand movements based on Riemann space according to claim 1, characterized in that: Step (5) specifically includes: (5.1) Calculate the data set and with respect to the identity matrix I n the distribution d and (5.2) Calculate the scaling parameter s: For the dataset in the covariance matrix perform the following scaling transformation: Among them, represents the covariance matrix after scaling transformation, and constitutes the data set

Images