Cross-Sample EEG Decoding Method Based on Joint Domain Alignment Tensor Subspace Learning

Abstract

The present invention discloses a cross-sample EEG decoding method based on joint domain alignment tensor subspace learning. The present invention represents the multi-dimensional information of EEG data using tensors, and uses the Tucker decomposition technique to obtain feature information in different dimensions and uses it as a shared subspace among samples. Then, to reduce the impact brought by domain differences, a set of alignment matrix sets is introduced to simultaneously align different sample domains to the tensor subspace, and graph regularization is used to maintain the local structures of the source domain and the target domain. Finally, an alternating iteration method is used to learn the optimal subspace, and a linear classifier is trained in the subspace to achieve the purpose of cross-sample decoding.

Description

Technical Field

[0001] The present invention belongs to the field of feature decoding, and relates to a cross-sample EEG decoding method based on graph-regularized joint domain align tensor space learning (JDATSL-GR). Background Art

[0002] Brain-computer interface (BCI) technology can directly realize the interaction between the brain and external devices, so it has broad application prospects in the field of medical health. Electroencephalogram (EEG) has been widely used in such rehabilitative BCIs due to its non-invasiveness and high temporal resolution. Such BCIs usually involve a data-driven EEG decoding model to obtain the brain intention of patients, so as to make external control and internal neuromodulation. Therefore, finding an effective model for decoding EEG signals is a crucial step. Due to the electroencephalogram differences between individuals, it is difficult to obtain a general model applicable to different populations. And during the training phase, it is necessary to collect enough EEG data related to specific tasks for new individuals, which may be time-consuming and impractical. Therefore, finding a decoding model applicable to most people has become a hot topic in current brain signal decoding.

[0003] In recent years, a method of feature transformation has been developed to transform the features of the source domain and / or target domain and reduce the differences, which has been widely used in cross-sample EEG decoding. For example, Morioka et al. used sparse coding for transmission, learned the sparse representations of multiple subjects, and transformed the electroencephalograms of new subjects into this space, or used data projection by finding a stationary subspace in the data of multiple subjects. These methods aim to mine and fix the common structure to cope with the variations, but they require a large number of sample features. A large number of studies have shown that there are multiple modal feature information in EEG signals. Utilizing the discriminant information between multiple dimensions through a small amount of EEG data is one of the forefront trends in decoding EEG signals. Tensor is a higher-order generalization of matrices and vectors, which is particularly suitable for representing multilinear relationships that cannot be naturally captured by vector and matrix algebra. In addition, the tensor decomposition algorithm can perfectly apply to multimodal EEG signals in algebraic form. It can not only optimize the iteration to obtain a subspace with smaller distribution differences, but also avoid the loss of dimensional information caused by vectorization during decomposition.

[0004] Subspace learning usually also takes into account the differences between samples. When the EEG domain differences between samples are too large, simply forcing the acquisition of a shared subspace often fails. Generally speaking, a simple linear transformation matrix can effectively reduce the impact of sample differences and increase robustness. And the graph regularization term is often used to maintain the local structure after the transformation of the source domain and the target domain. It utilizes the geometric properties of neighboring points to reduce the within-class distance of samples and make the within-class connections closer. Summary of the Invention

[0005] In view of the deficiencies of the prior art, the present invention proposes a cross-sample EEG decoding method based on joint domain alignment tensor subspace learning. First, the EEG data is represented by tensors, a set of alignment matrix sets is introduced in the Tucker decomposition, and the graph regularization term is used to maintain the local structure of the source domain and the target domain. Then, using the idea of alternating minimization, the subspace and the alignment matrix set are jointly iteratively optimized. Finally, a linear classifier is trained in the optimal subspace to achieve the purpose of cross-sample decoding.

[0006] To achieve the above objectives, the method of the present invention mainly includes the following steps:

[0007] Step 1. Data preprocessing

[0008] Collect electroencephalogram (EEG) data and perform preprocessing, and use complex wavelet transform to form tensor-form data of EEG, denoted by X;

[0009] Step 2. Acquisition of the shared subspace

[0010] Assume that there are N s samples in the given source domain. Each sample is denoted as a k-order tensor, and the N s samples are stacked into a (k + 1)-order tensor. Similarly, let be N t samples from the target domain; and k = 1, 2,..., K.

[0011] Obtain the initial tensor subspace using Tucker decomposition as follows:

[0012]

[0013] where [;] represents the 1 to k-mode product of G and U in the K-order tensor decomposition, g (i) and g s and g t are the core tensors of X s and X t , and U = {U (i)} i＝1…k represents the shared subspace of the source domain and the target domain;

[0014] Step 3. Joint domain alignment subspace

[0015] Initialize an alignment matrix set M, and align the source domain and the target domain to the tensor subspace simultaneously, generating the following optimization:

[0016]

[0017] where the alignment matrix set tensor subspace is the core tensor represented under the tensor subspace, I is the identity matrix;

[0018] To preserve the local structures of the source domain and the target domain, graph-based regularization is introduced; where let X = β[g; U]; where X = [X S , X t , g = [g s , g t , β = [β s , β t ; is the manifold regularization adaptation matrix; where the graph regularization is expressed as:

[0019]

[0020] where is a similarity matrix, which is defined as follows:

[0021]

[0022] where N p (X i ) is the p-nearest set of X i , σ is a scale function usually equal to 1; L = I - D -1 / 2 SD -1 / 2 is the normalized Laplacian matrix, where is 's diagonal matrix; thus the overall optimization problem is given:

[0023]

[0024] where λ is the regularization term weight;

[0025] Step 4. Alternating iterative optimization subspace

[0026] Since M and U (i) are coupled in the formula, it is expressed as a joint optimization problem, and the problem is decomposed into sub-problems using the alternating minimization scheme, and these sub-problems are iteratively optimized until convergence, and finally an approximate solution is obtained;

[0027] Step 5. Classification index evaluation

[0028] Project the labeled source domain data into the optimized tensor subspace obtained in step 4, as shown in the following formula:

[0029] g s = X s ×1U (1)T ×2U (2)T …× K U (i)T

[0030] Rely on the feature tensor g s Train a linear classifier. Similarly, project the target sample to be decoded as shown in the following formula:

[0031] g t = X t ×1U (1)T ×2U (2)T ...× K U (i)T

[0032] Use the feature tensor g t Classify and identify the effectiveness of this method.

[0033] Preferably, all signals in step 1 are sampled by an international standard 10-20 electrode distribution system; the data preprocessing includes wavelet denoising, electrocardiogram and electrooculogram rejection.

[0034] Preferably, the specific steps of the alternating iterative optimization subspace are as follows:

[0035] (1) Fix M, update U with β (i) ,g s ,g t

[0036] By introducing Z S = [X S ; M], Z t = [X t ; M], the overall optimization is converted into the following formula:

[0037]

[0038] Solve according to the existing Tucker decomposition algorithm;

[0039] (2) Fix U (i) ,g s ,g t , update M with β: Introduce the auxiliary variable W s = [g s ; U], W t = [g t ; U], and the sub-problem of M is obtained as follows:

[0040]

[0041] For each M (i) Optimize it; expand the above formula according to pattern - i to get as follows:

[0042]

[0043] where The symbol represents the Kronecker inner product,

[0044] Let Since M is row - orthogonal, in order to make the optimization problem in the standard orthogonal constraint, that is, to satisfy column - orthogonality, let P (i) = M (\i)T Substitute it into the above formula to get the following operations:

[0045]

[0046] Finally, the optimization problem of M is transformed into the optimization problem of P, as follows:

[0047]

[0048] (3) Fix M, U, g s , g t Update β

[0049] According to the auxiliary variable W = [g; U], expand W horizontally in the (k + 1)-mode to a matrix Therefore, the optimization of β in the objective function is as follows:

[0050]

[0051] where the above formula is solved according to the Lagrange multiplier method, and its Lagrangian function is

[0052] J = tr(λ(β T W k+1 LW (k+1)T β)+γ(I - β T β))

[0053] where γ = diag(γ1,…,γ N ) is the Lagrange multiplier, and the elements on its diagonal are the corresponding eigenvalues. Let We can get:

[0054] W k+1 LW (k+1)T = βγ

[0055] Finally, this formula is solved by generalized eigenvalue decomposition, and the problem of finding the optimal graph adaptation matrix β is transformed into the problem of N minimum eigenvectors.

[0056] Compared with many existing cross-sample EEG decoding algorithms, the present invention has the following characteristics:

[0057] Since EEG signals have multiple dimensions such as time, space, frequency and other characteristic information. Therefore, the present invention uses tensors to represent EEG data, making full use of the characteristic information between its dimensions. And the subspace learning is extended from matrices to tensor subspaces, avoiding the data loss problem caused by vectorization.

[0058] When facing excessive differences between samples, a matrix set is used to align and share the tensor subspace, and graph regularization is introduced to preserve the local structural characteristics of the source domain and the target domain. Finally, an alternating iteration strategy is used to optimize and obtain a shared tensor subspace with smaller sample differences, and the method is evaluated by classification metrics in this subspace. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is the flowchart of the implementation of the present invention;

[0060] Figure 2 is the EEG electrode naming diagram of the international 10-20 system in the embodiment of the present invention;

[0061] Figure 3 is the convergence diagram of the alternating iteration results of the decoding algorithm in the embodiment of the present invention;

[0062] Figure 4 is the algorithm convergence analysis diagram in the embodiment of the present invention;

[0063] Figure 5 is the comparison diagram between the tensor subspace aligned by graph regularization and joint domain in the algorithm of the embodiment of the present invention and the forced subspace. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0064] The following detailed description of the embodiments of the present invention is given on the premise of the technical solution of the present invention. The detailed implementation manners and specific operation processes are given, but the protection scope of the present invention is not limited to the following embodiments.

[0065] As Figure 1 shown, the present embodiment includes the following steps:

[0066] Step (1), electroencephalogram (EEG) data acquisition. Two sets of motor imagery (MI) datasets used in the present invention are from the fourth BCI competition, and two sets of event-related potentials (ERP) datasets are from the Kaggle competition. The specific relevant information of each dataset is shown in Table 1. For each of the four datasets, the EEG signals in the frequency band of 8 - 30 Hz are band-pass filtered using a fifth-order Butterworth filter. Then, the EEG signals between [0.5, 3] seconds after the cue occurs are used as the experimental data for one experiment. For the ERP datasets, the signals are used as experimental data at a time interval of 0.7 s after the start of the signal stimulation. The four datasets use the international 10 - 20 system EEG electrode nomenclature, as Figure 2 shown.

[0067] Table 1 Statistical information of relevant datasets

[0068]

[0069]

[0070] Step 2, obtaining the shared subspace

[0071] The four datasets are respectively transformed using complex wavelet transform to obtain data in the range of 8 - 30 Hz, with an interval of 1 Hz, so there is frequency information of 23 Hz. And it is expressed in tensor form. The specific data can be obtained according to Table 1: For example, MI1 is represented as data information of 750 * 22 * 23 (time * channel * frequency) in tensor form, where the datasets of 8 people are used as the source domain target X s , and the remaining one person is used as the target domain X t , and Tucker decomposition is used as follows:

[0072]

[0073] The matrix factor U = {U (i)} i＝1…k represents the shared subspace of the source domain and the target domain; as Figure 3 shown. In addition, the fitting error of each initial subspace to the original data is controlled at about 90%. The formula for the fitting error is as follows:

[0074]

[0075] Step 3, graph-regularized joint-domain alignment subspace

[0076] Assume the matrix set M T M = I. Align the tensor subspaces of the four datasets using the appropriate M matrix set, and the following formula is obtained:

[0077]

[0078] And by using graph regularization to preserve the local structures of the source domain and the target domain, the final optimization formula of the joint domain alignment subspace of graph regularization is

[0079]

[0080] Figure 3 When λ → 0 or when λ → +∞, the average classification result decreases. Especially after λ is greater than 10, the performance degradation is very obvious. This is because ignoring the geometric characteristics of the source domain and target domain datasets completely or magnifying their geometric characteristics of the source domain and target domain to align the shared subspace will both cause adverse interference. It can be seen that the performance of this method is the best when λ is around 1.

[0081] Step 4, alternately iteratively optimize the subspace

[0082] Since the shared subspace U to be solved is coupled with the matrix set M, the present invention uses surrogate minimization to decompose the problem into sub-problems and iteratively optimizes these sub-problems until convergence to obtain an approximate solution U. In addition, in order to verify that the alternating iteration scheme can converge effectively, Figure 4 it is shown that the four datasets can all tend to be stable after about 3 - 4 iterations of the algorithm.

[0083] Step 5, classification metric evaluation

[0084] Project the labeled source domain data into the optimized tensor subspace obtained in Step 4, as shown in the following formula:

[0085] g s = X s ×1U (1)T ×2U (2)T …× K U (i)T

[0086] Relying on the feature tensor g s train a linear classifier. Similarly, project the target samples to be decoded as shown in the following formula:

[0087] g t = X t ×1U (1)T ×2U (2)T …× K U (i)T

[0088] Moreover, the data of the four datasets were compared using the classical cross-sample decoding method as the baseline method, and Table 2 details the obtained results. Among them, the name of the present invention is represented in yellow font, and the method without using alignment is represented in green name. It can be seen that JDATSL has a smaller mean square error than NTSL and other baseline methods. The results of the NTSL method and the JADTSL method for each subject in the four datasets are as Figure 5 shown. On the two types of datasets of MI and ERP, each subject can obtain better results with the JDATSL method than with NTSL. And Figure 5 demonstrates that this method has better universality for different populations.

[0089] Table 2 Average classification accuracy (%) and standard deviation (% in parentheses).

[0090]

[0091] The content described in the embodiments of this specification is only an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms stated in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A cross-sample EEG decoding method based on joint domain alignment tensor subspace learning, characterized in that The method specifically includes the following steps: Step 1. Data preprocessing Collect electroencephalogram (EEG) data and perform preprocessing, and use complex wavelet transform to form tensor-form data of EEG, which is represented by X; Step 2. Obtaining the shared subspace Suppose there are N samples in the source domain p samples Each sample is denoted as a k - order tensor, and the N p samples are stacked into a (k + 1)-order tensor Similarly, let be N samples from the target domain; q samples Use Tucker decomposition to obtain the initial tensor subspace, as shown below: Where [;] represents the K-order tensor decomposition of G and U (i) The 1-to-k-mode product, g s and g t Yes X s and X t The core tensor of U = {U (i) } i＝1…k Represents the shared subspace of the source domain and the target domain; Step 3. Aligning the subspaces in the joint domain Initialize a set of alignment matrices M, and align the tensor subspaces of the source domain and the target domain simultaneously, resulting in the following optimization: Among them, the alignment matrix set tensor subspace is the core tensor represented under the tensor subspace, and I is the identity matrix; To preserve the local structures of the source domain and the target domain, graph-based regularization is introduced; let X = β[g; U]; where X = [X S , X t , g = [g s , g t , β = [β s , β t ; is the manifold regularization adaptation matrix; then the graph regularization is expressed as: s.t.β T β = I, Among them is a similarity matrix, which is defined as follows: where N p (X i ) is the p-nearest set of X i , σ is a scale function usually equal to 1; L = I - D -1 / 2 SD -1 / 2 is the normalized Laplacian matrix, where is a diagonal matrix; thus the overall optimization problem is given as follows: where λ is the weight of the regularization term; Step 4. Alternately iteratively optimizing the subspace Since M and U (i) are coupled in the equation, it is formulated as a joint optimization problem. The problem is decomposed into sub-problems using the alternating minimization scheme, and these sub-problems are iteratively optimized until convergence, and finally an approximate solution is obtained; Step 5. Evaluating the classification index Project the labeled source domain data into the optimized tensor subspace obtained in Step 4, as shown in the following formula: g s = X s × 1U (1)T × 2U (2)T … × K U (i)T Relying on the feature tensor g s Train a linear classifier. Similarly, project the target sample to be decoded as follows: g t = X t × 1U (1)T × 2U (2)T … × K U (i)T Utilize the feature tensor g t Classify and identify the effectiveness of this method.

2. The cross-sample EEG decoding method based on joint domain alignment tensor subspace learning according to claim 1, characterized in that: All signals in Step 1 are sampled by the internationally standard 10-20 electrode distribution system; data preprocessing includes wavelet denoising, electrocardiogram (ECG) and electrooculogram (EOG) removal.

3. The cross-sample EEG decoding method based on joint-domain alignment tensor subspace learning according to claim 1, characterized in that: The specific steps of the alternately iteratively optimizing the subspace are as follows: (1) Fix M and update U for β (i) , g s , g t By introducing Z S = [X S ; M], Z t = [X t ; M] is overall optimized and converted into the following formula; Solve according to the existing Tucker decomposition algorithm; (2) Fix U (i) , g s , g t , β Update M: Introduce auxiliary variable W s = [g s ; U], W t = [g t ; U] to obtain the sub-problem of M as follows: For each M (i) Optimize it; expand the above formula according to pattern - i to get the following: wherein the symbol represents a Kronecker product Let Since M is row-orthogonal, in order to put the optimization problem in the standard orthogonal constraint, that is, to satisfy column orthogonality, let P (i) = M (\i)T Substituting into the above formula gives the following operations: Finally, the optimization problem of M is converted into the optimization problem of P, as shown below: (3) Fix M, U, g s , g t Update β Expand \(W = [g; U]\) into a matrix in the \((k + 1)\)-mode horizontally according to the auxiliary variable \(W\). Therefore, the optimization of \(\beta\) in the objective function is shown as follows: s.t.β T β = I, where the above formula is solved according to the Lagrange multiplier method, and its Lagrangian function is J = tr(λ(β T W k+1 LW (k+1)T β) + γ(I - β T β)) where γ = diag(γ1,…,γ N ) is the Lagrange multiplier, and the elements on its diagonal are the corresponding eigenvalues. Let We can obtain: W k+1 LW (k+1)T = βγ Finally, this formula is solved by generalized eigenvalue decomposition, and the problem of finding the optimal graph adaptation matrix β is converted into the problem of N minimum eigenvectors.

Images