An EEG signal recognition method based on metric transfer learning

Abstract

The present invention relates to an electroencephalogram (EEG) signal recognition method based on metric transfer learning. Two projection matrices are used to map source domain samples and target domain samples into a shared subspace, while aligning the marginal probability distributions and conditional probability distributions simultaneously, minimizing the distance between the source domain and the target domain to reduce the distribution difference. Then, a graph structure model is studied to preserve the structural relationship of samples from high-dimensional to low-dimensional. Subsequently, the Mahalanobis distance metric is used to calculate the metric matrix for the source domain samples with known labels in the shared space. Finally, the source domain samples are weighted using the estimated density ratio method, a loss function is defined under the metric matrix, and the loss is minimized. The present invention can improve the cross-subject / session transfer learning effect, shorten the calibration time of traditional brain-computer interfaces (BCIs), provide a new guiding method for EEG signal recognition research, and utilize the label information of EEG source domain data. Metric learning is used to analyze the relationship between labeled samples and their neighboring samples, and measure the similarity between the two.

Description

Technical Field

[0001] The present invention relates to an electroencephalogram (EEG) signal recognition method based on metric transfer learning, belonging to the technical field of pattern recognition and being a transfer learning method integrating metric learning, which can improve the EEG decoding performance, reduce the sample distribution difference between the source domain and the target domain, and realize the EEG signal classification with high reliability and short calibration time. Background Art

[0002] As a bridge between brain science and information science research, the research on Brain-Computer Interface (BCI) has received attention in the cutting-edge research fields of various countries. The brain, as the center controlling human activities such as thoughts, behaviors, and emotions, analyzes and processes information obtained from the external environment and completes communication with the outside world through neuromuscular pathways. However, diseases such as spinal cord injury, amyotrophic lateral sclerosis, stroke, Parkinson's disease, and brain trauma often cause damage or weakening of the central nervous system function, resulting in varying degrees of disorders in perception, sensation, speech, movement, etc. On the one hand, the breakthrough of BCI technology is expected to achieve functional compensation and reconstruction by directly establishing high-precision information interaction and control between the brain and external devices; on the other hand, the active rehabilitation training based on BCI technology can enhance neural remodeling, promote the recovery of the patient's limb motor function, and improve the patient's quality of life and happiness index, which is of great significance to patients, families, and society.

[0003] The BCI system can be regarded as a pattern recognition system, and the most fundamental purpose is to obtain a high accuracy rate for external device control, and the signal recognition effect directly affects the recognition performance of the system. Common learning algorithms include linear discriminant analysis, support vector machine, Bayesian classifier, hidden Markov model, neural network, and combined classifier, etc. However, these algorithms essentially belong to supervised learning. To obtain an offline supervised classifier with a high recognition rate, supervised learning algorithms require a large amount of training and calibration of the subjects, and the brain electrodes are vulnerable to noise interference and there are large individual differences. In order to reduce the training time of the subjects and improve the accuracy and adaptability, many research teams have successively started to study the theory and methods of transfer learning to find a general algorithm model applicable to all subjects.

[0004] The idea of transfer learning is to apply the knowledge or patterns learned in a certain field or task to different but related fields or problems. At present, the research on transfer learning methods for BCI mainly focuses on the following three scenarios: transfer learning across subjects, transfer learning across time periods, and transfer learning across tasks.

[0005] In cross-subject / session transfer learning, He et al. proposed a Euclidean-space Alignment (EA) method. Different from Riemannian Alignment (RA), EA aligns by centering the covariance of each object through a reference matrix. The aligned features can be classified in the Euclidean space, showing greater superiority. Zhang and Wu proposed a Manifold Embedded Knowledge Transfer (MEKT) method, which extracts tangent space features from the covariance centralized by RA or EA to achieve transfer by minimizing the joint probability distribution of the source domain and the target domain, and uses the Domain Transferability Estimation (DTE) method to identify the most beneficial source domain. In cross-task transfer learning, He and Wu also proposed a Label Alignment (LA) method, which can handle the situation where the source domain and the target domain of different tasks have different label spaces. The above BCI transfer learning methods have achieved rich research results. However, in cross-scenario EEG transfer learning, the label information of the source domain is not fully utilized, and the transfer learning effects under different target domains vary greatly.

[0006] Metric transfer learning is a new research field that combines metric learning and transfer learning. Metric learning minimizes the intra-class distance and maximizes the inter-class distance, which can better assist transfer learning in improving the performance of the classifier. Zha et al. used image datasets of multiple source domains to help perform metric learning on the target domain, adaptively learning weights to reflect the contribution of each source metric to the target metric. Mahya et al. proposed a metric transfer learning method with geometric knowledge embedding, which learns an appropriate distance metric while finding a new feature representation, and uses instance weighting methods and graph optimization to maximize sample differences for accurate classification of target samples. However, these metric transfer learning methods do not explicitly reduce the differences between different source domains and target domains during the learning process, and the Euclidean distance metric method has the drawback of treating the components of each dimension of the data equally and losing distribution information, which will lead to insufficient generalization ability of the learning model and reduce the classification performance of the transfer learning model. Summary of the Invention

[0007] The object of the present invention is to provide an EEG signal recognition method based on metric transfer learning to measure the sample similarity in transfer learning, aiming at the deficiencies of existing EEG transfer learning methods.

[0008] To achieve the above object, the technical solution of the present invention is as follows:

[0009] An EEG signal recognition method based on metric transfer learning, comprising the following steps:

[0010] Step 1: Align the marginal probability distribution and the conditional probability distribution for both the labeled EEG source domain data and the unlabeled EEG target domain data simultaneously;

[0011] Specifically: Use two projection matrices to map the samples of the source domain and the target domain into the shared subspace, while aligning the marginal probability distribution and the conditional probability distribution, and minimizing the distance between the source domain and the target domain. The MMD distance is:

[0012]

[0013] where n s , n t are the numbers of samples in the EEG signal source domain and the target domain, Z s and Z t are transformation matrices on the latent space. Since the distribution matching is carried out in the RKHS space, the kernel method is introduced. The kernel matrix K = φ(x) T φ(x), and use Z = φ(x)P to kernelize the principal component analysis through the non - linear mapping φ(x) into the common space, obtaining the following expression:

[0014] D MMD (X s , X t ) H = tr(P T KMK T P) (2)

[0015] where M represents the MMD matrix, and P is a transformation matrix, which is used for both kernel principal component analysis and mapping K into the common subspace.

[0016] Step 2: Construct the graph Laplacian operator;

[0017] Specifically: Use the graph Laplacian operator to preserve the structural relationship of the samples mapped from the high - dimensional space to the low - dimensional space. The graph Laplacian operator can be obtained by minimizing the following formula:

[0018]

[0019] Step 3: Calculate the propagation error of the metric matrix A for the labeled source domain samples in the shared space using the Mahalanobis distance metric; Specifically: The sample pairs after mapping the source domain samples and the target domain samples into the shared subspace are x i and x j , and the corresponding propagation error formula is:

[0020]

[0021] Among them is a positive semi - definite matrix;

[0022] Step 4: Use the estimated density ratio method to weight the source - domain samples, define a loss function under the metric matrix, and minimize the loss;

[0023] Specifically: First, under the sample weight ω and the metric matrix A, use all intra - class and inter - class samples to define a loss function:

[0024] l(f,A,ω)=l in (A,ω)-l out (A,ω) (5)

[0025] wherein, l in is the sum of intra - class weighted differences, and l out is the sum of inter - class weighted differences.

[0026] Combined with steps (1) - (4), the objective function of the final metric transfer learning is:

[0027]

[0028] Here, the first term of the objective function represents the propagation error of the metric matrix A; the second term represents the joint probability distribution alignment and the graph Laplacian operator; the third term represents the loss function of the intra - class weighted difference and the inter - class weighted difference of the samples.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0030] Traditional cross - scenario EEG transfer learning methods do not make full use of the label information of the source domain, and the transfer learning effects under different target domains vary greatly; moreover, existing metric transfer learning methods do not explicitly reduce the differences between different source domains and target domains during the learning process, the generalization ability of the learning model is insufficient, and the Euclidean distance metric method has the drawback of treating the components of each dimension of the data equally and losing distribution information. In view of these problems, the present invention proposes a new metric transfer learning method, which combines the graph - structure model and the Mahalanobis distance metric to calculate the metric matrix with traditional transfer learning methods, further reduces the distance between similar samples and increases the distance between dissimilar samples under the distance metric, and improves the cross - subject / session transfer learning effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0032] Figure 1 This is the flowchart of the EEG signal recognition method based on metric transfer learning of the present invention. Detailed implementation manners

[0033] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0034] As Figure 1 shown, the implementation of the present invention mainly includes 4 steps: (1) Map the samples into the shared subspace, and align the marginal probability distribution and the conditional probability distribution of the labeled EEG source domain data and the unlabeled EEG target domain data simultaneously; (2) Construct the graph Laplacian operator; (3) Calculate the propagation error of the metric matrix A for the labeled source domain samples in the shared space using the Mahalanobis distance metric; (4) Weight the source domain samples using the estimated density ratio method, define the loss function under the metric matrix, and minimize the loss.

[0035] Next, each step will be described in detail.

[0036] Step 1: Map the samples into the shared subspace, and align the marginal probability distribution and the conditional probability distribution of the labeled EEG source domain data and the unlabeled EEG target domain data simultaneously;

[0037] Divide the EEG signals into the labeled source domain data D s ={(X s , Y s )(Y s is the label corresponding to X s )} and the unlabeled target domain data D t ={X t}. Match the two distributions according to the maximum mean discrepancy (MMD) and the reproducing kernel Hilbert space (RKHS). The MMD distance based on the RKHS is:

[0038]

[0039] where n s , n t are the sample numbers of the EEG signal source domain and the target domain, and Z s and Z t are the transformation matrices on the latent space. Equation (1) can be further deduced as:

[0040]

[0041] Where M represents the MMD matrix.

[0042]

[0043] Since the distribution matching is carried out in the RKHS space, the kernel method is introduced. The kernel matrix K = φ(x) T φ(x). Using Z = φ(x)P, the principal component analysis is kernelized into the common space through the nonlinear mapping φ(x), and the expression after distribution alignment is obtained:

[0044] D MMD (X s , X t ) H = tr(P T KMK T P) (4)

[0045] P is a transformation matrix [P s ; P t , which is used for both kernel principal component analysis and mapping K to the common subspace.

[0046] Step 2: Construct the graph Laplacian operator;

[0047] Using the graph Laplacian operator to preserve the structural relationship of samples mapped from the high-dimensional space to the low-dimensional space, the graph Laplacian operator can be obtained by minimizing the following formula:

[0048]

[0049] Where, G i is the new feature representation of x i in the shared subspace, and W is the adjacency matrix. Let G = P T K, then formula (5) can be rewritten as:

[0050] tr(P T KLK T P) (6)

[0051] Based on the general construction method of the graph Laplacian, the graph Laplacian operator can be obtained as L = D - W, where D is a diagonal matrix.

[0052] Step 3: Calculate the propagation error of the metric matrix A for the source domain samples with existing labels in the shared space using the Mahalanobis distance metric;

[0053] After mapping the source domain samples and the target domain samples to the shared subspace and aligning the distributions using matrix projection, for the source domain samples D with existing labels in the shared spaces = {(X s , Y s )} Calculate the metric matrix using Mahalanobis distance metric. The propagation error of the metric matrix is:

[0054]

[0055] (x i , x j ) is a sample pair of source domain samples with existing labels, is a positive semi - definite matrix, called the Mahalanobis matrix. For the source domain samples D s = {(X s , Y s )}, obtain a positive semi - definite matrix Establish the connection between the eigenvectors of the samples, which can preserve the similarity relationship between the samples, that is, further reduce the distance between similar samples and increase the distance between dissimilar samples.

[0056] Step 4: Weight the source domain samples using the estimated density ratio method, define a loss function under the metric matrix, and minimize the loss;

[0057] First, under the sample weights ω and the metric matrix A, define a loss function using all intra - class and inter - class sample pairs:

[0058]

[0059] where, l in is the sum of intra - class weighted differences, and l out is the sum of inter - class weighted differences.

[0060] Combined with steps (1) - (4), the objective function of the final metric transfer learning is:

[0061]

[0062] Here, the first term of the objective function represents the propagation error of the metric matrix A; the second term represents the alignment of the joint probability distribution and the graph Laplacian operator; the third term represents the loss function of the intra - class weighted difference and inter - class weighted difference of the samples. For the objective function of metric transfer learning established in formula (9), the constraint conditions are:

[0063]

[0064] H is the central matrix, I is the identity matrix, 1 n is an n - dimensional all - one vector.

[0065] According to the objective function and constraint conditions of metric transfer learning, combined with the reproducing kernel theory, the weights ω, matrix A and P are learned using matrix operations and iterative optimization algorithms. Specifically, in the iteration, the matrix P and ω are first updated using the gradient descent method, and then the matrix A is updated, thereby updating the value of the objective function in each iteration until its change is less than the determined threshold ε. Finally, classification is performed using the k-nearest neighbor classifier under the metric matrix A. For parameter optimization problems such as the selection of parameters k, λ, η, ε, kernel parameters, etc., the leave-one-out method is used to optimize the parameters k, λ, η, ε, and the kernel function parameters are selected by the kernel-object registration method.

[0066] The above has described the embodiments of the present invention in detail in conjunction with the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, without departing from the principle and spirit of the present invention, various changes, modifications, substitutions and variations to these embodiments still fall within the protection scope of the present invention.

Claims

1. A method for electroencephalogram signal recognition based on metric transfer learning, characterized in that: It includes the following steps: Step 1: Map the samples into the shared subspace, and simultaneously align the marginal probability distribution and the conditional probability distribution for the labeled EEG source domain data and the unlabeled EEG target domain data; Step 2: Construct the graph Laplacian operator; Utilize the graph Laplacian operator to preserve the structural relationship of the samples mapped from the high-dimensional space to the low-dimensional space. The graph Laplacian operator can be obtained by minimizing the following formula: Among them, G i is the new feature representation of x i in the shared subspace, W is the adjacency matrix, let G = P T K can rewrite the above formula as: tr(P T KLK T P) The graph Laplacian can be obtained as L = D - W, where D is a diagonal matrix, and P refers to the transformation matrix [P s ; P t , and K refers to the kernel matrix; Step 3: Calculate the propagation error of the metric matrix A for the source domain samples with existing labels in the shared space using the Mahalanobis distance metric; After mapping the source domain samples and target domain samples to the shared subspace by matrix projection and aligning their distributions, for the source domain samples D s ={(X s , Y s )} with known labels, the Mahalanobis distance metric is used to calculate the metric matrix, and the propagation error of the metric matrix is: (x i , x j ) is a sample pair of source domain samples with existing labels, is a positive semi - definite matrix; Step 4: Weight the source domain samples using the estimated density ratio method, define a loss function under the metric matrix, and minimize the loss; Under the sample weights and the metric matrix, define a loss function using all intra-class and inter-class sample pairs: where l in is the sum of the weighted intra-class differences, and l out is the sum of the weighted inter-class differences; The objective function of the final metric transfer learning is obtained as: The first term of the objective function represents the propagation error of the metric matrix A; the second term represents the joint probability distribution alignment and the graph Laplacian operator; the third term represents the loss function of the intra-class weighted difference and the inter-class weighted difference of the samples; For the objective function of the metric transfer learning established according to the above formula, the constraint condition is: and ω(x i ) > 0 H is the central matrix and I is the identity matrix, 1 n is an n-dimensional all-one vector.

2. The electroencephalogram signal recognition method based on metric transfer learning according to claim 1, characterized in that: The specific steps of Step 1 include: Using two projection matrices to map the samples of the labeled source domain and the unlabeled target domain into the shared subspace, while performing marginal probability distribution alignment and conditional probability distribution alignment, minimizing the distance between the source domain and the target domain, and the MMD distance is as follows: where n s and n t are the numbers of samples in the source domain and target domain of the electroencephalogram signals, Z s and Z t are transformation matrices in the latent space, and the distribution matching is performed in the RKHS space. The kernel method is introduced, and the kernel matrix K = φ(x) T φ(x). The principal component analysis is kernelized into the common space through the nonlinear mapping φ(x) using Z = φ(x)P, and the following expression is obtained: D MMD (X s ,X t ) H = tr(P T KMK T P) where M represents the MMD matrix, and P is a transformation matrix, which is used for both kernel principal component analysis and mapping K into the common subspace.

Images