A Method of EEG Metric Transfer Learning Based on Domain Generalization

Abstract

The present invention relates to an electroencephalogram (EEG) metric transfer learning method based on domain generalization. Taking EEG signals as the research object, based on the existing metric transfer learning methods, the idea of domain generalization is further introduced to enhance the generalization ability of the model. The present invention intends to study the kernel representation learning method, and combine the four aspects of overall divergence, inter-domain divergence, intra-class divergence, and inter-class divergence to find a feature transformation matrix that can maximize the inter-class divergence and overall divergence, and minimize the intra-class divergence and inter-domain divergence. The research method and achievements of the present invention can further enrich the content of EEG signal recognition algorithms and have wide application value.

Description

Technical Field

[0001] The present invention relates to an EEG metric transfer learning method based on domain generalization. The present invention belongs to the field of brain-computer interface transfer learning, and involves the calculation of the overall divergence, inter-domain divergence, intra-class divergence, and inter-class divergence of EEG data. Finally, the domain generalization performance is analyzed according to the calculated divergence, so as to obtain a model with stronger generalization ability. Background Art

[0002] The core of brain-computer interface is to enable the machine to "read" the brain's intentions, so first of all, a way is needed to record and measure brain activity. In current research, EEG signals are a relatively effective way to record brain activity, and most studies hope to detect and analyze signals that can reflect different brain states in the shortest possible time.

[0003] The current detection technologies mainly include electroencephalogram (EEG), electrocorticogram (ECoG), magnetoencephalogram (EMG), functional magnetic resonance imaging (fMRI), positron emission tomography (PET), etc. Among them, EEG is relatively simple, fast, cheap, does not cause damage to the human body, and has a high temporal resolution, so it has become one of the most important signal acquisition methods for BCI. However, in the actual application of BCI systems, it is a very difficult task to obtain sufficient representative samples that can cover the entire data space. Therefore, in the traditional field of pattern recognition and machine learning, it is usually assumed that the training data and test data come from the same feature space and obey the same data distribution. On the one hand, because the training process is relatively tedious, if the training time is too long, it will bring a lot of workload and mental burden to the subjects. In some cases, it is even not feasible to train patients for a long time. Therefore, the original labeled samples are usually small and insufficient to train a good classifier; on the other hand, EEG data is time-varying, and EEG data collected at different collection times and different states show large differences, which makes the data distribution obtained in different individuals, trials, and tasks have large inconsistencies, and the calibration time of machine learning model parameters is long and the adaptability is weak. Therefore, how to reduce the training time of subjects and improve accuracy and adaptability has become an important issue that needs to be urgently solved in the practical application of BCI systems.

[0004] Metric transfer learning is a new research area that combines metric learning and transfer learning, aiming to find a suitable method to reveal the relationships between data. Metric learning usually requires a large amount of label information to achieve satisfactory performance. In transfer learning, although the target domain cannot provide enough labeled samples for training, other labeled data can be added to the source domain for training, which enables transfer learning to well meet the requirements of metric learning, achieve the minimization of intra-class distance and the maximization of inter-class distance, and better assist transfer learning in improving the performance of the classifier. At present, there have been studies applying metric transfer learning to electroencephalogram recognition. However, these metric transfer learning methods do not explicitly reduce the differences between different source domains and target domains during the learning process, resulting in insufficient generalization ability of the learned models. When the data distribution in the target domain varies greatly, the classification performance of the transfer learning model cannot achieve good results.

[0005] Domain Generalization (DG) aims to learn a model with strong generalization ability from several datasets (domains) with different data distributions, so as to achieve excellent performance on unknown test sets. Currently, the main methods are divided into data manipulation, learning strategy, and representation learning. Among them, domain generalization methods based on representation learning have received extensive attention and have been successfully applied in fields such as computer vision and image analysis. Blanchard et al. first applied the kernel method to domain generalization and used the semi - positive definite kernel learning method to learn a domain - invariant kernel from the training data. Erfani et al. proposed an Elliptical Summary Randomisation (ESRand) method, which consists of a randomized kernel and elliptical data. ESRand projects each domain into an ellipse to represent domain information and then uses a similarity metric to calculate distances. Li et al. proposed a new domain generalization and domain adaptation method based on sample support vector machines, that is, a kernel - norm - based regularizer is introduced into the objective function to make the sample support vector machine produce a low - rank output for training samples and increase the domain adaptability of the target domain. The Transfer Component Analysis (TCA) method proposed by Pan et al. is a classic transfer method for data distribution adaptation. Similar to the core idea of TCA, the Domain - Invariant Component Analysis (DICA) method is one of the classic methods for domain generalization using kernels, and its goal is to find a feature transformation kernel function to minimize the distribution differences between all data in the feature space. The Scatter Component Analysis (SCA) method proposed by Ghifary et al. uses the kernel method to learn domain - invariant representations, and adopts Fisher discriminant analysis to minimize the differences between representations from the same category and the same domain, and maximize the differences between representations from different categories and different domains. The domain generalization method proposed by Dissanayake et al. uses a set of known base domains to represent an invisible domain and then uses classifier fusion to classify the invisible domain, achieving good classification accuracy on a phonocardiogram database containing normal and abnormal sounds. In the BCI field, researchers have conducted preliminary research on the generalization ability of electroencephalogram recognition models and achieved some research results. Han et al. explored the zero - calibration problem of BCI in practical situations from the perspective of domain generalization, and the results showed that the explicit domain generalization algorithm outperformed the empirical risk minimization method.Ma et al. proposed a new adversarial domain generalization framework DResNet, in which the domain information is used to learn two components of the weights: the unbiased weights common among subjects and the subject-specific biased weights. This method does not require any data from the test subjects and can generalize well to multiple test subjects simultaneously. Omedes et al. explored the use of low-frequency features to improve the generalization ability of ERP-BCI and demonstrated that there is a stable pattern in the frequency domain that allows the classifier to generalize between tasks. Wu et al. first applied active learning to the regression problem of BCI and studied the generalization ability of BCI regression models. Summary of the Invention

[0006] An object of the present invention is to provide an electroencephalogram metric transfer learning method based on domain generalization for the difficult problem of how to improve the generalization ability of the transfer learning model in the research of electroencephalogram metric transfer learning.

[0007] In the transfer learning method for brain-computer interfaces, the generalization ability of the model obtained by performing transfer learning on the distribution characteristics of a single target domain is weak. Especially in cross-task transfer learning, the huge data distribution differences often lead to a significant decline in transfer performance. To solve the problem of the weak generalization ability of the transfer learning model in brain-computer interfaces, the present invention introduces domain generalization to improve the performance and achieve the purpose of training the model once and using it multiple times.

[0008] To achieve the above object, the technical solution of the present invention is:

[0009] An electroencephalogram metric transfer learning method based on domain generalization, comprising the following steps:

[0010] Step 1: Calculate the overall divergence of the electroencephalogram data.

[0011] Specifically:

[0012] Suppose there are n source domains The divergence of the distribution of the source domain data X under the mapping φ is defined as:

[0013]

[0014] where, ||·|| H is the norm in RKHS; μ D is the mean mapping in the RKHS space.

[0015] Define the domain mean of all source domains as: The overall divergence is expressed as By mapping all the original data, we get Φ = [φ(x1),..., φ(x n )] T, after subtracting the mean, the covariance matrix can be obtained: cov(Φ) = Φ T Φ, the overall divergence can be further expressed as Using the kernel method, find the feature transformation matrix P, and the optimization objective of the overall divergence under the mapping φ is as follows:

[0016]

[0017] s.t. P T KP = I (2)

[0018] Among them, Tr(P T KKP) represents the sum of the diagonal elements of the matrix P T KKP.

[0019] Step 2: Measure the similarity between different domains.

[0020] Specifically: The inter-domain divergence is essentially the maximum mean difference (Maximum Mean Discrepancy, MMD) between different source domains. Similarly, use the kernel method to solve the feature transformation matrix P of the MMD matrix and optimize the following objective function:

[0021]

[0022] Among them, M is the MMD matrix.

[0023] Step 3: Solve the within-class divergence and between-class divergence:

[0024] Specifically: Use the Fisher linear discriminant to combine the two divergences, find the feature mapping P, minimize the within-class distance of all source domains after feature transformation, and maximize the between-class distance. Thus, the within-class divergence is:

[0025]

[0026] Among them, m k and are the mapping mean of the k-th class and the overall mapping mean in the RKHS space respectively;

[0027] The between-class divergence is:

[0028]

[0029] Among them, K is the kernel matrix.

[0030] Solve the linear transformation P that optimizes the four divergences, maximize the between-class divergence and the overall divergence, and minimize the within-class divergence and the inter-domain divergence, and then obtain the final objective function:

[0031]

[0032] such that P T KP = I (6)

[0033] where β and δ are hyperparameters greater than 0.

[0034] Solve for P by generalized eigenvalue decomposition to obtain P* that maximizes the objective function, and embed it into the metric transfer learning objective function to establish a domain generalization algorithm for EEG transfer, and the obtained objective function is as follows:

[0035]

[0036] where is the domain generalization term introduced in metric transfer learning.

[0037] According to the above method, the projection matrix P* and the target domain projection matrix P are obtained in multiple source domains respectively t After that, the metric matrix A is updated using the same optimization solution method.

[0038] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0039] The traditional transfer learning method for brain-computer interfaces has weak model generalization ability when performing transfer learning for the distribution characteristics of a single target domain. Especially in cross-task transfer learning, the huge data distribution difference often leads to a significant decline in transfer performance, and there is currently no good method to improve this. To address this deficiency, this method performs domain generalization through kernel representation learning, that is, optimally solves the divergence between each source domain, and then combines it with the metric transfer learning objective function and optimally solves it. The finally obtained transfer learning model integrates the characteristics of each source domain, enabling the model to still have excellent performance in cross-task transfer learning and having a wider application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0041] Figure 1 is a flowchart of the EEG metric transfer learning method based on domain generalization of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0043] Considering the problem of weak generalization ability of the transfer model in brain-computer interface transfer learning, the present invention introduces the idea of domain generalization on the basis of metric transfer learning, further improves the model framework of metric transfer learning, and uses the domain generalization method in multiple source domains to fuse the distribution characteristics of different source domains, so that the trained model achieves the purpose of enhanced generalization ability, thereby enabling cross-task transfer learning. The implementation of the present invention mainly includes the following three steps: (1) calculating the overall divergence of electroencephalogram data; (2) measuring the similarity between different domains; (3) calculating the within-class divergence and between-class divergence.

[0044] Before introducing the specific steps of domain generalization, a brief description of the metric transfer learning algorithm based on the present invention is given first. First, study the graph structure model, use two projection matrices to map the source domain samples and target domain samples into the shared subspace, and at the same time perform marginal probability distribution alignment and conditional probability distribution alignment, minimize the distance between the source domain and the target domain to reduce the distribution difference, and keep the sample structure consistent; then, use the Mahalanobis distance metric to calculate the metric matrix for the source domain samples with existing labels in the shared space; finally, use the estimated density ratio method to weight the source domain samples, define the loss function under the metric matrix, and minimize the loss. The objective function of metric transfer learning can be obtained as follows:

[0045]

[0046] Among them, r(A) represents the propagation error of the metric matrix A; represents the joint probability distribution alignment and the graph Laplacian operator; ηl(f, A, ω) represents the loss function of the within-class weighted difference and between-class weighted difference of the samples.

[0047] For the metric transfer learning objective function established for Formula (1), based on the constraint condition (Formula (2)) and the reproducing kernel theory, the weights, matrix A, and P are learned using matrix operations and iterative optimization algorithms. Specifically, in the iteration, first, the gradient descent method is used to update matrix P and ω, and then matrix A is updated, thereby updating the value of the objective function in each iteration until its change is less than the determined threshold ε. Finally, a k-nearest neighbor classifier is used for classification under the metric matrix A. For parameter optimization problems such as the selection of parameters k, λ, η, ε, and kernel parameters, the leave-one-out method is proposed to optimize parameters k, λ, η, ε, and the kernel function parameters are selected through the kernel-target registration method. The expression of the constraint condition (Formula (2)) is as follows:

[0048]

[0049] and ω(x j ) > 0 (2)

[0050] where Tr(P T KKP) represents the sum of the diagonal elements of matrix P T KKP.

[0051] Next, the specific steps related to domain generalization in the present invention will be formally introduced.

[0052] Step 1: Calculate the overall divergence of EEG data

[0053] Suppose there are n source domains The divergence of the distribution of source domain data X under the mapping φ is defined as:

[0054]

[0055] where ||·|| H is the norm in the RKHS, and μ D is the mean mapping in the RKHS space.

[0056] Define the domain mean of all source domains as: The overall divergence is expressed as By mapping all the original data, we get Φ = [φ(x1),..., φ(x n )] T , after subtracting the mean operation, the covariance matrix can be obtained: cov(Φ) = Φ T Φ, and the overall divergence can be further expressed as Using the kernel method, find the feature transformation matrix P, and the optimization objective of the overall divergence under the mapping φ is as follows:

[0057]

[0058] s.t. P T KP = I (4)

[0059] where Tr(P T KKP) represents the sum of the diagonal elements of matrix P T KKP.

[0060] Step 2: Measure the similarity between different domains

[0061] The inter-domain divergence is essentially the Maximum Mean Discrepancy (MMD) between different source domains. Similarly, use the kernel method to solve the eigen-transformation matrix P of the MMD matrix and optimize the following objective function:

[0062]

[0063] where M is the MMD matrix.

[0064] Step 3: Solve the within-class divergence and between-class divergence

[0065] Use Fisher's linear discriminant to combine the two divergences, find the eigen-mapping P, minimize the within-class distance of all source domains after eigen-transformation, and maximize the between-class distance. Thus, the within-class divergence is:

[0066]

[0067] where m k and are the mapped mean of the k-th class and the overall mapped mean in the RKHS space, respectively.

[0068] The between-class divergence is:

[0069]

[0070] where K is the kernel matrix.

[0071] Solve the linear transformation P that optimizes the four divergences, maximize the between-class divergence and the overall divergence, and minimize the within-class divergence and the inter-domain divergence, and then obtain the final objective function:

[0072]

[0073] s.t. P T KP = I (8)

[0074] where β and δ are hyperparameters greater than 0.

[0075] Solve for P by generalized eigenvalue decomposition to obtain P* that maximizes the objective function, and embed it into the metric transfer learning objective function to establish a domain generalization algorithm for EEG transfer. The resulting objective function is as follows:

[0076]

[0077] Among them, is the domain generalization term introduced in metric transfer learning.

[0078] According to the above method, the projection matrix P* in multiple source domains and the projection matrix P in the target domain are obtained respectively. t After that, the metric matrix A is updated using the same optimization solution method.

[0079] The above has described the embodiments of the present invention in detail in conjunction with the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, without departing from the principle and spirit of the present invention, various changes, modifications, substitutions, and variations made to these embodiments still fall within the protection scope of the present invention.

Claims

1. A method for electroencephalogram metric transfer learning based on domain generalization, characterized in that: It includes the following steps: Step 1: Obtain the electroencephalogram (EEG) signal features. Assume that there are n source domains for the obtained EEG signal features, and then solve the overall divergence of the EEG signal features. Step 2: Measure the similarity between different domains. Step 3: Solve the within-class divergence and between-class divergence. Use the Fisher linear discriminant to combine the two divergences, find the feature mapping P, minimize the within-class distance of all source domains after feature transformation, and maximize the between-class distance, so as to obtain the within-class divergence as: wherein, m k and are the mapping mean mapping of the k-th class and the overall mapping mean in the RKHS space, respectively, The between-class divergence is: Among them, K is the kernel matrix; Solve the linear transformation P that optimizes the four divergences, maximize the between-class divergence and the overall divergence, and minimize the within-class divergence and the inter-domain divergence, and then obtain the final objective function: s.t.P T KP = I Among them, β and δ are hyperparameters greater than 0. Solve P by generalized eigenvalue decomposition to obtain P* when the objective function is maximized, and embed it into the metric transfer learning objective function to establish a domain generalization algorithm for EEG transfer. The objective function is as follows: Among them, r(A) represents the propagation error of the metric matrix A; represents the domain generalization term introduced in metric transfer learning; ηl(f, A, ω) represents the loss function of the weighted intra-class difference and the weighted inter-class difference of samples; Find the projection matrix P* in multiple source domains and the projection matrix P in the target domain t After that, update the metric matrix A using an optimization solution method 2. The electroencephalogram metric transfer learning method based on domain generalization according to claim 1, characterized in that: The specific content of Step 1 includes: Suppose there are n source domains The divergence of the distribution of the source domain data X under the mapping φ is defined as: where, ∥·∥ H is the norm in the RKHS; μ D is the mean mapping in the RKHS space; Define the domain means of all source domains as: The overall divergence is expressed as Through Map all the original data to obtain Φ = [φ(x1), …, φ(x n )] T , after subtracting the mean, obtain the covariance matrix: cov(Φ) = Φ T Φ, and the overall divergence is further expressed as Using the kernel method, find the feature transformation matrix P. The optimization objective of the overall divergence under the mapping φ is as follows: Among them, Tr(P T KKP) represents the sum of the diagonal elements of the matrix P T KKP.

3. A method for electroencephalogram metric transfer learning based on domain generalization according to claim 1, characterized in that: The specific content of Step 2 includes: The inter-domain divergence is essentially the maximum mean difference between different source domains. Similarly, use the kernel method to solve the feature transformation matrix P of the MMD matrix and optimize the following objective function: Among them, M is the MMD matrix.

Images