Time-lock phase-lock electroencephalogram data migration method and system

Abstract

The invention relates to the technical field of brain-computer interfaces, and discloses a time-locked phase-locked electroencephalogram data migration method and system. The migration method comprises the following steps: configuring a source data set, a known label target data set and an unknown label target data set; aligning the three data sets for the first time by applying an edge distribution covariance alignment method; sequentially performing low-dimensional time domain feature extraction, time domain feature migration and spatial domain feature migration on samples in the source data set after the first-time alignment by applying a conditional distribution mean alignment method to obtain a feature set after conditional distribution mean alignment is completed; background electroencephalogram extraction and background electroencephalogram migration are sequentially carried out by applying a conditional distribution covariance alignment method, and a feature set after conditional distribution covariance alignment is obtained; and summing samples corresponding to the feature set after conditional distribution mean alignment and the feature set after conditional distribution covariance alignment to complete electroencephalogram data migration. According to the method, the time-lock and phase-lock characteristics of the electroencephalogram signals can be effectively utilized, and the electroencephalogram data migration performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interface technology, and in particular to a time-locked and phase-locked electroencephalogram (EEG) data migration method and system. Background Art

[0002] Brain-computer interface (BCI) provides a direct communication pathway between the brain and the external environment without relying on peripheral nerves, which can replace, repair, enhance, supplement or improve normal nervous system functions. The BCI system collects physiological signals generated during brain thinking activities and converts them into recognizable machine instructions, thereby realizing the direct expression of thinking content. BCI can provide a device control method for the disabled and the elderly without the involvement of muscles and peripheral nervous systems, thereby improving their quality of life and independent living ability; it can provide a brain state monitoring method for ordinary healthy people to help ordinary people adjust the way they use their brains; it can enhance the cognition of people working in complex links such as astronauts and pilots, and improve their cognitive ability to efficiently complete delicate and complex tasks. At present, BCI technology is developing rapidly and has shown potential application value in the fields of medicine, aerospace, entertainment, etc.

[0003] Electroencephalogram (EEG) has the advantages of high temporal resolution and high technical maturity, and has been widely used in the field of BCI. This type of BCI system places sensors on the surface of the scalp or inside the skull to collect EEG signals generated by brain thinking activities, identify EEG signals through decoding algorithms, establish a mapping relationship between EEG signals and thinking activities, and then convert thinking activities into machine instructions, ultimately achieving direct control of external devices such as computers and wheelchairs or brain state monitoring. Currently, EEG signals commonly used in BCI systems include steady-state visual evoked potential (SSVEP), P300 potential (Potential 300), error-related potential (ErrP), motor imagery (MI), etc. Accurately and efficiently completing the recognition task of these specific EEG signals through decoding algorithms is the core link in establishing a practical brain-computer interface system. The decoding algorithms used for these EEG signals include Discriminative canonical pattern matching (DCPM) algorithm, Discriminative spatial pattern (DSP) algorithm, Shrinkage linear discriminant analysis (SKLDA) algorithm, Task related component analysis (TRCA) algorithm, etc. These decoding algorithms all require EEG data training to learn relevant EEG signal features, and then use them for EEG recognition. Due to the significant variability of EEG signals between different users, it is usually difficult to establish a universal decoding algorithm that is widely applicable to the user group. Traditional brain-computer interface calibration modes such as Figure 1 As shown, EEG samples of a specific user are collected to train a decoding algorithm suitable for that individual. Usually, researchers need to collect enough EEG samples for training the decoding algorithm. When EEG samples are insufficient, the trained EEG decoding algorithm will encounter a small sample problem, causing the algorithm to fail. However, collecting EEG samples is inefficient and time-consuming, and the time cost will increase exponentially with the number of instructions of the brain-computer interface system. Therefore, the time cost required to collect EEG samples greatly limits the practical application of the BCI system.

[0004] Nowadays, EEG data migration methods are usually used to make full use of existing data and reduce the time required to collect EEG samples. The idea is to use EEG samples of multiple users collected in previous experiments as the source domain, and individual EEG samples of users for which the EEG decoding model is to be established as the target domain. The source domain EEG samples are mathematically transformed so that the statistical distribution difference between the source domain EEG samples and the target domain EEG samples is reduced after the transformation. In the EEG data migration method, how to reduce the difference in joint distribution between the source domain and the target domain is the key issue of data migration. If the EEG data migration method cannot effectively reduce the distribution difference between the source domain and the target domain, or even causes the distribution difference between the two domains to increase, it will cause negative transfer. At this time, the knowledge learned by the decoding algorithm in the source domain has a negative effect on sample recognition in the target domain, and the effect of using data migration is worse than not using data migration. It is usually hoped that the data migration method can make a certain statistical distribution of the source domain the same as the corresponding distribution of the target domain, that is, to achieve alignment of the source domain and the target domain. Ideally, this statistical distribution should be a joint distribution, and the data migration method should be able to achieve alignment of the joint distribution of the source domain with the joint distribution of the target domain.

[0005] Existing transfer learning methods generally approximate the joint distribution through marginal distribution and conditional distribution. Their technical routes can be roughly divided into marginal distribution data migration, conditional distribution data migration, joint distribution data migration, and dynamic distribution data migration. However, since most of these technologies will change the data dimension, or do not directly operate on the time domain and spatial domain characteristics of EEG signals, or are based on Riemannian geometry, they cannot truly achieve the alignment of the joint distribution between the source domain and the target domain, resulting in limited application scenarios. Summary of the invention

[0006] The purpose of the present invention is to provide a time-locked and phase-locked EEG data migration method and system, which can effectively process the source domain EEG data, starting from the expected distribution and covariance matrix, to achieve the joint distribution alignment of the source domain EEG data and the target domain EEG data, thereby improving the recognition performance of user intentions.

[0007] In order to achieve the above object, the present invention adopts the following technical scheme:

[0008] In a first aspect, the present invention provides a time-locked and phase-locked EEG data migration method, comprising the following steps:

[0009] Configure source dataset, known label target dataset, and unknown label target dataset;

[0010] Apply the marginal distribution covariance alignment method to align the source data set, the target data set with known labels, and the target data set with unknown labels to obtain the source data set after the first alignment, the target data set with known labels after the first alignment, and the target data set with unknown labels after the first alignment;

[0011] Apply the conditional distribution mean alignment method to perform low-dimensional time domain feature extraction, time domain feature migration and spatial domain feature migration on the samples in the source data set after the first alignment, so that the time domain and spatial domain feature distribution of the samples in the source data set after the first alignment is similar to the samples of the corresponding category in the target data set with known labels after the first alignment, and obtain the feature set after the conditional distribution mean alignment is completed;

[0012] Apply the conditional distribution covariance alignment method to perform background EEG extraction and background EEG migration in sequence, so that the conditional distribution of the background EEG covariance matrix of each category in the source data set after the first alignment is aligned with the corresponding category in the target data set with known labels after the first alignment, and obtain the feature set after the conditional distribution covariance alignment is completed;

[0013] The samples corresponding to the feature set after the conditional distribution mean is aligned and the feature set after the conditional distribution covariance is aligned are summed to complete the EEG data migration.

[0014] As a possible implementation method, the marginal distribution covariance alignment method includes the following sub-steps:

[0015] Calculate the covariance matrix of the samples in the source data set and the covariance matrix of the samples in the target data set with known labels, and obtain the covariance matrix set corresponding to the source data set and the covariance matrix set corresponding to the target data set with known labels respectively;

[0016] Calculate the geometric mean of the elements in the covariance matrix set corresponding to the source data set And the geometric mean of the covariance matrix set elements corresponding to the known label target data set

[0017] Applying the geometric mean Transform the samples in the source dataset to obtain the first aligned source dataset and apply the geometric mean The known label target dataset and the unknown label target dataset are transformed respectively to obtain the known label target dataset after the first alignment and the unknown label target dataset after the first alignment.

[0018] As a possible implementation method, the samples in the source dataset are transformed in the following way:

[0019]

[0020] in, represents the samples in the source dataset after the first alignment; Represents the samples in the source dataset; Represents the source dataset;

[0021] The known label target dataset and the unknown label target dataset are transformed as follows:

[0022]

[0023]

[0024] in, represents the samples in the target dataset with known labels after the first alignment; is a sample in the target dataset with known labels; It is a target dataset with known labels; For unknown label target dataset.

[0025] As a possible implementation method, low-dimensional time domain feature extraction includes the following sub-steps:

[0026] Solve the first spatial filter for each category sample in the source data set after the first alignment and the target data set with known labels after the first alignment;

[0027] The first spatial filter corresponding to each category of samples is used to extract the common low-dimensional time domain features of each corresponding category of samples in the source data set after the first alignment and the target data set with known labels after the first alignment, and the low-dimensional time domain features of the samples in the source data set after the first alignment are obtained. Low-dimensional time domain features of samples in the target dataset with known labels after the first alignment

[0028] As a possible implementation method, time domain feature migration includes the following sub-steps:

[0029] A feature mapping matrix corresponding to each category of samples is configured, and the feature mapping matrix is ​​used to map the low-dimensional time domain features of each category of samples in the source data set after the first alignment to the corresponding category in the target data set with known labels after the first alignment to complete the time domain feature migration.

[0030] As a possible implementation, the feature mapping matrix is:

[0031]

[0032] Among them, α is the regularization coefficient; the low-dimensional time domain features of all samples in the source dataset after the first alignment Form a collection, is the mean of the covariance matrix of the i-th sample, is the mean of the i-th class samples; the low-dimensional time domain features of all samples in the target dataset with known labels after the first alignment Form a collection, is the mean of the samples in the i-th category; is the identity matrix.

[0033] As a possible implementation method, spatial feature migration includes the following sub-steps:

[0034] Solve the spatial migration matrix of each category of samples;

[0035] Each category's spatial domain migration matrix is ​​used to transform each corresponding category sample after completing the time domain feature migration to obtain a feature set after the conditional distribution mean is aligned.

[0036] As a possible implementation method, background EEG extraction includes the following sub-steps:

[0037] Solve the second spatial filter corresponding to each category sample in the source data set after the first alignment;

[0038] Using a second spatial filter to perform spatial filtering on each category of samples in the source data set after the first alignment, so as to obtain low-dimensional time domain features of each category of samples;

[0039] Calculate the spatial domain migration matrix corresponding to each category of samples in the source data set after the first alignment, and use each category of spatial domain migration matrix to transform each corresponding category of samples of the low-dimensional time domain features obtained in the previous step to obtain the task-related time domain features of each sample in the source data set after the first alignment;

[0040] Calculate the background EEG of each sample in the source dataset after the first alignment Right now j∈i,i∈{1,…,L}, where Represents the sample with sequence number j in the source dataset after the first alignment; Represents the task-related time domain features of the sample with sequence number j in the source dataset after the first alignment.

[0041] As a possible implementation method, background EEG migration is performed in the following way: j∈i,i∈{1,…,L}, where is the background EEG after migration; U i is the transformation matrix, in, Task-related temporal features of all samples in the source dataset after first alignment Form a collection, is the mean of the samples in the i-th category; is the mean of the covariance matrix of the i-th sample in the first aligned known label target dataset; the background EEG of all samples in the source dataset after the first alignment Constitute a set, Q is the mean of the covariance matrix of the i-th sample in the set.

[0042] In a second aspect, the present invention provides a time-locked and phase-locked EEG data migration system, comprising:

[0043] A storage unit, used to store a source data set, a target data set with known labels, and a target data set with unknown labels;

[0044] A marginal distribution covariance alignment unit, which applies a marginal distribution covariance alignment method to align a source data set, a target data set with known labels, and a target data set with unknown labels, to obtain a source data set after first alignment, a target data set with known labels after first alignment, and a target data set with unknown labels after first alignment;

[0045] A conditional distribution mean alignment unit, which applies a conditional distribution mean alignment method to sequentially perform low-dimensional time domain feature extraction, time domain feature migration, and spatial domain feature migration on the samples in the source data set after the first alignment, so that the time domain and spatial domain feature distributions of the samples in the source data set after the first alignment are similar to those of the samples of the corresponding categories in the target data set with known labels after the first alignment;

[0046] The conditional distribution covariance alignment unit applies the conditional distribution covariance alignment method to perform background EEG extraction and background EEG migration in sequence, so that the conditional distribution of the background EEG covariance matrix of each category of the source data set after the first alignment is aligned with the corresponding category in the target data set with known labels after the first alignment;

[0047] The summing unit sums the result obtained by the conditional distribution mean alignment unit and the result obtained by the conditional distribution covariance alignment unit to complete the EEG data migration.

[0048] Compared with the prior art, the beneficial effects of the present invention are:

[0049] 1. The time-locked and phase-locked EEG data migration method proposed in the present invention includes three steps: marginal distribution covariance alignment, conditional distribution mean alignment, and conditional distribution covariance alignment. The EEG is decomposed into two parts: time-domain and spatial-domain features and background EEG. After the initial migration of the marginal distribution covariance is completed, special migration operations are performed on the time-domain and spatial-domain features and the background EEG respectively, which can realize the joint distribution alignment of the source domain and the target domain EEG samples.

[0050] 2. Compared with the existing technical solutions that only design EEG data migration technology for covariance matrix features, the present invention proposes technical solutions from three aspects: time domain features, spatial domain features and covariance matrix features of EEG data. It can more effectively utilize the time-locked and phase-locked features in EEG signals, which is conducive to improving the performance of EEG data migration and promoting the development of the field of transfer learning.

[0051] 3. The time-locked and phase-locked EEG data migration method proposed in the present invention is based on the mathematical mechanism of EEG signal generation and takes the basic EEG model as the technical starting point. It can better ensure the rationality and effectiveness of EEG data processing and solve the problem of differences in EEG data under different experimental conditions and between different user groups.

[0052] 4. The time-locked and phase-locked EEG data migration method proposed in the present invention does not change the data dimension, maintains the original structure of the data, and has an excellent migration effect. Compared with the existing technology that changes the data migration dimension and has a narrow usage scenario, the present invention can be seamlessly integrated into the existing spatial filtering method, statistical learning method and deep learning method, which means that both traditional machine learning algorithms and the latest deep learning models can benefit from the migrated data of the present invention, achieving higher classification accuracy and lower experimental costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0054] Figure 1 It is a flow chart of the traditional brain-computer interface calibration mode in the background technology of the present invention;

[0055] Figure 2 and Figure 3 This is a flow chart of the overall solution of the EEG data migration method in an embodiment of the present invention;

[0056] Figure 4 This is a flow chart of the time-locked and phase-locked EEG data migration method provided by the present invention in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, words such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. For example, the first threshold and the second threshold are only used to distinguish different thresholds, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0058] It should be noted that, in the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0059] In the present invention, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The following at least one item (items) or similar expressions thereof refer to any combination of these items, including any combination of single items (items) or plural items (items). For example, at least one item (items) of a, b or c can mean: a, b, c, the combination of a and b, the combination of a and c, the combination of b and c, or the combination of a, b and c, where a, b, c can be single or multiple.

[0060] The EEG data migration method can make full use of the existing EEG data to reduce the time of EEG sample collection. The overall idea of ​​this method can be found in Figure 2 and Figure 3 First, define the source domain and target domain. The source domain is the EEG samples of multiple users collected in the previous experiments, and the target domain is the individual EEG samples of users for which the EEG decoding model is to be established. The idea of ​​the EEG transfer method is to perform mathematical transformation on the source domain EEG samples so that the statistical distribution difference between the source domain EEG samples and the target domain EEG samples is reduced after the transformation. The source domain EEG samples can also be mathematically transformed to transform the source domain data into the target domain data, and then the transformed source domain data and target domain data are used to train the individual EEG samples of users for which the EEG decoding model is to be established, thereby greatly reducing the time of collecting the current user's EEG data.

[0061] Some commonly used EEG signals have the characteristics of time-locked and phase-locked, such as P300, SSVEP, ErrP, lateralized visual evoked potential, etc. According to the basic macroscopic model of EEG generation, the observed values ​​of these EEG signals on a single sensor can be described as the linear superposition of multiple task-related EEG source signals on a single sensor:

[0062] x(m)=As(m)+n(m); (1)

[0063] in, is the EEG data observation value recorded by the multi-channel sensor at time m, and also contains the location information of the physical space where each channel sensor is located; is the source aliasing matrix, which describes the coupling strength of each brain source to each sensor and contains the spatial information of the EEG signal; is the value of the task-related EEG signal at time m, which contains the time domain information of the EEG signal; is the value of the noise at time m; represents the set of real numbers, N sis the number of sensors, N t is the number of sampling points, N k is the number of brain power sources. The model obeys the following assumptions: ①n(m)~N(0,∑), that is, in represents the mathematical expectation, Cov[·] represents the covariance matrix; ②n(m) and As(m) are independent of each other; ③s(m) are linearly independent.

[0064] It can be seen from formula (1) that the user's time-locked and phase-locked EEG signal x(m)~N(As(m),∑), where As(m) mainly describes the time domain and spatial domain characteristics of the EEG signal, and ∑ mainly describes the energy and spatial domain characteristics. For two different users, the difference in their EEG data mainly comes from two sources, one is the difference in As(m), and the other is the energy difference in ∑. Due to the strong noise interference in the EEG data, the noise energy occupies a major position in the covariance matrix of the time-locked and phase-locked EEG samples. It is usually difficult to capture the characteristics of the EEG source signal, and EEG decoding needs to rely on the characteristics of As(m). The present invention comprehensively considers the time domain characteristics, spatial domain characteristics and covariance matrix characteristics of EEG data, and proposes a time-locked and phase-locked EEG data migration method and system, which can effectively process the source domain EEG data and realize the joint distribution alignment of the source domain EEG data and the target domain EEG data.

[0065] In a first aspect, the present invention provides a time-locked and phase-locked EEG data migration method, see Figure 4 , including the following steps:

[0066] Configure source dataset, known label target dataset, and unknown label target dataset;

[0067] As an example, configure the source dataset as follows:

[0068]

[0069] Configuring a known label target dataset as follows:

[0070]

[0071] Configuring unknown label target dataset as follows:

[0072]

[0073] The complete target dataset is denoted as Then there is i is the category number and L is the total number of categories. The total number of samples of the i-th category is recorded as have The total number of samples of the i-th category is recorded as have

[0074] Apply the marginal distribution covariance alignment method to align the source data set, the target data set with known labels, and the target data set with unknown labels to obtain the source data set after the first alignment, the target data set with known labels after the first alignment, and the target data set with unknown labels after the first alignment;

[0075] As a possible implementation method, the marginal distribution covariance alignment method includes the following sub-steps:

[0076] Calculate the covariance matrix of the samples in the source data set and the covariance matrix of the samples in the target data set with known labels, and obtain the covariance matrix set corresponding to the source data set and the covariance matrix set corresponding to the target data set with known labels respectively;

[0077] As an example, the covariance matrix of samples in the source dataset is calculated as follows:

[0078]

[0079] The covariance matrix of samples in the known label target dataset is calculated as follows:

[0080]

[0081] Get the source dataset The corresponding covariance matrix set is: Get a known labeled target dataset The corresponding covariance matrix set is:

[0082] Calculate the geometric mean of the elements in the covariance matrix set corresponding to the source data set And the geometric mean of the covariance matrix set elements corresponding to the known label target data set

[0083] As an example, the Euclidean mean is calculated The geometric mean of the elements in as well as The geometric mean of the elements in

[0084]

[0085] Alternatively, the logarithmic Euclidean mean, the Riemann mean, or other geometric mean calculation methods may be used, and the present invention does not make any specific limitation thereto.

[0086] Applying the geometric mean Transform the samples in the source dataset to obtain the first aligned source dataset and apply the geometric mean The known label target dataset and the unknown label target dataset are transformed respectively to obtain the known label target dataset after the first alignment and the unknown label target dataset after the first alignment.

[0087] As a possible implementation method, the samples in the source dataset are transformed in the following way:

[0088]

[0089] in, represents the samples in the source dataset after the first alignment; Represents the samples in the source dataset; Represents the source dataset;

[0090] The known label target dataset and the unknown label target dataset are transformed as follows:

[0091]

[0092] in, represents the samples in the target dataset with known labels after the first alignment; is a sample in the target dataset with known labels; It is a target dataset with known labels; For unknown label target dataset.

[0093] After transformation, the source dataset is obtained after the first alignment as follows:

[0094]

[0095] After transformation, we obtain the target dataset with known labels after the first alignment. as follows:

[0096]

[0097] After transformation, the unknown label target dataset is obtained after the first alignment as follows:

[0098]

[0099] So far, the target dataset All data migration has been completed, and the source data set after the first alignment is continued. Perform conditional distribution mean alignment and conditional distribution covariance alignment.

[0100] Apply the conditional distribution mean alignment method to perform low-dimensional time domain feature extraction, time domain feature migration and spatial domain feature migration on the samples in the source data set after the first alignment, so that the time domain and spatial domain feature distribution of the samples in the source data set after the first alignment is similar to the samples of the corresponding category in the target data set with known labels after the first alignment, and obtain the feature set after the conditional distribution mean alignment is completed;

[0101] The core object of the conditional distribution mean alignment operation is First, extract the samples The time domain features of each category sample in The temporal and spatial feature distributions of each category of samples in the The temporal and spatial feature distributions of the samples of the corresponding categories in are similar, thus reducing and The difference in the means of the conditional distributions.

[0102] As a possible implementation method, low-dimensional time domain feature extraction includes the following sub-steps:

[0103] Solve the corresponding source data set after the first alignment and known label target dataset after first alignment The first spatial filter for each class sample in There are L first spatial filters in total, spatial filter W i It needs to be obtained under reasonable assumptions. The goal of designing the spatial filter here is to extract and The common low-dimensional time domain features of the i-th sample in . The objective function of the spatial filter corresponding to the optimization problem has various forms, which are not limited here. Depending on the objective function, the L spatial filters can be L different spatial filters or the same spatial filters. The ultimate goal is to extract and The common low-dimensional time domain features of the i-th class samples in .

[0104] As an example, the spatial filter can extract the low-dimensional time domain features of the EEG sample. The process is expressed as follows:

[0105] s(m)=W T x(m); (15)

[0106] in, is the EEG data observation value recorded by the multi-channel sensor at time m, and also contains the location information of each channel sensor in the physical space. Since the dimension of x(m) is reduced during the spatial filtering process, it no longer contains the location information of the multi-channel sensor in the physical space, and becomes a task-related EEG power signal s(m) containing only time domain features. Therefore, spatial filtering can be used to extract low-dimensional time domain features of EEG data.

[0107] As a first example of constructing the first spatial filter, the objective function of the optimization problem is:

[0108]

[0109] in, Indicates taking the parameter that minimizes the value of the expression in the brackets tr(·) is the trace operator, which means taking the trace of the matrix in the brackets; θ is the mixing coefficient, which is used to mix the source dataset and the target dataset information. for The Euclidean mean of the i-th class samples in , for The Euclidean mean of the covariance matrix of the i-th sample in , for The Euclidean mean of the i-th class samples in , for The Euclidean mean of the covariance matrix of the i-th sample in .

[0110] The calculation of is as follows:

[0111]

[0112] The solution of equation (16) can be obtained by the following generalized eigenvalue problem:

[0113]

[0114] in, is the eigenvalue matrix, diag(·) means constructing a diagonal matrix using the elements in the brackets; λ1 is the eigenvalue with sequence number 1, and the eigenvector corresponding to λ1 is the matrix The first column vector in . Take the first N k The eigenvectors corresponding to the largest eigenvalues ​​constitute W i , N k ∈[1,N s ]. When using this method to construct a spatial filter, a first spatial filter should be constructed for each category, and finally L different first spatial filters are obtained.

[0115] As a second example of constructing the first spatial filter, the objective function of the optimization problem is:

[0116]

[0117] In the formula, for The inter-class scatter matrix of for The intra-class scatter matrix of for The inter-class scatter matrix of for The intra-class scatter matrix of .

[0118] The calculation of is as follows:

[0119]

[0120] In the above formula, express The Euclidean mean of the Euclidean means of each class of samples in ;

[0121]

[0122] In the above formula, express The Euclidean mean of the Euclidean means of the samples in each class.

[0123] The calculation of is as follows:

[0124]

[0125] It can be obtained from formula (17): It can be obtained from formula (19).

[0126] The solution of formula (22) can be obtained through the following generalized eigenvalue problem:

[0127]

[0128] in, is the eigenvalue matrix, diag(·) means constructing a diagonal matrix with the elements in the brackets, λ1 is the eigenvalue with the serial number 1, and the corresponding eigenvector matrix The first column of eigenvectors in . Generally, the first N k The eigenvectors corresponding to the largest eigenvalues ​​constitute W. When this method is used to construct the first spatial filter, all categories use the same spatial filter, that is, W1 = W2 = ... = W i =…=W L =W.

[0129] The first spatial filter corresponding to each category of samples is used to extract the common low-dimensional time domain features of each corresponding category of samples in the source data set after the first alignment and the target data set with known labels after the first alignment, and the low-dimensional time domain features of the samples in the source data set after the first alignment are obtained. Low-dimensional time domain features of samples in the target dataset with known labels after the first alignment

[0130] As an example, using the first spatial filter W i For the dataset Each sample in is spatially filtered to obtain Low-dimensional time domain features of samples in As shown below:

[0131]

[0132] In the formula, For the dataset The low-dimensional time domain features of the j-th sample in;

[0133] Use the first spatial filter W i For the dataset Each sample in is spatially filtered to obtain Low-dimensional time domain features of samples in As shown below:

[0134]

[0135] In the formula, For the dataset The low-dimensional time domain features of the j-th sample in .

[0136] As a possible implementation method, time domain feature migration includes the following sub-steps:

[0137] Configure the feature mapping matrix corresponding to each category sample i Acting on a dataset The low-dimensional time domain feature set of the i-th sample in The purpose is to transform the dataset The low-dimensional time domain feature map of category i is mapped to the dataset The corresponding category.

[0138] For the dataset Each sample in Perform feature mapping to obtain the corresponding low-dimensional time domain features, as shown in the following formula:

[0139]

[0140] In the formula, It is a dataset The low-dimensional time domain features of the jth sample in The result after feature mapping is completed.

[0141] The feature mapping requires a total of L feature mapping matrices P i The feature map matrix P can be calculated using different methods i , there is no restriction here, the purpose is to The low-dimensional time domain features of category i in are aligned to the dataset The corresponding category.

[0142] As a possible implementation, the feature mapping matrix P i for:

[0143]

[0144] Among them, α is the regularization coefficient; the low-dimensional time domain features of all samples in the source dataset after the first alignment Form a collection, is the mean of the covariance matrix of the i-th sample, is the mean of the i-th class samples; the low-dimensional time domain features of all samples in the target dataset with known labels after the first alignment Form a collection, is the mean of the samples in the i-th category; is the identity matrix.

[0145] As a way to calculate the feature map matrix P i For example, the optimization problem is expressed as follows:

[0146]

[0147] in,‖·‖ F is the Frobenius norm; α is the regularization coefficient; for The set of low-dimensional time domain features of the i-th sample in, is the Euclidean mean of its elements; As shown below:

[0148]

[0149] for The set of low-dimensional time domain features of the i-th sample in, for The Euclidean mean of the elements in ∑ (s) for The elements in correspond to the Euclidean means of the time-domain covariance matrix. and Calculated by the following formula:

[0150]

[0151] Formula (34) is a typical least squares problem, and its solution is the feature mapping matrix P shown in Formula (33): i .

[0152] The feature mapping matrix is ​​used to map the low-dimensional time domain features of each category of samples in the source dataset after the first alignment to the corresponding category in the target dataset with known labels after the first alignment to complete the time domain feature migration. It should be explained that after feature mapping, The amplitude may be reduced compared to that before feature mapping, and a scaling factor needs to be found Adjust its amplitude. The low-dimensional time domain features of each sample in Scaling is performed to complete the time domain feature migration, as shown in the following formula:

[0153]

[0154] for The jth sample in which the time domain feature migration is completed.

[0155] As a method to calculate the scaling factor p i For example, the optimization problem is expressed as follows:

[0156]

[0157] in, Calculated by formula (35), For the set {f j ,j∈i}, is expressed as follows:

[0158]

[0159] The optimal solution to the optimization problem is the scaling factor p i It can be obtained by the following formula:

[0160]

[0161] Samples in After time domain feature migration, it becomes Dimensions are N s ×N t Change to N k ×N t . for The set of i-th class samples that completes the time domain feature migration in , for The spatial feature migration aims to solve the spatial migration matrix Use this matrix to act on the set The elements in the restored sample dimension are N s ×N t , and the dataset middle The spatial characteristics of the sample are assigned to the restored sample.

[0162] As a possible implementation method, spatial feature migration includes the following sub-steps:

[0163] Solve the spatial migration matrix of each category of samples;

[0164] As an example, solve the spatial domain migration matrix A of each category of samples i , the optimization problem is expressed as follows:

[0165]

[0166] In the formula, It is a dataset The set of samples of the i-th category in , where the Euclidean mean of the covariance matrix corresponding to the element is yes The low-dimensional time domain feature set of the i-th class sample in , where the Euclidean mean of the covariance matrix corresponding to the element is and It can be obtained by the following formula:

[0167]

[0168] Formula (42) is a typical least squares optimization problem, and its optimal solution is the spatial migration matrix A i for:

[0169]

[0170] Here, pinv(·) means to find the pseudo-inverse of the matrix in the brackets.

[0171] Each category's spatial domain migration matrix is ​​used to transform each corresponding category sample after completing the time domain feature migration to obtain a feature set after the conditional distribution mean is aligned.

[0172] As an example, the obtained spatial migration matrix Acts on The corresponding samples in are shown as follows:

[0173]

[0174] in, That is The jth sample in that completes the mean alignment of the conditional distribution.

[0175] It should be pointed out that the conditional distribution mean alignment result The row rank is equal to N k , and before alignment The row rank is equal to N s , by N k ≤N s It can be seen that the alignment of conditional distribution means leads to The rank of is reduced, and the following steps will solve this problem.

[0176] The conditional distribution covariance alignment method is applied to perform background EEG extraction and background EEG migration in sequence, so that the conditional distribution of the background EEG covariance matrix of each category in the source data set after the first alignment is aligned with the corresponding category in the target data set with known labels after the first alignment, and the feature set after conditional distribution covariance alignment is obtained; the core object of the conditional distribution covariance alignment method is First, extract the samples The background EEG covariance matrix corresponding to each category in is then transformed mathematically to The conditional distribution of the background EEG covariance matrix of each category in The corresponding categories are aligned.

[0177] Medium Sample The background EEG is recorded as Requires background EEG The row rank and Different methods can be used to calculate background EEG This embodiment does not limit the specific solution method. Background EEG of the sample The dataset consisting of its category labels is As shown below:

[0178]

[0179] As a possible implementation method, the present invention proposes a method for calculating background EEG using a spatial filter, comprising the following sub-steps:

[0180] Solve the second spatial filter corresponding to each category sample in the source data set after the first alignment;

[0181] As an example of constructing a second spatial filter, the objective function of the optimization problem is:

[0182]

[0183] in, Indicates taking the parameter that minimizes the value of the expression in the brackets tr(·) is the trace operator, which means taking the trace of the matrix in the brackets; θ is the mixing coefficient, which is used to mix the source domain and target domain information. for The Euclidean mean of the i-th class samples in , for The Euclidean mean of the covariance matrix of the i-th sample in , for The Euclidean mean of the i-th class samples in , for The Euclidean mean of the covariance matrix of the i-th sample in .

[0184] Formula (48) is the same as the part on the right side of the equal sign in Formula (16). The meaning of each symbol has been explained in Formula (16) and will not be repeated here. Calculated by formula (17), Calculated by formula (19), Calculated by formula (18), Calculated by formula (20). The second spatial filter It can be obtained from formula (21). It should be noted that the first N k2 The eigenvector corresponding to the largest eigenvalue is composed of

[0185] Using a second spatial filter to perform spatial filtering on each category of samples in the source data set after the first alignment, so as to obtain low-dimensional time domain features of each category of samples;

[0186] As an example, using a second spatial filter For the dataset Each sample in is spatially filtered as shown below:

[0187]

[0188] In the formula, For the dataset The low-dimensional time domain features of the j-th sample in .

[0189] Calculate the spatial domain migration matrix corresponding to each category of samples in the source data set after the first alignment, and use each category of spatial domain migration matrix to transform each corresponding category of samples of the low-dimensional time domain features obtained in the previous step to obtain the task-related time domain features of each sample in the source data set after the first alignment;

[0190] As an example, let’s calculate the dataset The spatial domain migration matrix corresponding to each category of samples in The corresponding optimization problem is expressed as follows:

[0191]

[0192] Formula (50) is similar to formula (42), so It can be obtained by the method shown in formula (43) to formula (45), which will not be described here.

[0193] Using the Spatial Transfer Matrix Obtain the task-related temporal features of each sample in the source dataset after the first alignment As shown below:

[0194]

[0195] Calculate the background EEG of each sample in the source dataset after the first alignment Right now j∈i,i∈{1,…,L}, where Represents the sample with sequence number j in the source dataset after the first alignment; Represents the task-related time domain features of the sample with sequence number j in the source dataset after the first alignment.

[0196] Next, background EEG migration is performed. The background EEG migration is designed to Find a transformation matrix for each category sample in Act on the background EEG of the corresponding category Thereby reducing The background covariance matrix of each category in The difference between the background covariance matrices of the corresponding categories in .

[0197] As a possible implementation method, background EEG migration is performed in the following way:

[0198]

[0199] in, is the background EEG after migration; U i is the transformation matrix, in, Task-related temporal features of all samples in the source dataset after first alignment Form a collection, is the mean of the samples in the i-th category; is the mean of the covariance matrix of the i-th sample in the first aligned known label target dataset; the background EEG of all samples in the source dataset after the first alignment Constitute a set, Q is the mean of the covariance matrix of the i-th sample in the set.

[0200] As a way to calculate the transformation matrix U i For example, in most EEG data migration scenarios, The total number of samples N (t) It is difficult to accurately estimate the background EEG covariance matrix of each category. Therefore, this example proposes a method that does not require estimation. The background EEG covariance matrix can be used to calculate the transformation matrix U i The optimization problem corresponding to this method is:

[0201]

[0202] in, Calculated by formula (43), is a sample belonging to category i A collection of is the mean of its elements. Calculated by the following formula:

[0203]

[0204] From the assumption ② of the model in formula (1), we can know that and are unrelated to each other, so formula (53) can be simplified as:

[0205]

[0206] in, and It is expressed by the following formula:

[0207]

[0208] The solution of formula (55) is the transformation matrix U to be calculated i ,Right now:

[0209]

[0210] The samples corresponding to the feature set after the conditional distribution mean is aligned and the feature set after the conditional distribution covariance is aligned are summed to complete the EEG data migration.

[0211] As an example, the feature set after conditional distribution mean alignment is expressed by equation (46), and the feature set after conditional distribution covariance alignment is expressed by equation (52). The final migration result of EEG data is the sum of the two equations, as shown in the following equation:

[0212]

[0213] In the formula, For the source dataset The final result of completing the time-locked and phase-locked EEG data migration for the jth sample in is: The dataset consisting of its category labels is denoted as

[0214]

[0215] In a second aspect, the present invention provides a time-locked and phase-locked EEG data migration system, comprising:

[0216] A storage unit, used to store a source data set, a target data set with known labels, and a target data set with unknown labels;

[0217] A marginal distribution covariance alignment unit, which applies a marginal distribution covariance alignment method to align a source data set, a target data set with known labels, and a target data set with unknown labels, to obtain a source data set after first alignment, a target data set with known labels after first alignment, and a target data set with unknown labels after first alignment;

[0218] A conditional distribution mean alignment unit, which applies a conditional distribution mean alignment method to sequentially perform low-dimensional time domain feature extraction, time domain feature migration, and spatial domain feature migration on the samples in the source data set after the first alignment, so that the time domain and spatial domain feature distributions of the samples in the source data set after the first alignment are similar to those of the samples of the corresponding categories in the target data set with known labels after the first alignment;

[0219] The conditional distribution covariance alignment unit applies the conditional distribution covariance alignment method to perform background EEG extraction and background EEG migration in sequence, so that the conditional distribution of the background EEG covariance matrix of each category of the source data set after the first alignment is aligned with the corresponding category in the target data set with known labels after the first alignment;

[0220] The summing unit sums the result obtained by the conditional distribution mean alignment unit and the result obtained by the conditional distribution covariance alignment unit to complete the EEG data migration.

[0221] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the drawings, etc. In the specification, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the specification. Certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0222] Although the present invention has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present invention. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present invention and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present invention. Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, the present invention is intended to include such modifications and variations if they fall within the scope of the present invention and its equivalents.

Claims

1. A time-locked and phase-locked EEG data migration method, characterized in that: The steps include: Configure source dataset, known label target dataset, and unknown label target dataset; Apply the marginal distribution covariance alignment method to align the source data set, the target data set with known labels, and the target data set with unknown labels to obtain the source data set after the first alignment, the target data set with known labels after the first alignment, and the target data set with unknown labels after the first alignment; Applying the conditional distribution mean alignment method to sequentially perform low-dimensional time domain feature extraction, time domain feature migration, and spatial domain feature migration on the samples in the source data set after the first alignment, so that the time domain and spatial domain feature distributions of the samples in the source data set after the first alignment are similar to those of the samples of the corresponding categories in the target data set with known labels after the first alignment, and obtaining a feature set after the conditional distribution mean alignment is completed; Apply the conditional distribution covariance alignment method to perform background EEG extraction and background EEG migration in sequence, so that the conditional distribution of the background EEG covariance matrix of each category in the source data set after the first alignment is aligned with the corresponding category in the target data set with known labels after the first alignment, and obtain the feature set after the conditional distribution covariance alignment is completed; The samples corresponding to the feature set after the conditional distribution mean is aligned and the feature set after the conditional distribution covariance is aligned are summed to complete the EEG data migration.

2. The time-locked and phase-locked EEG data migration method according to claim 1, characterized in that: The marginal distribution covariance alignment method comprises the following sub-steps: Calculate the covariance matrix of the samples in the source data set and the covariance matrix of the samples in the target data set with known labels, and obtain the covariance matrix set corresponding to the source data set and the covariance matrix set corresponding to the target data set with known labels respectively; Calculate the geometric mean of the elements in the covariance matrix set corresponding to the source data set And the geometric mean of the covariance matrix set elements corresponding to the known label target data set Applying the geometric mean Transform the samples in the source dataset to obtain the first aligned source dataset and apply the geometric mean The known label target dataset and the unknown label target dataset are transformed respectively to obtain the known label target dataset after the first alignment and the unknown label target dataset after the first alignment.

3. The time-locked and phase-locked EEG data migration method according to claim 2, characterized in that: The samples in the source dataset are transformed as follows: in, represents the samples in the source dataset after the first alignment; Represents the samples in the source dataset; Represents the source dataset; The known label target dataset and the unknown label target dataset are transformed as follows: in, represents the samples in the target dataset with known labels after the first alignment; is a sample in the target dataset with known labels; It is a target dataset with known labels; For unknown label target dataset.

4. The time-locked and phase-locked EEG data migration method according to claim 1, characterized in that: The low-dimensional time domain feature extraction comprises the following sub-steps: Solve the first spatial filter for each category sample in the source data set after the first alignment and the target data set with known labels after the first alignment; The first spatial filter corresponding to each category of samples is used to extract the common low-dimensional time domain features of each corresponding category of samples in the source data set after the first alignment and the target data set with known labels after the first alignment, so as to obtain the low-dimensional time domain features of the samples in the source data set after the first alignment. Low-dimensional time domain features of samples in the target dataset with known labels after the first alignment 5. The time-locked and phase-locked EEG data migration method according to claim 4, characterized in that: The time domain feature migration includes the following sub-steps: A feature mapping matrix corresponding to each category of samples is configured, and the feature mapping matrix is ​​used to map the low-dimensional time domain features of each category of samples in the source data set after the first alignment to the corresponding category in the target data set with known labels after the first alignment to complete the time domain feature migration.

6. The time-locked and phase-locked EEG data migration method according to claim 5, characterized in that: The feature mapping matrix is: Among them, α is the regularization coefficient; the low-dimensional time domain features of all samples in the source dataset after the first alignment Form a collection, is the mean of the covariance matrix of the i-th sample, is the mean of the i-th class samples; the low-dimensional time domain features of all samples in the target dataset with known labels after the first alignment Form a collection, is the mean of the samples in the i-th category; is the identity matrix.

7. The time-locked and phase-locked EEG data migration method according to claim 6, characterized in that: The spatial feature migration includes the following sub-steps: Solve the spatial migration matrix of each category of samples; Each category's spatial domain migration matrix is ​​used to transform each corresponding category sample after completing the time domain feature migration to obtain a feature set after the conditional distribution mean is aligned.

8. The time-locked and phase-locked EEG data migration method according to claim 1, characterized in that: Background EEG extraction includes the following sub-steps: Solve the second spatial filter corresponding to each category sample in the source data set after the first alignment; Using the second spatial filter to perform spatial filtering on each category of samples in the source data set after the first alignment, so as to obtain low-dimensional time domain features of each category of samples; Calculate the spatial domain migration matrix corresponding to each category of samples in the source data set after the first alignment, and use each category of spatial domain migration matrix to transform each corresponding category of samples of the low-dimensional time domain features obtained in the previous step to obtain the task-related time domain features of each sample in the source data set after the first alignment; Calculate the background EEG of each sample in the source dataset after the first alignment Right now in, Represents the sample with sequence number j in the source dataset after the first alignment; Represents the task-related time domain features of the sample with sequence number j in the source dataset after the first alignment.

9. The time-locked and phase-locked EEG data migration method according to claim 8, characterized in that: Background EEG migration was performed as follows: in, is the background EEG after migration; U i is the transformation matrix, in, Task-related temporal features of all samples in the source dataset after first alignment Form a collection, is the mean of the samples in the i-th category; is the mean of the covariance matrix of the i-th sample in the first aligned known label target dataset; the background EEG of all samples in the source dataset after the first alignment Constitute a set, Q is the mean of the covariance matrix of the i-th sample in the set.

10. A time-locked and phase-locked EEG data migration system, characterized in that: include: A storage unit, used to store a source data set, a target data set with known labels, and a target data set with unknown labels; A marginal distribution covariance alignment unit, which applies a marginal distribution covariance alignment method to align a source data set, a target data set with known labels, and a target data set with unknown labels, to obtain a source data set after first alignment, a target data set with known labels after first alignment, and a target data set with unknown labels after first alignment; A conditional distribution mean alignment unit, which applies a conditional distribution mean alignment method to sequentially perform low-dimensional time domain feature extraction, time domain feature migration, and spatial domain feature migration on the samples in the source data set after the first alignment, so that the time domain and spatial domain feature distributions of the samples in the source data set after the first alignment are similar to those of the samples of the corresponding categories in the target data set with known labels after the first alignment; The conditional distribution covariance alignment unit applies the conditional distribution covariance alignment method to perform background EEG extraction and background EEG migration in sequence, so that the conditional distribution of the background EEG covariance matrix of each category of the source data set after the first alignment is aligned with the corresponding category in the target data set with known labels after the first alignment; The summing unit sums the result obtained by the conditional distribution mean alignment unit and the result obtained by the conditional distribution covariance alignment unit to complete the EEG data migration.

Images