An electroencephalogram transfer learning classification method and system based on an aligned reference source domain

Abstract

The application belongs to the field of electroencephalogram signal processing and classification, and particularly discloses an electroencephalogram transfer learning classification method and system based on an aligned reference source domain, which comprises the following steps: obtaining electroencephalogram signals of multiple subjects, calculating the Euclidean distance between each electroencephalogram data for each subject, taking the subject with high intra-class similarity and low inter-class similarity as a reference source domain, and taking the remaining subjects as a target domain to construct a training set; training a classification model based on the reference source domain through the training set, wherein the classification model comprises a feature extractor, a feature resolver and a classifier, the feature extractor extracts reference standard features and target domain features according to the electroencephalogram data of the reference source domain and the target domain; the feature resolver performs feature unwinding on the target domain features to obtain class-related features and domain-invariant features, so that the class-related features are aligned to the reference standard features; and the classifier determines the class of the electroencephalogram data of the target domain according to the class-related features. The application can effectively improve the classification accuracy of electroencephalogram signals.

Description

Technical Field

[0001] This invention belongs to the field of electroencephalogram (EEG) signal processing and classification, and more specifically, relates to an EEG transfer learning classification method and system based on aligned reference source domains. Background Technology

[0002] Brain-computer interfaces (BCIs) establish a direct interface between the brain and external devices without relying on surrounding muscles or nerves. In recent years, non-invasive electroencephalography (EEG) has become one of the most widely used methods in BCI systems due to its high temporal resolution and portability. EEG can be divided into evoked and spontaneous EEG based on its generation method. Evoked EEG (such as P300 and steady-state visual evoked potentials) relies on external stimuli and ignores the user's spontaneous will, while spontaneous EEG (such as motor imagery, MI) requires no external stimulation; users simply imagine body movements, making it more private and easier to accept. This has led to widespread attention on MI in the development of EEG-based BCIs.

[0003] Decoding EEG signals can identify a user's motor intentions, helping them control physical or virtual devices and thus improving their ability to communicate and interact with the outside world. This novel information interaction technology has been successfully applied in fields such as robot control, human-computer interaction, and medical rehabilitation. However, the high inter-trial variability, low signal-to-noise ratio, and sensitivity to noise in EEG signals make decoding motor intentions difficult, and the scarcity of data further complicates the task.

[0004] Considering the subjects as domains, models can be trained using data from multiple domains through transfer learning. However, since EEG is composed of inherent background neural activity and task-state EEG, there are significant distributional differences in the feature space. Mitigating this domain bias is a crucial step in EEG classification task transfer learning. Furthermore, typical transfer learning focuses on learning feature representations of the target domain while neglecting the class correlation distribution in the source domain, which can significantly lead to negative transfer to the target domain. Summary of the Invention

[0005] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a brainwave transfer learning classification method and system based on aligned reference source domain, the purpose of which is to improve the classification accuracy of brainwave signals.

[0006] To achieve the above objectives, according to one aspect of the present invention, a brainwave transfer learning classification method based on aligned reference source domains is proposed, comprising the following steps:

[0007] Model training phase:

[0008] EEG signals were acquired from multiple subjects, each subject's EEG signals including multiple categories of known EEG data;

[0009] For each subject's EEG signal, the Euclidean distance between each EEG data point is calculated to measure the similarity of the EEG data. Subjects with high intra-class similarity and low inter-class similarity are used as the reference source domain; the remaining subjects are used as the target domain, and a training set is constructed based on their EEG data and corresponding categories.

[0010] Based on the reference source domain, a classification model is trained using a training set to obtain a trained classification model. The classification model includes a feature extractor F, a feature decomposer D, and a classifier C. The feature extractor F is used to extract reference standard features from the EEG data of the reference source domain and the target domain, respectively. and target domain features Feature decomposer D is used to decompose features in the target domain. Perform feature unwrapping to obtain class-related features f cr With domain-invariant feature f di and make class-related features f cr To reference standard features Alignment; Classifier C is used to align based on class-related features f cr Determine the category of EEG data in the target domain;

[0011] Model application phase:

[0012] The EEG data to be classified is input into a trained classification model to obtain the category to which the EEG data belongs.

[0013] As a further preferred option, the following overall loss function L is used when training the classification model. RSTLNet (θ,δ,η):

[0014]

[0015] Among them, L cls Let L be the cross-entropy loss function. CORAL Let L be the second-order central moment loss function. DCCA Let α be the overall correlation loss function, and β1 and β2 be weighted hyperparameters.

[0016] As a further preferred option, the second-order central moment loss function L... CORAL The calculation formula is:

[0017]

[0018] in, Let F represent the Frobenius norm of the variance matrix, d represent the number of output neurons in the feature extractor F, and C represent the number of output neurons in the feature extractor F. r C t These represent the feature covariance matrices of the reference source domain and the target domain, respectively, determined based on the features of the reference standard and the target domain.

[0019] As a further preferred option, the overall correlation loss function L... DCCA The calculation formula is:

[0020]

[0021] Where, σ k (T) represents the k-th singular value of matrix T, where K represents the number of singular values ​​used to calculate matrix T. Matrix T is defined as: This represents the covariance matrix of the reference source domain EEG. This represents the covariance matrix between the source domain EEG and the target domain EEG. The covariance matrix of the target domain EEG; w S w T Let represent the affine transformation matrices of the reference source domain and the target domain, respectively; These represent the corresponding optimal affine transformation matrices, represents the parameters of the target domain network to be optimized; 1 represents a column vector in which all elements are equal to 1.

[0022] As a further preferred option, the cross-entropy loss function L... cls The calculation formula is:

[0023]

[0024] Where, N b This represents the number of samples in each mini-batch during training. This represents the i-th input EEG data in the target domain. The one-hot tag vector, D δ (*) (1) The class-related features f decomposed by feature decomposer D are represented by... cr F θ (*) indicates a feature extractor with parameters, C η (*) indicates a classifier with parameters; θ, δ, and η represent the network parameters of the feature extractor F, feature decomposer D, and classifier C, respectively.

[0025] As a further preferred option, the weighted hyperparameters are α = 1.0, β1 = 0.01, and β2 = 2.0.

[0026] As a further optimization, for each subject's EEG data, the Euclidean distance between each EEG data point is calculated to measure the distribution of the subject's EEG data. Subjects with high intra-class similarity and low inter-class similarity are used as the reference source domain. Specifically:

[0027] For each subject, each acquired EEG data point is represented as a spatial covariance matrix. The Euclidean distance between each spatial covariance matrix is ​​calculated to determine the distribution evaluation index rCD for that subject. The subject with the largest rCD value is taken as the reference source domain.

[0028] The distribution evaluation index rCD = CHS / DBI, where CHS represents the Calinski-Harabasz index and DBI represents the Davies-Bouldin index.

[0029] As a further preferred embodiment, the feature extractor F includes a spatial convolution module, a temporal convolution module, and a feature fusion module. The spatial convolution module is used to acquire the spatial features of the EEG data, the temporal convolution module is used to acquire the temporal features of the EEG data, and the feature fusion module is used to fuse the spatial and temporal features to obtain and output the spatiotemporal hybrid features.

[0030] As a further preferred option, the feature decomposer D uses pointwise convolution to unwrap the features, which includes a one-dimensional convolution kernel of size 1, a batch normalization operation, and a ReLU activation function.

[0031] According to another aspect of the present invention, an EEG transfer learning classification system based on an aligned reference source domain is provided, comprising a processor for executing the above-described EEG transfer learning classification method based on an aligned reference source domain.

[0032] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages:

[0033] 1. This invention avoids the negative transfer effect caused by poor source domains by selecting reference source domains with high intra-class similarity and low inter-class similarity. At the same time, it promotes the alignment and migration of class-related features of the target domain to reference standard features, making the overall data distribution more similar and effectively improving the classification accuracy of EEG signals.

[0034] 2. Quantitatively assess the data distribution of each subject in the Riemann space, and select reference subjects with high intra-class similarity and low inter-class similarity through the rCD index. This allows for the optimal selection of subjects and the acquisition of reference source subjects with better data distribution. This avoids the negative transfer effect of source domains with poor data distribution on the target domain and greatly improves the transfer effect on the target domain.

[0035] 3. This invention enriches class discrimination features by untangling class-related features, and adopts an alignment transfer learning strategy that combines covariance calibration based on second-order statistics with deep canonical correlation analysis of the overall data to promote the alignment of class-related features with reference standards. Specifically, this is manifested in closer center distances and more similar overall data distribution, thereby improving classification accuracy. Attached Figure Description

[0036] Figure 1 This is a flowchart of the EEG transfer learning classification method based on aligned reference source domains according to an embodiment of the present invention;

[0037] Figure 2 Figures (a) to (d) are schematic diagrams of feature visualization in the BCIC IV IIa dataset according to embodiments of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0039] This invention provides a brainwave transfer learning classification method based on aligned reference source domains, such as... Figure 1 As shown, it includes the following steps:

[0040] (1) Determine the reference source domain;

[0041] EEG signals from multiple subjects were acquired. Each subject's EEG signals included multiple known categories of EEG data, i.e., N EEG data points were acquired for each subject, representing K categories. The EEG data of each subject were analyzed, and Euclidean distance was calculated in Euclidean space. The recalculated distribution coefficient (rCD) was used to identify high-performing subjects with high intra-class similarity and low inter-class similarity, and these subjects were selected as the reference source domain. The remaining subjects were used as the target domain, and a training set was constructed based on their EEG data and corresponding categories; 70% of the EEG data could be used as the training set, and the remaining 30% as the test set.

[0042] Furthermore, in step (1), for any subject, the spatial covariance matrix is ​​used... Σi To encode and represent EEG data, that is: in This represents the EEG data matrix of the i-th EEG test, where m is the number of EEG channels, and T... s The data length in a single EEG test is given; then the covariance matrices Σ1,...,Σ of the N tests are calculated. N Euclidean average MR for:

[0043]

[0044] in Represents different covariance matrices (Σ) A ,Σ B The Euclidean distance between ) and ajh It is a matrix Σ A The element in the j-th row and h-th column, b jh It is a matrix Σ B The element in the j-th row and h-th column.

[0045] The Calinski-Harabasz Score (CHS) and Davies-Bouldin Index (DBI) are used to evaluate the two Euclidean spaces for the spatial covariance matrix. CHS quantifies the clustering effect by the ratio of inter-class distance to intra-class distance (the higher the CHS, the better the clustering effect). DBI measures the degree of separation between classes by the ratio of intra-class diameter to inter-class distance (the lower the DBI, the better the separation effect).

[0046] The specific expressions of these two indicators in Euclidean space are as follows:

[0047]

[0048] Where K is the number of classes, and n is the total number of samples. k Let C be the number of samples in the k-th class. k Let C represent all samples of class k. h Let n represent all samples of class h. h c represents the number of samples in class h. k Indicates that through M R The calculated k-th class center, c E Indicates that through M R The calculated global center, c h Indicates that through M R The calculated h-th class center.

[0049] Based on CHS and DBI, a distribution evaluation index rCD is further constructed:

[0050] rCD = CHS / DBI

[0051] A higher rCD reflects greater intra-class similarity and inter-class dispersion. Based on this, we identify high-performing participants with high intra-class similarity and low inter-class similarity (i.e., participants with the highest rCD) and use them as the reference source domain.

[0052] In this embodiment, for the nine participants in the BCIC IV IIa dataset, the rCD metric reflects that participant A03 has a better original data distribution. The rCD metric for participant A03... A03 =8.565) compared to A02(rCD) A02 =0.046) is several times higher. Therefore, subject A03 is set as the reference standard source domain.

[0053] (2) Train the classification model;

[0054] Based on the reference source domain, the classification model is trained using the training set to obtain a trained classification model. The classification model includes a feature extractor F, a feature decomposer D, and a classifier C, as detailed below:

[0055] (2-1) Extract reference standard features from the reference source domain and target domain networks respectively using feature extractor F. and target domain features

[0056] Furthermore, the feature extractor F comprises three modules: a spatial convolution module, a temporal convolution module, and a feature fusion module. These are all sequence processing modules based entirely on simple and compact 1D convolutional kernels. Specifically, the spatial convolution module starts with conventional one-dimensional convolution and performs batch normalization; the temporal convolution module uses depthwise convolution to filter temporal signals, employs batch normalization and ReLU activation functions to further process temporal features, and applies average pooling layers and dropout layers to improve the sparsity of the data representation; the feature fusion module uses depthwise separable convolution to learn more effective spatiotemporal hybrid features, performs optimal feature combinations through pointwise convolution, uses average pooling layers for dimensionality reduction, and performs flattening operations to achieve feature extraction.

[0057] Finally, reference standard features were extracted from the EEG data of the source and target domains using feature extractor F. and target domain features Right now:

[0058]

[0059] in, and Let θ represent the EEG data from the reference source domain and the target domain, respectively, and let θ represent the network parameters of the feature extractor F.

[0060] (2-2) Target domain features Feature decoupling is performed using feature decomposer D to obtain class-related features f. cr With domain-invariant feature f di The introduction of covariance calibration based on second-order statistics and deep canonical correlation analysis of the overall data makes the class correlation feature f cr To reference standard features Alignment.

[0061] Furthermore, the feature decomposer D employs pointwise convolution for feature unwrapping, consisting of a one-dimensional convolution kernel of size 1, batch normalization, and a ReLU activation function. This unwrappes features from the target domain. Decompose to obtain class-related features f crWith domain-invariant feature f di ,Right now:

[0062]

[0063] Where δ represents the network parameters of the feature decomposer D.

[0064] (2-3) Class-related features f cr The target domain EEG signals are classified using classifier C.

[0065] Furthermore, when training the classification model, all losses are jointly optimized, and all network parameters are trained in an end-to-end manner. The overall objective function during training is expressed as:

[0066]

[0067] Among them, L cls Let L be the cross-entropy loss function. CORAL Let L be the second-order central moment loss function. DCCA Let be the overall correlation loss function, and α, β1, and β2 be the weighted hyperparameters for the contribution of each loss function.

[0068] Specifically, for the second-order central moment loss function and the overall correlation loss function:

[0069] The reference standard features obtained through feature extractor F Represented in matrix form The class-related features f after decoupling the feature decomposer D cr Represented in matrix form Where d is the number of output neurons of the feature extractor F, which is the same as the number of output neurons of the feature decomposer D. and These represent the number of samples in a mini-batch of the reference source domain and the target domain, respectively (generally speaking). By combining covariance calibration based on second-order statistics with deep canonical correlation analysis of the overall data, the class correlation feature f is made more accurate. cr To reference standard features Alignment:

[0070] In order to minimize the feature sample set of the reference source domain Feature sample set related to the target domain The second-order statistic distance is used, and CORAL is introduced as the distance loss function. The second-order central moment loss function is:

[0071]

[0072] in, Let C represent the Frobenius norm of the variance matrix. The eigencovariance matrices of the source and target domains are referenced. r and C t It is given by the following formula:

[0073]

[0074] Where 1 is a column vector in which all elements are equal to 1.

[0075] To maximize the feature sample set of the reference source domain Feature sample set related to the target domain To assess the overall correlation, we further introduce correlation coefficient loss using deep canonical correlation analysis (DCCA). Let the central data matrix of the reference source domain be... (Similarly, the target domain) Canonical correlation analysis (CCA) is used to find linear projection pairs between two matrices. This makes them have the highest correlation:

[0076]

[0077] in and r is the regularization constant r (same as) Assume r r >0, therefore It is positive definite. The goal of DCCA is to learn the parameters τ of the target domain network. T and w r and w t The affine transformation makes To maximize the value, the above equation can be rewritten as the following constraint optimization formula:

[0078]

[0079] According to the solution method of CCA singular value decomposition, and The total correlation of the first k components is a matrix The sum of the first k singular values ​​is the sum of the first k singular values. Therefore, the goal of DCCA is to maximize the sum of the first k singular values ​​of matrix T by minimizing the following loss function, resulting in the overall correlation loss function:

[0080]

[0081] Where σ k (T) is the k-th singular value of matrix T. This represents the covariance matrix of the reference source domain EEG. This represents the covariance matrix between the source domain EEG and the target domain EEG. The covariance matrix of the target domain EEG; w S w T Let represent the affine transformation matrices of the reference source domain and the target domain, respectively.

[0082] Specifically, for the cross-entropy loss function:

[0083] Class-related features f of the target domain cr Used as input to the target domain classification layer C, the classifier is trained by minimizing the cross-entropy loss:

[0084]

[0085] in, Indicates target domain input EEG The one-hot tag vector, D δ (*) (1) The class-related features f decomposed by feature decomposer D are represented by... cr η represents the network parameters of classifier C.

[0086] (3) Classification of EEG data;

[0087] The EEG data to be classified is input into a trained classification model to obtain the category to which the EEG data belongs.

[0088] The effects of the present invention are verified through specific embodiments below:

[0089] The intermediate layer features are outlined, and the t-SNE algorithm is also used to visualize the high-dimensional embedded space of each layer, thereby visualizing the process of the network learning the distribution of brain features. Figure 2 As shown, (a) illustrates the relatively disordered EEG features after the spatial convolution module; (b) shows that after processing by the temporal convolution module, data of the same class become closer together; (c) shows the more compact EEG features of data of the same class after the feature fusion module; and (d) shows the EEG features obtained after processing by the feature decomposer D combined with an alignment transfer learning strategy based on covariance calibration of second-order statistics and deep canonical correlation analysis of the overall data. This makes the distance between different classes more separated, highlighting the effectiveness of the proposed EEG transfer learning classification method based on aligned reference source domain in reducing intra-class distance and increasing inter-class distance. Ultimately, a four-class classification accuracy of 83.25% was achieved.

[0090] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A brainwave transfer learning classification method based on aligned reference source domains, characterized in that, Includes the following steps: Model training phase: EEG signals were acquired from multiple subjects, each subject's EEG signals including multiple categories of known EEG data; For each subject's EEG signal, the Euclidean distance between each EEG data point is calculated to measure the similarity of the EEG data. Subjects with high intra-class similarity and low inter-class similarity are used as the reference source domain; the remaining subjects are used as the target domain, and a training set is constructed based on their EEG data and corresponding categories. The specific method for determining the reference source domain is as follows: For each subject, each acquired EEG data point is represented as a spatial covariance matrix, and the Euclidean distance between each spatial covariance matrix is ​​calculated to determine the distribution evaluation index for that subject. rCD ;Pick rCD The subject with the largest value is used as the reference source domain; the distribution evaluation index , CHS This represents the Calinski-Harabasz index. DBI This represents the Davies-Bouldin index; Based on the reference source domain, the classification model is trained using a training set to obtain a trained classification model; the classification model includes a feature extractor. F Feature decomposer D and classifier C Among them, feature extractor F This is used to extract reference standard features from EEG data of the reference source domain and the target domain, respectively. and target domain features Characteristic decomposer D Used for target domain features Perform feature unwrapping to obtain class-related features. Domain-invariant features and make class-related features To reference standard features Alignment; Classifier C Used based on class-related features Determine the category of EEG data in the target domain; Specifically, this will be achieved through a feature extractor. F The obtained reference standard features Represented in matrix form Feature decomposer D Decoupled class-related features Represented in matrix form ,in d It is a feature extractor F The number of output neurons, and the feature decomposer D The number of output neurons is the same. and Let represent the number of samples in a mini-batch of the reference source domain and the target domain, respectively; combining covariance calibration based on second-order statistics with deep canonical correlation analysis of the overall data, the class correlation characteristics are... To reference standard features Alignment; In order to minimize the feature sample set of the reference source domain Feature sample set related to the target domain The second-order statistical distance is used, and CORAL is introduced as the distance loss. To maximize the feature sample set of the reference source domain Feature sample set related to the target domain To assess the overall correlation, we further introduce deep canonical correlation analysis (DCCA) to determine the correlation coefficient loss; the corresponding overall correlation loss function is... The calculation formula is: in, Representation matrix T The k Large singular values, K denotes the matrix T The matrix T used to calculate the number of singular values ​​is defined as follows: , This represents the covariance matrix of the reference source domain EEG. This represents the covariance matrix between the source domain EEG and the target domain EEG. Represents the covariance matrix of the target domain EEG; , Let represent the affine transformation matrices of the reference source domain and the target domain, respectively; , These represent the corresponding optimal affine transformation matrices, This represents the parameters of the target domain network to be optimized; 1 represents a column vector in which all elements are equal to 1. Model application phase: The EEG data to be classified is input into a trained classification model to obtain the category to which the EEG data belongs.

2. The EEG transfer learning classification method based on aligned reference source domain as described in claim 1, characterized in that, When training the classification model, the following overall loss function is used. : in, Let cross-entropy be the loss function. It is the second-order central moment loss function. The overall correlation loss function is... , , For weighted hyperparameters; , , These represent feature extractors. F Feature decomposer D、 Classifier C Network parameters.

3. The EEG transfer learning classification method based on aligned reference source domain as described in claim 2, characterized in that, Second-order central moment loss function The calculation formula is: in, The Frobenius norm represents the variance matrix. d Feature extractor F The number of output neurons, , These respectively represent the characteristics according to the reference standard. The reference source domain and the feature covariance matrix of the target domain are determined by the characteristics of the target domain.

4. The EEG transfer learning classification method based on aligned reference source domain as described in claim 2, characterized in that, Cross-entropy loss function The calculation formula is: in, This represents the number of samples in each mini-batch during training. Indicates the target domain. i Input EEG data One-hot label vectors, Characteristic decomposer D Decomposition of class-related features , This represents a feature extractor with parameters. This represents a classifier with parameters.

5. The EEG transfer learning classification method based on aligned reference source domain as described in claim 2, characterized in that, Weighted hyperparameters , , .

6. The EEG transfer learning classification method based on aligned reference source domain as described in claim 1, characterized in that, The feature extractor F It includes a spatial convolution module, a temporal convolution module, and a feature fusion module. The spatial convolution module is used to obtain the spatial features of EEG data, the temporal convolution module is used to obtain the temporal features of EEG data, and the feature fusion module is used to fuse spatial and temporal features to obtain spatiotemporal hybrid features and output them.

7. The EEG transfer learning classification method based on aligned reference source domains as described in any one of claims 1-6, characterized in that, Feature decomposer D Feature unwrapping is performed using pointwise convolution, which includes a one-dimensional convolution kernel of size 1, batch normalization, and the ReLU activation function.

8. A brainwave transfer learning classification system based on aligned reference source domains, characterized in that, Includes a processor for executing the EEG transfer learning classification method based on aligned reference source domains as described in any one of claims 1-7.

Images