Multi-source cross-subject motor imagery electroencephalogram signal classification method based on depth domain adaptation

Abstract

The invention relates to the technical field of motor imagery electroencephalogram signal decoding, in particular to a multi-source cross-subject motor imagery electroencephalogram signal classification method based on depth domain adaptation, and aims to solve the problem of large data distribution difference between different subjects, the method comprises the following steps: firstly, preprocessing electroencephalogram signals and aligning Euclidean spatial data; secondly, key features are extracted by adopting a regularization common space mode method, and signal feature representation is optimized; and finally, constructing a deep learning model based on a multi-layer perceptron, and respectively aligning marginal distribution and category information in combination with the central moment difference loss and the contrast domain difference loss to realize high-precision cross-subject classification. The method is suitable for brain-computer interface scenes such as exercise rehabilitation and auxiliary equipment control, and has high robustness and practicability.

Description

Technical Field

[0001] The present invention relates to the technical field of motor imagery electroencephalogram signal decoding, and particularly to a multi-source cross-subject motor imagery electroencephalogram signal classification method based on deep domain adaptation. Background Technique

[0002] A Brain-Computer Interface (BCI) is an innovative technology that directly connects the human brain with a computer or other electronic devices to enable communication between the human brain and external devices. By bypassing the traditional neuromuscular pathway, BCI directly reads and interprets the user's brain activities, allowing the brain to directly communicate and interact with external devices. This technology provides higher autonomy and quality of life for individuals with limited mobility and has broad application prospects in fields such as medicine, rehabilitation, and human-computer interaction.

[0003] BCI systems typically use a non-invasive and portable method to acquire brain signals, namely electroencephalography (EEG). EEG collects inhibitory and excitatory postsynaptic potential signals generated by cortical nerve cell activities by placing an electrode array on the scalp surface. These signals reflect the neural activity patterns in specific regions of the brain. EEG has the advantages of being non-invasive, safe, and portable during the signal acquisition process, making it one of the mainstream technologies in current BCI research and applications. A typical BCI system includes multiple core modules such as signal acquisition, preprocessing, feature extraction, classification, and control. First, the system acquires EEG signals through the electrode array, and then removes interference signals such as electromyographic noise and eye movement artifacts through preprocessing techniques such as filtering and denoising. Next, the system extracts key features related to the task and classifies them through machine learning algorithms to translate these features into instructions that external devices can execute. Finally, these instructions are used to control external devices such as wheelchairs, robotic arms, or computer interfaces. In a BCI system, whether the model can learn from the EEG data of one subject and accurately identify the brain activities of another subject mainly depends on the ability of the machine learning model to handle the distribution differences between the training and test data. Most supervised learning algorithms are based on a simplified assumption that the training data and test data are independently and identically distributed. However, in real-world applications, the neural activity differences and different attention levels among individuals make the EEG signals show significant variability, making it difficult to transfer the data collected from one subject to another. Therefore, when a traditional learning model is trained in one scenario and tested in another scenario, its performance is often affected, and this challenge is even a test for deep learning models.

[0004] To improve the reliability of the MI-BCI system in daily use, researchers have proposed transfer learning methods. As a common machine learning technique, transfer learning trains the system in situations with limited data. Its goal is to extract useful knowledge from other domains and transfer it to the current task. Different from other machine learning methods, transfer learning does not require the training and test data to be exactly the same. The flexibility of this method allows it to use data from similar backgrounds for training without having to prepare a new training set every time there is a change in the data distribution. Effectively using transfer learning can not only improve the classification accuracy but also significantly reduce the calibration time required for the BCI system.

[0005] Due to the uniqueness of each person's brain structure and the non-stationary characteristics of brain signals, transfer learning in brain-computer interfaces is no small matter. It is these challenging characteristics that have promoted the development of domain adaptation techniques. As a branch of statistical learning theory, domain adaptation takes into account the distribution changes between the training data and the test data, corresponding to the source domain and the target domain respectively. The idea of domain adaptation is that although the source domain and the target domain come from different individuals, they are consistent in terms of the nature of the task. Therefore, the rich labeled information in the source domain can be used to guide the training of the classifier in the target domain, especially when the labeled data in the target domain is scarce or completely missing. This method not only helps to shorten the calibration time of the BCI system but also improves the accuracy and reliability of cross-subject applications.

[0006] The deficiencies of existing domain adaptation methods include:

[0007] First, previous methods usually only focus on the alignment of the overall distributions between the source domain and the target domain, and achieve transfer by minimizing the distribution difference between domains. However, this method ignores the importance of class information, which may lead to incorrect alignment of the features of different classes in the target domain, thereby having an adverse impact on the classification performance.

[0008] Second, when using a multi-layer perceptron as the backbone network, existing methods often do not fully consider the mutual relationships and interaction information between features. The features in electroencephalogram signals have complex temporal dependencies and spatial correlations. Simply processing them through a simple fully connected layer may lose these important information structures, thus reducing the model's ability to express complex signal patterns and classification performance.

[0009] Third, in the past, all source domains were merged into a single source domain and then domain adaptation was used for distribution alignment. It ignores the non-stationarity of each brain source domain itself, causing interference (that is, the electroencephalogram data of different people follows different marginal distributions), and directly merging them into a new source domain makes it impossible to determine whether its new marginal distribution still follows the electroencephalogram data distribution, thus bringing a large deviation. Summary of the Invention

[0010] To solve the technical problem of poor accuracy in electroencephalogram (EEG) signal classification, the present invention proposes a multi-source cross-subject motor imagery EEG signal classification method based on deep domain adaptation. First, the collected EEG signal data is preprocessed, including removing noise and artifacts, to improve the signal quality and enhance the reliability of the data. Subsequently, the EA alignment technique is applied to the preprocessed EEG signals to reduce the distribution differences between different subjects and improve the comparability of the data. On this basis, the RCSP method is used to extract key features to further optimize the signal representation and retain the core information of the motor imagery task. Based on the extracted features, a deep learning network model based on a multi-layer MLP is constructed, and the multi-source domain adaptation technique is combined to achieve the cross-subject classification task. To effectively address the distribution differences between different subjects, two loss functions are designed in the network: First, the central moment difference loss, which is used to reduce the marginal distribution differences between the source domain and the target domain and align the feature distributions as a whole; Second, the contrast domain difference loss, which is used to narrow the class differences between the same-class samples in the source domain and the target domain, thus ensuring the alignment at the class level. To achieve the above object, the technical solution adopted by the present invention is as follows:

[0011] Obtain motor imagery EEG data, preprocess the EEG data, and obtain the preprocessed EEG signal data;

[0012] Perform Euclidean space data alignment on the preprocessed EEG signal data;

[0013] Extract features from the aligned data using the RCSP technique;

[0014] Construct an MLP network model;

[0015] Based on the constructed MLP network model, train the source domain data, classify the features of the target subject, and achieve the classification of the motor imagery EEG signals of the target subject.

[0016] Optionally, the preprocessing of the EEG data using the EEGLAB toolbox includes:

[0017] Use a causal 50th-order 8 - 30 Hz finite impulse response bandpass filter to remove muscle artifacts and DC drift;

[0018] Extract the EEG signals from [0.5, 3.5] seconds after the cue appears as trials.

[0019] Optionally, the Euclidean space data alignment of the preprocessed EEG signal data includes:

[0020] Assume that the subject has performed n trials, and X i represents the EEG signal of the i-th experiment, and its corresponding calculation formula is:

[0021]

[0022] is the arithmetic mean of all covariance matrices of a subject, and alignment is performed. The corresponding formula is:

[0023]

[0024] After alignment, the average covariance matrix of all n aligned trials is:

[0025]

[0026] The mean of the covariance matrices of all subjects is equal after alignment with the identity matrix.

[0027] Optionally, extracting features from the aligned data using the RCSP technique includes:

[0028] Using X i ∈R N×S to represent the electroencephalogram signal of the i-th experiment, where N and S represent the number of channels and the number of samples respectively; calculating the normalized covariance matrix and the pairwise covariance matrix The corresponding formulas are:

[0029]

[0030] tr(·) is a function for calculating the trace of a matrix, and conv(·) is a function for calculating the pairwise covariance of each channel;

[0031] The regularized average spatial covariance matrix The corresponding formula is:

[0032]

[0033] where 0 ≤ α, β ≤ 1, m is the number of training experiments, and I is the identity matrix;

[0034] The composite covariance matrix and its eigenvalue decomposition are:

[0035]

[0036] where U0 is the eigenvector matrix, λ is the eigenvalue of the composite covariance matrix and U0 is arranged in descending order according to the corresponding eigenvalue λ; the whitening transformation matrix is:

[0037]

[0038] RCSP is extracted based on the diagonalization of the whitened covariance matrix, and the corresponding formula is:

[0039]

[0040] Among them, S c1 and S c2 are similarly decomposed to obtain the same feature matrix, and the corresponding formula is:

[0041] S c1 = Bλ c1 B T , S c2 = Bλ c2 B T , λ c1 + λ c2 = 1.

[0042] The calculation formula corresponding to the projection matrix of RCSP is:

[0043] W = B T P,

[0044] Select the first n columns and the last n columns to form the projection matrix W 2n , and the eigenvector X i of a single experiment is transformed into the following form:

[0045] f i = log(Var(W 2n X i ))

[0046] Optionally, the construction of the MLP network model includes:

[0047] Using the extracted RCSP to input into a 3-layer MLP network to further extract the common features among subjects;

[0048] Input the feature cross-pooling layer, and the features interact pairwise;

[0049]

[0050] Using the features of the feature cross-pooling layer for input into the hidden layer, and the hidden layer learns high-order feature interactions in a non-linear manner to further learn deep features;

[0051] z1 = σ1(W1f BI (v x ) + b1)

[0052] z2 = σ2(W2z2 + b2)

[0053] …

[0054] z L = σ L (W L z L-1 + b L )

[0055] Among them, L represents the number of hidden layers, W i , b i , σ i respectively represent the weight matrix, bias vector, and activation function of the L-th layer.

[0056] Optionally, training the source domain data with the constructed MLP network model and classifying the features of the target subject includes:

[0057] During the training process, introducing the central moment difference loss function to measure the difference in the marginal distributions of the overall data between the source domain and the target domain;

[0058] Assume X i and X j are two feature samples, and their corresponding formula is:

[0059]

[0060] Among them, E is the expectation of X, c k is the k-th central moment;

[0061] On the basis of achieving the overall distribution alignment, further reducing the distribution difference between the source domain and the target domain at the class level through the contrastive domain discrepancy loss function (CDD). This loss function calculates the distribution similarity of each class in the feature space and aligns the feature distributions of the source domain and the target domain category by category. By optimizing the consistency of the feature distributions at the class level, the classification accuracy of the target domain samples is further improved.

[0062] For any two classes c1 and c2, the formula corresponding to the distance between them is:

[0063]

[0064] Among them,

[0065] Among them, g(·) is the kernel function, G θ is the network feature extractor; when c1 = c2, it measures the difference in the same subdomain; when c1 ≠ c2, it measures the difference between the two subdomains, and the formula corresponding to the LSD loss is:

[0066]

[0067] The formula corresponding to the overall loss function is:

[0068] L total = L CLS + L CMD + L CDD+L CENT ,

[0069] where L CLS is the cross-entropy loss function, and L CENT is the conditional entropy loss function.

[0070] The present invention has the following beneficial effects:

[0071] The multi-source cross-subject motor imagery EEG signal classification method based on deep domain adaptation of the present invention first preprocesses the collected EEG signal data, including removing noise and artifacts, to improve the signal quality and enhance the reliability of the data. Subsequently, the EA alignment technology is applied to the preprocessed EEG signals to reduce the distribution differences between different subjects and improve the comparability of the data. On this basis, the RCSP method is used to extract key features to further optimize the signal representation and retain the core information of the motor imagery task. Based on the extracted features, a deep learning network model based on a multi-layer MLP is constructed, and the multi-source domain adaptation technology is combined to achieve the cross-subject classification task. To effectively address the distribution differences between different subjects, two loss functions are designed in the network: First, the central moment difference loss, which is used to reduce the marginal distribution differences between the source domain and the target domain and align the feature distributions as a whole; Second, the contrast domain difference loss, which is used to narrow the class differences between the same-class samples in the source domain and the target domain, thus ensuring the alignment at the class level. In summary, the accuracy of EEG signal classification is improved. Description of the Drawings

[0072] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0073] Figure 1 is a flowchart of a multi-source cross-subject motor imagery EEG signal classification method based on deep domain adaptation of the present invention;

[0074] Figure 2 is a schematic diagram showing the subject accuracy of the present invention on the dataset BCI IV 1;

[0075] Figure 3 is a schematic diagram showing the subject accuracy of the present invention on the dataset BCI IV 2a;

[0076] Figure 4 is a detailed flowchart of the overall method of the present invention;

[0077] Figure 5Schematic diagram for comparison between the present invention and prior methods. Detailed implementation manners

[0078] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation manners, structures, features and their effects of the technical solutions proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0079] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0080] Reference Figure 1 , shows the flow of some embodiments of a multi-source cross-subject motor imagery EEG signal classification method based on deep domain adaptation according to the present invention. The multi-source cross-subject motor imagery EEG signal classification method based on deep domain adaptation includes the following steps:

[0081] Step S1, acquire motor imagery EEG data, preprocess the EEG data, and obtain the preprocessed EEG signal data.

[0082] In some embodiments, motor imagery EEG data can be acquired, the EEG data can be preprocessed, and the preprocessed EEG signal data can be obtained.

[0083] The data sets used in the present invention are the data of the 4th Brain-Computer Interface Competition BCI IV 1 and BCI IV 2a. The BCI IV 1 data comes from 7 healthy subjects and is captured with 59 electrodes at a sampling rate of 100 Hz. Each subject selects two categories from the three categories of left hand, right hand and foot, and each category has 100 trials. The BCI IV 2a data comes from 9 healthy subjects. The data has 25 channels in total, of which 22 channels are EEG and 3 channels are EOG. The data of the three EOG channels is not involved in classification. The signal is sampled at 250 Hz (250 samples per second) and band-pass filtered between 0.5 Hz and 100 Hz. The sensitivity of the amplifier is set to 100 μV. An additional 50 Hz notch filter is enabled to suppress line noise. The data includes two types of motor tasks of left hand and right hand, and each category has 72 trials.

[0084] As an example, the EEG data is preprocessed using the EEGLAB toolbox, which can specifically include the following steps:

[0085] In the first step, a causal 50th-order 8 - 30Hz finite impulse response bandpass filter is used to remove muscle artifacts and DC drifts, thereby obtaining a clearer MI signal.

[0086] In the second step, the EEG signals within [0.5, 3.5] seconds after the cue appears are extracted as trials.

[0087] Step S2: Align the preprocessed EEG signal data in Euclidean space.

[0088] In some embodiments, the preprocessed EEG signal data can be aligned in Euclidean space (Euclidean Space Data Alignment, EA) to make the data distributions among different subjects more consistent.

[0089] As an example, aligning the preprocessed EEG signal data in Euclidean space may include the following steps:

[0090] Assume that the subject has conducted n trials, and X i represents the EEG signal of the i-th experiment, and its corresponding calculation formula is:

[0091]

[0092] is the arithmetic mean of all covariance matrices of a subject, and alignment is performed, and its corresponding formula is:

[0093]

[0094] After alignment, the average covariance matrix of all n aligned trials is:

[0095]

[0096] That is, the means of the covariance matrices of all subjects are equal after alignment with the identity matrix. Therefore, the covariance matrix distributions of different subjects are more similar, which will greatly facilitate the deep domain adaptation in subsequent steps.

[0097] Step S3: Extract features from the aligned data using the RCSP technique.

[0098] In some embodiments, the Regularized Common Spatial Pattern (RCSP) technique can be used to extract features from the aligned data.

[0099] As an example, this step may include the following steps:

[0100] Using X i ∈R N×SThe electroencephalogram (EEG) signal representing the i-th experiment, where N and S represent the number of channels and the number of samples respectively. First, calculate the normalized covariance matrix and the pairwise covariance matrix The corresponding formulas are as follows:

[0101]

[0102] tr(·) is a function for calculating the trace of a matrix, and conv(·) is a function for calculating the pairwise covariance of each channel.

[0103] Then, regularize the average spatial covariance matrix The corresponding formula is:

[0104]

[0105] where 0 ≤ α, β ≤ 1, m is the number of training experiments, and I is the identity matrix.

[0106] The composite covariance matrix and its eigenvalue decomposition are:

[0107]

[0108] where U0 is the eigenvector matrix, λ is the eigenvalue of the composite covariance matrix and U0 is arranged in descending order of the corresponding eigenvalue λ. The whitening transformation matrix is:

[0109]

[0110] RCSP is extracted based on the diagonalization of the whitened covariance matrix, and the corresponding formula is:

[0111]

[0112] where S c1 and S c2 can also be decomposed to obtain the same eigenmatrix, and the corresponding formulas are:

[0113] S c1 = Bλ c1 B T , S c2 = Bλ c2 B T , λ c1 + λ c2 = 1.

[0114] The calculation formula for the projection matrix of RCSP is:

[0115] W = B T P,

[0116] Select the first n columns and the last n columns to form the projection matrix W 2n , the eigenvector X of a single experiment i can be transformed into the following form:

[0117] f i = log(Var(W 2n X i ))

[0118] Step S4, construct an MLP network model

[0119] In some embodiments, a Multilayer Perceptron (MLP) network model can be constructed to achieve multi-source domain adaptation and reduce the distribution difference between each source domain and the target domain

[0120] As an example, this step may include the following steps:

[0121] Use the extracted RCSP input to further extract the common features among subjects through a 3-layer MLP network, ensuring that the information among different subjects is shared as much as possible

[0122] Input the feature cross-pooling layer, and interact pairwise among the features

[0123]

[0124] Use the features of the feature cross-pooling layer for the input hidden layer, and the hidden layer learns high-order feature interactions in a non-linear manner to further learn deep features

[0125] z1 = σ1(W1f BI (v x ) + b1)

[0126] z2 = σ2(W2z2 + b2)

[0127] …

[0128] z L = σ L (W L z L-1 + b L )

[0129] where L represents the number of hidden layers, W i , b i , σ i represent the weight matrix, bias vector, and activation function of the L-th layer respectively

[0130] Step S5, train the source domain data based on the constructed MLP network model, classify the features of the target subject, and achieve the classification of the motor imagery EEG signals of the target subject

[0131] In some embodiments, the source domain data can be trained based on the MLP network model to classify the characteristics of the target subject, so as to achieve accurate classification of the motor imagery EEG signals of the target subject.

[0132] As an example, this step may include the following steps:

[0133] During the training process, the Central Moment Discrepancy (CMD) loss function is introduced to measure the difference in the overall data marginal distributions between the source domain and the target domain. This loss function constrains the features learned by the network by calculating the distance between the centers of the feature distributions of the source domain and the target domain, making the distributions of the source domain and the target domain closer in the feature space. This optimization strategy effectively reduces the global distribution bias between the source domain and the target domain, thereby enhancing the cross-domain transfer ability of the model.

[0134] Assume X i and X j are two feature samples, and their corresponding formula is:

[0135]

[0136] where E is the expectation of X, and c k is the k-th central moment.

[0137] On the basis of achieving the overall distribution alignment, the Contrastive Domain Discrepancy (CDD) loss function is further used to reduce the distribution difference between the source domain and the target domain at the category level. This loss function calculates the distribution similarity of each category in the feature space and aligns the feature distributions of the source domain and the target domain category by category. By optimizing the consistency of the feature distributions at the category level, the classification accuracy of the target domain samples is further improved.

[0138] For any two classes c1 and c2, the formula corresponding to the distance between them is:

[0139]

[0140] where

[0141] where g(·) is the kernel function, and G θ is the network feature extractor. When c1 = c2, it measures the difference in the same subdomain. When c1 ≠ c2, it measures the difference between the two subdomains. The formula corresponding to the LSD loss is:

[0142]

[0143] Finally, the formula corresponding to the overall loss function of the method of the present invention is:

[0144] L total = L CLS + L CMD + L CDD + L CENT ,

[0145] wherein, L CLS is the cross-entropy loss function, and L CENT is the conditional entropy loss function.

[0146] It should be noted that the results of the simulation experiment are as follows:

[0147] As shown in Table 1, when the method proposed in the present invention is compared with the traditional method, the experimental results also show the advantages of the present invention.

[0148] Table 1

[0149]

[0150] That is, Table 1 shows the result comparison (average precision and standard deviation between subjects) between the present invention and other existing methods.

[0151] The experimental results show that the method proposed in this paper performs excellently in the cross-subject motor imagery EEG signal classification task. Compared with the traditional method and other deep learning models, it significantly improves the classification accuracy. On the BCI IV 1 dataset, the average precision of the present invention reaches 92.79% ± 4.46, which is better than the traditional FBCSP-LDA (82.56% ± 11.74) and the deep learning models EEGNet (87.92% ± 8.75), ConvNet (87.56% ± 14.24), and the domain adaptation model DRDA (90.83% ± 14.65). On the more challenging BCI IV 2a dataset, the present invention still performs the best, with an average precision reaching 77.55% ± 16.11, further verifying its generalization and robustness.

[0152] The several methods used in the comparative experiment have their own characteristics: FBCSP-LDA is a traditional method that extracts features through filter bank common spatial pattern (FBCSP) and uses linear discriminant analysis (LDA) for classification, but it has poor adaptability to cross-subject tasks; EEGNet is a lightweight spatio-temporal convolutional network designed specifically for EEG signals, which can effectively extract features, but it has limited cross-domain generalization ability; ConvNet, as a classic convolutional neural network, is good at capturing local features, but it lacks robustness to changes in inter-domain distributions; DRDA uses deep domain adaptation technology to improve classification performance by aligning the feature distributions of the source domain and the target domain, but it may be sensitive to small sample data.

[0153] Such as Figure 2The figure shows the accuracy differences of the method proposed by the present invention among different subjects on the dataset BCI IV 1. From Figure 2 it can be seen that the overall classification accuracy of the present invention on the BCI IV 1 dataset is relatively good. The accuracy of each subject is distributed above 80%, and the average accuracy is close to the medium and high levels of all subjects, reflecting the stability and applicability of the model. Among them, the accuracy of subject A5 is the highest, approaching 100%; the accuracy of subject A2 is the lowest, slightly lower than that of other subjects, which may be related to individual characteristics or data quality.

[0154] As Figure 3 shown, it is the accuracy difference of the method proposed by the present invention among different subjects on the dataset BCI IV 2a. From Figure 3 it can be seen that the overall classification performance of the present invention on the BCI IV 2a dataset is relatively good, and the average accuracy is higher than 70%. However, the classification accuracy differences among subjects are relatively large, showing significant individual differences. The accuracy of subject A3 is close to 100%, showing the best performance, indicating that the model has strong adaptability to its data; while the accuracy of subject A5 is lower than 50%, significantly lower than that of other subjects, which may also be related to its data characteristics or signal quality.

[0155] Figure 4 It is the detailed flowchart of the overall method of the present invention. Figure 5 It is the comparison chart of the present invention and the previous method.

[0156] In summary, first, the collected EEG signal data is preprocessed, including removing noise and artifacts, to improve the signal quality and enhance the reliability of the data. Subsequently, the EA alignment technology is applied to the preprocessed EEG signals to reduce the distribution differences among different subjects and improve the comparability of the data. On this basis, the RCSP method is used to extract key features, further optimizing the signal representation and retaining the core information of the motor imagery task. Based on the extracted features, a deep learning network model based on a multi-layer MLP is constructed, and a multi-source domain adaptation technology is combined to achieve cross-subject classification tasks. To effectively address the distribution differences among different subjects, the present invention designs two loss functions in the network: First, the central moment difference loss, which is used to reduce the marginal distribution differences between the source domain and the target domain and align the feature distributions as a whole; Second, the contrast domain difference loss, which is used to narrow the class differences between the same-class samples in the source domain and the target domain, thereby ensuring the alignment at the class level.

[0157] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A multi-source cross-subject motor imagery EEG signal classification method based on deep domain adaptation, characterized in that: The following steps are involved: Acquire motor imagery EEG data, pre-process the EEG data, and obtain pre-processed EEG signal data; Perform Euclidean space data alignment on the preprocessed EEG signal data; Extract features from the aligned data using RCSP technology; Build an MLP network model; The source domain data is trained based on the constructed MLP network model, the characteristics of the target subjects are classified, and the classification of the motor imagery EEG signals of the target subjects is achieved.

2. The multi-source cross-subject motor imagery EEG signal classification method based on deep domain adaptation according to claim 1 is characterized in that: The EEG data was preprocessed using the EEGLAB toolbox, including: A causal 50th-order 8-30 Hz finite impulse response bandpass filter was used to remove muscle artifacts and DC drift; The EEG signals at [0.5, 3.5] seconds after the cue appeared were extracted as the trial.

3. The multi-source cross-subject motor imagery EEG signal classification method based on deep domain adaptation according to claim 1 is characterized in that: The step of performing Euclidean space data alignment on the preprocessed EEG signal data includes: Assume that the subject performs n trials, X i represents the EEG signal of the ith experiment, and its corresponding calculation formula is: is the arithmetic mean of all covariance matrices of a subject and performs alignment, the corresponding formula is: After alignment, the average covariance matrix of all n aligned trials is: The covariance matrices of all subjects were aligned to be equal.

4. The multi-source cross-subject motor imagery EEG signal classification method based on deep domain adaptation according to claim 1 is characterized in that: The method of extracting features from the aligned data using the RCSP technology includes: Use X i ∈R N×S represents the EEG signal of the ith experiment, where N and S represent the number of channels and samples respectively; calculate the normalized covariance matrix and the pairwise covariance matrix The corresponding formula is: tr(·) is a function for calculating the trace of a matrix, and conv(·) is a function for calculating the pairwise covariance of each channel; Regularized mean spatial covariance matrix The corresponding formula is: Where 0≤α, β≤1, m is the number of training experiments, and I is the unit matrix; The composite covariance matrix and its eigenvalue decomposition are: Among them, U0 is the eigenvector matrix, λ is the composite covariance matrix The eigenvalues ​​of , and U0 are arranged in descending order according to the corresponding eigenvalues ​​λ; the whitening transformation matrix is: RCSP is extracted based on the diagonalization of the whitened covariance matrix, and its corresponding formula is: Among them, S c1 and S c2 The same feature matrix is ​​decomposed in the same way, and its corresponding formula is: S c1 =Bλ c1 B T ,S c2 =Bλ c2 B T ,l c1 +λ c2 =1. The calculation formula corresponding to the projection matrix of RCSP is: W=B T P, Select the first n columns and the last n columns to form the projection matrix W 2n , the feature vector X of a single experiment i Transformed into the following form: f i =log(Var(W 2n X i ))。 5. The multi-source cross-subject motor imagery EEG signal classification method based on deep domain adaptation according to claim 1, characterized in that: The constructing of the MLP network model includes: The extracted RCSP was used to input into a 3-layer MLP network to further extract common features across subjects; Input feature cross-pooling layer, features interact with each other; The features of the feature cross-pooling layer are used to input the hidden layer. The hidden layer learns high-order feature interactions in a nonlinear way and further learns deep features. z1=σ1(W1f BI (in x )+b1) z2=σ2(W2z2+b2) … With L =σ L (IN L With L-1 +b L ) Among them, L represents the number of hidden layers, W i , b i , σ i They represent the weight matrix, bias vector and activation function of the Lth layer respectively.

6. The multi-source cross-subject motor imagery EEG signal classification method based on deep domain adaptation according to claim 1, characterized in that: The constructed MLP network model is used to train the source domain data and classify the characteristics of the target subjects, including: During the training process, the central moment difference loss function is introduced to measure the difference in the marginal distribution of the overall data in the source domain and the target domain; Assume X i and X j are two feature samples, and the corresponding formula is: Where E is the expectation of X, c k is the central moment of order k; For any two classes c1 and c2, the distance between them corresponds to the formula: in, where g(·) is the kernel function, G θ is a network feature extractor; when c1=c2, it measures the difference in the same subdomain; when c1≠c2, it measures the difference between two subdomains. The formula corresponding to the LSD loss is: The formula corresponding to the overall loss function is: L total =L CLS +L CMD +L CDD +L CENT , Among them, L CLS is the cross entropy loss function, L CENT is the conditional entropy loss function.

Images