Personalized motor imagery prediction method based on cognitive electroencephalogram

Abstract

The invention provides a personalized motor imagery prediction method based on cognitive electroencephalogram, which comprises the following steps: S1, preprocessing cognitive electroencephalogram, including baseline correction, band-pass filtering and independent component analysis; s2, extracting time-frequency spatial domain characteristics of the cognitive electroencephalogram; s3, constructing a time-frequency-space domain brain network of cognitive electroencephalogram by adopting different electroencephalogram synchronous quantization modes according to the characteristics of different domains; s4, constructing a multi-domain graph convolutional network prediction model to extract topological connection features of the multi-domain brain network, and inputting the topological connection features into a multi-domain fusion module for fusion to obtain output features; and S5, inputting the output features into a classifier to obtain a prediction result of a single action, and throwing a personalized motor imagery action through a voting module. According to the method, the feasibility of realizing personalized motor imagery prediction by applying the correlation between the cognitive electroencephalogram and the motor imagery is proved. Meanwhile, a practical method is provided for solving the performance problem caused by the difference between individuals.

Description

Technical Field

[0001] The present invention belongs to the technical field of biomedical engineering brain-computer interface, and in particular relates to a personalized motor imagery prediction method based on cognitive EEG. Background Art

[0002] A brain-computer interface (BCI) is a new information exchange and control technology that establishes a connection between the human brain and electronic devices such as computers, independent of peripheral nerves and related muscle tissue. Motor imagery (MI)-based BCIs require subjects to imagine movement to generate event-related synchronization / desynchronization features in the brain. These BCIs are used to identify movement intentions and convert these intentions into rehabilitation-assistance commands or interventions to re-establish the rehabilitation circuit between the individual and the external environment. Due to its active rehabilitation training nature and objective results, MI-BCI technology has been widely used in stroke rehabilitation and the regeneration of damaged neurons.

[0003] However, individual differences in the training of MI-BCI systems lead to high fluctuations in the EEG signals generated by different subjects, which makes it difficult for the established decoding model to accurately predict the intentions of a specific subject. Therefore, it is crucial to find an indicator that can quantify individual MI differences. Action is an important factor affecting individual differences, and different actions induce different MI performance. Specifically, strong actions can induce higher MI classification accuracy, while weak actions induce relatively low MI classification accuracy. At the same time, a positive correlation between cognitive tasks and MI has been shown. Specifically, strong actions can induce higher event-related potential amplitudes, while weak actions are the opposite. The above studies demonstrate the feasibility of using cognitive tasks to predict high-performance MI actions to optimize MI-BCI performance.

[0004] Considering that both cognitive tasks and MI involve brain neural activities that collaborate with multiple brain regions, building a network model can specifically quantify multi-region EEG signals to achieve more accurate predictions. Graph convolutional network (GCN) is a practical method to solve the above problems. It extends the concept of traditional convolution kernels to the graph domain and allows learning node features based on topological connections with adjacent nodes in the graph, showing a powerful ability to explore interactions between different brain structures. In addition, current research mainly uses single-domain features as input to convolutional networks. However, the complexity and diversity of EEG make it impossible for single-domain features to fully and accurately capture the activity characteristics of the brain. Related studies have shown that the fusion of different feature domains can better capture the intricate interactions between different EEG features and improve the model's ability to process complex, multi-dimensional data. Summary of the Invention

[0005] In view of this, the present invention proposes a personalized motor imagery prediction method based on cognitive EEG, which solves the problem of individual differences in MI-BCI by designing a graph convolution prediction model based on a multi-domain brain network. The personalized motor imagery prediction method based on the multi-domain brain network graph convolution algorithm proposed in the present invention is feasible. This method proves the feasibility of decoding and prediction by applying the correlation between brain areas activated by different paradigms. At the same time, it also reflects the value of feature fusion and GCN, providing a practical method for achieving personalized prediction and solving performance problems caused by individual differences.

[0006] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] A personalized motor imagery prediction method based on cognitive EEG comprises the following steps:

[0008] S1. Preprocess cognitive EEG, including baseline correction, bandpass filtering, and independent component analysis;

[0009] S2. Extract the time-frequency-spatial features of cognitive EEG;

[0010] S3. Based on the characteristics of different domains, different EEG synchronization quantization methods are used to construct the time-frequency-spatial domain brain network of cognitive EEG;

[0011] S4. Build a multi-domain graph convolutional network prediction model to extract the topological connection features of the multi-domain brain network, input the topological connection features into the multi-domain fusion module for fusion, and obtain the output features;

[0012] S5. Input the output features into the classifier to obtain the prediction results of a single action, and then cast a personalized motor imagination action through the voting module.

[0013] Furthermore, in step S1, the cognitive EEG is preprocessed in sequence, including 0.5-20 Hz third-order zero-phase Butterworth band-pass filtering, independent component analysis, and 1.2 s time window capture.

[0014] Furthermore, in step S2, multi-domain feature extraction is used to extract time-frequency-space features from the EEG signal preprocessed in step S1, including the following steps:

[0015] S201, extracting the downsampled component of cognitive EEG as a time domain feature;

[0016] S202, extracting power spectrum density of different frequency bands of cognitive EEG as frequency domain features;

[0017] S203, extracting the common spatial component of cognitive EEG as a spatial domain feature;

[0018] Furthermore, in step S3, the Pearson correlation coefficient, phase-locking value, and Manhattan distance are used to construct a multi-domain brain network in the time-frequency and spatial domains according to the characteristics and distribution of different domains, including the following steps:

[0019] S301. Use Pearson correlation coefficient to construct time-domain brain network based on the time-domain characteristics of cognitive EEG;

[0020] S302, using the phase-locked value to construct a frequency domain brain network based on the frequency domain characteristics of cognitive EEG;

[0021] S303. Use Manhattan distance to construct a spatial brain network based on cognitive EEG spatial features;

[0022] Furthermore, in step S4, a multi-domain graph convolutional network prediction model is constructed to extract the topological connection features of the multi-domain brain network, and the topological connection features are input into the multi-domain fusion module for fusion to obtain output features, including the following steps:

[0023] S401. Construct a multi-domain graph convolutional network prediction model to extract the topological connection features of the multi-domain brain network;

[0024] The multi-domain graph convolutional network includes five modules: time domain graph convolution, frequency domain graph convolution, spatial domain graph convolution, multi-domain feature fusion and voting;

[0025] The time domain graph convolution, frequency domain graph convolution and spatial domain graph convolution have the same structure, which includes an input layer, a graph convolution layer, a global pooling layer and a Dropout layer;

[0026] S402: Input the extracted topological connection features into a multi-domain fusion module for fusion to obtain output features;

[0027] Furthermore, in step S5, the cognitive features are input into the Softmax layer. The Softmax layer normalizes the feature vectors corresponding to each action and outputs a probability distribution representing the likelihood that the input features belong to each candidate action intensity. Ultimately, the system selects the intensity with the highest probability as the predicted result, and a voting module then selects the personalized motor imagery action. The number of strong action votes received by all actions is compared, and the action with the highest number of votes is selected as the subject's personalized MI action.

[0028] A computer-readable storage medium stores a computer program, characterized in that when the computer program is executed by a processor, a personalized motor imagery prediction method based on cognitive EEG is implemented.

[0029] Compared with the existing technology, the personalized motor imagery prediction method based on cognitive EEG described in the present invention has the following advantages:

[0030] (1) By constructing a multi-domain brain network, the present invention can more effectively capture the differences in motor imagery between individuals and improve the decoding accuracy of MI tasks;

[0031] (2) Combining time domain, frequency domain and spatial domain information, using Pearson correlation coefficient, phase-locking value and Manhattan distance to construct a multi-domain brain network, achieve complementary fusion of multimodal features and improve the generalization ability of the model;

[0032] (3) By analyzing the correlation between brain regions activated by different brain-computer interface paradigms, a personalized prediction model is established, so that the MI-BCI system can better adapt to the EEG characteristics of different individuals and reduce the impact of individual differences on system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which constitute part of the present invention, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0034] Figure 1 This is a flowchart of a personalized motor imagery prediction method based on cognitive EEG according to an embodiment of the present invention;

[0035] Figure 2 This is a network structure diagram of the graph convolutional network model described in an embodiment of the present invention;

[0036] Figure 3 A graph showing specific network parameters of the graph convolutional network model described in an embodiment of the present invention; DETAILED DESCRIPTION

[0037] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0038] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0039] like Figures 1 to 3 As shown, a personalized motor imagery prediction method based on cognitive EEG includes the following steps:

[0040] S1. Preprocess cognitive EEG, including baseline correction, bandpass filtering, and independent component analysis;

[0041] S2. Extract the time-frequency-spatial features of cognitive EEG;

[0042] S3. Based on the characteristics of different domains, different EEG synchronization quantization methods are used to construct the time-frequency-spatial domain brain network of cognitive EEG;

[0043] S4. Build a multi-domain graph convolutional network prediction model to extract the topological connection features of the multi-domain brain network, input the topological connection features into the multi-domain fusion module for fusion, and obtain the output features;

[0044] S5. Input the output features into the classifier to obtain the prediction results of a single action, and then cast a personalized motor imagination action through the voting module.

[0045] Use multiple feature extraction methods to extract multi-domain cognitive EEG; select appropriate EEG synchronization quantization methods based on the characteristics and distribution of features in different domains to build a multi-domain brain network; use the multi-domain brain network as input, use graph convolutional networks to predict the strength of actions, and cast strong actions through weighted voting, thereby improving the performance of MI-BCI.

[0046] In step S1, cognitive EEG is preprocessed in sequence, including 0.5-20 Hz third-order zero-phase Butterworth bandpass filtering, independent component analysis, and 1 s time window capture. At this time, the sample shape of each action and each trial is 19x1000, representing 19 electrode channels and 1000 time points.

[0047] In step S2, the time-frequency and spatial domain features are extracted respectively through downsampling, discrete wavelet transform and co-spatial pattern to construct multi-domain features;

[0048] S201. Extract the downsampled component of the cognitive EEG as a time domain feature, where the downsampling factor is set to 25, which means that the mean (or other statistics) is calculated once every 25 data points and the mean is used as the new data point. After downsampling, the time domain feature shape of each trial of each action is obtained to be 19x40;

[0049] S202. Decompose the EEG signal using a four-layer Daubechies wavelet to obtain signals in four frequency bands: delta (1-4 Hz), theta (4-8 Hz), alpha (8-13 Hz), and beta (13-22 Hz). The power spectral density (PSD) of the four frequency bands is obtained as frequency domain features using the Welch method. The time domain feature shape of each trial of each movement is 19x4.

[0050] S203. Use the common space pattern to construct a spatial filter for extracting spatial distribution components. The spatial filter can obtain an optimized discrimination direction and achieve the best classification effect. The spatial feature shape obtained for each action and each trial is 19x1.

[0051] In step S3, the Pearson correlation coefficient, phase-locking value, and Manhattan distance are used to construct a multi-domain brain network in the time-frequency and spatial domains according to the characteristics and distribution of different domains;

[0052] S301. Use the Pearson correlation coefficient to measure the linear correlation between the two channels, and then construct a brain network based on time domain features. The shape of the time domain brain network obtained for each action and each trial is 19x19;

[0053] S302, using the phase-locked value to calculate the instantaneous phase difference between the frequency features of the two channels, and then constructing a brain network based on the frequency domain features. The frequency domain brain network shape obtained for each action and each trial is 19x19;

[0054] S303. Use Manhattan distance to calculate the sum of the distances between two points along the coordinate axis, and then construct a brain network based on spatial features. The shape of the spatial brain network obtained for each action and each trial is 19x19.

[0055] In step S4, a multi-domain graph convolutional network prediction model is constructed to extract the topological connection features of the multi-domain brain network, and the topological connection features are input into the multi-domain fusion module for fusion to obtain output features;

[0056] S401. Construct a multi-domain graph convolutional network prediction model to extract the topological connection features of the multi-domain brain network;

[0057] The multi-domain graph convolutional network includes five modules: time domain graph convolution, frequency domain graph convolution, spatial domain graph convolution, multi-domain feature fusion and voting;

[0058] The time domain graph convolution, frequency domain graph convolution and spatial domain graph convolution have the same structure, which includes an input layer, a graph convolution layer, a global pooling layer and a Dropout layer;

[0059] When the model is run, all the data from one subject is first selected as the test set, and the data from all other subjects is used as the training set. For each input brain network, and for each input multi-domain brain network, node-level features are first processed through a graph convolutional layer. Assume that the input graph G = {V, E, A} has S nodes, each node has F features, where V and E represent the set of nodes and edges, respectively. is a weighted adjacency matrix. Then the graph signal is defined for all nodes The graph Laplacian matrix can be obtained by L = DA, where D∈RS×S is the degree matrix of G. Each element in D represents the degree of the corresponding node, which can be obtained by D ii =∑ j A ij The normalized graph Laplacian operator is defined as:

[0060]

[0061] in is the identity matrix. Next, we use the singular value decomposition to To break it down:

[0062]

[0063] in yes The eigenvector matrix of is a diagonal matrix whose elements represent The eigenvalues of S-1 ,.

[0064] Then, let g θ For the graph convolution kernel constrained by θ, the graph convolution is defined by the following formula:

[0065] g θ *x=Ug θ (∧)Ux T x

[0066] where g θ (∧) represents g θ It is a function determined by ∧, and its form is:

[0067] g θ (∧)=diag(g(λ0),…,g(λ S-1 ))

[0068] However, direct decomposition The eigenvalues of g are very time-consuming. To solve this problem, Chebyshev polynomials are used to approximate g θ , which is defined as follows:

[0069]

[0070] where K is the order of the Chebyshev polynomial, is the mth order Chebyshev polynomial, and furthermore, It can be expressed as:

[0071]

[0072] where λ max express The maximum eigenvalue of . In our framework, K = 2 is selected. The propagation rule of the graph convolutional layer is defined as follows:

[0073]

[0074] Among them G (l) represents the output of the l-th layer graph convolution, Θ (l) Represents the trainable weight matrix of this layer. G (l-1) is the output of the previous layer of graph convolution.

[0075] Then, for higher-level representation, a rectified linear unit (ReLU) is set. ReLU is a commonly used activation function defined as follows:

[0076] f(x)=max(0,x)

[0077] Its main goal is to introduce nonlinearity so that the neural network can learn complex nonlinear features. A Dropout layer (Dropout-1) is added after GC-1, which randomly cuts the number of output neurons with probability Pdrop to alleviate the overfitting of features. Another graph convolution layer (GC-2) is used for further feature learning. Another Dropout layer (Dropout-2) is added after the GC-2 layer. Then, a read function is implemented to aggregate node features and obtain cognitive features.

[0078] S402: Input the extracted topological connection features into a multi-domain fusion module for fusion to obtain output features;

[0079] In step S5, the cognitive features are input into the Softmax layer. The Softmax layer normalizes the feature vectors corresponding to each action and outputs a probability distribution representing the likelihood that the input features belong to each candidate action intensity. Ultimately, the system selects the intensity with the highest probability as the predicted result, and a voting module then selects the personalized motor imagery action. The number of strong action votes received by all actions is compared, and the action with the highest number of votes is selected as the subject's personalized MI action.

[0080] During the training process of the time domain, frequency domain, and spatial domain GCN blocks, the learning rate of the module is set to 0.001, the number of training iterations is set to 1000, and the batch size is 480. The data of one subject is used as the test set, and the data of the other subjects are used as the training set.

[0081] Experimental findings demonstrate the feasibility of decoding and predicting motor imagery using correlations between brain regions activated by different paradigms using a personalized cognitive EEG-based motor imagery prediction method. This method also demonstrates the value of feature fusion and GCN, providing a practical approach for achieving personalized prediction and addressing performance issues caused by individual differences.

[0082] Those skilled in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and description of the present invention.

[0084] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A personalized motor imagery prediction method based on cognitive EEG, comprising the following steps: S1. Preprocess cognitive EEG, including baseline correction, bandpass filtering, and independent component analysis; S2. Extract the time-frequency-spatial features of cognitive EEG; S3. Based on the characteristics of different domains, different EEG synchronization quantization methods are used to construct the time-frequency-spatial domain brain network of cognitive EEG; S4. Build a multi-domain graph convolutional network prediction model to extract the topological connection features of the multi-domain brain network, input the topological connection features into the multi-domain fusion module for fusion, and obtain the output features; S5. Input the output features into the classifier to obtain the prediction results of a single action, and then cast a personalized motor imagination action through the voting module.

2. The method of claim 1, wherein: Preprocessing of cognitive data includes the following steps: S101, intercept the cognitive EEG in a fixed-second time window and extract cognitive data using baseline correction; S102, filtering the cognitive data using a Butterworth filter; S103. Using independent component analysis on the filtered data to eliminate artifacts caused by electromyography and electrooculography.

3. The method of claim 2, wherein: In step S2, time-frequency and space-time feature extraction methods are used to extract features in the time-frequency and space-time domains, including the following steps: S201, extracting the downsampled component of cognitive EEG as a time domain feature; S202, extracting power spectrum density of different frequency bands of cognitive EEG as frequency domain features; S203: extracting the common spatial component of cognitive EEG as a spatial domain feature.

4. The method of claim 3, wherein: In step S3, different EEG synchronization quantization methods are used according to the characteristics of different domains to construct a time-frequency-spatial domain brain network of cognitive EEG, including the following steps: S301. Use Pearson correlation coefficient to construct time-domain brain network based on the time-domain characteristics of cognitive EEG; S302, using the phase-locked value to construct a frequency domain brain network based on the frequency domain characteristics of cognitive EEG; S303. Use Manhattan distance to construct a spatial brain network based on cognitive EEG spatial features.

5. The method of claim 4, wherein: A multi-domain graph convolutional network prediction model is constructed to extract the topological connection features of the multi-domain brain network. The topological connection features are input into the multi-domain fusion module for fusion to obtain output features, including the following steps: S401. Construct a multi-domain graph convolutional network prediction model to extract the topological connection features of the multi-domain brain network; The multi-domain graph convolutional network includes five modules: time domain graph convolution, frequency domain graph convolution, spatial domain graph convolution, multi-domain feature fusion and voting; The time domain graph convolution, frequency domain graph convolution and spatial domain graph convolution have the same structure, which includes an input layer, a graph convolution layer, a global pooling layer and a Dropout layer; S402: Input the extracted topological connection features into a multi-domain fusion module for fusion to obtain output features.

6. The method of claim 5, wherein: The output features are input into the classifier to obtain the prediction results of a single action, and the voting module votes for the personalized motor imagery action, including the following steps: S501: Input the fused output features into the classifier, and predict a single action using the trained model to obtain a preliminary prediction result. S502: Perform a comprehensive analysis of the multiple prediction results through a voting module, and select personalized motor imagery actions based on the votes received by the multiple predicted actions.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the personalized motor imagery prediction method based on a multi-domain graph convolutional network as described in any one of claims 1 to 7.

Images