A unilateral limb motor imagery task electroencephalogram classification method

Abstract

The application discloses a unilateral limb motor imagery task electroencephalogram classification method, which sequentially comprises the following steps: A: constructing a unilateral limb motor imagery data set and preprocessing to obtain motor imagery electroencephalogram signals; B: using a deep neural network model to extract features and classify the motor imagery electroencephalogram signals to obtain a probability distribution matrix of the electroencephalogram signals; C: using a CSP method to process the motor imagery electroencephalogram signals to obtain a time-frequency graph data set, and then using a deep neural network model to extract features and classify the time-frequency graph data set to obtain a probability distribution matrix of the time-frequency graph signals; D: calculating an average clustering coefficient and optimizing the two probability distribution matrices; and then based on D-S evidence theory and using the two optimized probability distribution matrices to make a fusion decision to obtain a final classification result. The application can effectively improve the classification accuracy of unilateral limb motor imagery tasks and provides a data basis for the development of brain-computer interface technology.

Description

Technical Field

[0001] This invention relates to the field of biosignal recognition technology, and in particular to an EEG classification method for unilateral limb motor imagery tasks based on multimodal information fusion. Background Technology

[0002] Brain-computer interfaces (BCIs) establish an information exchange channel between the human brain and the external environment. They can directly read and analyze electrical signals generated by the brain to recognize user intentions and use EEG signals to achieve interaction between humans and external devices. The core technology is EEG signal recognition. Motor imagery tasks (MIs) simulate the mental process of movement without actual movement; that is, the brain imagines the entire movement without actually contracting the muscles. In the field of neurophysiology, there are many similarities between real movements and motion images in the peripheral autonomic nervous system and cortical potentials. MI, as an important paradigm of spontaneous BCI, has been widely used in the field of neurorehabilitation, providing rehabilitation treatment and motor assistance for disabled patients with movement disorders, such as those with brainstem injury, stroke, or muscular atrophy. These patients have intact brains but are unable to communicate with the outside world due to peripheral nerve damage. While the MI-BCI system cannot repair their own physiological neural transmission channels, it can transmit the brain's consciousness to controlled devices for control. Current research on motor imagery tasks often involves movements of different body parts, identifying the EEG signals generated by imagining movements of different body parts. However, compared to motor imagery tasks involving different body parts, combining motor imagery tasks of the same limb with corresponding body part movements is more natural and intuitive. For example, in arm rehabilitation training combined with a rehabilitation robot, imagining different arm movements and then having the robot perform the same movements for rehabilitation training is more conducive to training and rehabilitation and better suited to clinical applications. However, classifying and recognizing a single limb is more difficult and complex than recognizing different body parts. This is because the motor cortex regions activated when the same body part performs a motor task are very similar, which brings greater difficulties to decoding and recognition research. Therefore, effective feature extraction methods are crucial in the classification process.

[0003] The process of classifying and recognizing EEG signals includes EEG signal preprocessing, feature extraction, and feature classification. Effective feature extraction and a suitable feature classifier are crucial to the performance of identity recognition. Representative feature extraction methods for EEG signals can be categorized into time-domain analysis (including amplitude and mean), frequency-domain analysis (including power spectrum analysis and coherence analysis), time-frequency-domain analysis (including wavelet transform and empirical mode decomposition), and spatial-domain analysis (including co-spatial pattern method and independent component analysis). To improve the accuracy of biometric recognition, some studies combine different feature extraction methods to extract multi-dimensional features of EEG signals to represent them from multiple domains. In the feature classification of EEG signals, deep learning methods directly use the original signal as the model input for end-to-end training, eliminating the need for feature extraction, and are currently widely used. Due to the time-domain, frequency-domain, and spatial properties of EEG signals, convolutional neural networks and recurrent neural networks are commonly used as methods for biometric recognition.

[0004] Deep learning algorithms are increasingly being applied to EEG signal classification tasks. Current motor imagery EEG signal recognition typically uses raw EEG signals or transformed EEG frequency domain features for classification. Simple deep learning models can achieve good performance, thus attracting increasing attention from researchers. However, existing motor imagery EEG signal recognition methods often neglect the inherent connections between signals from different domains, failing to consider the feature diversity and correlation between signals from different domains and modalities, thus lacking more comprehensive feature information. Therefore, fusing information from different features of EEG signals is crucial for improving the accuracy of EEG signal classification. Summary of the Invention

[0005] The purpose of this invention is to provide an EEG classification method for unilateral limb motor imagery tasks. This method utilizes a convolutional neural network to extract and classify features from preprocessed raw EEG signals and two-dimensional time-frequency maps obtained using continuous wavelet transform (CWT). The results obtained from the classification of the two modalities are then fused for decision-making. This method can effectively improve the classification accuracy of unilateral limb motor information tasks, thereby providing a data foundation for the development of brain-computer interface technology.

[0006] The present invention adopts the following technical solution:

[0007] A method for classifying unilateral limb motor imagery tasks using electroencephalograms (EEGs) includes the following steps:

[0008] A: Construct a unilateral limb motor imagery dataset, and perform data preprocessing on the motor imagery EEG signals within the unilateral limb motor imagery dataset to obtain preprocessed motor imagery EEG signals.

[0009] B: The preprocessed motor imagery EEG signals obtained in step A are used for feature extraction and classification using a deep neural network model to obtain the probability distribution matrix P of the EEG signals. EEG Probability distribution matrix P EEG It includes the probability that the preprocessed motor imagery EEG signal belongs to the corresponding action category;

[0010] C: The preprocessed motor imagery EEG signal obtained in step A is spatially filtered using the CSP method to obtain the feature matrix of the EEG signal; then, the time-frequency graph dataset is obtained by selecting the feature matrix and performing continuous wavelet transform; finally, a deep neural network model is used to extract features and classify the time-frequency graph dataset to obtain the probability distribution matrix P of the time-frequency graph signal. TF Probability distribution matrix P TF It contains the probability that the time-frequency graph signal belongs to the corresponding action category;

[0011] D: Using the probability distribution matrix P of the EEG signal obtained in step B EEG And the probability distribution matrix P of the time-frequency signal obtained in step C. TF Calculate the clustering coefficient for each action category, and then apply the calculated average clustering coefficient to the probability distribution matrix P. EEG and P TF Optimization is performed to obtain the optimized probability distribution matrix P. EEG* and P TF* Furthermore, based on the DS evidence theory and utilizing the probability distribution matrix P... EEG* and P TF* A fusion decision is made to obtain the final classification result.

[0012] In step A, data preprocessing includes bandpass filtering, downsampling, channel selection, independent component analysis for artifact removal, and data selection of motor imagery EEG signals.

[0013] In step A, only the EEG signals of motor imagery under three actions—extending the arm forward, rotating the wrist to the left, and grasping a water cup—are selected from the unilateral limb motor imagery dataset.

[0014] In step B, in, and These represent the probabilities that a sample result belongs to the corresponding action category in a three-class classification experiment of electroencephalogram signals;

[0015] In step C, in, and These represent the probabilities that a sample result belongs to the corresponding action category in a three-class classification experiment using time-frequency graphs.

[0016] Step C includes the following specific steps:

[0017] C1: Spatial filtering of preprocessed motor imagery EEG signals is performed using the CSP algorithm;

[0018] When performing spatial filtering, the CSP projection matrix W is first calculated. Then, using the CSP projection matrix W and the preprocessed motor imagery EEG signal obtained in step A, the feature matrix Z is calculated. The calculation formula is as follows: Z M×T =W M×M E M×T Where W is the CSP projection matrix, M is the number of channels in the preprocessed EEG data, T is the data length, and E is the EEG data matrix of size M×T formed by converting the preprocessed motor imagery EEG signals.

[0019] C2: Select the feature matrix Z′ extracted by CSP;

[0020] C3: Based on the feature matrix Z′ obtained in step C2, time-frequency features are extracted using the CWT algorithm and transformed into a two-dimensional time-frequency graph. Morlet wavelet is selected as the basis function for wavelet transform, and finally the motion imagination time-frequency graph dataset is obtained.

[0021] The formula for wavelet transform is as follows:

[0022]

[0023]

[0024]

[0025] Here, Morlet is a commonly used basis function in wavelet transform, used to decompose the signal to be processed during the wavelet transform process; Φ(t) represents the basis function, ω represents the wavelet center frequency; t represents time; i represents the sequence that changes with time; Φ α，β (t) represents the basis function after scaling and translation transformation, α represents the scaling factor, β represents the time translation factor, CWT(α,β) represents the result after performing wavelet transform on the signal, and f(t) represents the EEG signal.

[0026] C4: Based on the obtained motion visualization time-frequency graph dataset, a convolutional neural network is used for feature extraction and classification in an image processing manner to obtain the probability distribution matrix P of the time-frequency graph signal. TF ,

[0027] In step C2, the feature matrix Z is selected. M×TThe first m rows and the last m rows of data are used as the feature matrix Z′ for CSP feature extraction; where 2m <M。

[0028] Step D includes the following specific steps:

[0029] D1: Based on the probability distribution matrix P EEG and P TF The probability of each action category is included, and the clustering coefficient CC of each action category data in the two probability distribution matrices is calculated. Then, the average clustering coefficient is obtained by averaging the clustering coefficients of all nodes in each action category data.

[0030] in, h represents the number of neighbors of a node; n represents the number of edges connecting all adjacent nodes of a node.

[0031] D2: The obtained probability distribution matrix P EEG and P TF Using the average clustering coefficients as weights, the two probability distribution matrices are re-optimized to obtain the optimized probability distribution matrix P. EEG * and P TF *;

[0032]

[0033] D3: Utilize the DS fusion rule to optimize the probability distribution matrix P EEG * and P TF * Perform a fusion decision to obtain a new probability distribution matrix P, and obtain the final classification result based on it;

[0034]

[0035] Where k is the evidence conflict factor, k = Σ B∩C＝φ P EEG (B)P TF (C), where A, B, and C represent probability distribution matrices P, P0, P1, P2, P3, P4, P5, P6, P7, P8, P9, P10, P11 EEG and P TF The three action categories.

[0036] When performing bandpass filtering, a fourth-order Butterworth bandpass filter was used to filter the raw EEG data from 8 to 30 Hz to obtain the frequency band of motor-related EEG signals. When performing downsampling, the EEG signals were downsampled to 250 Hz. When performing channel selection, 20 EEG channels from the sensorimotor cortex of the brain were selected for classification. When performing independent component analysis for artifact removal, EEGLAB was used to perform ICA calculation and artifact removal on the EEG signals.

[0037] In step B, the convolutional neural network structure used is EEG-CNN, which includes convolutional layers, pooling layers, and fully connected layers; two convolutional layers and a max pooling layer are combined to form a feature extraction module, and the one-dimensional convolutional kernel in the convolutional layer is used to extract the features of each channel as the feature map output by this layer.

[0038] In step C, the convolutional neural network structure is TF-CNN, and VGG16 is used as the basic network framework for extracting time-frequency map feature information.

[0039] During training, the convolutional neural networks in steps B and C use the Adam optimizer with β1 = 0.9 and β2 = 0.999 to update the trainable parameters of each network layer, with the initial learning rate set to 0.01.

[0040] The beneficial effects and advantages of this invention are as follows:

[0041] 1. This invention achieves classification and recognition of motor imagery EEG signals by fusing information from EEG and time-frequency maps. This invention converts sequential EEG signals into a two-dimensional time-frequency map containing rich time-frequency and spatial information, using it as a supplement to the EEG signal to achieve more complete decision-making and improve classification accuracy.

[0042] 2. In the conversion of time-frequency maps, the image is generated by stacking the channels. To fully utilize all channels of the EEG signal, this invention employs the CSP algorithm to spatially filter the preprocessed EEG signal to obtain a feature matrix. This feature matrix is ​​then used to replace the EEG signal to complete the time-frequency map conversion. This invention uses the CSP algorithm to concentrate the feature information in the generated feature matrix primarily at its head and tail. By selecting the first and last parts of the feature matrix as the signal for generating the time-frequency map, it can fully utilize the feature information from multiple channels and facilitate the stacking of time-frequency maps, achieving better classification results. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the process of the present invention;

[0044] Figure 2 This is a comparison chart of the classification performance of the DS fusion method described in this invention with EEG-CNN, TF-CNN, and CSP methods. Detailed Implementation

[0045] The present invention will now be described in detail with reference to the accompanying drawings and embodiments:

[0046] like Figure 1 As shown, the EEG classification method for unilateral limb motor imagery tasks described in this invention includes the following steps in sequence:

[0047] A: Construct a unilateral limb motor imagery dataset, and perform data preprocessing on the motor imagery EEG signals within the unilateral limb motor imagery dataset to obtain preprocessed motor imagery EEG signals.

[0048] In step A, the unilateral limb motor imagery dataset uses the "Multimodal signal dataset of 11 intuitive motor tasks of a single upper limb during multiple recordings" from the Giga DB dataset. This dataset includes arm extension movements in six directions (up, down, left, right, forward, and backward), grasping movements of three objects (cup, card, and ball), and motor imagery EEG signals under two wrist twisting movements (leftward and rightward rotation). Each of the above movements is randomly executed 50 times, and motor imagery EEG signals are sampled at a sampling rate of 2500Hz. The duration of each motor imagery movement is 4 seconds, and each subject performs three identical recording processes.

[0049] In step A, data preprocessing includes bandpass filtering, downsampling, channel selection, independent component analysis for artifact removal, and data selection of motor imagery EEG signals;

[0050] In this embodiment, bandpass filtering was used to limit the frequency range of EEG signals in the unilateral limb motor imagery dataset to 8-30Hz, and downsampling was performed to 250Hz. Channel selection was also performed, selecting only EEG signal data from 20 channels of the sensorimotor cortex for study. Independent component analysis (ICA) was performed on the EEG signals using the EEGLAB toolbox in MATLAB to remove interference signals caused by eye movements, muscle activity, etc. ICA, as a blind source separation algorithm, has wide applications in biomedical signal processing and image denoising. In the unilateral limb motor imagery dataset, only motor imagery EEG signals from three actions—forward arm extension, left wrist rotation, and grasping a water cup—were selected for classification.

[0051] B: The preprocessed motor imagery EEG signals obtained in step A are used for feature extraction and classification using a deep neural network model to obtain the probability distribution matrix P of the EEG signals. EEG Probability distribution matrix P EEG It includes the probability that the preprocessed motor imagery EEG signal belongs to the corresponding action category;

[0052] In this embodiment, since step A only selects motor imagery EEG signals for classification under three actions: forward arm extension, left wrist rotation, and grasping a water cup, therefore... in, and Let represent the probability that a sample result in a three-class classification experiment of EEG signals belongs to the corresponding action category. The superscript EEG is an abbreviation for electroencephalogram (EEG).

[0053] In step B, the preprocessed motor imagery EEG signals are converted into an M×T matrix. Then, a convolutional neural network is used to extract features and classify the matrix-form motor imagery EEG signals to obtain the probability distribution matrix of the EEG signals. Here, M is the number of channels, which is 20 in this embodiment, and T is the data length. The motor imagery EEG signals used in this paper are generated by three action stimuli: forward arm extension, left wrist rotation, and grasping a water cup. The EEG signals belong to different types of actions. The main purpose of the classification task is to identify the action category to which the EEG signals belong. The three-class classification experiment in this paper refers to the classification and recognition of the EEG signals corresponding to the three actions. Generally, the category with the highest probability value in the probability distribution matrix is ​​taken as the classification result. The three-class classification experiment is a conventional technique in this field and will not be described in detail here.

[0054] C: The preprocessed motor imagery EEG signal obtained in step A is spatially filtered using the CSP method to obtain the feature matrix of the EEG signal; then, the time-frequency graph dataset is obtained by selecting the feature matrix and performing continuous wavelet transform; finally, a deep neural network model is used to extract features and classify the time-frequency graph dataset to obtain the probability distribution matrix P of the time-frequency graph signal. TF Probability distribution matrix P TF It contains the probability that the time-frequency graph signal belongs to the corresponding action category;

[0055] In this embodiment, since step A only selects motor imagery EEG signals for classification under three actions: forward arm extension, left wrist rotation, and grasping a water cup, therefore... in, and These represent the probabilities that a sample result belongs to the corresponding action category in the time-frequency graph three-class classification experiment. The superscript TF is an abbreviation for time and frequency.

[0056] In this invention, the feature matrix of the obtained EEG signal is used to replace the EEG signal matrix for subsequent processing. Since the data of each row of the feature matrix is ​​used to replace the EEG data of each channel of the EEG signal matrix for continuous wavelet transform to generate a time-frequency graph dataset, the data of each row of the feature matrix is ​​referred to as a "virtual channel" in this invention.

[0057] Step C includes the following specific steps:

[0058] C1: Since the time-frequency image is composed of stacked channels, in order to fully utilize the feature information of all 20 channels in the time-frequency image, this invention uses the CSP algorithm to perform spatial filtering on the preprocessed motor imagery EEG signal. During spatial filtering, the CSP projection matrix W is first calculated, and then the feature matrix Z is calculated using the CSP projection matrix W and the preprocessed motor imagery EEG signal obtained in step A. The calculation formula is as follows:

[0059] Z M×T =W M×M E M×T ;

[0060] Where W is the CSP projection matrix, M is the number of channels in the preprocessed EEG data, T is the data length, and E is the EEG data matrix of size M×T formed by converting the preprocessed motor imagery EEG signals.

[0061] C2: Select the feature matrix extracted by CSP;

[0062] Since the feature matrix Z obtained in step C1 has an equivalence in feature information distribution, with the feature information mainly concentrated at the head and tail of the feature matrix Z, while the feature information in the middle is not obvious and can be ignored, this invention selects feature matrix Z... M×T The first m rows and the last m rows of data are used as the feature matrix Z′ for CSP feature extraction; where 2m <M;

[0063] C3: Based on the feature matrix Z′ obtained in step C2, time-frequency features are extracted using the CWT algorithm and transformed into a two-dimensional time-frequency map with a resolution of 64×64. The Morlet wavelet is selected as the basis function for wavelet transform, and finally the motion imagination time-frequency map dataset is obtained.

[0064] The formula for wavelet transform is as follows:

[0065]

[0066]

[0067]

[0068] Here, Morlet is a commonly used basis function in wavelet transform, used to decompose the signal to be processed during the wavelet transform process; Φ(t) represents the basis function, ω represents the wavelet center frequency; t represents time; i represents the sequence that changes with time; Φ α，β (t) represents the basis function after scaling and translation transformation, α represents the scaling factor, β represents the time translation factor, CWT(α,β) represents the result after performing wavelet transform on the signal, and f(t) represents the EEG signal.

[0069] C4: Based on the obtained motion visualization time-frequency graph dataset, a convolutional neural network is used for feature extraction and classification in an image processing manner to obtain the probability distribution matrix P of the time-frequency graph signal. TF ,

[0070] D: Using the probability distribution matrix P of the EEG signal obtained in step B EEG And the probability distribution matrix P of the time-frequency signal obtained in step C. TF Calculate the clustering coefficient for each action category, and then apply the calculated average clustering coefficient to the probability distribution matrix P. EEG and P TF Optimization is performed to obtain the optimized probability distribution matrix P. EEG* and P TF* Furthermore, based on the DS evidence theory and utilizing the probability distribution matrix P... EEG* and P TF* A fusion decision is made to obtain the final classification result;

[0071] In this invention, the probability distribution matrix P of the obtained electroencephalogram (EEG) signal EEG The probability distribution matrix P of the time-frequency signal TF In classification problems, evidence is called "evidence" and can be used to determine whether a classification result belongs to a certain category. Based on the DS evidence theory, uncertain reasoning can be completed to integrate multiple pieces of evidence into comprehensive evidence, that is, to obtain a new basic probability allocation and realize the fusion of different information.

[0072] The rules of composition for the DS evidence theory are as follows:

[0073] Suppose two independent probability distribution matrices, P obtained from EEG signal classification, are respectively... EEG And P obtained through time-frequency diagram classification TF Let P be the new probability distribution matrix generated by fusion, then the synthesis rule is defined as:

[0074]

[0075] Where k is the evidence conflict factor, k = ∑ B∩C＝φ P EEG (B)P TF (C), where A, B, and C represent probability distribution matrices P, P0, P1, P2, P3, P4, P5, P6, P7, P8, P9, P10, P11 EEG and P TF The three action categories are: when k=1, the evidence is completely conflicting, synthesis cannot be carried out, and the synthesis rule fails; when 0<k<1, the probability distribution matrices of different events are fused using the synthesis rule to realize the fusion decision process.

[0076] Individual anomalous pieces of evidence often lead to conflicting evidence, resulting in inaccurate or even erroneous fusion results. Furthermore, the two probability distribution matrices obtained through feature extraction and classification of EEG signals and time-frequency image signals are not identical and can still be optimized. To address this issue, this invention introduces clustering coefficients before the fusion decision, readjusts the basic probability assignments of each piece of evidence, and then uses fusion rules to make the fusion decision.

[0077] Clustering coefficients represent the degree of clustering of nodes of the same type. In a specific network, nodes of the same type tend to cluster together. For example, if node v1 is connected to node v2, and node v2 is connected to node v3, then node v3 is very likely connected to v1, or in other words, closely related. This phenomenon reflects the dense connection property between some nodes. The clustering coefficient of any node can be expressed as:

[0078]

[0079] Where h represents the number of all neighboring nodes of a node, i.e., the number of neighbors of the node; n represents the number of edges connecting all neighboring nodes of a node; the average clustering coefficient can be obtained by averaging the clustering coefficients of all nodes in the network. In this invention, the average clustering coefficient of each category is calculated. The probability distribution matrices obtained from each classifier are optimized, and the final optimized probability distribution matrix is ​​as follows:

[0080]

[0081] A higher clustering coefficient indicates a greater degree of node clustering, and thus plays a more significant role in the overall evaluation. Therefore, the clustering coefficient effectively represents the tightness of the connections between data groups; a higher value indicates tighter connections, more similar characteristics of the data, and a more accurate classification. Therefore, the clustering coefficient for each category can be used to represent its credibility. The evidence can be optimized by averaging the clustering coefficient, and then the optimized P-value can be used... EEG* and P TF* Replace the original probability distribution matrix P EEG and P TF By performing fusion decision-making to obtain the final classification result, the classification accuracy of unilateral limb movement information tasks can be effectively improved.

[0082] Example:

[0083] A: First, we selected the "Multimodal signal dataset of 11 intuitive motor tasks of a single upper limb during multiple recordings" from the Giga DB dataset to construct a unilateral limb motor imagery dataset.

[0084] This dataset, completed by Ji-Hoon Jeong et al. in 2020, includes not only EEG data but also electromyography (EMG) and electrooculogram (EOG) data. All data were collected synchronously in the same experimental environment without interference. The EEG signal acquisition equipment used in the dataset employed electrode placement according to the 10-20 international configuration, utilizing 60 EEG channels, 6 EMG channels, and 4 EOG channels at a sampling rate of 2500 Hz. The duration of the motor imagery was 4 seconds, with each subject undergoing three identical recording cycles, recording both actual and imagined motor data. This invention utilizes the raw EEG data from the imagined motor imagery within the dataset.

[0085] Then, the motor imagery EEG signals in the unilateral limb motor imagery dataset were preprocessed to obtain the preprocessed motor imagery EEG signals.

[0086] In this embodiment, data preprocessing includes bandpass filtering, downsampling, channel selection, independent component analysis for artifact removal, and data selection of motor imagery EEG signals. The tool used is the EEG signal analysis and processing toolbox EEGLAB in MATLAB.

[0087] Bandpass filtering: Considering that filtering EEG data can effectively reduce noise in the signal and improve the signal-to-noise ratio of the frequency band of interest; numerous studies have shown that the frequency band related to motion is 8–30 Hz. Therefore, this invention selects to use a 4th-order Butterworth bandpass filter to filter the raw EEG data from 8 to 30 Hz, thereby obtaining the frequency band of motion-related EEG.

[0088] Downsampling: Since the original EEG signal is sampled at a frequency of 2500Hz, the corresponding EEG data is huge, which will lead to a significant reduction in the speed of subsequent preprocessing and feature analysis. Therefore, in this invention, the EEG signal is downsampled to 250Hz.

[0089] Channel selection: Since the original EEG data in the constructed unilateral limb motor imagery dataset contains 60 EEG channels, 4 EOG channels, and 7 EMG channels, to obtain the final EEG data, we need to remove the 7 EMG channels and 4 EOG channels, resulting in 60 EEG channels. In this invention, these 60 EEG channels are further selected, with 20 channels from the sensorimotor cortex of the brain chosen for classification.

[0090] Independent Component Analysis (ICA) for Artifact Removal: Because the amplitude of EEG signals is very weak and susceptible to eye movement interference, the data becomes distorted. The simplest method is to directly remove these distorted components, but this loses the effective components of the data, causing significant errors in subsequent research and analysis. Therefore, Independent Component Analysis (ICA) is often used in the preprocessing stage of EEG signals to calculate and remove interference. ICA is a method used to find hidden factors or components in multivariate (multidimensional) statistical data, and it is the most commonly used blind source separation technique in EEG artifact removal. This method decomposes multichannel EEG data from different sources into independent components (ICs), determines whether the independent components are eye movement, electromyography (EMG), or other artifacts, and removes them if they are artifacts. The remaining independent components are then used to reconstruct EEG data free of eye movement and other artifacts, thus achieving the effect of removing EMG, EMG, and other artifacts. In this invention, EEGLAB is used to perform ICA calculations and artifact removal on EEG signals.

[0091] In the unilateral limb motor imagery dataset, only the motor imagery EEG signals of three actions—extending the arm forward, rotating the wrist to the left, and grasping a water cup—were selected for classification.

[0092] B: The preprocessed motor imagery EEG signals obtained in step A are used for feature extraction and classification using a deep neural network model to obtain the probability distribution matrix P of the EEG signals. EEG ,

[0093] Electroencephalogram (EEG) signal data is typically presented as a two-dimensional matrix consisting of signal channels and sampling points. Each row of the matrix represents the sampled data for each channel. This form of EEG signal data encompasses both its spatial and temporal characteristics. The EEG signal data used in this invention is processed for 4 seconds at a sampling frequency of 250 Hz, resulting in a data length of 1000. The final EEG signal data is processed into a 20×T matrix, where 20 represents the number of channels and T = 1000 represents the data length.

[0094] In this invention, the convolutional neural network structure used is EEG-CNN, which includes convolutional layers, pooling layers, and fully connected layers; wherein two convolutional layers and a max-pooling layer are combined to form a feature extraction module; the EEG signal E input to the network... M×TIt is an M x T matrix, where M represents the number of channels and T represents the length of the EEG signal. In this invention, instead of the commonly used two-dimensional convolution, one-dimensional convolution is used in the network construction. The kernel of a one-dimensional convolution moves only in one direction, and the calculation is performed only along the length of the signal, extracting temporal features from the EEG data. Compared to two-dimensional CNNs, this design has fewer model parameters and is more efficient. Classical CNNs rely on multiple layers of convolution and pooling to extract different basic features to improve classification accuracy. However, increasing the number of network layers causes the network parameters to increase exponentially, hindering rapid convergence and severely impacting network performance. Considering that our input is a relatively simple matrix vector, we use a shallower network depth, reducing the number of convolutional layers used. Only two sets of convolutional and pooling layers are used to form the feature extraction module to extract feature information from the input signal. The application of two convolutional layers allows the network to have more non-linear variations and adapt to complex patterns. The one-dimensional convolutional kernel in the convolutional layer is used to extract the features of each channel as the feature map output by this layer. In this embodiment, the size of the convolutional kernel is set to 3×1, and the stride is 1. After convolution, N can be obtained. w ×N f Feature map of the form; where N w Let N be a vector. f The number of convolutional kernels is denoted as 1; the max pooling layer is used to downsample the data output by the convolutional layer. In this embodiment, the pooling layer is selected with a kernel size of 2×1 and a stride of 2; the fully connected layer is used to flatten the features extracted by the convolutional layer.

[0095] C: The preprocessed motor imagery EEG signal obtained in step A is spatially filtered using the CSP method to obtain the feature matrix Z of the EEG signal. Feature matrix Z′ is obtained by selecting from Z, and continuous wavelet transform is performed on Z′ to obtain the time-frequency map dataset of motor imagery. Finally, the time-frequency map dataset is used to extract features and classify them using a deep neural network model to obtain the probability distribution matrix P of the time-frequency map signal. TF ,

[0096] In this invention, the conversion of the time-frequency diagram consists of the following four steps:

[0097] C1: Obtaining the characteristic matrix Z;

[0098] Since the conversion and generation of the time-frequency map are performed independently for each channel, the 20 EEG channels used in this invention cannot all be combined into a single image. Selecting only a few channels would result in information loss, losing much useful channel information. To effectively utilize the information from each channel, this invention preprocesses the data to extract features. Specifically, it uses the CSP method to perform spatial filtering on the EEG signals from the 20 EEG channels, obtaining virtual channels, and then generating the time-frequency map. CSP is a spatial filtering feature extraction algorithm for binary classification tasks. It can extract the spatial distribution components of each class from multi-channel EEG data. The basic principle of the algorithm is to use matrix diagonalization to find a set of optimal spatial filters for projection, maximizing the difference in variance between the two classes of signals, thereby obtaining feature vectors with high discriminative power. For three-class classification tasks, the "OVR" strategy (One vs Rest) can be used to extend CSP to achieve multi-class CSP feature extraction. In this invention, the feature matrix Z of the EEG signal is obtained based on the calculated CSP projection matrix W and the preprocessed EEG signal obtained in step A. The calculation formula is as follows:

[0099] Z M×T =W M×M E M×T ;

[0100] Where W is the CSP projection matrix, M is the number of channels of the preprocessed EEG data, M=20, T is the data length, and E is the EEG data matrix of size M×T formed by converting the preprocessed motor imagery EEG signals in step A.

[0101] C2: Select the feature matrix extracted by CSP;

[0102] In this embodiment, the feature matrix Z is selected. M×T The first and last rows of data (i.e., m=1) are used as the feature matrix for CSP feature extraction, resulting in the feature matrix Z′. 2×T This is used to replace the preprocessed motor imagery EEG signal data for time-frequency mapping conversion.

[0103] C3: Generation of time-frequency graph dataset;

[0104] Feature information was extracted from EEG signals using CWT and ultimately converted into a 64×64 resolution two-dimensional time-frequency map. This method was then used to process all EEG signals to obtain a time-frequency map dataset of motor imagery.

[0105] C4: Feature extraction and classification of time-frequency graph signals;

[0106] Based on the obtained motion visualization time-frequency graph dataset, a convolutional neural network is used to extract and classify features in an image processing manner, resulting in the probability distribution matrix P of the time-frequency graph signal. TF ,

[0107] The convolutional neural network architecture used is TF-CNN, which can be based on VGG16 as the basic network framework. VGG16 is a typical convolutional neural network structure, characterized by its inclusion of convolutional kernel computation and feedforward structures. Compared to other CNNs, VGG16 offers high accuracy and stability, thus gaining widespread application. Therefore, this invention uses the VGG16 network model for extracting time-frequency image features. The VGG16 network uses multiple consecutive convolutions, requiring fewer parameters than a single large convolutional kernel, while exhibiting more non-linear variations and adapting to more complex patterns. Furthermore, the concatenation of convolutional kernels and multiple feature extractions result in more refined features than a single convolutional kernel. It performs well in feature extraction for time-frequency images with relatively small differences. VGG16 contains 16 hidden layers (13 convolutional layers and 3 fully connected layers). The convolutional kernels used in the convolutional parts are all 3×3 with a stride of 1, and the max-pooling layers are 2×2 with a stride of 2.

[0108] In this invention, the training phase of the classification model used for EEG signals and time-frequency maps uses the Adam optimizer with β1 = 0.9 and β2 = 0.999 to update the trainable parameters of each network layer, with the initial learning rate set to 0.01.

[0109] D: Using the probability distribution matrix P of the EEG signal obtained in step B EEG And the probability distribution matrix P of the time-frequency signal obtained in step C. TF Calculate the clustering coefficient for each action category, and then apply the average clustering coefficient to the probability distribution matrix P. EEG and P TF Optimization is performed to obtain the optimized probability distribution matrix P. EEG* and P TF* Furthermore, based on the DS evidence theory and utilizing the probability distribution matrix P... EEG* and P TF* A fusion decision is made to obtain the final classification result;

[0110] D1: Based on the probability distribution matrix P EEG and P TF The probability of each action category is included, and the clustering coefficient CC of each action category data in the two probability distribution matrices is calculated. Then, the average clustering coefficient is obtained by averaging the clustering coefficients of all nodes in each action category data.

[0111] D2: The obtained probability distribution matrix P EEG and P TF Using the average clustering coefficients as weights, the two probability distribution matrices are re-optimized and corrected to obtain the optimized probability distribution matrix P. EEG * and P TF *;

[0112] D3: Utilize the DS fusion rule to optimize the probability distribution matrix P EEG * and P TF * Perform a fusion decision to obtain a new probability distribution matrix P, and obtain the final classification result based on it.

[0113] In this invention, the average classification accuracy of 25 subjects across three acquisition processes was tested. The classification results were compared using the method described in this invention (referred to as DS fusion result), the aforementioned network model for classifying EEG signals (referred to as EEG-CNN), the aforementioned network model for classifying time-frequency maps (referred to as TF-CNN), and the classic CSP algorithm most commonly used in EEG signal classification. The comparison figures are shown below. Figure 2 As shown, it is evident that the EEG classification method described in this invention is more effective than other methods.

Claims

1. A method for classifying unilateral limb motor imagery tasks using electroencephalograms, characterized in that, The steps are as follows: A: Construct a unilateral limb motor imagery dataset, and perform data preprocessing on the motor imagery EEG signals within the unilateral limb motor imagery dataset to obtain preprocessed motor imagery EEG signals. B: The preprocessed motor imagery EEG signals obtained in step A are used for feature extraction and classification using a deep neural network model to obtain the probability distribution matrix of the EEG signals. Probability distribution matrix It includes the probability that the preprocessed motor imagery EEG signal belongs to the corresponding action category; C: Spatial filtering is performed on the preprocessed motor imagery EEG signal obtained in step A using the CSP method to obtain the feature matrix of the EEG signal; Then, the time-frequency graph dataset is obtained by selecting the feature matrix and performing continuous wavelet transform. Finally, a deep neural network model is used to extract features and classify the time-frequency graph dataset to obtain the probability distribution matrix of the time-frequency graph signal. Probability distribution matrix It contains the probability that the time-frequency graph signal belongs to the corresponding action category; D: Using the probability distribution matrix of the EEG signals obtained in step B and the probability distribution matrix of the time-frequency signal obtained in step C. Calculate the clustering coefficient for each action category, and then apply the calculated average clustering coefficient to the probability distribution matrix. and Optimization is performed to obtain the optimized probability distribution matrix. and ; Based on the DS evidence theory and utilizing the probability distribution matrix and A fusion decision is made to obtain the final classification result; Step D includes the following specific steps: D1: Based on the probability distribution matrix and The probability of each action category included is used to calculate the clustering coefficient of each action category data in the two probability distribution matrices. Then, the average clustering coefficient is obtained by averaging the clustering coefficients of all nodes for each action category. ; in, ; Indicates the number of neighbors of a node; This represents the number of edges connecting all adjacent nodes of a given node. D2: The obtained probability distribution matrix and Using the average clustering coefficients as weights, the two probability distribution matrices are re-optimized to obtain the optimized probability distribution matrix. and ; = ， * = ； D3: Optimize the probability distribution matrix using the DS fusion rule. and A fusion decision is made to obtain a new probability distribution matrix P, and the final classification result is obtained based on this matrix. ； Where k is the evidence conflict factor. A, B, and C represent probability distribution matrices P, B, C, and C, respectively. and The three action categories.

2. The EEG classification method for unilateral limb motor imagery tasks according to claim 1, characterized in that: In step A, data preprocessing includes bandpass filtering, downsampling, channel selection, independent component analysis for artifact removal, and data selection of motor imagery EEG signals.

3. The EEG classification method for unilateral limb motor imagery tasks according to claim 1, characterized in that: In step A, only the EEG signals of motor imagery under three actions—extending the arm forward, rotating the wrist to the left, and grasping a water cup—are selected from the unilateral limb motor imagery dataset. In step B, ,in, These represent the probabilities that a sample result belongs to the corresponding action category in a three-class classification experiment of electroencephalogram signals; In step C, ,in, These represent the probabilities that a sample result belongs to the corresponding action category in a three-class classification experiment using time-frequency graphs.

4. The EEG classification method for unilateral limb motor imagery tasks according to claim 1, characterized in that, Step C includes the following specific steps: C1: Spatial filtering of preprocessed motor imagery EEG signals is performed using the CSP algorithm; When performing spatial filtering, the CSP projection matrix is ​​first calculated. Then, using the CSP projection matrix The feature matrix is ​​calculated from the preprocessed motor imagery EEG signal obtained in step A. ; The calculation formula is as follows: ;in, For CSP projection matrix, The number of channels in the preprocessed EEG data. For data length, The size of the preprocessed motor imagery EEG signal is formed by conversion. The EEG data matrix; C2: Select the feature matrix extracted by CSP. ; C3: Based on the feature matrix obtained in step C2 The CWT algorithm is used to extract time-frequency features and transform them into a two-dimensional time-frequency graph. Morlet wavelet is selected as the basis function for wavelet transform, and finally the motion imagination time-frequency graph dataset is obtained. The formula for wavelet transform is as follows: ； ； ； Among them, Morlet is a commonly used basis function in wavelet transform, which is used to decompose the signal to be processed during the wavelet transform process; ω represents the wavelet center frequency; t represents time; i represents the time-varying sequence. Let represent the basis functions after scaling and translation transformations, where α represents the scaling factor and β represents the time translation factor. This represents the result after performing a wavelet transform on the signal. Represents brain electrical signals; C4: Based on the obtained motion visualization time-frequency graph dataset, a convolutional neural network is used for feature extraction and classification in an image processing manner to obtain the probability distribution matrix of the time-frequency graph signal. , , These represent the probabilities that a sample result belongs to the corresponding action category in a three-class classification experiment using time-frequency graphs.

5. The EEG classification method for unilateral limb motor imagery tasks according to claim 4, characterized in that: In step C2, the feature matrix is ​​selected. The first m rows and the last m rows of data are used as the feature matrix for CSP feature extraction. ; where 2m < .

6. The EEG classification method for unilateral limb motor imagery tasks according to claim 2, characterized in that: When performing bandpass filtering, a fourth-order Butterworth bandpass filter was used to filter the raw EEG data from 8 to 30 Hz to obtain the frequency band of motor-related EEG signals. When performing downsampling, the EEG signals were downsampled to 250 Hz. When performing channel selection, 20 EEG channels from the sensorimotor cortex of the brain were selected for classification. When performing independent component analysis for artifact removal, EEGLAB was used to perform ICA calculation and artifact removal on the EEG signals.

7. The EEG classification method for unilateral limb motor imagery tasks according to claim 1, characterized in that: In step B, the convolutional neural network structure used is EEG-CNN, which includes convolutional layers, pooling layers, and fully connected layers; two convolutional layers and a max pooling layer are combined to form a feature extraction module, and the one-dimensional convolutional kernel in the convolutional layer is used to extract the features of each channel as the feature map output by this layer.

8. The EEG classification method for unilateral limb motor imagery tasks according to claim 1, characterized in that: In step C, the convolutional neural network structure is TF-CNN, and VGG16 is used as the basic network framework for extracting time-frequency map feature information.

9. The EEG classification method for unilateral limb motor imagery tasks according to claim 1, characterized in that: The convolutional neural networks in steps B and C are trained using... , The Adam optimizer updates the trainable parameters of each network layer, with the initial learning rate set to 0.01.

Images