A method for recognizing EEG features of motor imagery based on force changes during unilateral upper limb movement

Abstract

The present invention discloses a method for identifying EEG features of motor imagery during force changes in unilateral upper limb movement. The method comprises the following steps: Step 1: collecting EEG signals of a subject performing a motor imagery process under a paradigm-induced model during an EEG signal acquisition experiment, wherein the paradigm is designed to be a motor imagery process during force changes in unilateral upper limb movement; Step 2: preprocessing the EEG signals of each single experiment; Step 3: constructing a hybrid neural network for identifying EEG features of motor imagery during force changes in unilateral upper limb movement; and Step 4: training the entire hybrid neural network in a supervised manner. The present invention can be applied to the dynamic force interaction process between a robot and a patient in a brain-controlled rehabilitation robot system.

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interaction technology, and in particular to a method for recognizing EEG characteristics of motor imagery with force changes during unilateral upper limb movement. Background Art

[0002] In motor imagery brain-computer interface (BCI) technology, a rehabilitation robot is used as an external device to form a brain-controlled rehabilitation robot system. Through closed-loop feedback control, a new artificial neural pathway is established between the brain and the affected limb. This can help patients with brain injuries such as stroke achieve neural remodeling and improve limb motor function, and has great potential for clinical application. Existing BCIs typically use a motor imagery paradigm for simple limb movements, such as left and right hand and foot movements. The number of recognizable thought states induced is relatively limited, and only a small number of instructions can be provided to control the movement direction of the rehabilitation robot. However, rehabilitation robot-assisted training primarily emphasizes the dynamic interaction of forces between the robot and the patient. Due to the varying muscle strength and motor abilities of patients, the robot must provide force in the direction of movement based on the patient's desired force to assist the patient in completing limb movement training. Therefore, how to enable BCIs to induce and recognize the patient's thought state regarding force assistance and enable the brain to control the force of the rehabilitation robot's movement direction is of great value to the clinical application of brain-controlled rehabilitation robot systems.

[0003] Some researchers have designed static force-variation motor imagery paradigms for unilateral upper limb movements and have achieved the identification of EEG signal features corresponding to the brain states induced by these paradigms. For example, Xu et al. designed a right-hand grip force-variation motor imagery paradigm, introducing three levels of force (20%, 50%, and 80% of maximum voluntary contraction (MVC)) for right-hand fist clenching. They then used the Hilbert transform (HHT) and support vector machine (SVM) algorithms to extract features and classify subjects' EEG data. These studies expanded the number of recognizable mental state categories for motor imagery brain-computer interfaces by measuring the force of static limb movements. However, static force-variation motor imagery paradigms are difficult to apply to the dynamic force interaction between the robot and the patient in brain-controlled rehabilitation robotic systems. Furthermore, traditional methods for EEG feature recognition rely on manual feature extraction, resulting in limited feature learning and, as the number of possible categories increases, low classification accuracy. To address these issues, researchers have begun to investigate deep learning algorithms for EEG feature recognition. Yang et al. proposed using common spatial patterns (CSP) to extract EEG spatial features, and then using a convolutional neural network (CNN) to learn deep features. However, due to the poor ability of CNNs to learn EEG temporal features, feature learning was insufficient, resulting in poor classification performance. Ma et al. proposed a time-distributed attention network (TD-Atten) to identify EEG features. They used a sliding window approach to continuously slice EEG signals and then used a one-versus-rest filter bank common spatial pattern (OVR-FBCSP) algorithm to extract frequency-spatial features from the sliced EEG. They then used an attention mechanism and a long short-term memory (LSTM) network to further extract EEG temporal features. However, this network model still required manual EEG feature extraction, and classification accuracy was limited by the quality of feature extraction. Schirrmeister et al. adopted an end-to-end approach to automatically learn features from EEG data, avoiding manual feature extraction. Because EEG signals have strong temporal correlation, this approach primarily uses a temporal convolutional neural network (TCN) to extract the temporal state characteristics of EEG signals for classification. However, this end-to-end network uses a single-scale convolution kernel to extract EEG features, resulting in relatively simple feature information. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for identifying EEG characteristics of motor imagery with force changes under unilateral upper limb movement, which can be applied to the dynamic force interaction process between the robot and the patient in a brain-controlled rehabilitation robot system.

[0005] A method for recognizing EEG features of motor imagery based on force changes during unilateral upper limb movement, comprising the following steps:

[0006] Step 1: During the EEG signal acquisition experiment, EEG signals of the subject performing a motor imagery process under the induction of a paradigm, wherein the paradigm is designed to be a motor imagery process of force changes under unilateral upper limb movement;

[0007] Step 2: Preprocess the EEG signals of each single experiment;

[0008] Step 3: constructing a hybrid neural network for use in EEG feature recognition of motor imagery of force changes in unilateral upper limb movement, wherein the hybrid neural network includes an input layer, a dimension conversion layer, a multi-scale temporal convolutional network module, a spatial convolution module, a pooling layer, a dropout layer, an attention module, a tiling layer, and a fully connected layer connected in series in sequence;

[0009] Step 4: Train the entire hybrid neural network in a supervised manner.

[0010] Preferably, the acquisition experiment in step 1 includes multiple rounds, each round includes several single experiments, and each round of experiments is arranged as a group from small to large intensity, and the intensity of each round of motor imagery experiment is consistent.

[0011] Preferably, in step 2, bilateral mastoid averaging is first used for re-reference, then the EEG signal is baseline corrected, 0.5-100 Hz band-pass filtering and 50 Hz power frequency notch filtering are performed to remove vertical electrooculogram artifacts and bad blocks, and then the EEG signal is spatially filtered using a common average reference to improve the signal-to-noise ratio, and 8-30 Hz band-pass filtering is performed to retain the EEG signal in the frequency band related to motor imagery, and finally the EEG signal is downsampled.

[0012] Preferably, the step 3 is as follows:

[0013] Step 3-1: The input of the input layer is the preprocessed two-dimensional EEG signal h∈R N×M , R N×M represents a two-dimensional real number space, N and M are the number of leads and time points of the EEG signal, respectively;

[0014] Step 3-2: Dimension conversion layer transforms the two-dimensional EEG signal h∈R N×M Converted into three-dimensional EEG data h∈R 1×N×M, which has three dimensions: channel, height and width, R 1×N×M Represents three-dimensional real space;

[0015] Step 3-3: Construct a multi-scale temporal convolutional network module to perform three-dimensional EEG data h∈R 1×N×M Extract multidimensional fine-grained time-frequency domain features for each single experiment, then splice and fuse the extracted multidimensional fine-grained time-frequency domain features to obtain enhanced EEG signal time-frequency domain feature information;

[0016] Step 3-4: construct a spatial convolution module to continue extracting spatial features from the enhanced EEG signal time-frequency domain feature information to obtain EEG signal time-frequency-space domain feature information;

[0017] Steps 3-5: After the spatial convolution module, the pooling layer and the dropout layer are connected in series in sequence. The pooling layer aggregates the obtained EEG signal time-frequency-space domain feature information in the width dimension, and the dropout layer randomly discards the feature information to avoid the influence of noise data.

[0018] Steps 3-6: Construct an attention module and use a channel attention mechanism to focus on features related to the intensity motor imagery category in the channel dimension of the EEG signal time-frequency-spatial domain feature information after the pooling layer and the dropout layer, thereby suppressing feature redundancy.

[0019] Steps 3-7: After the attention module, the tiled layer and the fully connected layer are connected in series to further process the output features of the attention module.

[0020] Preferably, the multi-scale temporal convolutional network module is formed by connecting multiple sets of temporal convolutional layers and batch normalization layers in parallel, and then connecting them in series with a feature splicing and fusion layer. The temporal convolutional layers are respectively configured with three small-sized convolution kernels of sizes (1, ka), (1, kb), and (1, kc), and the size difference between the three convolution kernels is small.

[0021] Preferably, the spatial convolution module is formed by sequentially connecting a spatial convolution layer, a batch normalization layer, and a nonlinear activation layer in series.

[0022] Preferably, the pooling layer adopts an average pooling strategy, and uses a larger-sized pooling kernel to aggregate the time-frequency-space domain feature information of the EEG signal extracted by the multi-scale time convolution network module and the spatial convolution module in the local range of the pooling kernel in the width dimension, thereby reducing the information redundancy of the feature information in the width dimension.

[0023] Preferably, the dropout layer randomly drops feature information according to a certain dropout rate during the network training process to avoid the influence of noise data and prevent the network from overfitting.

[0024] Preferably, the attention module first uses global average pooling and global maximum pooling methods to aggregate the time-frequency-space domain feature information of the EEG signal after the pooling layer and the discard layer in the global range of height and width dimensions to obtain average pooling features and maximum pooling features. Secondly, the average pooling features and maximum pooling features are respectively obtained by a shared network composed of a multi-layer perceptron to obtain two channel attention maps. Again, the corresponding elements of the two channel attention maps are added to obtain the final channel attention map, and each element of the final channel attention map is operated to generate a channel attention weight. The input of the attention module is dynamically weighted by the channel attention weight to focus on features related to the intensity movement imagination category.

[0025] Preferably, in step 4, during the training process, the network is optimized using a cross entropy loss function as the objective function.

[0026] Beneficial effects of the present invention:

[0027] The hybrid neural network model constructed by the present invention performs feature recognition on the motor imagery EEG signals of subjects induced by the force variation motor imagery paradigm under unilateral upper limb movement. In view of the relatively subtle differences in the time-frequency domain features of the single-experiment motor imagery EEG signals corresponding to different forces, a multi-scale temporal convolutional network module is designed to extract fine-grained features of different dimensions from each single-experiment EEG signal after preprocessing, overcoming the problem that the features learned by the single-scale temporal convolutional neural network are relatively limited. A pooling layer is designed with a larger pooling kernel to aggregate the multi-dimensional time-frequency-space domain feature information learned by the multi-scale temporal convolutional network module and the spatial convolution module, thereby reducing the information redundancy of the feature information in the width dimension. An attention module is designed to dynamically weight the multi-dimensional time-frequency-space domain feature information after the information redundancy in the width dimension is reduced through the channel attention mechanism, further focusing on the features related to the force motor imagery category in the channel dimension, reducing the information redundancy of the feature information in the channel dimension, thereby improving the sensitivity of the hybrid neural network model to features, better fitting performance of the data, and improving classification performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a single experimental process of the force change movement imagery paradigm under unilateral upper limb movement state of the present invention.

[0029] Figure 2 It is a structural diagram of the hybrid neural network of the present invention.

[0030] Figure 3 It is a structural diagram of the multi-scale temporal convolutional network module in the hybrid neural network of the present invention.

[0031] Figure 4 It is a structural diagram of the spatial convolution module in the hybrid neural network of the present invention.

[0032] Figure 5 It is a structural diagram of the attention module in the hybrid neural network of the present invention. DETAILED DESCRIPTION

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0034] This embodiment designs the paradigm as a process of imagining movement with varying force under unilateral upper limb movement based on the action process of wiping a desktop in daily life. The force level of the imagination of the movement with varying force is designed according to the difficulty of wiping the desktop stains, which helps the subjects better imagine and thus increases the number of thinking categories of imagination of movement with varying force under unilateral upper limb movement.

[0035] A method for recognizing EEG features of motor imagery during force changes in unilateral upper limb movement, comprising the following steps:

[0036] Step 1: Collect the EEG signals of the subjects performing the motor imagery process under the induction of the designed paradigm. Three levels of strength (small strength, medium strength, and large strength) are introduced into the paradigm. The three levels of strength motor imagery scenes correspond to wiping dust, mud, and tea stains, three stains with different wiping difficulties. The motor imagery categories correspond to small strength imagination, medium strength imagination, and large strength imagination. In the EEG signal collection experiment, the strength of each round of motor imagery experiments is consistent. Each 3 rounds of experiments are arranged in groups according to small, medium, and large strengths. There are 24 single experiments in each round. The single experiment process is as follows: Figure 1 As shown, it is divided into 4 periods, totaling 16 seconds. The first period is the preparation period, in which a white circle appears in the center of the computer screen for 2 seconds, and the subject remains relaxed. The second period is the prompt period, in which the white circle disappears and a text prompt appears. This period lasts for 4 seconds, and the subject remains relaxed and does not make any movements. The third period is the motor imagery period, in which the text prompt disappears and the screen goes black for 6 seconds, and the subject performs a force motor imagery. The fourth period is the rest period, in which the word "rest" appears on the screen for 4 seconds, and the subject remains relaxed.

[0037] Step 2: Preprocess the EEG signals from each single experiment. First, use bilateral mastoid averaging for re-referencing. Next, perform baseline correction on the EEG signals, apply 0.5-100 Hz bandpass filtering and 50 Hz power frequency notch filtering to remove vertical eye artifacts and remove bad blocks. Then, spatially filter the EEG signals using common mean reference to improve the signal-to-noise ratio, and perform 8-30 Hz bandpass filtering to retain EEG signals in the frequency band related to motor imagery. Finally, downsample the EEG signals. The sampling rate is generally 128 Hz or 256 Hz.

[0038] Step 3: Construct a hybrid neural network for the EEG feature recognition of motor imagery with force changes in unilateral upper limb movement. The network consists of an input layer, a dimension conversion layer, a multi-scale temporal convolutional network module, a spatial convolution module, a pooling layer, a dropout layer, an attention module, a tiling layer, and a fully connected layer connected in series in sequence, as shown in the following example: Figure 2 The specific steps are as follows:

[0039] Step 3-1: The input of the input layer is the preprocessed two-dimensional EEG signal h∈R N×M , R N×M Represents a two-dimensional real number space, N and M are the number of leads and time points of the EEG signal, respectively.

[0040] Step 3-2: Dimension conversion layer transforms the two-dimensional EEG signal h∈R N×M Converted into three-dimensional EEG data h∈R 1×N×M , which has three dimensions: channel, height and width, R 1×N×M Represents three-dimensional real space.

[0041] Step 3-3: Construct a multi-scale temporal convolutional network module to perform three-dimensional EEG data h∈R 1×N×M Extract multi-dimensional fine-grained time-frequency domain features according to each single experiment. This module connects three sets of temporal convolution layers and batch normalization layers in parallel, and then connects them in series with a feature splicing and fusion layer, such as Figure 3 As shown. Among them:

[0042] Each group of temporal convolution layers and batch normalization layers connected in series includes one temporal convolution layer and one batch normalization layer. The three temporal convolution layers are respectively set with small-sized convolution kernels of sizes (1, ka), (1, kb), and (1, kc). The size difference between the three convolution kernels is small. The number of convolution kernels in each temporal convolution layer is 8, the step size is 1, and the activation function adopts a linear activation function. In this embodiment, the sizes of the small-sized convolution kernels are set to (1, 3), (1, 5), and (1, 7), and the size difference between the three convolution kernels is 2 respectively. The convolution operation process is as follows:

[0043] The input of each temporal convolutional layer is three-dimensional EEG data h∈R 1×N×M, the feature output extracted by a single convolution kernel is in b k are the weight matrix and bias of the kth convolution kernel, k = 1, 2, ... 8, f(·) is the linear activation function, and the output feature of each convolution layer is x ij ∈R 8×N×M .

[0044] Each time convolution layer is followed by a batch normalization layer in series to normalize the output features of the time convolution layer and regularize the model parameters to prevent network overfitting and improve the convergence speed of the network. The batch normalization process is as follows:

[0045] Define a batch of data as B={x1,x2...,x m}, m represents the number of single experiments corresponding to a batch, and the mean of a batch of data is calculated variance The standardized output is The denormalized output is z i =γy i +β, the constant ε is introduced to maintain the stability of numerical calculations and prevent the denominator from being equal to 0. In this embodiment, m is set to 16, that is, the data of 16 single experiments are regarded as a batch of data.

[0046] The purpose of the batch normalization layer is to speed up the convergence of network training, but this can lead to a decrease in the network's expressiveness. To prevent this, denormalization is added to the batch normalization process. Two adjustment parameters (scaling: γ and offset: β) are added to each neuron in the network. These two parameters are learned through training, which enhances the network's expressiveness.

[0047] The feature splicing and fusion layer is used to splice and fuse the multidimensional features extracted by the three sets of serial temporal convolution layers and batch normalization layers to obtain the enhanced EEG signal time-frequency domain feature information z c =[z 1 ,z 2 ,z 3 ],z c ∈R 24×N×M , z 1 ,z 2 ,z 3 These correspond to feature extraction of three groups of serially connected temporal convolutional layers and batch normalization layers.

[0048] Step 3-4: Construct a spatial convolution module to continue to enhance the EEG signal time-frequency domain feature information z c ∈R 24×N×MTo extract spatial features, the module consists of a spatial convolution layer, a batch normalization layer, and a nonlinear activation layer connected in series, as shown in the following example: Figure 4 The specific steps are as follows:

[0049] The convolution kernel size of the spatial convolution layer is (N, 1), where N is the number of EEG signal leads, the number of convolution kernels is 48, the step size is 1, and the activation function uses a linear activation function. The convolution operation process is as follows:

[0050] The input of the spatial convolution layer is the enhanced EEG signal time-frequency domain feature information z c ∈R 24×N×M , the feature output extracted by a single convolution kernel is in b k are the weight matrix and bias of the kth convolution kernel, k = 1, 2, ... 48, f(·) is the linear activation function, and the output feature of the spatial convolution layer is x ij ∈R 48×1×M .

[0051] A batch normalization layer is connected in series after the spatial convolution layer to normalize the output features of the spatial convolution layer, accelerate network learning, and achieve regularization. The batch normalization process is as follows:

[0052] Define a batch of data as B={x1,x2...,x m}, m represents the number of single experiments corresponding to a batch, and the mean of a batch of data is calculated variance The standardized output is The denormalized output is z i =γy i +β, introducing the constant ε to maintain numerical stability and prevent the denominator from being equal to 0. Denormalization is added to the batch normalization process, adding two adjustment parameters (scaling: γ and offset: β) to each neuron in the network. These two parameters are learned through training, enhancing the network's expressiveness. In this example, m is set to 16, meaning that the data from 16 single experiments is considered a batch.

[0053] The nonlinear activation layer is used to shorten the network training time and avoid the gradient vanishing or gradient exploding problems during network training. The nonlinear activation operation process is as follows:

[0054]

[0055] Steps 3-5: Add one pooling layer and one dropout layer in series after the spatial convolution module.

[0056] The pooling layer adopts an average pooling strategy and uses a larger size pooling kernel to compare the EEG signal time-frequency-space domain feature information p∈R extracted by the multi-scale temporal convolutional network module and the spatial convolution module. 48×1×M Aggregation processing is performed on the width dimension in the local range of the pooling kernel to reduce the information redundancy of the feature information in the width dimension. The size of the pooling kernel is (1, kd), the number of pooling kernels is 48, and the step size is 64. In this embodiment, the size of the pooling kernel is set to (1, 64).

[0057] During the training process of the network, the drop layer randomly discards feature information according to a certain drop rate to avoid the influence of noise data and prevent the network from overfitting. In this embodiment, the drop rate is set to 0.5. The output feature after the pooling layer and the drop layer is X s ∈R 48×1×M / 64 .

[0058] Steps 3-6: Construct an attention module and use the channel attention mechanism to focus on the features related to the intensity motor imagery category in the channel dimension of the EEG signal time-frequency-space domain feature information after the pooling layer and the discard layer, and suppress feature redundancy, such as Figure 5 As shown. The input of the attention module is the output feature X after the pooling layer and the discard layer s ∈R 48×1×M / 64 .

[0059] First, global average pooling and global maximum pooling methods are used to s Aggregate information globally in height and width dimensions to obtain average pooled features and max pooling features The global average pooling and global maximum pooling process are as follows:

[0060]

[0061]

[0062] in, Represents X s The information of the k-th channel feature in height and width,

[0063] Secondly, and Two channel attention maps T are obtained through a shared network avg ∈R 48×1×1 、T max ∈R 48×1×1 The shared network is a multilayer perceptron model with one hidden layer, and its processing is as follows:

[0064]

[0065]

[0066] Where W0∈R 48 / r×1 , W1∈R 48×48 / r The parameter matrices between the input and hidden layers of the shared network and the parameter matrices between the hidden and output layers are respectively. The number of neurons in the hidden layer is set to 48 / r, where r represents the scaling ratio. In this embodiment, r is set to 16.

[0067] Again, the two channel attention maps are added together to get the final channel attention map T avg+max ∈R 48×1×1 , for T avg+max Each element of is operated by sigmoid function to generate the channel attention weight M c ∈R 48×1×1 The sigmoid function operation process is as follows:

[0068]

[0069] Finally, the channel attention weight M c Input X to the attention module s Perform multiplication and weighting on each channel to obtain the output feature X f ∈R 48×1×M / 64 .

[0070] Steps 3-7: After the attention module, add one tiled layer and one fully connected layer in series.

[0071] The tiling layer is used to convert the output feature X of the attention module into f ∈R 48×1×M / 64 "Flatten" to get the one-dimensional feature vector Y = [y1,y2,…,y 48×1×M / 64 ]Enter the fully connected layer.

[0072] The fully connected layer maps the one-dimensional feature vector Y output by the tiled layer to the motor imagery category label space V = {v t |t=1,2,...,CL}, where t=1,2,...,CL is the label of the motor imagery category. In this embodiment, CL is 3, corresponding to the three categories of low-force imagery, medium-force imagery, and high-force imagery. Next, the softmax function is used to determine the posterior probability p(t|Y) of the feature vector Y belonging to motor imagery category t. The specific processing process is as follows:

[0073] p(t|Y)=softmax(V)=softmax(w*Y+b)

[0074] Where w∈R (48×1×M / 64)×CLis the weight coefficient of the fully connected layer, b is the bias term, and the softmax function operation process is as follows:

[0075]

[0076] Step 4: The entire hybrid neural network is trained in a supervised manner. During the training process, the cross entropy loss function The network was optimized for the objective function. In this example, CL was set to 3, corresponding to the three categories of low-intensity imagery, medium-intensity imagery, and high-intensity imagery. The number of training iterations was 500. To accelerate the convergence of the network model, the Adam optimizer was selected in this example, with a batch size of 16 and a learning rate of 0.001.

[0077] The network's generalization ability was measured using a four-fold cross-validation method. Each subject's EEG data was randomly divided into four equal parts, with each part split into training and test data at a 3:1 ratio. The three motor imagery EEG data samples were balanced across the training and test data. In each fold of cross-validation, one part of the training data was randomly shuffled and fed into the network model to train it. After the network model was trained, the test set was used to verify the classification accuracy of each subject.

[0078] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying EEG features of motor imagery during unilateral upper limb movement, characterized in that: The following steps are involved: Step 1: During the EEG signal acquisition experiment, EEG signals of the subject performing a motor imagery process under the induction of a paradigm, wherein the paradigm is designed to be a motor imagery process of force changes under unilateral upper limb movement; Step 2: Preprocess the EEG signals of each single experiment; Step 3: constructing a hybrid neural network for use in EEG feature recognition of motor imagery of force changes in unilateral upper limb movement, wherein the hybrid neural network includes an input layer, a dimension conversion layer, a multi-scale temporal convolutional network module, a spatial convolution module, a pooling layer, a dropout layer, an attention module, a tiling layer, and a fully connected layer connected in series in sequence; Step 4: Train the entire hybrid neural network in a supervised manner.

2. The method for recognizing EEG characteristics of motor imagery based on force changes in unilateral upper limb movement according to claim 1, characterized in that: The acquisition experiment in step 1 includes multiple rounds, each round includes several single experiments, and each round of experiments is grouped into a group and arranged from small to large intensity, and the intensity of each round of motor imagery experiments is consistent.

3. The method for recognizing EEG characteristics of motor imagery based on force changes during unilateral upper limb movement according to claim 1, characterized in that: The step 2 is specifically as follows: First, bilateral mastoid averaging was used for re-reference, followed by baseline correction of the EEG signal, 0.5-100 Hz band-pass filtering and 50 Hz power frequency notch filtering to remove vertical electrooculogram artifacts and eliminate bad blocks. The EEG signal was then spatially filtered using a common average reference to improve the signal-to-noise ratio, and an 8-30 Hz band-pass filter was performed to retain the EEG signal in the frequency band related to motor imagery. Finally, the EEG signal was downsampled.

4. The method for recognizing EEG characteristics of motor imagery based on force changes during unilateral upper limb movement according to claim 1, characterized in that: The step 3 is specifically as follows: Step 3-1: The input of the input layer is the preprocessed two-dimensional EEG signal h∈R N×M , R N×M represents a two-dimensional real number space, N and M are the number of leads and time points of the EEG signal, respectively; Step 3-2: Dimension conversion layer transforms the two-dimensional EEG signal h∈R N×M Converted into three-dimensional EEG data h∈R 1×N×M , which has three dimensions: channel, height and width, R 1×N×M Represents three-dimensional real space; Step 3-3: Construct a multi-scale temporal convolutional network module to perform three-dimensional EEG data h∈R 1×N×M Extract multidimensional fine-grained time-frequency domain features for each single experiment, then splice and fuse the extracted multidimensional fine-grained time-frequency domain features to obtain enhanced EEG signal time-frequency domain feature information; Step 3-4: construct a spatial convolution module to continue extracting spatial features from the enhanced EEG signal time-frequency domain feature information to obtain EEG signal time-frequency-space domain feature information; Steps 3-5: After the spatial convolution module, the pooling layer and the dropout layer are connected in series in sequence. The pooling layer aggregates the obtained EEG signal time-frequency-space domain feature information in the width dimension, and the dropout layer randomly discards the feature information to avoid the influence of noise data. Steps 3-6: Construct an attention module and use a channel attention mechanism to focus on features related to the intensity motor imagery category in the channel dimension of the EEG signal time-frequency-spatial domain feature information after the pooling layer and the dropout layer, thereby suppressing feature redundancy. Steps 3-7: After the attention module, the tiled layer and the fully connected layer are connected in series to further process the output features of the attention module.

5. The method for recognizing EEG characteristics of motor imagery based on force changes in unilateral upper limb movement according to claim 1 or 4, characterized in that: The multi-scale temporal convolutional network module is formed by connecting multiple groups of temporal convolutional layers and batch normalization layers in parallel, and then connecting them in series with feature splicing and fusion layers. The temporal convolutional layers are respectively provided with three convolution kernels of different sizes.

6. The method for recognizing EEG characteristics of motor imagery based on force changes in unilateral upper limb movement according to claim 1 or 4, characterized in that: The spatial convolution module is formed by sequentially connecting a spatial convolution layer, a batch normalization layer, and a nonlinear activation layer.

7. The method for recognizing EEG characteristics of motor imagery based on force changes in unilateral upper limb movement according to claim 1 or 4, characterized in that: The pooling layer adopts an average pooling strategy and uses a pooling kernel to aggregate the time-frequency-space domain feature information of the EEG signal extracted by the multi-scale temporal convolutional network module and the spatial convolution module in the width dimension within the local range of the pooling kernel.

8. The method for recognizing EEG characteristics of motor imagery based on force changes during unilateral upper limb movement according to claim 1, characterized in that: In step 4, during the training process, the network is optimized using the cross entropy loss function as the objective function.

Images