A method and system for unilateral upper limb motor imagery electroencephalographic decoding

Abstract

This invention discloses a method and system for decoding EEG signals of unilateral upper limb motor imagery, comprising: acquiring and preprocessing EEG signals; constructing a multi-scale temporal convolutional coding module to extract and encode multi-scale temporal features through parallel multi-branch convolution; constructing a spatiotemporally interleaved Transformer module to split input features through its internal spatiotemporally interleaved attention layer, modeling temporal self-attention and spatial dynamic attention in parallel, and fusing the output; classifying to obtain predicted probabilities; constructing a self-distillation training mechanism, establishing teacher and student networks with identical structures, updating only the student network parameters during training, with the teacher network parameters updated through the exponential moving average of the student network parameters; optimizing the student network using a weighted sum of supervised loss and self-distillation loss; and selecting the optimal student model for decoding the target subject based on validation set performance. This invention efficiently models the spatiotemporal dependence of EEG signals and, combined with self-distillation, suppresses cross-subject differences, significantly improving decoding accuracy.

Description

Technical Field

[0001] This invention relates to the field of electroencephalogram (EEG) signal processing technology, and in particular to a method and system for decoding unilateral upper limb motor imagery using EEG. Background Technology

[0002] Motor imagery brain-computer interfaces (BCIs) enable direct interaction between humans and external devices by recognizing motor intentions in electroencephalogram (EEG) signals, and have significant application value in stroke rehabilitation, neurological function training, and intelligent control. Existing unilateral upper limb motor imagery decoding technologies mainly fall into two categories: The first category is manual feature extraction methods. These methods typically extract time-frequency-space features based on common spatial patterns (CSP), filter banks, continuous wavelet transforms, etc., and then use classifiers such as support vector machines and multilayer perceptrons to complete the recognition. These methods rely on human experience, and feature extraction and classifier training are disconnected, resulting in limited overall optimization capabilities.

[0003] The second category is deep learning methods. Convolutional neural network methods can directly learn temporal and spatial features from raw EEG data, such as EEGNet, DeepConvNet, and LMDA. In recent years, models that incorporate Transformers into EEG decoding have also emerged, such as EEG-ViT, EEG-Conformer, and EEG-Deformer. These methods utilize self-attention mechanisms to model global dependencies, exhibiting stronger sequence modeling capabilities than traditional convolutional networks.

[0004] In cross-subject application scenarios, some studies have attempted to improve the generalization ability of models through knowledge distillation or self-distillation. That is, the teacher model is first trained on multi-subject source domain data, and then cross-subject knowledge is transferred to the student model to reduce the performance degradation caused by the distribution shift of new subjects.

[0005] (1) Unilateral upper limb motor imagery tasks differ from bilateral limb tasks. The brain regions activated in these tasks are highly overlapping, and the spatial differences between different categories are very subtle. Existing methods are unable to extract sufficiently discriminative fine-grained spatiotemporal features.

[0006] (2) Traditional convolutional models are limited by local receptive fields and cannot simultaneously take into account long-term time dependence and dynamic relationships between channels. Some existing Transformer models are often modeled in a serial manner of "time first, space second" or "space first, time second", which fails to fully reflect the spatiotemporal coupling and interactive evolution of EEG signals.

[0007] (3) Most existing EEG Transformers directly perform unified attention calculations for all spatiotemporal locations, which is computationally expensive and lacks the use of neurophysiological priors regarding the differences in the importance of EEG channels.

[0008] (4) Significant individual differences in EEG and severe data distribution shifts among different subjects result in insufficient generalization ability of the model in cross-subject scenarios; traditional knowledge distillation usually requires additional design of complex teacher networks, which is costly to train and inconvenient to deploy.

[0009] The existing technology has the following problems: 1. It is difficult to extract stable and distinguishable dynamic spatiotemporal features from highly overlapping unilateral upper limb motor imagery EEGs; 2. It is difficult to simultaneously model long-range time dependencies and channel spatial dynamics at a low computational cost; 3. It is difficult to effectively utilize the source domain knowledge of multiple subjects to improve cross-subject generalization performance. Summary of the Invention

[0010] The technical solution of this invention to solve the above-mentioned technical problems is to provide a method for unilateral upper limb motor imagery EEG decoding, comprising the following steps: Step S1: Acquire and preprocess EEG signals. Collect multi-channel EEG data of the subject performing a unilateral upper limb motor imagery task, preprocess the data, and extract the motor imagery time period as the input sample. Step S2: Construct a multi-scale temporal convolutional coding module. Input the preprocessed EEG samples into a parallel multi-branch temporal convolutional network. Each branch uses a convolutional kernel of different lengths to extract multi-scale temporal features. Then, fuse and encode the outputs of the multiple branches to form temporal embedding features. Step S3: Construct a spatiotemporal interleaved Transformer module, inputting the temporal embedded features into at least one spatiotemporal interleaved block; the spatiotemporal interleaved block performs feature splitting on the input features through its internal spatiotemporal interleaved attention layer, and then performs temporal self-attention modeling and spatial dynamic attention modeling in parallel, and finally fuses and outputs the modeling results; Step S4: Classification output. Global pooling is performed on the output of the last spatiotemporal interleaving block, and the result is input into the classification layer to obtain the predicted probability of the motion imagery category. Step S5: Construct a self-distillation training mechanism, and establish teacher and student networks with identical structures, both of which include the multi-scale temporal convolutional coding module, the spatiotemporal interleaved Transformer module, and the classification layer; Step S6: Teacher network parameters are updated. During training, only student network parameters are updated. The teacher network parameters are updated using the exponential moving average of the student network parameters. Step S7: Joint loss optimization, using the weighted sum of supervised classification loss and self-distillation loss as the total loss of the student network for optimization; Step S8: Model selection and testing. Select the optimal student network model based on the performance on the validation set for EEG decoding of the target subjects.

[0011] Furthermore, in step S2, the multi-scale temporal convolutional coding module preferably employs three parallel convolutional branches, with the convolutional kernel lengths set to the time lengths corresponding to 0.10, 0.125, and 0.25 times the sampling rate, respectively.

[0012] Furthermore, the spatiotemporal interleaved attention layer in step S3 specifically includes: Step S31: Feature splitting, dividing the input features into temporal branch features and spatial branch features along the feature dimension; Step S32: Temporal self-attention modeling. After transforming the temporal branch features, perform multi-head self-attention operation to learn the dynamic temporal dependencies. Step S33: Spatial dynamic attention modeling. After rearranging the spatial branch features, dynamic channel weights are generated through lightweight convolution, and adaptive weighting is applied to the channels at different time steps. Step S34: Spatiotemporal fusion, concatenating the temporal branch output and the spatial branch output along the feature dimension, and then completing the fusion through pointwise convolution.

[0013] Furthermore, in step S6, the teacher network parameter update formula is: Teacher parameter = momentum coefficient × old teacher parameter + (1 - momentum coefficient) × current student parameter.

[0014] Furthermore, the total loss function in step S7 is a weighted combination of supervised classification loss and distillation loss, with the distillation loss constructed using Kullback-Leibler divergence based on the temperature coefficient.

[0015] This invention also proposes a system for unilateral upper limb motor imagery EEG decoding, for implementing the method described above, comprising: Data preprocessing module: used to acquire and preprocess EEG signals, and output input samples; Multi-scale temporal convolutional coding module: used to extract multi-scale temporal features of input samples through a parallel multi-branch temporal convolutional network and output temporal embedding features; Spatiotemporal Interleaved Transformer Module: Contains at least one spatiotemporal interleaved block, used for modeling and fusing feature splitting, parallel temporal self-attention and spatial dynamic attention, and outputting spatiotemporal representation; The classification output module is used to perform global pooling and classification on spatiotemporal representations and output predicted probabilities. The self-distillation training module contains a teacher network and a student network with the same structure. During training, the teacher network is updated by exponential moving average, and the student network is optimized based on the joint loss of supervised classification loss and self-distillation loss.

[0016] Furthermore, the spatiotemporal interleaving block in the spatiotemporal interleaving Transformer module specifically includes: Normalization layer: Normalizes the input features; Spatiotemporal interleaved attention layer: used to split the normalized features, perform temporal self-attention modeling and spatial dynamic attention modeling in parallel, and fuse the two for output; Residual connection layer: The output of the spatiotemporal interleaving attention layer is residually connected to the input of the spatiotemporal interleaving block; Feedforward network layer: performs nonlinear transformation on the result after residual connection.

[0017] Compared with the prior art, the present invention has the following beneficial effects: (1) Improve the accuracy of EEG decoding of unilateral upper limb motor imagery. Experiments in the paper show that the average classification accuracy of the present invention reaches 65.46% on the CAS dataset and 55.76% on the KU dataset, both of which are better than the baseline convolutional network and Transformer methods compared.

[0018] (2) Enhance cross-subject generalization ability. By introducing a self-distillation mechanism, the model can effectively inherit the common knowledge across subjects obtained from multi-subject training and obtain more stable classification results on unknown target subjects.

[0019] (3) Enhanced spatiotemporal discriminative feature extraction capability. Ablation experiments show that the performance decreased significantly after removing the spatiotemporal interleaved attention module; the performance also decreased significantly after removing the self-distillation module, indicating that the key modules proposed in this invention all make important contributions to the performance improvement.

[0020] (4) Balancing performance and interpretability. Through feature visualization and scalp topology visualization, it can be found that the present invention can learn more compact intra-class features and more separated inter-class features, and can focus on key brain regions related to motor imagery.

[0021] (5) It has good scalability. This method can be extended to brain-computer interface control, stroke rehabilitation training, neurofeedback and other cross-subject EEG classification tasks. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0023] Figure 1This is a flowchart illustrating the steps of the method for unilateral upper limb motor imagery EEG decoding according to the present invention.

[0024] Figure 2 This is a schematic diagram of the spatiotemporal interleaving Transformer module described in this invention; Figure 3 This is a schematic diagram of the structure of the multi-scale temporal convolutional coding module described in this invention; Figure 4 This is a schematic diagram of the structure of the spatiotemporal interlacing block described in this invention; Figure 5 This is a normalized confusion matrix diagram of the STIT-SD of the present invention on dataset I (a) and dataset II (b); Figure 6 The following are subject performance graphs for the STIT-SD and baseline methods of this invention: (a) and (b) are subject accuracy heatmaps for all comparison methods on datasets I and II, respectively; (c) and (d) are the subject accuracy differences between STIT-SD and the strongest baseline method on datasets I and II, respectively. Figure 7 The following is a graph showing the impact of different STIT-SD module parameters on the average accuracy of this invention ((a) embedding dimension of the MST module, (b) number of headers of the TSA module, (c) depth of the STIT). Figure 8 The UMAP visualization for this invention illustrates the importance of introducing self-distillation for feature learning. Different colors represent different categories: (a) no SD in dataset I, (b) SD in dataset II, (c) no SD in dataset I, (d) SD in dataset II. Figure 9 This invention provides a class-activated terrain visualization map based on two datasets. Detailed Implementation

[0025] This invention proposes a method and system for decoding unilateral upper limb motor imagery EEG, aiming to improve the accuracy of unilateral upper limb motor imagery EEG decoding.

[0026] The method for unilateral upper limb motor imagery EEG decoding proposed in this invention will be described below in specific embodiments: Example 1: A method for EEG decoding of unilateral upper limb motor imagery, such as Figure 1 As shown, it includes the following steps: Step S1: Acquire and preprocess EEG signals. Collect multi-channel EEG data of the subject performing a unilateral upper limb motor imagery task, preprocess the data, and extract the motor imagery time period as the input sample. Step S2: Construct a multi-scale temporal convolutional coding module. Input the preprocessed EEG samples into a parallel multi-branch temporal convolutional network. Each branch uses a convolutional kernel of different lengths to extract multi-scale temporal features. Then, fuse and encode the outputs of the multiple branches to form temporal embedding features. Step S3: Construct a spatiotemporal interleaved Transformer module, inputting the temporal embedded features into at least one spatiotemporal interleaved block; the spatiotemporal interleaved block performs feature splitting on the input features through its internal spatiotemporal interleaved attention layer, and then performs temporal self-attention modeling and spatial dynamic attention modeling in parallel, and finally fuses and outputs the modeling results; Step S4: Classification output. Global pooling is performed on the output of the last spatiotemporal interleaving block, and the result is input into the classification layer to obtain the predicted probability of the motion imagery category. Step S5: Construct a self-distillation training mechanism, and establish teacher and student networks with identical structures, both of which include the multi-scale temporal convolutional coding module, the spatiotemporal interleaved Transformer module, and the classification layer; Step S6: Teacher network parameters are updated. During training, only student network parameters are updated. The teacher network parameters are updated using the exponential moving average of the student network parameters. Step S7: Joint loss optimization, using the weighted sum of supervised classification loss and self-distillation loss as the total loss of the student network for optimization; Step S8: Model selection and testing. Select the optimal student network model based on the performance on the validation set for EEG decoding of the target subjects.

[0027] Specifically, this application constructs a spatiotemporally interleaved Transformer and self-distillation framework, abbreviated as STIT-SD, for decoding unilateral motor imagery across subjects. This framework consists of two parts: a base (teacher) model and a student model. The base model is trained on multi-subject data to learn features that can generalize across individuals. Its structure comprises a multi-scale temporal convolutional (MST) encoder, a spatiotemporally interleaved Transformer (STI-Transformer), and a classification module.

[0028] The MST encoder first extracts temporal patterns from multiple scales and implicitly separates information from different EEG frequency bands. Subsequently, the STI-Transformer interactively models the relationship between temporal dynamics and spatial (channel dimension), focusing more on the spatiotemporal characteristics of unilateral motion imagery. Finally, the classification module maps the learned representations to category probabilities. To reduce the domain differences between the source subject group and the new target subjects, this application further introduces a self-knowledge distillation strategy. Specifically, the teacher model provides soft supervision for the student model, encouraging the student to inherit cross-subject knowledge while learning subject-specific features from the target data, such as... Figure 2 As shown, the structure consists of the following parts: (1) a multi-scale temporal (MST) convolutional encoder; (2) a spatiotemporally interleaved Transformer (STI-Transformer); and (3) a classification module. During the training phase, labeled EEG experimental data from all source subjects were simultaneously input into the teacher model and the student model. The student model was optimized using a combined loss function, where $S_L$ is the supervised cross-entropy loss relative to the true labels, and $S_D$ is the self-distillation loss used to align the student model's predictions with the softened output of the teacher model. The teacher parameters were updated using the exponential moving average of the student parameters. During the testing phase, EEG data from unseen target subjects were processed only through the trained student model to generate predicted labels.

[0029] The goal of a multi-scale temporal convolutional coding module is to capture informative temporal structures at different time scales, including both fast transient activities and slower oscillatory rhythms. From a frequency domain perspective, this design is equivalent to adaptively extracting information from multiple EEG rhythm frequency bands. Figure 3 As shown, let the EEG sample be... ,in For the number of trials, For the number of EEG channels, This represents the number of time sampling points. Based on prior neurophysiological knowledge and existing research, this application sets three sampling points corresponding to the sampling frequency. The time-proportional convolution branches have convolution kernel sizes of respectively , and Output for each branch Feature maps, of which This represents the total feature dimension. After three parallel convolutions, three feature tensors are obtained. They encode time information in different frequency bands and have spatial dimensions. and duration These feature maps are then concatenated along the feature dimension to form... The spliced ​​representation is then subjected to nonlinear activation and time-averaged pooling (to reduce the time resolution). This includes batch normalization. Learnable positional encoding is then introduced. The final tensor is obtained by injecting temporal sequence information and permuting the dimensions before inputting it into the Transformer. The entire multi-scale temporal encoding process can be summarized by equation (1).

[0030] (1); in and They represent the first The kernel size and padding parameters for each branch; This represents dimension permutation and positional encoding operations. For batch sizes of... The input is a tensor, and the final output is a tensor. .

[0031] Following multi-scale encoding, it is necessary to simultaneously model long-range temporal dependencies and spatial interactions between channels. Directly applying full self-attention to all spatiotemporal locations is not only computationally expensive but also fails to utilize the structural priors of EEG. Therefore, this application designs a spatiotemporally interleaved Transformer composed of multiple stacked STIT blocks to learn real-time dynamic spatiotemporal information in EEG data. Each STIT block consists of layer normalization, a Criss-CrossAttention-inspired STIA module, residual addition units, and a feedforward network.

[0032] Let X be the input of the l-th STI-Transformer block. (l-1) Its output X l Given from equations (2) and (3): (2); (3); Wherein, LN(·) represents layer normalization, STIA(·) is the proposed spatiotemporally interleaved attention module, and MLP(·) is a position-wise feedforward network.

[0033] Spatiotemporal Interleaved Attention (STIA): such as Figure 3 As shown, STIA consists of a Temporal Self-Attention (TSA) module, a Spatial Dynamic Attention (SDA) module, and a fusion module. TSA is used to capture temporal dependencies, SDA is used to learn channel correlations, and the fusion module aggregates information from both the temporal and channel dimensions.

[0034] For the input features after layer normalization First, it is divided into two sub-parts along the feature dimension: (4); in, Used for time attention, This is used for spatial attention. The goal of doing this is to explicitly allocate half of the feature capacity to temporal modeling and the other half to spatial modeling.

[0035] Temporal Self-Attention (TSA): In TSA, ... Divided into A time strip, of which Let the first... The query, key, and value projection dimensions of each attention head are all [missing information]. The corresponding projection matrix is , , Then the first The time band in the first Attention span based on height is calculated as follows: (5); (6); (7); Where Softmax is the softmax function. This indicates a splicing operation.

[0036] Spatial Dynamic Attention (SDA): In SDA, ... Remodeling This approach treats each time step as an independent sample, focusing primarily on the channel dimension. SDA learns the data dependency importance weights for each channel and time step through lightweight convolutions. (8); in, This represents element-wise multiplication. A pointwise convolution along the feature dimension is used to generate attention weights. .

[0037] Spatiotemporal Interleaved Attention (STIA): Acquiring and Then, the two are concatenated along the feature dimension and fused by applying a 1×1 convolution: (9); Compared to the complete criss-crossattention, the STIA proposed in this application significantly reduces the computational burden while retaining strong spatiotemporal modeling capabilities.

[0038] Classification module: Output of the last STIT block After global average pooling along the time dimension, a sequence-level representation is obtained; then it is input into a fully connected layer and mapped to the logits space corresponding to the number of MI task categories; finally, softmax is used to generate the probabilities of each category, which serve as the prediction results of the base model.

[0039] Self-distillation method: Even with a strong base model, performance can still degrade due to inter-subject differences when directly applied to unseen subjects. To mitigate this issue, this application introduces a self-knowledge distillation strategy: the teacher model trained on multi-subject data is treated as a cross-subject prior and used to guide the student model adapted to specific target subjects.

[0040] Teacher-student settings: such as Figure 2 As shown, the teacher and student networks share the same STIT backbone structure, including a multi-scale temporal encoder, a spatiotemporally interleaved Transformer block, and a classification head. (The last two sentences are incomplete and likely refer to separate data points.) and This represents the parameters of the student network and the teacher network. Both are initialized with the same parameters learned during the multi-subject basic training phase at the start of a subject-specific fit.

[0041] During the target subject fitting process, only the student network is updated directly through backpropagation; the teacher network is not updated through gradient descent, but rather through the exponential moving average (EMA) of the student parameters. (10); Where μ∈[0,1) is the momentum coefficient that controls the update rate. Such EMA updates make the teacher network equivalent to the time integration of the student network during the training iteration process, thus its predictions are more stable and less noisy, and can be used as a soft objective for self-distillation.

[0042] Let the input sample and its true label be respectively. and Teachers and students The output logits are respectively and Based on these logits, using temperature parameters Construct the softened probability distribution: (11); (12); in, and These represent the distribution of teachers and students, respectively. One element, Number of categories. Temperature parameter. Controlling the smoothness of the probability distribution makes the outputs of teachers and students more similar in form, which is conducive to the transfer of implicit information.

[0043] Self-distillation loss: Distillation loss L_SD is defined as the Kullback-Leibler (KL) divergence between the softened teacher and student distributions, in the following form: (13) in, and These are the soft probability vectors for teachers and students, respectively. This is a common scaling factor.

[0044] In addition, student networks also use real-name tags. We obtain the supervised loss by accepting standard cross-entropy supervision. : (14) in, For student networks The first probability vector of time One element, For one-hot tags.

[0045] Ultimately, the total loss of the student model is: (15) Among them, hyperparameters Used to balance monitoring loss and self-distillation loss.

[0046] STIT-SD training process: First, all labeled trials for all training subjects are divided into training and validation sets. The teacher and student networks share the same STIT backbone and are initialized with the same parameters. For each mini-batch, both processes the same input; the student network updates gradients based on a combination of supervised loss and self-distillation loss, while the teacher network updates using the EMA of the student parameters according to Equation (10). After each epoch, the student network is evaluated on the validation set, and the checkpoint with the best validation performance is retained as the final model.

[0047] Algorithm 1: STIT-SD Training Process: Input: Training set Validation set

[0048] Output: Trained student network

[0049] 1) Initialize STIT backbone parameters

[0050] 2) Set the parameters for students and teachers: 3) For each training epoch: 4) To Each mini-batch : 5) (Student logits) 6)

[0051] 7) (Teacher logits) 8)

[0052] 9)

[0053] 10) Calculate according to formula (14)

[0054] 11) Calculate according to formula (13)

[0055] 12) Calculate the total loss according to formula (15).

[0056] 13) Based on Backpropagation update

[0057] 14) Update according to equation (10)

[0058] 15) End 16) In Upper assessment And save the best checkpoint 17) End.

[0059] Furthermore, in step S2, the multi-scale temporal convolutional coding module preferably employs three parallel convolutional branches, with the convolutional kernel lengths set to the time lengths corresponding to 0.10, 0.125, and 0.25 times the sampling rate, respectively.

[0060] Furthermore, the spatiotemporal interleaved attention layer in step S3 specifically includes: Step S31: Feature splitting, dividing the input features into temporal branch features and spatial branch features along the feature dimension; Step S32: Temporal self-attention modeling. After transforming the temporal branch features, perform multi-head self-attention operation to learn the dynamic temporal dependencies. Step S33: Spatial dynamic attention modeling. After rearranging the spatial branch features, dynamic channel weights are generated through lightweight convolution, and adaptive weighting is applied to the channels at different time steps. Step S34: Spatiotemporal fusion, concatenating the temporal branch output and the spatial branch output along the feature dimension, and then completing the fusion through pointwise convolution.

[0061] Furthermore, in step S6, the teacher network parameter update formula is: Teacher parameter = momentum coefficient × old teacher parameter + (1 - momentum coefficient) × current student parameter.

[0062] Furthermore, the total loss function in step S7 is a weighted combination of supervised classification loss and distillation loss, with the distillation loss constructed using Kullback-Leibler divergence based on the temperature coefficient.

[0063] Example 2: A system for unilateral upper limb motor imagery EEG decoding, for implementing the method described above, comprising: Data preprocessing module: used to acquire and preprocess EEG signals, and output input samples; Multi-scale temporal convolutional coding module: used to extract multi-scale temporal features of input samples through a parallel multi-branch temporal convolutional network and output temporal embedding features; Spatiotemporal Interleaved Transformer Module: Contains at least one spatiotemporal interleaved block, used for modeling and fusing feature splitting, parallel temporal self-attention and spatial dynamic attention, and outputting spatiotemporal representation; The classification output module is used to perform global pooling and classification on spatiotemporal representations and output predicted probabilities. The self-distillation training module contains a teacher network and a student network with the same structure. During training, the teacher network is updated by exponential moving average, and the student network is optimized based on the joint loss of supervised classification loss and self-distillation loss.

[0064] Furthermore, the spatiotemporal interleaving block in the spatiotemporal interleaving Transformer module specifically includes: Normalization layer: Normalizes the input features; Spatiotemporal interleaved attention layer: used to split the normalized features, perform temporal self-attention modeling and spatial dynamic attention modeling in parallel, and fuse the two for output; Residual connection layer: The output of the spatiotemporal interleaving attention layer is residually connected to the input of the spatiotemporal interleaving block; Feedforward network layer: performs nonlinear transformation on the result after residual connection.

[0065] It should be noted that: Transformer: Transformer Network; STI: Spatiotemporal Interleaved; STIA: Spatiotemporal Interleaved Attention; MST: Multi-scale Temporal Convolution; SD: Self-Distillation; EMA: Exponential Moving Average.

[0066] Example 3: A method for EEG decoding of unilateral upper limb motor imagery includes the following steps: Step 1: Select the CAS public dataset. Each participant was given three tasks: resting, hand motor imagery, and elbow motor imagery. After preprocessing, each participant received EEG data from 900 trials, 62 channels, and 800 sampling points.

[0067] Step 2: Use leave-one-out cross-validation, where one participant is used as the target test participant and the rest are used as source domain participants; the source domain data is further divided into training set and validation set.

[0068] Step 3: Input each sample into the multi-scale temporal convolutional coding module of the present invention, and extract multi-scale rhythmic features through three temporal convolutional branches with different kernel lengths.

[0069] Step 4: Input the encoded features into a two-layer spatiotemporal interleaved Transformer module, and perform temporal self-attention, spatial dynamic attention and fusion operations in each layer.

[0070] Step 5: Construct the teacher network and student network with identical initial parameters; during training, update the student network parameters using a weighted sum of cross-entropy loss and distillation loss supervised by real labels.

[0071] Step 6: The teacher network parameters are updated using an exponential moving average of the student network parameters until the student model with the best performance on the validation set is selected after training is completed.

[0072] Step 7: The best student model was used to classify the target subject test set, and the average accuracy was 65.46%, the F1-macro was 64.60%, and the Kappa was 0.48.

[0073] This embodiment demonstrates that the present invention can effectively perform cross-subject classification of unilateral upper limb motor imagery tasks involving different joints.

[0074] Example 4: A method for EEG decoding of unilateral upper limb motor imagery includes the following steps: Step 1: Select three types of hand grasping motor imagery task data from the first session in the KU public dataset. After preprocessing, each participant obtained EEG data from 150 trials, 60 channels, and 1000 sampling points.

[0075] Step 2: Use the same leave-one-out training and testing strategy as in Example 1.

[0076] Step 3: Input the sample into the model of this invention, and use the multi-scale temporal convolution module to extract the temporal context features of different frequency bands.

[0077] Step 4: Learn the fine-grained spatiotemporal differences between different grasping tasks through the spatiotemporal interleaving Transformer module.

[0078] Step 5: Use the self-distillation mechanism to suppress the interference of cross-subject noise and individual differences on the decision boundary of the student model.

[0079] Step 6: Tests show that the present invention achieves an average accuracy of 55.76%, an F1-macro score of 54.66%, and a Kappa value of 0.34 on this dataset, which is higher than the baseline method.

[0080] This embodiment demonstrates that the present invention is also well applicable to EEG classification problems with more subtle spatial activation differences and more difficult task differentiation.

[0081] 5. Verification Experiment: 5.1 Dataset: This paper uses two publicly available datasets to evaluate the effectiveness of the proposed model: the CAS dataset and the KU dataset.

[0082] Dataset I (CAS): The CAS dataset was used to record EEG images of motor imagery at different joints of the same upper limb. A total of 25 healthy subjects without prior MI-BCI experience participated in the experiment. EEG was acquired at 1000 Hz using a 64-channel gel electrode cap (standard 10–20 system) and a NeuroscanSynAmps2 amplifier. The reference electrode was located at the left mastoid process, and the electrode impedance was maintained below 10 kΩ. Real-time EMG monitoring of the forearm and upper arm muscles ensured that no actual movement occurred during the subjects' imagery.

[0083] Each participant completed three tasks: resting (eyes open and relaxed), hand MI (kinesthetic imagery of hand movements), and elbow MI (kinesthetic imagery of elbow movements). Each task consisted of 300 trials, with the imagery phase of each trial lasting 4 seconds. Raw EEG data were preprocessed using EEGLAB, including common mean reference (CAR), 0.1–40 Hz bandpass filtering, baseline correction, automatic artifact removal (AAR), and downsampling to 200 Hz. Sixty-two EEG channels were retained in the experiment, resulting in a data size of 900 × 62 × 800 (trials × channels × time points) per participant.

[0084] Dataset II (KU): The KU dataset focuses on fine motor skills of the same upper limb (right arm), including motor execution (ME) and motor imagery (MI) recordings. This dataset consists of 82,500 trials from 25 healthy subjects across 3 sessions. Each recording includes 60-channel EEG, 7-channel EMG, and 4-channel EOG. The experiment comprises 11 tasks covering three categories of upper limb movements: arm extension in 6 directions, hand grasping of 3 objects, and wrist rotation in 2 categories. Each task was recorded in 50 trials, each including 3 seconds of visual / textual cues, 4 seconds of task execution, and 4 seconds of rest. EEG was acquired using a BrainAmp amplifier at 2500 Hz, with electrode impedance maintained below 15 kΩ.

[0085] This study used tri-class hand grasping MI data from the first session of each subject. Preprocessing was also performed in EEGLAB, including 0.1–40Hz bandpass filtering, baseline correction, ICA artifact removal, and downsampling to 250Hz. The processed dataset for each subject was 150 × 60 × 1000 (trials × channels × time points).

[0086] 5.2 Baseline Methods: This paper compares STIT-SD with the following baseline methods: (i) three CNN-based models: EEGNet, deepConvNet, and LMDA; (ii) three Transformer-based models: EEG-ViT, EEG-Deformer, and EEG-Conformer.

[0087] EEGNet: A compact CNN architecture for EEG-BCI that combines temporal convolutions and depthwise separable convolutions to extract interpretable features and performs well across various BCI paradigms.

[0088] deepConvNet: A deep convolutional network consisting of four convolutional blocks, with a special regularization design, can directly process raw EEG and is a strong baseline for end-to-end MI decoding.

[0089] LMDA: Lightweight Multi-Attention EEG Decoding Network. It combines channel attention modules and deep attention modules to capture multi-scale dependencies and integrates the ConvNet backbone with the separable convolutions of EEGNet.

[0090] EEG-ViT: A visual Transformer-based EEG decoding model. It divides EEG time series into overlapping time patches, maps them to tokens, and inputs them into a pure Transformer encoder, thus enabling the original ViT architecture to process EEG.

[0091] EEG-Conformer: A compact CNN-Transformer hybrid model that can simultaneously model local and global information. Its one-dimensional convolutions extract low-level temporal and spatial features, while its self-attention layer captures long-range temporal dependencies.

[0092] EEG-Deformer: A CNN-Transformer network for coarse-to-fine temporal dynamic modeling. It introduces a hierarchical coarse-to-fine Transformer (HCT) module to model multi-scale temporal patterns and uses a dense information cleansing (DIP) module to iteratively optimize the temporal representation.

[0093] 5.3 Experimental Setup: This paper employs a generalized participant-independent evaluation protocol to ensure that no test participant information is used during training. For all datasets, a leave-one-out-of-subjects (LOSO) cross-validation strategy is used: in each tradeoff, the data of one participant is used as the test set, and the data of the remaining participants are used as the source domain data; the source domain data is further divided into training and validation sets at 80% and 20% respectively.

[0094] Model performance was evaluated using classification accuracy (Acc), macro-average F1 score (F1-macro), and Cohen's skappa coefficient (κappa). Let TP, TN, FP, and FN represent true positives, true negatives, false positives, and false negatives, respectively. The evaluation metrics are defined as follows: (16); (17); (18); in, Number of categories, subscript Indicates the first kind, This represents the expected accuracy at the random level. For each method, the index is first calculated for each participant, and then averaged across all participants.

[0095] To assess the statistical significance of the performance difference between STIT-SD and the baseline method, this paper employs the Wilcoxon signed-rank test based on subject-by-subject results. This test is a nonparametric method for paired data and does not require the underlying distribution to follow a normal distribution. A p-value less than 0.05 is considered statistically significant.

[0096] 5.4 Implementation Details: All experiments were implemented using PyTorch and run on an NVIDIA RTX 3090 GPU. The optimizer used was AdamW with an initial learning rate of 1×10⁻⁶. -3 The weight decays to 1×10 -5 The learning rate was scheduled using a cosine annealing strategy. The batch size for all datasets was set to 128, and the dropout rate was set to 0.5 to mitigate overfitting. Each model was trained for 200 epochs and tested using the best model on the validation set (kappa).

[0097] For STIT-SD, the number of convolutional kernels in the Multi-Scale Temporal (MST) module was set to d=120 across all datasets; in STI-Transformer, L=2 blocks were used, and the number of temporal attention heads n_head=4; for self-distillation, the loss weight parameter α=0.2 and the temperature τ=4.0 were set. All baseline models used the same data partitioning and the same early stopping / optimal validation criterion to ensure fair comparison.

[0098] Table 1. Overall performance comparison of different methods on the two datasets: ; Normalized confusion matrices of STIT-SD on datasets I(a) and II(b); 5.5 Overall Performance Comparison: First, the overall performance of STIT-SD was compared with all baseline algorithms, and the results are shown in Table 1. The original paper used paired t-tests to assess the statistical significance of performance differences and used the Benjamini-Hochberg procedure to correct the p-values ​​to control for false discovery rate. As shown in Table 1, STIT-SD achieved the best results across all three metrics on both datasets.

[0099] On dataset I, STIT-SD achieves a 1.93% improvement in average accuracy compared to the strongest baseline, EEG-Deformer, with a statistically significant difference after correction (p<0.01). On the more challenging dataset II, STIT-SD also outperforms EEG-Conformer by 1.84% (p<0.05), while maintaining consistent gains on F1-macro and Kappa datasets. This demonstrates that our proposed method not only improves average accuracy but also delivers a more balanced and reliable overall improvement.

[0100] also, Figure 5 Normalized confusion matrices for both datasets are presented. It can be seen that diagonal elements dominate for all categories; misclassification mainly occurs between the two classes of MI in dataset I and between the three gripping types in dataset II, indicating that STIT-SD exhibits a relatively balanced inter-class classification ability on both datasets.

[0101] To further verify that the performance improvement was not dominated by a few abnormal subjects, the original paper also reported the subject-specific accuracy and F1 score in supplementary tables A.1 and A.2, and... Figure 6 The results were visualized in the study. The results showed that STIT-SD consistently outperformed the strongest baseline on most participants across both datasets, indicating that its performance gains were a stable improvement across participants rather than a dataset-specific or participant-specific phenomenon.

[0102] 5.6 Ablation Experiment: To examine the contributions of STIT's two core modules—STI-Attention and Self-Distillation—to the model, this paper conducted an ablation study, removing the respective modules and observing their impact on the classification results of the two datasets. The results are shown in Table 2. Overall, removing STI-Attention had the greatest impact on performance; further comparison of the roles of TSA and SDA shows that TSA is more critical. After removing TSA, all three metrics on both datasets decreased significantly, indicating that for the unilateral upper limb MI task, more attention should be paid to changes in temporal features. Removing the self-distillation module resulted in the second largest performance decrease, indicating that self-distillation, as an effective regularization strategy, can enhance the model's generalization ability in cross-subject MI tasks. These results demonstrate that each module of STIT-SD synergistically improves the model's predictive ability.

[0103] Table 2: Ablation experiments of the STIT-SD attention module on two datasets; ; 5.7 Impact of MST Convolutional Kernels: Temporal convolutional kernels are a key factor determining the effectiveness of local feature extraction. In STIT-SD, multi-scale temporal (MST) convolutions are used to capture contextual information of the original MI-EEG across different frequency bands. To evaluate the impact of convolutional kernel configurations, the original paper tested three settings: (1, 0.1f_s), (1, 0.1f_s & 0.15f_s), and (1, 0.1f_s & 0.15f_s & 0.25f_s). The results show that the three-scale configuration achieves the highest accuracy and Kappa on both datasets; while the performance difference between the single-scale and dual-scale configurations is small, indicating that if the frequency band corresponding to the newly added convolutional kernel is similar to the existing settings, the additional benefits are limited. This means that for unilateral upper limb MI decoding, explicitly covering multiple and sufficiently separated frequency bands is important, and the MST module can effectively utilize the complementary temporal information between these frequency bands.

[0104] Table 3 shows the impact of different MST convolution kernel settings on dataset I; ; Table 4 shows the impact of different MST convolution kernel settings on dataset II; ; 5.8 Impact of Hyperparameters; This paper further analyzes the impact of several key hyperparameters in STIT-SD on decoding performance, including the embedding dimension d, the number of STI-Attention heads n_heads, and the depth L of the STI-Transformer. The results are as follows: Figure 7 As shown, the embedding dimension d controls the dimension of the learned feature space, n_heads determines how many complementary attention subspaces are used to model spatiotemporal dependencies, and L controls the degree of refinement of hierarchical features.

[0105] In the embedding dimension experiments, d was increased from 30 to 120 in steps of 30. The results showed that both datasets achieved optimal performance at d=60; further increasing d led to a slight decrease in accuracy for dataset I, while the decrease was more pronounced on the more challenging KU dataset, indicating that excessively high dimensionality may introduce overfitting. Regarding the number of attention heads, dataset I achieved the highest accuracy with n_heads=4, while dataset II benefited more from a larger number of heads (n_heads=6), reflecting its higher task complexity. Finally, Figure 7 (c) shows that when the Transformer depth exceeds L=2, neither dataset gains further benefits, and the optimal setting is L=2. Overall, STIT-SD is quite sensitive to structural hyperparameters, and the appropriate selection of feature dimensions and attention depth is crucial for handling MI tasks of varying difficulty.

[0106] 5.9 Visualization Analysis: To demonstrate the interpretability of STIT-SD, the original paper visualizes it from two perspectives: first, the deep feature distribution based on UMAP; and second, the scalp topology map reflecting spatial-temporal characteristics. UMAP (Uniform Manifold Approximation and Projection) is a novel manifold learning dimensionality reduction method. Figure 8 The effect of the self-distillation module on the feature distribution of two datasets is demonstrated. After introducing SD, both datasets show larger inter-class distances and smaller intra-class distances, indicating that SD makes the features extracted by the model more discriminative; while without SD, the inter-class distance decreases, indicating that the model's generalization ability is weak.

[0107] In addition, the original paper also provides topological visualizations on two datasets, including the original EEG scalp image and the category activation topology (CAT) map obtained after calculating the contribution of each channel to the classification result using Grad-CAM. The results are shown below. Figure 9 Compared to the original EEG, the CAT plot reveals the key channels the model focuses on in different tasks. For dataset I, the elbow and hand tasks show significant differences in their relative resting states. Although their spatial activations are quite similar, there are still subtle differences in electrode importance. For dataset II, the spatial variations of the three hand MI tasks are relatively smaller, which is related to the characteristics of the tasks themselves, but they still differ in activation intensity. This is the key to the model distinguishing these fine-grained tasks, and it also shows that the model has indeed learned discriminative features.

[0108] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for EEG decoding of unilateral upper limb motor imagery, characterized in that, Includes the following steps: Step S1: Acquire and preprocess EEG signals. Collect multi-channel EEG data of the subject performing a unilateral upper limb motor imagery task, preprocess the data, and extract the motor imagery time period as the input sample. Step S2: Construct a multi-scale temporal convolutional coding module. Input the preprocessed EEG samples into a parallel multi-branch temporal convolutional network. Each branch uses a convolutional kernel of different lengths to extract multi-scale temporal features. Then, fuse and encode the outputs of the multiple branches to form temporal embedding features. Step S3: Construct a spatiotemporal interleaved Transformer module, inputting the temporal embedded features into at least one spatiotemporal interleaved block; the spatiotemporal interleaved block performs feature splitting on the input features through its internal spatiotemporal interleaved attention layer, and then performs temporal self-attention modeling and spatial dynamic attention modeling in parallel, and finally fuses and outputs the modeling results; Step S4: Classification output. Global pooling is performed on the output of the last spatiotemporal interleaving block, and the result is input into the classification layer to obtain the predicted probability of the motion imagery category. Step S5: Construct a self-distillation training mechanism, and establish teacher and student networks with identical structures, both of which include the multi-scale temporal convolutional coding module, the spatiotemporal interleaved Transformer module, and the classification layer; Step S6: Teacher network parameters are updated. During training, only student network parameters are updated. The teacher network parameters are updated using the exponential moving average of the student network parameters. Step S7: Joint loss optimization, using the weighted sum of supervised classification loss and self-distillation loss as the total loss of the student network for optimization; Step S8: Model selection and testing. Select the optimal student network model based on the performance on the validation set for EEG decoding of the target subjects.

2. The method for unilateral upper limb motor imagery EEG decoding according to claim 1, characterized in that, In step S2, the multi-scale temporal convolutional coding module preferably uses three parallel convolutional branches, with the convolutional kernel lengths set to the time lengths corresponding to 0.10, 0.125, and 0.25 times the sampling rate, respectively.

3. The method for unilateral upper limb motor imagery EEG decoding according to claim 1, characterized in that, The spatiotemporal interleaved attention layer in step S3 specifically includes: Step S31: Feature splitting, dividing the input features into temporal branch features and spatial branch features along the feature dimension; Step S32: Temporal self-attention modeling. After transforming the temporal branch features, perform multi-head self-attention operation to learn the dynamic temporal dependencies. Step S33: Spatial dynamic attention modeling. After rearranging the spatial branch features, dynamic channel weights are generated through lightweight convolution, and adaptive weighting is applied to the channels at different time steps. Step S34: Spatiotemporal fusion, concatenating the temporal branch output and the spatial branch output along the feature dimension, and then completing the fusion through pointwise convolution.

4. The method for unilateral upper limb motor imagery EEG decoding according to claim 1, characterized in that, In step S6, the formula for updating the teacher's network parameters is: Teacher parameter = momentum coefficient × old teacher parameter + (1 - momentum coefficient) × current student parameter.

5. The method for unilateral upper limb motor imagery EEG decoding according to claim 1, characterized in that, The total loss function in step S7 is a weighted combination of supervised classification loss and distillation loss, with the distillation loss constructed using Kullback-Leibler divergence based on the temperature coefficient.

6. A system for unilateral upper limb motor imagery EEG decoding, characterized in that, To implement the method according to any one of claims 1 to 5, comprising: Data preprocessing module: used to acquire and preprocess EEG signals, and output input samples; Multi-scale temporal convolutional coding module: used to extract multi-scale temporal features of input samples through a parallel multi-branch temporal convolutional network and output temporal embedding features; Spatiotemporal Interleaved Transformer Module: Contains at least one spatiotemporal interleaved block, used for modeling and fusing feature splitting, parallel temporal self-attention and spatial dynamic attention, and outputting spatiotemporal representation; The classification output module is used to perform global pooling and classification on spatiotemporal representations and output predicted probabilities. The self-distillation training module contains a teacher network and a student network with the same structure. During training, the teacher network is updated by exponential moving average, and the student network is optimized based on the joint loss of supervised classification loss and self-distillation loss.

7. The system according to claim 6, characterized in that, The spatiotemporal interleaving block in the spatiotemporal interleaving Transformer module specifically includes: Normalization layer: Normalizes the input features; Spatiotemporal interleaved attention layer: used to split the normalized features, perform temporal self-attention modeling and spatial dynamic attention modeling in parallel, and fuse the two for output; Residual connection layer: The output of the spatiotemporal interleaving attention layer is residually connected to the input of the spatiotemporal interleaving block; Feedforward network layer: performs nonlinear transformation on the result after residual connection.

Images