Brain electrical signal decoding model training method, brain electrical signal decoding method, device and storage medium

Abstract

This application provides a training method, a decoding method, a device, and a storage medium for an electroencephalogram (EEG) signal decoding model. The method acquires and preprocesses multi-channel EEG signals to obtain EEG samples. A spatiotemporal encoder encodes the EEG samples to obtain spatiotemporal representations. At least one feature fusion module fuses the spatiotemporal representations to obtain feature representations. A classifier classifies the feature representations to obtain a classification prediction result for finger movements and determines the classification loss. A projection module projects the feature representations to obtain projected features and determines the supervised contrastive loss. A joint loss is constructed based on the classification loss and the supervised contrastive loss, and the parameters are iteratively updated until a preset training termination condition is met, resulting in a trained EEG signal decoding model. This application can improve the decoding accuracy of finger-level movement EEG signals.

Description

Technical Field

[0001] This application relates to the field of brain-computer interface technology, and in particular to a training method, brain signal decoding method, device and storage medium for an electroencephalogram (EEG) signal decoding model. Background Technology

[0002] Brain-computer interfaces (BCIs) establish an information exchange channel between the brain and external devices by collecting and analyzing brain activity, without relying on conventional peripheral neuromuscular pathways. Electroencephalography (EEG), with its non-invasive nature, high temporal resolution, and relatively low cost, is a commonly used neural signal in BCIs. Converting motor-related EEG signals into machine-executable commands can be used in scenarios such as assisted control, intelligent human-computer interaction, and robotic operation.

[0003] Current motor EEG decoding primarily focuses on coarse-grained motor categories such as the hands and upper limbs. For fine-grained motor intentions at the finger level, the neural activation areas in the sensorimotor cortex of different fingers, especially adjacent fingers, highly overlap. Furthermore, scalp EEG is affected by volumetric conduction, low signal-to-noise ratio, and non-stationarity, making the discernible differences between categories even more subtle. Therefore, finger-level motor EEG signals need to simultaneously preserve channel spatial differences, frequency characteristics, and contextual dependencies over a longer time span.

[0004] Deep learning methods can directly perform end-to-end representation learning from EEG time sequences. Convolutional neural networks excel at extracting local spatiotemporal patterns, but their receptive field expansion depends on layer stacking, limiting their ability to model long-range temporal dependencies. Self-attention-based models can establish global connections, but they incur high computational and storage overhead as sequence length increases. The selective state-space model Mamba has near-linear sequence modeling complexity, making it suitable for handling long EEG sequences. However, standard Mamba models causally in sequence, primarily utilizing unidirectional context, and existing methods typically use it as a single temporal module.

[0005] On the other hand, existing spatial filtering often learns a set of fixed spatial parameters shared across trials, making it difficult to explicitly adapt to changes in the responses of different EEG channels in each trial. This can easily lead to the loss of subtle channel-specific information during spatial aggregation. Spatial, spectral, and temporal features have complementary roles in motor EEG. If they are modeled separately and then simply spliced ​​or fused at a fixed ratio, it is difficult to adaptively select effective information for different trials.

[0006] Furthermore, the neural responses among finger-level motor EEG categories are highly similar, and using conventional classification loss alone to primarily constrain prediction results is insufficient to adequately limit the intra-class distribution and inter-class boundaries of the latent feature space. Therefore, there is a need for an EEG decoding method that can coordinate the modeling of trial-dependent spatial information, frequency variations, local temporal patterns, and bidirectional long-range dependencies within a unified framework, and further enhance feature discriminativeness. Summary of the Invention

[0007] This application discloses a training method, a method for decoding brain signals, a device, and a storage medium for a brain signal decoding model, which can solve the problem of low accuracy in decoding finger-level motor brain signals.

[0008] In a first aspect, embodiments of this application provide a training method for an electroencephalogram (EEG) signal decoding model, the method comprising: Acquire multi-channel EEG signals; The multi-channel EEG signals are preprocessed to obtain EEG samples; The EEG samples are encoded using a spatiotemporal encoder to obtain a spatiotemporal representation; The spatiotemporal representation is fused by at least one feature fusion module to obtain a feature representation for decoding. The feature representation is classified by a classifier to obtain the classification prediction result of the finger movement category, and the classification loss is determined based on the classification prediction result. The feature representation is projected using a projection module to obtain projected features, and the supervised contrast loss is determined based on the projected features. A joint loss is constructed based on the classification loss and the supervised contrast loss, and the parameters of the spatiotemporal encoder, the feature fusion module, the classifier and the projection module are iteratively updated based on the joint loss until the preset training termination condition is met, so as to obtain the trained EEG signal decoding model.

[0009] In some possible embodiments of this application, the step of encoding the EEG sample using a spatiotemporal encoder to obtain a spatiotemporal representation includes: The EEG sample is convolutionally transformed by temporal convolution in the spatiotemporal encoder to obtain a local temporal representation. The local time representation is processed by the dynamic spatial adaptive filter in the spatiotemporal encoder to determine the channel response corresponding to the local time representation, and channel recalibration weights are generated based on the channel response. The recalibrated features are obtained by weighting the channel recalibration weights and the local time representations using an element-wise weighting method. The recalibrated features are processed by hierarchical channel space modeling to obtain cross-channel representations; The cross-channel representation is subjected to max pooling time downsampling, and convolutional representation and sequence representation are obtained by dimensionality transformation, wherein the spatiotemporal representation includes the convolutional representation and the sequence representation.

[0010] In some possible embodiments of this application, the spatiotemporal representation is fused using at least one feature fusion module to obtain a feature representation for decoding, including: The convolutional representation is subjected to local convolution to obtain local convolutional features; The sequence representation is modeled in a bidirectional Mamba selective state space to obtain bidirectional temporal features; The sequence representation is subjected to learnable spectral gating modulation to obtain spectral modulation features; The bidirectional timing features and the spectral modulation features are aggregated to obtain spectral-timing features; Gating weights are generated based on the local convolutional features and the spectral-temporal features; The local convolutional features and the spectral-temporal features are weighted and fused according to the gating weights to obtain the feature representation.

[0011] In some possible embodiments of this application, the step of performing bidirectional Mamba selective state-space modeling on the sequence representation to obtain bidirectional temporal features includes: The sequence representation is normalized to obtain a normalized sequence representation; Mamba sequence modeling is performed on the normalized sequence representation according to the first time sequence to obtain the first time sequence feature; The normalized sequence representation is reordered according to a second time order to obtain a reordered sequence representation, wherein the second time order is the opposite of the first time order. Perform Mamba sequence modeling on the reordered sequence representation to obtain the modeling output; The modeling output is reordered according to the first time sequence to obtain the second time sequence feature; The first time series feature and the second time series feature are concatenated and linearly projected to obtain the bidirectional time series feature.

[0012] In some possible embodiments of this application, the step of performing learnable spectral gating modulation on the sequence representation to obtain spectral modulation features includes: The sequence representation is normalized to obtain a normalized sequence representation; The normalized sequence representation is subjected to a Fast Fourier Transform to obtain a frequency domain representation. The learnable spectrum gating is used to adaptively modulate different frequency components in the frequency domain representation to obtain the modulated frequency domain representation. The frequency domain characterization after modulation is subjected to inverse fast Fourier transform to obtain the spectral modulation features.

[0013] In some possible embodiments of this application, the aggregation of the bidirectional timing features and the spectral modulation features to obtain spectral-timing features includes: Multiplying the spectral modulation features by the trainable coefficients yields the weighted spectral modulation features; The normalized sequence representation, the bidirectional temporal features, and the weighted spectral modulation features are aggregated to obtain aggregated features; The aggregated features are downsampled to obtain spectral-temporal features that match the local convolutional branches.

[0014] In some possible embodiments of this application, the step of classifying the feature representation using a classifier to obtain a classification prediction result for the finger movement category, and determining the classification loss based on the classification prediction result, includes: The classifier receives the feature representation, maps the feature representation to the finger movement category space, obtains the classification prediction result of the finger movement category, and uses cross-entropy classification loss to determine the classification loss; The step of projecting the feature representation through a projection module to obtain projected features, and determining the supervised contrast loss based on the projected features, includes: The feature representation is received through the projection module, the feature representation is mapped to the normalized latent space to obtain the projected features, and the supervised contrast loss is determined by the supervised contrast loss. The construction of the joint loss based on the classification loss and the supervised comparison loss includes: The classification loss and the supervised contrast loss are weighted and summed to obtain the joint loss, wherein the weights of the supervised contrast loss increase from the initial weight value to the target weight value during training.

[0015] Secondly, embodiments of this application provide a method for decoding electroencephalogram (EEG) signals, the method comprising: Acquire multi-channel EEG signals of the target; The target multichannel EEG signal is preprocessed to obtain the target EEG sample; The target EEG sample is input into a trained EEG signal decoding model, which includes a trained spatiotemporal encoder, a feature fusion module, and a classifier. The classifier outputs the corresponding finger movement category; The trained EEG signal decoding model is obtained by the training method of the EEG signal decoding model.

[0016] Thirdly, embodiments of this application provide an electronic device, which includes a processor and a memory. When the processor executes a computer program stored in the memory, it implements the training method for the EEG signal decoding model and the EEG signal decoding method.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing at least one instruction, which, when executed by a processor, implements the training method for the EEG signal decoding model and the EEG signal decoding method.

[0018] This application provides a training method, decoding device, and storage medium for an EEG signal decoding model. The method acquires and preprocesses multi-channel EEG signals to obtain EEG samples. A spatiotemporal encoder encodes the EEG samples to obtain spatiotemporal representations. At least one feature fusion module fuses these spatiotemporal representations to obtain feature representations. A classifier classifies the feature representations to obtain a classification prediction result for finger movement categories and determines the classification loss. A projection module projects the feature representations to obtain projected features and determines a supervised contrastive loss. A joint loss is constructed based on the classification loss and the supervised contrastive loss, and parameters are iteratively updated until a preset training termination condition is met, resulting in a trained EEG signal decoding model. This technical solution utilizes a dynamic spatial adaptive filter to preserve subtle spatial differences related to finger-level movements, a feature fusion module to collaboratively extract local temporal details, frequency variations, and bidirectional long-range context, and a supervised contrastive loss to enhance the compactness of similar EEG features and the separability between different categories, thereby improving the decoding accuracy of finger-level movement EEG signals. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments are briefly described below. Obviously, the drawings described below are only some embodiments of this application, and those skilled in the art can obtain other forms of drawings based on these drawings without creative effort.

[0020] Figure 1 is a flowchart of the training method for the EEG signal decoding model provided in the embodiments of this application; Figure 2 is a detailed flowchart of step S104 of the training method for the EEG signal decoding model provided in the embodiment of this application; Figure 3 is a flowchart of an electroencephalogram (EEG) signal decoding method provided in an embodiment of this application; Figure 4 is a schematic diagram of the structure of the training device for the EEG signal decoding model provided in the embodiment of this application; Figure 5 is a structural diagram of the electronic device provided in an embodiment of this application. Detailed Implementation

[0021] In the following description, specific details such as particular system architectures, network modules, and signal processing procedures are set forth for illustrative purposes and not for limitation, in order to understand the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. Unnecessary detailed descriptions will not be provided for acquisition devices, general-purpose computing platforms, and conventional signal processing procedures well-known in the art.

[0022] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, steps, operations, elements, components and / or a collection thereof.

[0023] It should also be understood that the terms "first" and "second" used in this application are only used to distinguish different processing stages or modules and should not be construed as indicating or implying relative importance; "one embodiment" or "some embodiments" means that a particular feature, structure or step described in connection with that embodiment may be included in at least one embodiment of this application.

[0024] Please refer to Figure 1 The figure shows a flowchart of the training method for the EEG signal decoding model provided in one implementation of this application. As shown in the figure, the training method for the EEG signal decoding model includes the following steps: Step S101: Acquire multi-channel EEG signals.

[0025] In this embodiment, the multi-channel EEG signal is an EEG signal collected from multiple locations on the scalp of the subject using a non-invasive EEG acquisition device. The multi-channel EEG signal can be a time-domain voltage signal collected by multiple electrodes placed at different locations on the scalp. For example, a 32-channel, 64-channel, or 128-channel EEG acquisition device can be used, with electrodes arranged according to the international 10-20 system, to collect the EEG signals of the subject while performing different finger movement imagery or motor execution tasks. The multi-channel EEG signal can also be a continuously recorded EEG signal. For example, during a series of finger movement tasks performed by the subject, the EEG signals of the entire experimental process can be continuously recorded to obtain multi-channel EEG data with continuous time. The multi-channel EEG signal can also be an EEG signal received by a computer device through a communication interface. For example, the computer device can receive multi-channel EEG signals from the EEG acquisition device via wired or wireless communication, or it can receive pre-stored EEG data files through an input / output interface.

[0026] Step S102: Preprocess the multi-channel EEG signals to obtain EEG samples.

[0027] In this embodiment, after acquiring multi-channel EEG signals, the multi-channel EEG signals need to be preprocessed to obtain EEG samples. Specifically, preprocessing the multi-channel EEG signals to obtain EEG samples includes: acquiring task time stamp information corresponding to the multi-channel EEG signals; extracting multiple initial EEG samples from the multi-channel EEG signals based on the task time stamp information; performing bandpass filtering and channel dimension standardization on the multiple initial EEG samples to obtain intermediate EEG samples; and resampling the intermediate EEG samples according to a preset sampling rate to obtain the EEG samples.

[0028] In practical implementation, preprocessing involves improving signal quality and converting the format of multi-channel EEG signals. This can include one or more of the following: task-based truncation, bandpass filtering, channel-dimensional normalization, and resampling. Task time stamp information is used to identify the time position corresponding to each task in the multi-channel EEG signals, and can include the stimulus moment and / or action moment in the experimental paradigm. The task time stamp information can be recorded synchronously with the EEG signals by the EEG acquisition system. For example, while recording the EEG signals, the EEG acquisition system receives trigger signals sent by the experimental control program through a parallel or serial port and records the moment corresponding to the trigger signal as the task time stamp information. Based on the task time stamp information, multiple initial EEG samples can be extracted from the multi-channel EEG signals. During extraction, data segments can be extracted from continuous EEG signals according to a preset time window, starting from the task time stamp information. For example, the EEG signal 2 seconds after the cue stimulus can be extracted as the initial EEG sample, or the EEG signal 1 second before and 2 seconds after the action moment can be extracted as the initial EEG sample. If the experiment involves multiple trials, a corresponding number of initial EEG samples can be extracted. During extraction, transitional segments at the beginning and / or end of each initial EEG sample can be removed, such as removing EEG data in the first 0.2 seconds after the cue stimulus appears, to avoid interference from visual evoked potentials on motor-related EEG.

[0029] Then, bandpass filtering and channel-dimensional standardization are performed on multiple initial EEG samples to obtain intermediate EEG samples. Bandpass filtering can use a Butterworth bandpass filter from 0.5 to 40 Hz to suppress task-irrelevant low-frequency drift and high-frequency noise. Channel-dimensional standardization can use z-score standardization, which calculates the mean and standard deviation of each EEG channel across all time sampling points, subtracts the mean from each time sampling point value of that channel, and divides by the standard deviation. Intermediate EEG samples can be filtered and standardized but not yet resampled, retaining the original sampling rate. Finally, the intermediate EEG samples are resampled according to a preset sampling rate to obtain the final EEG samples. The preset sampling rate can be set according to subsequent computational needs. For example, if the original sampling rate is 1000 Hz, and the subsequent model processing requires a sampling rate of 250 Hz, the intermediate EEG samples can be resampled from 1000 Hz to 250 Hz. The preset sampling rate can also be set to 128 Hz, 250 Hz, or 512 Hz, etc. Resampling can be performed using linear interpolation, spline interpolation, or desampling after anti-aliasing filtering. After the above processing, the obtained EEG samples can be in two-dimensional matrix form; for example, an EEG sample can be represented as... Where C represents the number of EEG channels and L represents the number of time sampling points, and the EEG sample has two main dimensions: EEG channels and time sampling points.

[0030] In other embodiments, preprocessing may also include artifact removal, electrooculography (EOG) interference suppression, baseline correction, or bad track interpolation. For example, independent component analysis may be used to remove EOG artifacts, or surface Laplacian transform may be used to enhance spatial locality.

[0031] Step S103: Encode the EEG sample using a spatiotemporal encoder to obtain a spatiotemporal representation.

[0032] In this embodiment, after obtaining the EEG sample, it needs to be input into a spatiotemporal encoder to extract a spatiotemporal representation. The spatiotemporal encoder is a neural network module used to extract spatiotemporal features from the EEG sample. The spatiotemporal encoder may include temporal convolution and a dynamic spatial adaptive filter. Specifically, encoding the EEG sample using the spatiotemporal encoder to obtain a spatiotemporal representation includes: performing a convolutional transformation on the EEG sample using temporal convolution to obtain a local temporal representation; processing the local temporal representation using a dynamic spatial adaptive filter to determine the channel response corresponding to the local temporal representation and generating channel recalibration weights based on the channel response; weighting the channel recalibration weights and the local temporal representation using an element-wise weighted method to obtain recalibrated features; processing the recalibrated features using hierarchical channel space modeling to obtain a cross-channel representation; performing max-pooling temporal downsampling on the cross-channel representation, and obtaining convolutional representations and sequence representations through dimensionality transformation, wherein the spatiotemporal representation includes both convolutional representations and sequence representations.

[0033] In practical implementation, temporal convolution can be a neural network layer that performs convolution operations on EEG samples along the time dimension. It can be one-dimensional or two-dimensional convolution. For example, two-dimensional convolution can be used, treating the EEG sample as a two-dimensional matrix of channel × time. The convolution kernel slides along the time dimension to extract local temporal waveform features on each EEG channel. Convolutional transformation can also be combined with batch normalization (BN) and exponential linear unit (ELU). For example, batch normalization and exponential linear unit activation can be sequentially applied to the convolutional transformation result to obtain multi-kernel temporal features as local temporal representations. Local temporal representations can be expressed as: , Where X represents EEG samples, Conv2D represents two-dimensional convolution, BN represents batch normalization, and ELU represents exponential linear unit activation. This is a local time representation.

[0034] The dynamic spatial adaptive filter is a neural network module used to generate channel weights based on the channel responses of the current EEG sample. Specifically, adaptive average pooling can be performed on the local temporal representation along the time dimension to obtain channel descriptors. Then, lightweight bottleneck mapping and sigmoid function processing are applied to the channel descriptors to generate channel recalibration weights. For example, the channel recalibration weights can be expressed as: , in, For adaptive average pooling along the time dimension, For lightweight bottleneck mapping, This represents the Sigmoid function. Channel weights are recalibrated. Channel responses can be the average activation level of each EEG channel in the current EEG sample over the time dimension. For example, in a finger movement task, the response of sensorimotor cortex-related channels may be stronger, while the response of other channels may be weaker. The dynamic spatial adaptive filter can generate different channel weights based on these response differences.

[0035] The recalibrated features are obtained by element-wise weighting of the channel recalibration weights and the local temporal representation. Element-wise weighting can be achieved by multiplying the channel recalibration weights and the local temporal representation element-wise along the channel dimension, for example: , in For local time representation, Recalibrate the weights for the channels. For element-wise multiplication, These are the recalibrated features. If a certain EEG channel has a larger weight, the local temporal features of that channel are enhanced; if a certain EEG channel has a smaller weight, the local temporal features of that channel are suppressed. The recalibrated features can be multi-core temporal features that enhance task-related channel information, such as highlighting sensorimotor cortical channels related to finger movements and suppressing task-irrelevant channels.

[0036] Next, hierarchical channel space modeling is required for the recalibrated features to obtain cross-channel representations. Hierarchical channel space modeling can be a process of progressively convolving and aggregating the recalibrated features along the EEG channel sequence. Specifically, it can include at least two levels of channel-wise convolution. The first level of channel-wise convolution can extract local channel group features with overlapping receptive fields along a predetermined EEG channel sequence, and the second level of channel-wise convolution can aggregate the local channel group features into cross-channel representations. Channel-wise convolution can be a one-dimensional convolution along the EEG channel dimension, for example, performing one-dimensional convolution on adjacent channel groups according to the channel sequence of the international 10-20 system to extract the spatial relationships between local channel groups.

[0037] After obtaining the cross-channel representation, max-pooling temporal downsampling is performed on the cross-channel representation, and convolutional representation and sequence representation are obtained through dimensionality transformation. Max-pooling temporal downsampling can be achieved by max-pooling the cross-channel representation along the time dimension, for example, reducing the number of time sampling points to half or a quarter of the original. Dimensional transformation can convert the downsampled cross-channel representation into convolutional representation suitable for local convolutional branch processing and sequence representation suitable for spectral-temporal hybrid branch processing, respectively. For example, the convolutional representation can retain the channel dimension and time dimension for subsequent local convolution; the sequence representation can recombine the channel dimension and time dimension for subsequent bidirectional Mamba selective state space modeling and learnable spectral gating modulation. Spatiotemporal representations include convolutional representations and sequence representations, which are respectively input into different branches in the subsequent feature fusion module.

[0038] In a specific finger movement EEG decoding scenario, the EEG sample is a two-dimensional matrix of 64 channels × 500 time sampling points. Temporal convolution transforms the EEG sample along the time dimension, and combined with batch normalization and exponential linear unit activation, yields multi-kernel temporal features of 64 channels × 500 time sampling points, serving as a local temporal representation. A dynamic spatial adaptive filter performs adaptive average pooling along the time dimension on the local temporal representation, obtaining a 64-dimensional channel descriptor. Lightweight bottleneck mapping and a sigmoid function are then used to generate 64-dimensional channel recalibration weights. The 64-dimensional channel recalibration weights are element-wise multiplied with the local temporal representation of 64 channels × 500 time sampling points to obtain the recalibrated features. Two-stage channel-wise convolution is performed on the recalibrated features: the first stage extracts features from adjacent channel groups, and the second stage aggregates them into cross-channel representations. Max pooling temporal downsampling was applied to the cross-channel representations, reducing the number of temporal sampling points from 500 to 125. Dimensional transformation was then used to obtain convolutional and sequential representations with 64 channels × 125 temporal sampling points, respectively. The spatiotemporal representations include both the convolutional and sequential representations. Therefore, the dynamic spatial adaptive filter can generate channel recalibration weights based on the channel responses of the current EEG sample, enhancing task-related channels and thus preserving subtle spatial differences related to finger-level movements.

[0039] In an alternative implementation, the spatiotemporal representation can be a feature tensor that preserves both spatial and temporal information of the EEG channels, and includes convolutional representation and sequence representation. For example, convolutional representation can be used for subsequent local convolutional branches to extract local waveform details, and sequence representation can be used for subsequent spectral-temporal hybrid branches to extract bidirectional long-range temporal features and frequency modulation features.

[0040] Step S104: The spatiotemporal representation is fused using at least one feature fusion module to obtain a feature representation for decoding.

[0041] In this embodiment, after obtaining the spatiotemporal representation, it needs to be fused using at least one feature fusion module to obtain a feature representation for decoding. The feature fusion module is a neural network module for multi-branch feature extraction and fusion of the spatiotemporal representation. This feature fusion module may include a local convolutional branch, a spectrum-temporal hybrid branch, and an adaptive gated fusion unit. Since the spatiotemporal representation includes convolutional representation and sequence representation, the local convolutional branch can receive the convolutional representation and perform local convolution on it to obtain local convolutional features; the spectrum-temporal hybrid branch can receive the sequence representation, perform bidirectional Mamba selective state-space modeling on it to obtain bidirectional temporal features, and perform learnable spectrum-gated modulation on it to obtain spectrum modulation features. The bidirectional temporal features and spectrum modulation features are then aggregated to obtain spectrum-temporal features; the adaptive gated fusion unit receives the local convolutional features and spectrum-temporal features, generates gating weights based on both, and performs weighted fusion according to the gating weights to obtain the feature representation. Specifically, the spatiotemporal representation is fused through at least one feature fusion module to obtain the feature representation used for decoding. For a detailed description of the feature representation obtained, please refer to the corresponding figures and descriptions below, which will not be repeated here.

[0042] In one embodiment, the feature fusion module can be multiple modules stacked in a hierarchical manner. For example, the sequence representation and convolutional representation output by the previous feature fusion module can be used as the input of the next feature fusion module to obtain hierarchical feature representations containing local, spectral, and long-range temporal information step by step.

[0043] Step S105: The feature representation is classified by a classifier to obtain the classification prediction result of the finger movement category, and the classification loss is determined based on the classification prediction result.

[0044] In this embodiment, after obtaining the feature representation, it needs to be input into a classifier to obtain the classification prediction result of the finger movement category and determine the classification loss. The classifier is a neural network module used to map the feature representation to a category space, and may include fully connected layers and a softmax function. Specifically, the classifier can map the feature representation to the finger movement category space through fully connected layers and a softmax function to obtain the classification prediction result of the finger movement category. The finger movement category space is a set of finger categories to be identified, and may include at least two different finger categories induced by motor imagination or motor execution, such as six categories: left thumb, left index finger, left middle finger, right thumb, right index finger, and right middle finger. The classification prediction result can be a prediction vector consistent with the number of finger categories to be identified. For example, if there are five finger categories to be identified, the classifier outputs a 5-dimensional prediction vector, with each dimension corresponding to the prediction probability of one finger category. Then, the cross-entropy classification loss is used to determine the classification loss. The cross-entropy classification loss can be expressed as... ,in For real labels, To predict probabilities.

[0045] Step S106: The feature representation is projected using the projection module to obtain projected features, and the supervised contrast loss is determined based on the projected features.

[0046] In this embodiment, during the training phase, in addition to the classification loss, a supervised contrastive loss also needs to be determined to constrain the geometric relationship of features between EEG samples of the same class and EEG samples of different classes. Specifically, the feature representation is projected using a projection module to obtain projected features, and the supervised contrastive loss is determined based on the projected features. The projection module is used to map the feature representation to a normalized latent space to constrain the geometric relationship of features between EEG samples of the same class and EEG samples of different classes. For example, the projection module is a neural network module used to map the feature representation to the normalized latent space. The normalized latent space can be the unit hypersphere space where the projected features are located. Specifically, the projection module can map the feature representation to the normalized latent space through at least one fully connected layer and L2 normalization. For example, the projection module can include two fully connected layers, with the first layer followed by a ReLU activation function and the second layer followed by L2 normalization, mapping the feature representation to projected features on the unit hypersphere.

[0047] In one implementation, the supervised contrastive loss can be a loss function used to constrain EEG samples of the same class to be close to each other and EEG samples of different classes to be far apart. For example, for each anchor sample, the supervised contrastive loss can bring it closer to samples of the same class in the latent space and push it further away from samples of different classes in the latent space.

[0048] In one embodiment, the geometric relationship of features can be a distance relationship or a similarity relationship between features in a normalized latent space. For example, the cosine similarity between projected features of EEG samples of the same category can be high, while the cosine similarity between projected features of EEG samples of different categories can be low.

[0049] Step S107: Construct a joint loss based on classification loss and supervised contrast loss, and iteratively update the parameters of the spatiotemporal encoder, feature fusion module, classifier and projection module based on the joint loss until the preset training termination condition is met, and obtain the trained EEG signal decoding model.

[0050] In this embodiment, after determining the classification loss and the supervised contrast loss separately, they need to be combined into a unified joint loss. The model parameters are then iteratively updated based on this joint loss until a preset training termination condition is met, thereby obtaining a trained EEG signal decoding model. The joint loss is a weighted sum of the classification loss and the supervised contrast loss. For example, the joint loss can be expressed as: , in Cross-entropy classification loss, To monitor and compare losses, This is used for supervised contrastive loss weights. The supervised contrastive loss weights can be increased from initial weight values ​​to target weight values ​​during training. For example, The value can be set to 0 in the early stages of training, and gradually increased to 0.1 as the number of training rounds increases, so that the model first obtains a stable classification ability, and then the feature geometric structure constraints are gradually strengthened.

[0051] After constructing the joint loss, the parameters of the spatiotemporal encoder, feature fusion module, classifier, and projection module are iteratively updated based on the joint loss. Parameter updates can be performed using backpropagation and gradient descent based on the joint loss. For example, the gradient of the joint loss with respect to each parameter in the spatiotemporal encoder, feature fusion module, classifier, and projection module can be calculated, and the Adam (Adaptive Moment Estimation) optimizer can be used to update each parameter. Iterative updates continue until a preset training termination condition is met, resulting in a trained EEG signal decoding model. The preset training termination condition may include one or more of the following: convergence of the joint loss, a decrease in the joint loss being less than a preset loss threshold, the number of training epochs reaching a preset number, or the classification accuracy of the classifier on the validation set reaching a preset accuracy threshold.

[0052] In a specific finger movement EEG decoding scenario, feature representations are input to both the classifier and the projection module. The classifier maps the feature representations to a space of five finger movement categories, resulting in five prediction vectors, and uses cross-entropy classification loss to determine the classification loss. The projection module maps the feature representations to a normalized latent space, obtaining projected features, and uses supervised contrastive loss to determine the supervised contrastive loss. The classification loss and the supervised contrastive loss are weighted and summed to obtain the joint loss, where the weights of the supervised contrastive loss gradually increase from 0 to 0.1 using cosine scheduling. Based on the joint loss, the parameters of the spatiotemporal encoder, feature fusion module, classifier, and projection module are iteratively updated until the joint loss converges or the number of training epochs reaches a preset number, resulting in a trained EEG signal decoding model. Through this method, the classification loss is determined by the classifier to learn the category decision boundary, the supervised contrastive loss is determined by the projection module to constrain the geometric relationships of features, and the joint loss is constructed by weighted summation to jointly optimize classification ability and feature discriminativeness.

[0053] Through the above steps, the training method of the EEG signal decoding model provided in this application enables the generation of channel recalibration weights based on the channel response of the current EEG sample using a dynamic spatial adaptive filter, which can preserve and highlight subtle spatial differences related to finger-level movements; the collaborative extraction of local temporal details, frequency changes, and bidirectional long-range context through local convolutional branches and spectral-temporal hybrid branches in the feature fusion module avoids reliance on a single feature domain; adaptive fusion of samples is performed through an adaptive gating fusion unit, allowing different EEG samples to select a more appropriate information ratio based on their own features; and supervised contrastive loss is used to assist classification training, further enhancing the compactness of similar EEG features and the separability between different categories, thereby improving the decoding accuracy of finger-level movement EEG signals.

[0054] Please refer to Figure 2 This is a detailed flowchart of step S104 of the training method for the EEG signal decoding model provided in one implementation of this application. As shown in the figure, it includes the following steps: Step S210: Perform local convolution on the convolution representation to obtain local convolution features.

[0055] In this embodiment, after obtaining the spatiotemporal representation, the convolutional representation is processed by a local convolution branch in the feature fusion module to obtain local convolutional features. This feature fusion module includes a local convolution branch, a spectrum-temporal hybrid branch, and an adaptive gated fusion unit. Local convolution is a convolution operation used to extract local temporal waveform features. In one embodiment, the local convolution branch may sequentially include random inactivation, one-dimensional convolution, batch normalization, exponential linear unit activation, and max pooling. For example, random inactivation, one-dimensional convolution, batch normalization, exponential linear unit activation, and max pooling can be sequentially performed on the spatiotemporal representation to obtain local convolutional features. In one embodiment, the local convolutional features may be features that preserve local waveform details in finger-level EEG signals. For example, local convolutional features can capture short-term waveform changes in EEG signals, such as local modulation of sensorimotor rhythms.

[0056] Step S220: Perform bidirectional Mamba selective state-space modeling on the sequence representation to obtain bidirectional temporal features.

[0057] In this embodiment, while performing local convolution on the convolutional representation through a local convolution branch, a bidirectional Mamba selective state space model is performed on the sequence representation through a spectral-temporal hybrid branch in the feature fusion module to obtain bidirectional temporal features. The feature fusion module includes a local convolution branch, a spectral-temporal hybrid branch, and an adaptive gating fusion unit. The local convolution branch can sequentially employ random inactivation, one-dimensional convolution, batch normalization, exponential linear unit activation, and max pooling to preserve the local fine-grained waveform in the finger movement EEG, which can be represented as: , in, The convolutional representation received by the local convolutional branch includes the EEG channel dimension and the time dimension; This represents a random deactivation operation, used to set some features to zero with a preset probability during training to suppress overfitting; This represents a one-dimensional convolution operation performed along the time dimension, used to extract local temporal waveform features from the convolution representation; This indicates a batch normalization operation, used to normalize the output of a one-dimensional convolution to stabilize the feature distribution and accelerate training. This represents the exponential linear unit activation function, used to perform a nonlinear transformation on batch-normalized features; This represents the max pooling operation, which is used to downsample the activated features along the time dimension to reduce the time dimension while preserving significant local responses. This indicates that the locally convolutional features obtained after processing by the local convolutional branch match the spectral-temporal features output by the spectral-temporal hybrid branch in the time dimension, so that the subsequent adaptive gated fusion unit can perform weighted fusion. This formula represents the initial processing of the convolutional representation... Random deactivation is performed, followed by one-dimensional convolution, batch normalization, exponential linear unit activation, and max pooling in sequence to finally obtain local convolutional features. .

[0058] The bidirectional Mamba selective state-space model is a selective state-space model that includes forward and reverse processing paths. The forward processing path performs Mamba sequence modeling on the normalized sequence representation according to the original time order (e.g., the first time order), obtaining the first temporal feature of the forward output. The reverse processing path reorders the sequence according to a second time order (reverse of the first time order), performs Mamba sequence modeling, and then restores the original time order, obtaining the second temporal feature of the reverse output. The first and second temporal features are concatenated and linearly projected to obtain the bidirectional temporal feature, which can be represented as follows: Specifically, bidirectional Mamba selective state-space modeling of the sequence representation to obtain bidirectional temporal features includes: normalizing the sequence representation to obtain a normalized sequence representation; performing Mamba sequence modeling on the normalized sequence representation according to a first time order to obtain a first temporal feature; reordering the normalized sequence representation according to a second time order to obtain a reordered sequence representation, wherein the second time order is the reverse of the first time order; performing Mamba sequence modeling on the reordered sequence representation to obtain a modeling output; reordering the modeling output according to the first time order to obtain a second temporal feature; and concatenating and linearly projecting the first and second temporal features to obtain bidirectional temporal features. .

[0059] In practical implementation, normalizing the sequence representation to obtain a normalized sequence representation can include: normalizing the sequence representation along the feature dimension through layer normalization or batch normalization, so that the feature mean of each time position is 0 and the variance is 1, thus obtaining a normalized sequence representation. Normalized sequence representation can reduce the differences in numerical distribution between different EEG samples, improve model training stability, and is suitable for input into the Mamba selective state-space model.

[0060] The forward processing path may include at least one Mamba selective state space module. For example, the forward processing path may process the normalized sequence representation sequentially in time order t=1,2,...,L, update the state through the selective state space model, and output the forward hidden state sequence to obtain the first temporal feature. The first temporal feature may be a temporal representation that includes forward context information. For example, the output at each time point in the first temporal feature may depend on all inputs before that time point.

[0061] The reverse processing path may include a reordering operation, at least one Mamba selective state-space module, and a sequence restoration operation. For example, the reverse processing path may first reorder the normalized sequence representation according to a second time order, which is the reverse of the first time order, resulting in a reordered sequence representation; then, perform Mamba sequence modeling according to the reordered time order to obtain the modeling output; finally, reorder the modeling output according to the first time order to restore the original time order, resulting in a second temporal feature. The second temporal feature can be a temporal representation that includes backward context information. For example, the output at each time point in the second temporal feature may depend on all inputs after that time point.

[0062] By concatenating the first and second time-series features and performing linear projection, a bidirectional time-series feature is obtained. Concatenation can be an operation that combines first and second temporal features along a feature dimension. For example, if both the first and second temporal features are feature sequences of dimension D, then concatenation yields a feature sequence of dimension 2D. Linear projection can be a process that maps the concatenated features to the target dimension using a fully connected layer. For example, a fully connected layer can map concatenated features of dimension 2D to bidirectional temporal features of dimension D. Bidirectional temporal features can be noncausal long-range temporal representations that simultaneously incorporate both forward and backward contexts. For example, features at each time point in a bidirectional temporal feature can utilize both preceding and subsequent EEG signal information.

[0063] In an alternative implementation, Mamba sequence modeling can be based on a selective state-space model. For example, Mamba sequence modeling can efficiently model long-range temporal dependencies by selectively remembering and forgetting information in the sequence through a selection mechanism of input dependencies.

[0064] It is understandable that, in a specific finger movement EEG decoding scenario, normalized sequence representation... This is a feature sequence with 64 channels and 125 time sampling points. The forward processing path processes the normalized sequence representation sequentially from t=1 to 125, obtaining the first temporal feature. The reverse processing path first reorders the normalized sequence representation according to a second temporal order (reverse of the first), then performs Mamba sequence modeling in the reordered temporal order to obtain the modeling output. Next, it reorders the modeling output according to the first temporal order to restore the original temporal order, obtaining the second temporal feature. The first and second temporal features are concatenated along the feature dimension to obtain a concatenated feature with doubled dimensions. This concatenated feature is then mapped back to the original dimension via linear projection, resulting in a bidirectional temporal feature. In this way, bidirectional temporal features can utilize both forward and backward contextual information simultaneously, improving the ability to model long-range temporal dependencies.

[0065] Step S230: Perform learnable spectral gating modulation on the sequence representation to obtain spectral modulation features.

[0066] In this embodiment, while performing bidirectional Mamba selective state-space modeling of the sequence representation through a spectrum-time hybrid branch, it is also necessary to perform learnable spectrum-gated modulation on the sequence representation through the spectrum-time hybrid branch to obtain spectrum modulation features. Learnable spectrum-gated modulation is a process of adaptively modulating the frequency components of the EEG signal in the frequency domain. In an optional implementation, learnable spectrum-gated modulation may include Fast Fourier Transform, learnable spectrum gating, and Inverse Fast Fourier Transform. Specifically, performing learnable spectrum-gated modulation on the sequence representation to obtain spectrum modulation features includes: normalizing the sequence representation to obtain a normalized sequence representation. The normalized sequence representation is subjected to a Fast Fourier Transform (FFT) to obtain a frequency domain representation. Different frequency components in the frequency domain representation are adaptively modulated using learnable spectral gating to obtain a modulated frequency domain representation. The modulated frequency domain representation is then subjected to an Inverse Fast Fourier Transform (IFT) to obtain the spectral modulation features. .

[0067] In the specific implementation, the sequence representation is first normalized along the feature dimension through layer normalization or batch normalization, so that the feature mean of each time position is 0 and the variance is 1, thus obtaining the normalized sequence representation. To reduce the numerical distribution differences between different EEG samples, a Fast Fourier Transform (FFT) is performed on the normalized sequence representation along the time dimension, transforming it from the time domain to the frequency domain, resulting in a frequency domain representation containing amplitude and phase information of different frequency components. Next, each frequency component in the frequency domain representation is multiplied by the corresponding gating parameter in the learnable spectral gating to adjust the weights of different frequency components, obtaining the modulated frequency domain representation. For example, for finger movement EEG signals, frequency components related to sensorimotor rhythms can be assigned larger gating weights, while noise-related frequency components can be assigned smaller gating weights. Finally, an Inverse Fast Fourier Transform (IFFT) is performed on the modulated frequency domain representation along the time dimension, transforming it back to the time domain to obtain the spectral modulation features. This is used for subsequent aggregation with normalized sequence representation and bidirectional temporal features.

[0068] In one embodiment, learnable spectrum gating can employ frequency-point parameters, segmented frequency band parameters, or a gating function dynamically generated by a neural network. Frequency-point parameters can assign a trainable weight to each frequency point; segmented frequency band parameters can divide the frequency axis into multiple frequency bands and assign a trainable weight to each band; a dynamically generated gating function can dynamically output gating weights based on input features. The parameters of the aforementioned learnable spectrum gating can be updated during training via backpropagation, enabling the model to adaptively select frequency components useful for the task.

[0069] In a specific EEG decoding scenario involving finger movements, the sequence representation is a feature sequence with 64 channels × 125 time sampling points. Layer normalization is performed on the sequence representation to obtain a normalized sequence representation. Perform a Fast Fourier Transform (FFT) along the time dimension to obtain the frequency domain representation; perform adaptive modulation using learnable spectrum gating to obtain the modulated frequency domain representation; perform an Inverse Fast Fourier Transform (IFFT) to obtain the spectral modulation features. In this way, spectral modulation features can enhance frequency components related to finger movements and suppress noise-related frequency components, thereby improving the discriminative power of subsequent feature fusion.

[0070] Step S240: Aggregate the bidirectional timing features and the spectral modulation features to obtain the spectral-timing features.

[0071] In this embodiment of the application, bidirectional time-series features are obtained through a spectrum-time hybrid branch. and spectral modulation characteristics Next, the two need to be aggregated with the normalized sequence representation to obtain the spectrum-time features. Specifically, aggregating the bidirectional time-series features and the spectrum modulation features to obtain the spectrum-time-series features includes: multiplying the spectrum modulation features by the trainable coefficients to obtain the weighted spectrum modulation features; aggregating the normalized sequence representation, the bidirectional time-series features, and the weighted spectrum modulation features to obtain the aggregated features; and downsampling the aggregated features to obtain the spectrum-time-series features that match the local convolutional branch. This aggregation process can be represented as: , in These are trainable coefficients. It is a spectral modulation feature. This refers to the weighted spectral modulation characteristics.

[0072] In the specific implementation, the spectral modulation features are first multiplied by trainable coefficients to obtain the weighted spectral modulation features. The trainable coefficients are learnable parameters used to control the weights of the spectral modulation features; they can be scalar coefficients or vector coefficients. For example, trainable coefficients... It can be a scalar, multiplying the overall spectral modulation characteristics by... This yields the weighted spectral modulation features; alternatively, it can be a channel-by-channel vector, with each feature channel corresponding to a coefficient. The weighted spectral modulation features can be expressed as... ,in The first feature is the spectral modulation feature. Then, the normalized sequence representation, bidirectional temporal features, and weighted spectral modulation features are aggregated to obtain the aggregated feature. Aggregation is the process of combining multiple features into a unified feature representation, which can include adding the normalized sequence representation, bidirectional temporal features, and weighted spectral modulation features to obtain the aggregated feature. For example, the aggregated feature can be represented as... ,in For normalized sequence representation, It has bidirectional time-series characteristics. The aggregation process can also include residual connections, such as using normalized sequence representations as the residual basis, bidirectional temporal features and weighted spectral modulation features as residual increments, which are then added together to obtain the aggregated features. Next, the aggregated features are downsampled to obtain spectral-temporal features that match the local convolutional branches. Downsampling is a process to reduce the temporal dimension of the features, which can be achieved through max pooling. For example, max pooling can be performed on the aggregated features along the temporal dimension to reduce the number of time sampling points to a dimension that matches the output of the local convolutional branches. The spectral-temporal features can be feature representations aligned with the local convolutional features in both the temporal and feature dimensions. For example, if the local convolutional features are 64 channels × 62 time sampling points, the spectral-temporal features can also be downsampled to 64 channels × 62 time sampling points so that the subsequent adaptive gating fusion unit can perform element-wise weighted fusion.

[0073] Step S250: Generate gate weights based on local convolutional features and spectral-temporal features.

[0074] In this embodiment, after obtaining local convolutional features through a local convolutional branch and spectral-temporal features through a spectral-temporal hybrid branch, an adaptive gating fusion unit needs to generate gating weights based on the local convolutional features and the spectral-temporal features. The gating weights are dynamic weights used to control the fusion ratio of the local convolutional features and the spectral-temporal features. In one embodiment, the adaptive gating fusion unit can concatenate the local convolutional features and the spectral-temporal features, and then perform linear mapping and Sigmoid function processing on the concatenated features to obtain the gating weights. For example, the gating weights can be expressed as: , in, For spectral-temporal characteristics, Local convolution features and For trainable parameters, This is the Sigmoid function.

[0075] Step S260: The local convolutional features are weighted and fused with the spectral and temporal features according to the gating weights to obtain the feature representation.

[0076] In this embodiment, after generating the gating weights, the local convolutional features and the spectral-temporal features need to be weighted and fused according to the gating weights to obtain the feature representation. Specifically, the local convolutional features and the spectral-temporal features can be weighted and summed according to the gating weights, for example: , in, For gating weights, For spectral-temporal characteristics, Local convolution features This is an element-wise multiplication. Through this method, the adaptive gating fusion unit can dynamically adjust the relative proportions of local convolutional features and spectral-temporal features based on the current EEG sample, thereby obtaining a feature representation that fuses local details and global spectral-temporal information.

[0077] It is understandable that in a specific finger movement EEG decoding scenario, the spatiotemporal representation is a feature tensor with 64 channels × 125 time sampling points. The local convolutional branch sequentially performs random deactivation, one-dimensional convolution, batch normalization, exponential linear unit activation, and max pooling on the spatiotemporal representation to obtain local convolutional features. The spectral-temporal hybrid branch normalizes the spatiotemporal representation to obtain a normalized sequence representation. A forward processing path performs Mamba sequence modeling in the original chronological order to obtain a forward output. A reverse processing path flips the normalized sequence representation in time, performs Mamba sequence modeling again, and restores the original chronological order to obtain a reverse output. The forward and reverse outputs are concatenated and linearly projected to obtain bidirectional temporal features. Simultaneously, a Fast Fourier Transform (FFT) is performed on the normalized sequence representation to obtain a frequency domain representation. Adaptive modulation is performed using learnable spectral gating, followed by an Inverse FFT to obtain spectral modulation features. The spectral modulation features are multiplied by trainable coefficients to obtain weighted spectral modulation features. The normalized sequence representation, bidirectional temporal features, and weighted spectral modulation features are aggregated to obtain aggregated features. Downsampling of the aggregated features yields spectral-temporal features. The adaptive gating fusion unit concatenates local convolutional features with spectral-temporal features, generates gating weights through linear mapping and a Sigmoid function, and weights the local convolutional features and spectral-temporal features according to the gating weights to obtain the feature representation.

[0078] Through the above steps, the training method of the EEG signal decoding model provided in this application enables the preservation of local temporal details through local convolutional branches, the extraction of bidirectional long-range temporal dependence and frequency modulation information through spectral-temporal hybrid branches, and the adaptive fusion of samples through adaptive gating fusion units, thereby obtaining a feature representation that simultaneously contains local, spectral, and bidirectional long-range temporal information.

[0079] Please refer to Figure 3 The figure shows a flowchart of a brainwave signal decoding method provided in one implementation of this application. As shown, the brainwave signal decoding method includes the following steps: Step S310: Acquire the target multichannel EEG signal.

[0080] In this embodiment, the target multichannel EEG signal is the multichannel EEG signal to be decoded during the testing or online use phase. The target multichannel EEG signal can be acquired in real time using a non-invasive EEG acquisition device. For example, in online brain-computer interface applications, multichannel EEG signals can be acquired in real time when the subject performs a finger movement imagery task.

[0081] Step S320: Preprocess the target multichannel EEG signal to obtain the target EEG sample.

[0082] In this embodiment, the preprocessing method is the same as step S102 in the training phase, which can perform task-based segmentation, bandpass filtering, channel dimension standardization, and resampling on the target multi-channel EEG signal to obtain the target EEG sample.

[0083] Step S330: Input the target EEG sample into the trained EEG signal decoding model. The trained EEG signal decoding model includes a trained spatiotemporal encoder, a feature fusion module, and a classifier.

[0084] In this embodiment, the trained EEG signal decoding model is obtained by the above-described training method. In an optional implementation, the trained EEG signal decoding model may not include a projection module. For example, during the testing or online use phase, the projection module and supervised contrast loss are not executed; only the trained spatiotemporal encoder, feature fusion module, and classifier are retained for forward decoding.

[0085] Step S340: The classifier outputs the corresponding finger movement category.

[0086] The finger movement categories include at least two distinct finger categories induced by motor imagery or motor execution. In an optional implementation, the classifier can output the finger movement category corresponding to the target EEG sample. For example, the classifier can output specific finger categories such as the left index finger or the right thumb.

[0087] In a specific finger movement brain-computer interface application scenario, the computer device acquires the target multi-channel EEG signal, preprocesses the target multi-channel EEG signal to obtain the target EEG sample, inputs the target EEG sample into a trained EEG signal decoding model, and the classifier outputs the corresponding finger movement category. The trained EEG signal decoding model includes a trained spatiotemporal encoder, feature fusion module, and classifier, but does not include a projection module.

[0088] In a specific application, the computer device decodes the target EEG sample using a trained EEG signal decoding model through the above steps S310 to S340, and the classifier outputs the corresponding finger movement category for fine-grained movement intention recognition in non-invasive brain-computer interfaces.

[0089] Through the above steps, the EEG signal decoding method provided in this application enables forward decoding to be performed using only the trained spatiotemporal encoder, feature fusion module, and classifier during the testing or online use phase, without performing the projection module and supervised contrast loss, thereby reducing the computational load of online decoding and improving decoding efficiency.

[0090] Figure 4This is a structural diagram of the training device (hereinafter referred to as the "training device") for the EEG signal decoding model provided in this application embodiment. The training device 400 may include multiple functional modules composed of computer program segments. The computer programs of each program segment in the training device 400 may be stored in the memory of an electronic device and executed by at least one processor to perform (see details). Figure 1 (Description) Training function and EEG signal decoding function of the EEG signal decoding model.

[0091] In this embodiment, the training device 400 can be divided into multiple functional modules according to the functions it performs. These functional modules may include: an acquisition module 401, a preprocessing module 402, an encoding module 403, a fusion module 404, a classification module 405, a projection module 406, and a training module 407. The term "module" in this application refers to a series of computer program segments that can be executed by at least one processor and perform a fixed function, stored in memory. In this embodiment, the training device 400 can be used to implement, for example... Figure 1 The gain control method for the electronic device shown is illustrated. Figure 1 As shown, the training device 400 is applied to electronic devices (such as...) Figure 5 In the electronic device shown, the training device 400 includes: The acquisition module 401 is used to acquire multi-channel EEG signals; The preprocessing module 402 is used to preprocess the multi-channel EEG signals to obtain EEG samples; Encoding module 403 is used to encode the EEG sample using a spatiotemporal encoder to obtain a spatiotemporal representation; The fusion module 404 is used to perform fusion processing on the spatiotemporal representation through at least one feature fusion module to obtain a feature representation for decoding; The classification module 405 is used to classify the feature representation through a classifier to obtain the classification prediction result of the finger movement category, and to determine the classification loss based on the classification prediction result; The projection module 406 is used to perform projection processing on the feature representation through the projection module to obtain projected features, and to determine the supervised contrast loss based on the projected features; The training module 407 is used to construct a joint loss based on the classification loss and the supervised contrast loss, and to iteratively update the parameters of the spatiotemporal encoder, the feature fusion module, the classifier and the projection module based on the joint loss until the preset training termination condition is met, so as to obtain a trained EEG signal decoding model.

[0092] Figure 5This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. As shown in the figure, the electronic device 1 can be a server, computer, mobile phone, tablet computer, smart wearable device, augmented reality / virtual reality device, laptop computer, netbook, etc. The electronic device 1 may include a communication module 101, a memory 102, a processor 103, an input / output (I / O) interface 104, and a bus 105. The processor 103 is coupled to the communication module 101, the memory 102, and the I / O interface 104 via the bus 105.

[0093] Communication module 101 may include a wired communication module and / or a wireless communication module. The wired communication module may provide one or more wired communication solutions such as Universal Serial Bus (USB) and Controller Area Network (CAN). The wireless communication module may provide one or more wireless communication solutions such as Wireless Fidelity (Wi-Fi), Bluetooth (BT), mobile communication networks, Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR).

[0094] Memory 102 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM). The RAM can be directly read and written by the processor 103 and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data. The RAM may include static random-access memory (SRAM), dynamic random-access memory (DRAM), synchronous dynamic random-access memory (SDRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.

[0095] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 103. Non-volatile memory can include disk storage devices and flash memory.

[0096] The memory 102 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 103. The one or more computer programs include multiple instructions, which, when executed by the processor 103, enable a training method and a brainwave signal decoding method for an EEG signal decoding model to be executed on the electronic device 1.

[0097] In other embodiments, the electronic device 1 further includes an external memory interface for connecting to an external memory to expand the storage capacity of the electronic device 1.

[0098] Processor 103 may include one or more processing units, such as application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0099] Processor 103 provides computing and control capabilities, for example, processor 103 is used to execute computer programs stored in memory 102.

[0100] I / O interface 104 is used to provide a channel for user input or output. For example, I / O interface 104 can be used to connect various input and output devices, such as mouse, keyboard, touch device, display screen, etc., so that users can enter information or visualize information.

[0101] Bus 105 is used at least to provide a channel for communication between communication modules 101, memory 102, processor 103, and I / O interface 104 in electronic device 1.

[0102] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 1. In other embodiments of this application, the electronic device 1 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0103] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, and the method implemented when the program instructions are executed can refer to the methods in the above embodiments of this application.

[0104] The computer-readable storage medium can be the internal memory of the electronic device described in the above embodiments, such as the hard disk or memory of the electronic device. Alternatively, the computer-readable storage medium can be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device.

[0105] In some embodiments, the computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function, etc.; and the data storage area may store data created based on the use of the electronic device, etc.

[0106] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0107] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0108] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0109] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0110] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A training method for an electroencephalogram (EEG) signal decoding model, characterized in that, The method includes: Acquire multi-channel EEG signals; The multi-channel EEG signals are preprocessed to obtain EEG samples; The EEG samples are encoded using a spatiotemporal encoder to obtain a spatiotemporal representation; The spatiotemporal representation is fused by at least one feature fusion module to obtain a feature representation for decoding. The feature representation is classified by a classifier to obtain the classification prediction result of the finger movement category, and the classification loss is determined based on the classification prediction result. The feature representation is projected using a projection module to obtain projected features, and the supervised contrast loss is determined based on the projected features. A joint loss is constructed based on the classification loss and the supervised contrast loss, and the parameters of the spatiotemporal encoder, the feature fusion module, the classifier and the projection module are iteratively updated based on the joint loss until the preset training termination condition is met, so as to obtain the trained EEG signal decoding model.

2. The training method for the EEG signal decoding model according to claim 1, characterized in that, The process of encoding the EEG sample using a spatiotemporal encoder to obtain a spatiotemporal representation includes: The EEG sample is convolutionally transformed by temporal convolution in the spatiotemporal encoder to obtain a local temporal representation. The local time representation is processed by the dynamic spatial adaptive filter in the spatiotemporal encoder to determine the channel response corresponding to the local time representation, and channel recalibration weights are generated based on the channel response. The recalibrated features are obtained by weighting the channel recalibration weights and the local time representations using an element-wise weighting method. The recalibrated features are processed by hierarchical channel space modeling to obtain cross-channel representations; The cross-channel representation is subjected to max pooling time downsampling, and convolutional representation and sequence representation are obtained by dimensionality transformation, wherein the spatiotemporal representation includes the convolutional representation and the sequence representation.

3. The training method for the EEG signal decoding model according to claim 2, characterized in that, The spatiotemporal representations are fused using at least one feature fusion module to obtain feature representations for decoding, including: The convolutional representation is subjected to local convolution to obtain local convolutional features; The sequence representation is modeled in a bidirectional Mamba selective state space to obtain bidirectional temporal features; The sequence representation is subjected to learnable spectral gating modulation to obtain spectral modulation features; The bidirectional timing features and the spectral modulation features are aggregated to obtain spectral-timing features; Gating weights are generated based on the local convolutional features and the spectral-temporal features; The local convolutional features and the spectral-temporal features are weighted and fused according to the gating weights to obtain the feature representation.

4. The training method for the EEG signal decoding model according to claim 3, characterized in that, The bidirectional Mamba selective state-space modeling of the sequence representation yields bidirectional temporal features including: The sequence representation is normalized to obtain a normalized sequence representation; Mamba sequence modeling is performed on the normalized sequence representation according to the first time sequence to obtain the first time sequence feature; The normalized sequence representation is reordered according to a second time order to obtain a reordered sequence representation, wherein the second time order is the opposite of the first time order. Perform Mamba sequence modeling on the reordered sequence representation to obtain the modeling output; The modeling output is reordered according to the first time sequence to obtain the second time sequence feature; The first time series feature and the second time series feature are concatenated and linearly projected to obtain the bidirectional time series feature.

5. The training method for the EEG signal decoding model according to claim 3, characterized in that, The process of performing learnable spectral-gated modulation on the sequence representation to obtain spectral modulation features includes: The sequence representation is normalized to obtain a normalized sequence representation; The normalized sequence representation is subjected to a Fast Fourier Transform to obtain a frequency domain representation. The learnable spectrum gating is used to adaptively modulate different frequency components in the frequency domain representation to obtain the modulated frequency domain representation. The frequency domain characterization after modulation is subjected to inverse fast Fourier transform to obtain the spectral modulation features.

6. The training method for the EEG signal decoding model according to claim 5, characterized in that, The aggregation of the bidirectional time-series features and the spectral modulation features to obtain the spectral-time-series features includes: Multiplying the spectral modulation features by the trainable coefficients yields the weighted spectral modulation features; The normalized sequence representation, the bidirectional temporal features, and the weighted spectral modulation features are aggregated to obtain aggregated features; The aggregated features are downsampled to obtain spectral-temporal features that match the local convolutional branches.

7. The training method for the EEG signal decoding model according to claim 1, characterized in that, The step of classifying the feature representation using a classifier to obtain a classification prediction result for the finger movement category, and determining the classification loss based on the classification prediction result, includes: The classifier receives the feature representation, maps the feature representation to the finger movement category space, obtains the classification prediction result of the finger movement category, and uses cross-entropy classification loss to determine the classification loss; The step of projecting the feature representation through a projection module to obtain projected features, and determining the supervised contrast loss based on the projected features, includes: The feature representation is received through the projection module, the feature representation is mapped to the normalized latent space to obtain the projected features, and the supervised contrast loss is determined by the supervised contrast loss. The construction of the joint loss based on the classification loss and the supervised comparison loss includes: The classification loss and the supervised contrast loss are weighted and summed to obtain the joint loss, wherein the weights of the supervised contrast loss increase from the initial weight value to the target weight value during training.

8. A method for decoding electroencephalogram (EEG) signals, characterized in that, The method includes: Acquire multi-channel EEG signals of the target; The target multichannel EEG signal is preprocessed to obtain the target EEG sample; The target EEG sample is input into a trained EEG signal decoding model, which includes a trained spatiotemporal encoder, a feature fusion module, and a classifier. The classifier outputs the corresponding finger movement category; The trained EEG signal decoding model is obtained by training the EEG signal decoding model according to any one of claims 1 to 7.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the training method of the EEG signal decoding model as described in any one of claims 1 to 7 and the EEG signal decoding method as described in claim 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, which, when executed by a processor, implements the training method of the EEG signal decoding model as described in any one of claims 1 to 7 and the EEG signal decoding method as described in claim 8.

Images