Brain-computer interface instruction issuing method, device and equipment based on regulation enhancement simulation

Abstract

The invention relates to the technical field of brain-computer interfaces, and provides a brain-computer interface instruction issuing method, device and equipment based on regulation enhancement simulation, and the method comprises the steps that an electroencephalogram decoding model comprises an encoder, a feature enhancer and a task classifier, the encoder encodes a real-time electroencephalogram signal to obtain compression representation before nerve regulation, and the feature enhancer is used for classifying the compression representation before nerve regulation; the feature enhancer performs feature enhancement on the compression representation to obtain enhanced representation, and the task classifier classifies the enhanced representation to obtain an electroencephalogram decoding result. According to the method, a feature enhancer is obtained by combining training of a state discriminator based on a sample electroencephalogram signal collected before nerve regulation and a real state label after nerve regulation, and the feature enhancer is driven to learn a feature migration relation between a compression feature before nerve regulation and a feature after nerve regulation; the feature characterization capability of an electroencephalogram decoding model on electroencephalogram signals is remarkably improved, so that the decoding robustness on weak stimulation signals is enhanced on the premise of not depending on high-intensity external stimulation.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interface technology, and in particular to a method, apparatus and device for issuing brain-computer interface commands based on modulated augmentation simulation. Background Technology

[0002] Brain-Computer Interface (BCI) is an emerging discipline arising from the intersection of neuroscience and information science. It studies how to establish direct communication and control channels between the brain and external devices, enabling information exchange between the brain and devices. This allows for the replacement, restoration, enhancement, supplementation, or improvement of the interaction between the central nervous system and the internal and external environment, showing broad application prospects in fields such as medical rehabilitation, intelligent interaction, and national defense. The workflow of a BCI system mainly includes three core steps: First, acquiring the user's neurophysiological signals through invasive or non-invasive techniques; second, decoding the user's intentions encoded in brain activity through signal processing steps such as feature extraction and conversion algorithms; and finally, converting the decoding results into specific device control commands according to the specific application scenario, realizing direct interaction between the brain and the external environment.

[0003] Steady-State Visual Evoked Potentials (SSVEP) are an exogenous electroencephalographic response. When the human eye fixates on a periodically flashing visual stimulus, the occipital lobe of the brain generates neural oscillations synchronized with the stimulus frequency and its harmonic frequencies. Studies have shown that visual stimuli in the 1–90 Hz range can induce SSVEP responses. Among these, the response amplitudes induced by low-frequency (4–12 Hz) and mid-frequency (12–30 Hz) stimuli are significantly higher than those in the high-frequency range (>30 Hz), making these two frequency bands the most widely used in SSVEP-BCI systems. Benefiting from the high response amplitude and signal-to-noise ratio (SNR) of SSVEP, SSVEP-BCI systems offer advantages such as a large instruction set and high information transfer rate (ITR). They exhibit optimal performance among EEG (electroencephalography)-based BCI systems and are commonly used in brain-computer interface forms such as online typing interfaces and control function selection interfaces.

[0004] Current high-speed SSVEP-BCI systems typically use the 8–15.8 Hz frequency band as the primary encoding frequency. Stimulation in this band easily induces a high signal-to-noise ratio brain response, which is beneficial for achieving high-precision decoding. However, such stimulation is often accompanied by a strong flickering sensation, resulting in a poor user experience and easily causing visual fatigue, thus limiting the long-term stable operation of the system. To improve user comfort, recent studies have attempted to use weak stimulation methods such as peripheral visual field stimulation and high-frequency stimulation. Although these methods can alleviate discomfort to some extent, the low signal-to-noise ratio of the induced brain response and poor inter-command discrimination still present challenges in terms of decoding accuracy.

[0005] In summary, although the existing SSVEP-BCI system has made significant progress in transmission rate, it still faces challenges in improving user comfort and system stability, especially in key technical bottlenecks such as the insufficient ability of the decoding model to extract features of weak stimulus signals and low command recognition accuracy. Summary of the Invention

[0006] This invention provides a brain-computer interface command issuance method, apparatus, and device based on modulation enhancement simulation, which solves the defects of existing decoding models in terms of insufficient ability to extract features of weak stimulus signals and low command recognition accuracy.

[0007] This invention provides a brain-computer interface command issuance method based on modulated augmentation simulation, comprising the following steps: Acquire the user's real-time brainwave signals; The real-time EEG signal is input into the EEG decoding model to obtain the EEG decoding result output by the EEG decoding model; Based on the EEG decoding results, commands are issued; The EEG decoding model includes an encoder, a feature enhancer, and a task classifier; the encoder is used to encode the real-time EEG signal to obtain a compressed representation before neural modulation; the feature enhancer is used to enhance the compressed representation to obtain an enhanced representation; the task classifier is used to classify the enhanced representation to obtain the EEG decoding result. The feature enhancer is obtained by training a state discriminator based on sample EEG signals collected before neural modulation and real state labels after neural modulation.

[0008] According to the present invention, a brain-computer interface command issuance method based on modulation enhancement simulation includes the following training steps for the feature enhancer: An initial EEG decoding model including an initial feature enhancer is obtained, and the parameters of the initial encoder in the initial EEG decoding model are initialized based on the parameters of the pre-trained encoder in the pre-trained EEG decoding model; the parameters of the state discriminator are initialized based on the parameters of the pre-trained state discriminator in the pre-trained EEG decoding model. The sample EEG signal collected before the neural modulation is input into the initial EEG decoding model. The initial encoder encodes the sample EEG signal to obtain a compressed representation of the sample before neural modulation. The initial feature enhancer enhances the compressed representation of the sample to obtain an enhanced representation of the sample. The state discriminator discriminates the enhanced representation of the sample to obtain a state prediction result. Based on the difference between the state prediction result and the true state label, a state discrimination loss is determined, and the initial feature enhancer is trained based on the state discrimination loss to obtain the feature enhancer.

[0009] According to the present invention, a brain-computer interface command issuance method based on modulation enhancement simulation includes the following steps for acquiring the pre-trained EEG decoding model: The sample EEG signal set is acquired, including a first EEG signal set collected before neural modulation and a second EEG signal set collected after neural modulation, as well as an original EEG decoding model; the original EEG decoding model includes an original encoder, an original decoder, an original task classifier, and an original state discriminator; The sample EEG signal set is input into the original EEG decoding model. The original encoder encodes the sample EEG signal set to obtain compressed features. The original task classifier classifies the compressed features to obtain the predicted category distribution. The original state discriminator performs state prediction on the compressed features to obtain the state prediction probability. Based on the difference between the predicted category distribution and the label categories corresponding to the sample EEG signal set, the task classification loss is determined; The state classification loss is determined based on the difference between the state prediction probability and the true state label corresponding to the sample EEG signal set. Based on the task classification loss and the state classification loss, a target loss is determined, and the original EEG decoding model is trained based on the target loss to obtain the pre-trained EEG decoding model.

[0010] According to the present invention, a brain-computer interface command issuance method based on modulation enhancement simulation is provided, wherein determining the target loss based on the task classification loss and the state classification loss includes: Based on the correlation between the internal dimensions of the compressed features, the orthogonality constraint loss is determined; The target loss is determined based on the task classification loss, the state classification loss, and the orthogonality constraint loss.

[0011] According to the present invention, a brain-computer interface command issuance method based on modulation enhancement simulation is provided, the method further comprising: The sample EEG signal set is frequency domain transformed to obtain frequency domain input features; The frequency domain input features are locally masked to obtain the frequency domain masked features; The compressed features are obtained by encoding the frequency domain mask features using the original encoder; The original decoder performs spectral reconstruction on the compressed features to obtain the reconstructed features; The reconstruction loss is determined based on the difference between the reconstructed features and the frequency domain input features; The target loss is determined based on the reconstruction loss, the task classification loss, the state classification loss, and the orthogonality constraint loss.

[0012] According to the present invention, a brain-computer interface command issuance method based on modulation enhancement simulation is provided, wherein the encoder is a harmonic alignment encoder; The harmonic alignment encoder includes a fundamental frequency feature extraction module, a harmonic frequency feature extraction module, and a self-attention feature mining module; the fundamental frequency feature extraction module is used to extract fundamental frequency features from the real-time EEG signal; the harmonic frequency feature extraction module is used to extract harmonic frequency features from the real-time EEG signal; the self-attention feature mining module is used to perform self-attention calculation based on the fundamental frequency features and the harmonic frequency features to obtain the compressed representation.

[0013] According to the present invention, a brain-computer interface command issuance method based on modulation enhancement simulation is provided, wherein the fundamental frequency feature extraction module performs convolution calculation on the fundamental frequency part of the real-time EEG signal through a first convolution kernel with a first dilation rate to obtain the fundamental frequency feature; The frequency octave feature extraction module obtains the frequency octave features by performing convolution calculation on the frequency octave portion of the real-time EEG signal using a second convolution kernel with a second dilation rate. Wherein, the second convolution kernel has a larger size on the frequency axis than the first convolution kernel, and the second dilation rate is greater than the first dilation rate.

[0014] The present invention also provides a brain-computer interface command issuing device based on modulation enhancement simulation, comprising the following units: The acquisition unit is used to acquire the user's real-time EEG signals; The input unit is used to input the real-time EEG signal into the EEG decoding model to obtain the EEG decoding result output by the EEG decoding model; The instruction issuing unit is used to issue instructions based on the EEG decoding results; The EEG decoding model includes an encoder, a feature enhancer, and a task classifier; the encoder is used to encode the real-time EEG signal to obtain a compressed representation before neural modulation; the feature enhancer is used to enhance the compressed representation to obtain an enhanced representation; the task classifier is used to classify the enhanced representation to obtain the EEG decoding result. The feature enhancer is obtained by training a state discriminator based on sample EEG signals collected before neural modulation and real state labels after neural modulation.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the brain-computer interface instruction issuing method based on modulation enhancement simulation as described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the brain-computer interface instruction issuing method based on modulation enhancement simulation as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the brain-computer interface instruction issuing method based on modulation enhancement simulation as described above.

[0018] This invention provides a brain-computer interface (BCI) command issuance method, apparatus, and device based on modulation enhancement simulation. The EEG decoding model includes an encoder, a feature enhancer, and a task classifier. The encoder encodes real-time EEG signals to obtain a compressed representation before neural modulation. The feature enhancer enhances the compressed representation to obtain an enhanced representation. The task classifier classifies the enhanced representation to obtain the EEG decoding result. In this method, the feature enhancer is trained using sample EEG signals acquired before neural modulation and real-state labels after neural modulation, in conjunction with a state discriminator. This training method drives the feature enhancer to learn the feature transfer relationship between the compressed features before neural modulation and the features after neural modulation, simulating the enhancement effect of neural modulation. This significantly improves the EEG decoding model's ability to represent EEG signals by features, thereby enhancing the system's robustness in decoding weak stimulus signals without relying on high-intensity external stimuli. Ultimately, while ensuring user comfort, it improves the accuracy of BCI command recognition and the overall stability of the system. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the brain-computer interface command issuance method based on modulation enhancement simulation provided by the present invention.

[0021] Figure 2 This is a flowchart of the acquisition process for the sample EEG signal set provided by the present invention.

[0022] Figure 3 This is a training diagram of the pre-trained EEG decoding model provided by the present invention.

[0023] Figure 4 This is a training diagram of the EEG decoding model provided by the present invention.

[0024] Figure 5 This is a schematic diagram of the harmonic alignment encoder provided by the present invention.

[0025] Figure 6 This is a schematic diagram of the overall architecture of the EEG signal processing system provided by the present invention.

[0026] Figure 7 This is a schematic diagram of the brain-computer interface command issuing device based on modulation enhancement simulation provided by the present invention.

[0027] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] The terms "first," "second," etc., used in this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and that the objects distinguished by "first," "second," etc., are generally of the same class.

[0030] Among related technologies, neuromodulation technology has been proven in recent years to effectively enhance the neural response of the visual cortex of the brain, showing significant potential in improving BCI performance under weak stimulation conditions. However, neuromodulation experiments typically require expensive equipment and strict safety regulations, and the technical barriers, time, and economic costs limit its widespread application in everyday brain-computer interface (BCI) settings.

[0031] Figure 1 This is a flowchart illustrating the brain-computer interface command issuance method based on modulated augmentation simulation provided by the present invention, as shown below. Figure 1 As shown, the method includes steps 110, 120 and 130.

[0032] Step 110: Obtain the user's real-time EEG signals.

[0033] Specifically, in a specific implementation scenario, this invention can be applied to a steady-state visual evoked potential (SSVEP) brain-computer interface system. First, it can acquire the user's real-time electroencephalogram (EEG) signals. These real-time EEG signals refer to electrical signals reflecting the user's brain neural activity, acquired in real-time from the user's scalp using an EEG acquisition device, such as an EEG cap with multiple electrodes. For example, when a user gazes at a visual stimulus flashing at a specific frequency on a computer screen, their visual cortex will generate a response synchronized with the stimulus frequency and its harmonic frequencies, i.e., an SSVEP signal, which is a typical type of EEG signal.

[0034] Here, after acquiring the user's real-time EEG signal, a series of preprocessing operations can be performed on the real-time EEG signal, such as filtering operations to remove power frequency interference and baseline drift, bad lead interpolation, downsampling, independent component analysis (ICA), etc. The embodiments of the present invention do not specifically limit this.

[0035] Independent component analysis was used to remove artifacts such as eye movement and electromyography.

[0036] Step 120: Input the real-time EEG signal into the EEG decoding model to obtain the EEG decoding result output by the EEG decoding model; Step 130: Based on the EEG decoding results, issue commands; The EEG decoding model includes an encoder, a feature enhancer, and a task classifier; the encoder is used to encode the real-time EEG signal to obtain a compressed representation before neural modulation; the feature enhancer is used to enhance the compressed representation to obtain an enhanced representation; the task classifier is used to classify the enhanced representation to obtain the EEG decoding result. The feature enhancer is obtained by training a state discriminator based on sample EEG signals collected before neural modulation and real state labels after neural modulation.

[0037] Specifically, after acquiring the user's real-time EEG signal, the real-time EEG signal can be input into the EEG decoding model to obtain the EEG decoding result output by the EEG decoding model.

[0038] The EEG decoding model comprises an encoder, a feature enhancer, and a task classifier. The encoder encodes real-time EEG signals to obtain a compressed representation before neural modulation. The feature enhancer enhances the compressed representation to obtain an enhanced representation. The task classifier classifies the enhanced representation to obtain the EEG decoding result. The role of the EEG decoding model is to map complex, low signal-to-noise ratio real-time EEG signals to clear, computer-understandable instruction categories. The EEG decoding result output by this model is the recognition result of the user's intention. For example, among 40 flashing targets, if the EEG decoding model determines that the user is looking at the 5th target, the EEG decoding result is "target 5".

[0039] Here, the encoder's role is to extract and encode features from the input real-time EEG signal, compressing it from a high-dimensional raw signal space to a low-dimensional, more information-dense feature space, thus obtaining a compressed representation before neural modulation. Here, "before neural modulation" refers to the user's EEG signal in its natural state during actual use, without physical neural modulation, such as transcranial magnetic stimulation (TMS).

[0040] Compression representation is a feature that retains the key features most relevant to the task in the original EEG signal while filtering out some noise and irrelevant information. Here, key features can be frequency and phase information in the SSVEP signal, etc., and this embodiment of the invention does not specifically limit them.

[0041] In specific implementation, the encoder can adopt a variety of neural network structures. The encoder can be a cascaded multilayer convolutional neural network (CNN), a deep neural network (DNN), a combination of CNN and DNN, or a Transformer, etc. The embodiments of the present invention do not make specific limitations on this.

[0042] Here, the feature enhancer receives the compressed representation from the encoder before neural modulation and enhances it to simulate a brainwave signal characteristic similar to that after physical neural modulation—that is, an enhanced representation. It should be understood that physical neural modulation can enhance the activity of specific brain regions, thereby improving the signal-to-noise ratio and decodeability of brainwave signals. However, performing physical neural modulation on users in daily use is impractical. The feature enhancer of this invention achieves the gain effect of neural modulation in a virtual or simulated manner by transforming the feature space.

[0043] Here, the feature enhancer can be a feedforward neural network, a residual network (RS), or other modules capable of learning nonlinear mappings, etc., and the embodiments of the present invention do not specifically limit it in this regard.

[0044] Augmented representations can be defined as compressed features that simulate ideal brain response states after undergoing nonlinear transformations within a feature space. Specifically, augmented representations retain, on the one hand, the core task information inherent in the compressed representation before neural modulation, which is directly related to the user's intention. For example, in the SSVEP task, this core task information indicates the target frequency the user is focusing on. On the other hand, through the mapping effect of feature enhancers, augmented representations enhance the saliency and discriminability of this core task information while suppressing interference from noise and irrelevant features. Thus, at the feature level, they simulate and reflect signal state information with high signal-to-noise ratio and high decodeability, similar to that after physical neural modulation. Therefore, augmented representations are high-quality features that both contain the user's original intention and possess feature distribution characteristics under ideal decoding conditions, providing easier-to-discriminate input for subsequent task classifiers.

[0045] Here, the task classifier receives the enhanced representations from the feature enhancer, performs final classification on these representations, and outputs the EEG decoding results. Because the input enhanced representations simulate high-quality, high signal-to-noise ratio signal features, the task classifier can perform the classification task more accurately and stably. Typically, the task classifier consists of one or more fully connected (FC) layers and a final classification output layer.

[0046] To enable the feature augmenter to learn how to simulate the effects of neuromodulation, it is trained in conjunction with a state discriminator, based on sample EEG signals acquired before neuromodulation and real-state labels after neuromodulation. The joint state discriminator training employs an adversarial or generative training strategy. Specifically, during the training phase, an additional auxiliary model, the state discriminator, is introduced. This discriminator is trained to distinguish between two types of EEG features: signals from before neuromodulation (Pre-state) and signals from after neuromodulation (Post-state). The input to the feature augmenter is the feature representation obtained by the encoder processing the sample EEG signals acquired before neuromodulation; the output is the augmented feature. During training, the augmented feature is assigned a label from the post-modulation (Post-state) as a supervisory signal. The optimization goal of the feature augmenter is to ensure that the augmented features output by the feature augmenter are recognized by the state discriminator as features belonging to the post-modulation state.

[0047] Through this mechanism, the feature enhancer is guided to learn the mapping relationship between pre-neuromodulation features and post-neuromodulation features, thereby simulating the enhancement effect of neural modulation.

[0048] Here, the state discriminator is similar to a binary classifier, used to determine whether a given compressed representation originates from a pre-modulation (Pre) or post-modulation (Post) state. This state discriminator is a simple binary classification network, with the compressed representation output by the encoder as its input. The output is a probability value. , This represents the probability that the representation comes from the Post state. The specific structure of the state discriminator can be several fully connected layers plus a sigmoid activation function; however, this embodiment of the invention does not impose specific limitations on this.

[0049] The steps for acquiring the sample EEG signals are as follows: First, a 40-target instruction system was designed based on mid-to-high frequency visual stimulation. Centered on a 50Hz visual stimulus, the system was expanded forward and backward in 0.2Hz increments to form the 40-target instruction system, with a phase interval of 0.5π between adjacent instructions. Users were required to fixate on the response locations according to the on-screen prompts, thereby inducing brain responses under different targets. After completing a set of SSVEP tasks, users underwent transcranial magnetic stimulation (TMS) neuromodulation to enhance their visual attention, thereby improving the signal-to-noise ratio of brain responses in subsequent SSVEP tasks and facilitating the reconstruction of high-quality brain signals in subsequent decoding and reconstruction tasks. After the modulation was completed, SSVEP data of the user under high visual attention levels was collected again.

[0050] Finally, after obtaining the EEG decoding results, commands can be issued based on these results.

[0051] Understandably, the feature enhancer is trained using sample EEG signals acquired before neural modulation and real-state labels after neural modulation, in conjunction with a state discriminator. This allows for virtual enhancement of weak, latent SSVEP signals in new subjects under zero-training conditions, thereby improving decoding accuracy and robustness without additional physical modulation. This approach effectively reduces the user burden, enhances user experience, and ensures long-term stable system operation.

[0052] The method provided in this invention includes an EEG decoding model comprising an encoder, a feature enhancer, and a task classifier. The encoder encodes real-time EEG signals to obtain a compressed representation before neural modulation. The feature enhancer enhances the compressed representation to obtain an enhanced representation. The task classifier classifies the enhanced representation to obtain the EEG decoding result. In this method, the feature enhancer is trained using sample EEG signals acquired before neural modulation and real-state labels after neural modulation, in conjunction with a state discriminator. This training method drives the feature enhancer to learn the feature transfer relationship between the compressed features before neural modulation and the features after neural modulation, simulating the enhancement effect of neural modulation. This significantly improves the EEG decoding model's ability to represent EEG signals by features, thereby enhancing the system's robustness in decoding weak stimulus signals without relying on high-intensity external stimuli. Ultimately, while ensuring user comfort, it improves the accuracy of brain-computer interface command recognition and the overall stability of the system.

[0053] Based on the above embodiments, the training steps of the feature enhancer include: Step 210: Obtain an initial EEG decoding model including an initial feature enhancer, and initialize the parameters of the initial encoder in the initial EEG decoding model based on the parameters of the pre-trained encoder in the pre-trained EEG decoding model; the parameters of the state discriminator are initialized based on the parameters of the pre-trained state discriminator in the pre-trained EEG decoding model. Step 220: Input the sample EEG signal collected before neural modulation into the initial EEG decoding model. The initial encoder encodes the sample EEG signal to obtain a compressed representation of the sample before neural modulation. The initial feature enhancer enhances the compressed representation of the sample to obtain an enhanced representation of the sample. The state discriminator discriminates the enhanced representation of the sample to obtain a state prediction result. Step 230: Based on the difference between the state prediction result and the true state label, determine the state discrimination loss, and train the initial feature enhancer based on the state discrimination loss to obtain the feature enhancer.

[0054] Specifically, firstly, an initial EEG decoding model including an initial feature enhancer is obtained, and then the parameters of the initial encoder in the initial EEG decoding model are initialized based on the parameters of the pre-trained encoder in the pre-trained EEG decoding model. The parameters of the state discriminator are initialized based on the parameters of the pre-trained state discriminator in the pre-trained EEG decoding model.

[0055] In addition, the parameters of the initial decoder in the initial EEG decoding model can be initialized based on the parameters of the pre-trained decoder in the pre-trained EEG decoding model, and the parameters of the initial task classifier in the initial EEG decoding model can be initialized based on the parameters of the pre-trained task classifier in the pre-trained EEG decoding model.

[0056] Here, the only difference in model structure between the initial EEG decoding model and the pre-trained EEG decoding model is that the initial EEG decoding model includes an initial feature enhancer, which is positioned between the initial encoder and the initial decoder. In other words, the initial EEG decoding model includes an initial encoder, an initial feature enhancer, an initial task classifier, an initial state discriminator, and an initial decoder. The pre-trained EEG decoding model includes a pre-trained encoder, a pre-trained decoder, a pre-trained task classifier, and a pre-trained state discriminator.

[0057] Here, the parameters of the initial feature enhancer can be preset or randomly generated, and this embodiment of the invention does not impose specific limitations on this.

[0058] After obtaining the initial EEG decoding model, the sample EEG signals collected before neural modulation can be input into the initial EEG decoding model. The initial encoder encodes the sample EEG signals to obtain a compressed representation of the sample before neural modulation. The initial feature enhancer enhances the compressed representation to obtain an enhanced representation. The state discriminator then determines the state of the enhanced representation to obtain a state prediction result. Here, the state prediction result is a probability value between 0 and 1, representing the confidence level in judging that the enhanced representation belongs to the post-neural modulation state.

[0059] Finally, after obtaining the state prediction results, the state discrimination loss can be determined based on the difference between the state prediction results and the true state labels, and the initial feature enhancer can be trained based on the state discrimination loss to obtain the feature enhancer.

[0060] It is understandable that the greater the difference between the state prediction result and the true state label, the greater the state discrimination loss; the smaller the difference between the state prediction result and the true state label, the smaller the state discrimination loss.

[0061] Here, the true state label can be set to 1, and the state discrimination loss can be the binary classification cross-entropy loss. The formula for the state discrimination loss is as follows:

[0062] in, Indicates the loss in the state determination. Indicates the actual state label. This represents the state prediction result, i.e., the output probability of the state discriminator.

[0063] It should be noted that, during backpropagation gradient calculation, in this embodiment of the invention, the parameters of the initial encoder, initial decoder, initial task classifier, and state discriminator are kept fixed (frozen), and the gradient is only used to update the parameters of the initial feature enhancer. Through continuous iteration of this process, the initial feature enhancer is forced to learn how to adjust its mapping function so that its output sample enhancement representation becomes increasingly closer to the real neurally modulated features in terms of feature distribution. This makes it increasingly difficult for the state discriminator to distinguish between true and false signals, and eventually the state discrimination loss tends to converge. After training, this optimized initial feature enhancer is the final desired feature enhancer.

[0064] The method provided in this invention obtains an initial EEG decoding model including an initial feature enhancer, and initializes the parameters of the initial encoder in the initial EEG decoding model based on the parameters of the pre-trained encoder in the pre-trained EEG decoding model. The parameters of the state discriminator are initialized based on the parameters of the pre-trained state discriminator in the pre-trained EEG decoding model. This initialization strategy provides a high-quality starting point for the training of the initial EEG decoding model and accelerates its convergence. Simultaneously, the adversarial signal provided by the state discriminator directly guides the feature enhancer to learn the feature space mapping from before to after neural modulation. This mechanism ensures that the feature enhancer can efficiently and accurately learn the ability to simulate neural modulation effects, thereby significantly improving the feature extraction performance and decoding robustness of the final EEG decoding model for weak stimulus signals in real-world scenarios.

[0065] Based on the above embodiments, the step of obtaining the pre-trained EEG decoding model includes: Step 310: Obtain a sample EEG signal set including a first EEG signal set collected before neural modulation and a second EEG signal set collected after neural modulation, as well as an original EEG decoding model; the original EEG decoding model includes an original encoder, an original decoder, an original task classifier, and an original state discriminator. Step 320: Input the sample EEG signal set into the original EEG decoding model. The original encoder encodes the sample EEG signal set to obtain compressed features. The original task classifier classifies the compressed features to obtain the predicted category distribution. The original state discriminator performs state prediction on the compressed features to obtain the state prediction probability. Step 330: Determine the task classification loss based on the difference between the predicted category distribution and the label categories corresponding to the sample EEG signal set; Step 340: Determine the state classification loss based on the difference between the predicted state probability and the true state label corresponding to the sample EEG signal set; Step 350: Based on the task classification loss and the state classification loss, determine the target loss, and train the original EEG decoding model based on the target loss to obtain the pre-trained EEG decoding model.

[0066] Specifically, firstly, a sample set of EEG signals, including a first set of EEG signals collected before neural modulation and a second set of EEG signals collected after neural modulation, as well as the original EEG decoding model, can be obtained.

[0067] The first set of EEG signals can be represented by EEG-Pre data, which is the SSVEP EEG data collected from the user before undergoing TMS or other neuromodulation techniques. The second set of EEG signals can be represented by EEG-Post data, which is the EEG signal set collected from the same user after receiving TMS, when their attention and other states are improved. These two datasets constitute the sample EEG signal set.

[0068] In an alternative embodiment, Figure 2 This is a flowchart of the sample EEG signal set acquisition process provided by the present invention, as follows: Figure 2 As shown, using the SSVEP neural modulation dataset, a 40-target high-frequency SSVEP visual stimulation interface was designed. Six blocks of SSVEP data were collected, with 40 trials in each block. The 40 targets appeared once in a random order. Each trial consisted of a 0.5s cue + 6s visual stimulation + 0.5s rest. The first EEG signal set was recorded as EEG-Pre. Then, rTMS neural modulation was performed on the subjects. A 5 Hz rTMS stimulation was applied to the left brain V1 of the subjects, with a total of 1200 pulses. This improved the signal-to-noise ratio of the subsequent SSVEP response by enhancing the subjects' visual attention. After the rTMS stimulation, the above SSVEP task-state data collection was repeated, with the procedure consistent with Pre. The second EEG signal set was recorded as EEG-Post.

[0069] The original EEG decoding model is a complete model architecture used for pre-training, whose parameters are typically randomly initialized at the start of training. The original EEG decoding model includes an original encoder, an original decoder, an original task classifier, and an original state discriminator.

[0070] After obtaining the sample EEG signal set, the sample EEG signal set can be input into the original EEG decoding model, and the original encoder can encode the sample EEG signal set to obtain compressed features.

[0071] To guide compression features To more effectively capture task features related to stimulus categories in SSVEP signals, this embodiment of the invention introduces a parallel-trained original task classifier. This original task classifier uses... As input, output the predicted class distribution. ,in This represents the number of frequency categories for the SSVEP task. The original task classifier is jointly trained with the original encoder, and backpropagation prompts the encoder to extract latent features that are discriminative for the SSVEP task category, thereby improving the accuracy of downstream instruction recognition.

[0072] The original state discriminator performs state prediction on the compressed features to obtain the state prediction probability.

[0073] Here, the original encoder can be a cascaded multilayer convolutional neural network, a deep neural network, a combination of CNN and DNN, or a Transformer, etc. The embodiments of the present invention do not specifically limit this.

[0074] Here, the specific structure of the original state discriminator can be several fully connected layers plus a sigmoid activation function, and the embodiments of the present invention do not specifically limit this.

[0075] After obtaining the predicted category distribution and state prediction probabilities, the task classification loss can be determined based on the difference between the predicted category distribution and the label categories corresponding to the sample EEG signal set. Here, the task classification loss is the categorical cross-entropy loss, and the formula for the task classification loss is as follows:

[0076] in, Indicates task classification loss. This indicates the label category corresponding to the sample EEG signal set, i.e., the true one-hot encoded label. This indicates the predicted class distribution. For example, if there are 40 SSVEP targets, then the actual one-hot encoded labels would be 0, 1, 2, 3, ... 39, a total of forty classes.

[0077] It is understandable that the greater the difference between the predicted category distribution and the label category corresponding to the sample EEG signal set, the greater the task classification loss; the smaller the difference between the predicted category distribution and the label category corresponding to the sample EEG signal set, the smaller the task classification loss.

[0078] Furthermore, the state classification loss can be determined based on the difference between the predicted state probability and the true state label corresponding to the sample EEG signal set. The formula for the state classification loss is as follows:

[0079] in, Represents the state classification loss. Indicates the actual state label. This represents the probability of state prediction.

[0080] It is understandable that the greater the difference between the state prediction probability and the true state label corresponding to the sample EEG signal set, the greater the state classification loss; the smaller the difference between the state prediction probability and the true state label corresponding to the sample EEG signal set, the smaller the state classification loss.

[0081] Finally, after obtaining the task classification loss and the state classification loss, the target loss can be determined based on the task classification loss and the state classification loss. The original EEG decoding model can then be trained based on the target loss, and the trained original EEG decoding model can be used as a pre-trained EEG decoding model.

[0082] Here, the target loss can be determined based on the sum of the task classification loss and the state classification loss, or it can be determined based on the weighted sum of the task classification loss and the state classification loss. This embodiment of the invention does not specifically limit the method.

[0083] It should be noted that the methods described in the above embodiments are not limited to the SSVEP paradigm, but can also be applied to other brain-computer interface paradigms, such as motor imagery (MI) and event-related potentials (P300). As long as two different states, such as low concentration / high concentration and fatigue / awake state, can be collected, the same methods as those in the embodiments of this invention can be used for model training and command issuance.

[0084] The method provided in this invention, by combining task classification loss and state classification loss to train the original EEG decoding model for multiple tasks, effectively solves the problems of insufficient model feature learning and weak generalization ability in single-task training. Task classification loss ensures that the pre-trained EEG decoding model masters basic EEG command decoding capabilities, while state classification loss forces the pre-trained EEG decoding model to distinguish the intrinsic differences between signals before and after neural modulation at the feature level, guiding the original encoder to extract general features that are sensitive to and discriminative of changes in neural modulation state. This multi-task collaborative optimization mechanism ensures that the final pre-trained encoder and pre-trained state discriminator contain high-quality, robust feature prior knowledge, laying a solid foundation for the efficient training of subsequent feature enhancers, thereby improving the overall performance and stability of the EEG decoding system.

[0085] Based on the above embodiments, step 350, determining the target loss based on the task classification loss and the state classification loss, includes: Step 351: Determine the orthogonality constraint loss based on the correlation between the internal dimensions of the compressed features; Step 352: Determine the target loss based on the task classification loss, the state classification loss, and the orthogonality constraint loss.

[0086] Specifically, to improve compression features To address the structural decoupling and interpretability of neural activity, embodiments of the present invention further add orthogonality constraints to each dimension. Orthogonality constraints promote... The different dimensions are independent and non-redundant, avoiding multiple dimensions from expressing the same EEG feature components. Assume that the compressed features of NNN samples in a mini-batch are concatenated into a matrix. The goal of orthogonality constraints is to make That is, the covariance matrix is ​​close to the identity matrix. The formula for the orthogonality constraint loss is as follows:

[0087] in, This represents the loss due to orthogonality constraints. Denotes the Frobenius norm. The identity matrix is ​​used. This orthogonality constraint loss, by penalizing non-orthogonal components, guides the encoder to learn more physically interpretable and more separable feature dimensions. This allows different dimensions to capture different neural components in the EEG signal, such as frequency bands and modulation changes, thereby improving the interpretability and stability of the pre-trained EEG decoding model.

[0088] After obtaining the orthogonality constraint loss, the target loss can be determined based on the task classification loss, state classification loss, and orthogonality constraint loss.

[0089] Here, the target loss can be determined based on the sum of task classification loss, state classification loss, and orthogonality constraint loss, or based on the weighted sum of task classification loss, state classification loss, and orthogonality constraint loss.

[0090] Based on the above embodiments, the method further includes: Step 410: Perform frequency domain transformation on the sample EEG signal set to obtain frequency domain input features; Step 420: Perform local masking on the frequency domain input features to obtain frequency domain masked features; Step 430: The frequency domain mask features are encoded by the original encoder to obtain the compressed features; Step 440: The original decoder performs spectral reconstruction on the compressed features to obtain the reconstructed features; Step 450: Determine the reconstruction loss based on the difference between the reconstructed features and the frequency domain input features; Step 460: Determine the target loss based on the reconstruction loss, the task classification loss, the state classification loss, and the orthogonality constraint loss.

[0091] Specifically, to enhance the neural network's attention to the frequency harmonics information in SSVEP, a frequency domain transformation is performed on the sample EEG signal set to obtain frequency domain input features. That is, Fourier transform or spectral analysis is applied to the sample EEG signals of each trial to generate frequency domain input features. .

[0092] To enhance the robustness of the pre-trained EEG decoding model to local information loss and to construct a reconstruction task, the frequency domain input features are locally masked to obtain frequency domain masked features. Specifically, a portion of the frequency domain input features is set to 0 or random noise to simulate information loss. The frequency domain masked features can be used... express.

[0093] Furthermore, the frequency domain mask features are encoded by the original encoder to obtain compressed features. The compressed features are then reconstructed using the spectrum by the original decoder to obtain reconstructed features. This process reconstructs the masked frequency domain signal. The original decoder structure is symmetrical to the original encoder and is constructed using dedilation convolution and Transformer modules.

[0094] After obtaining the reconstructed features, the reconstruction loss can be determined based on the difference between the reconstructed features and the frequency domain input features. The formula for the reconstruction loss is as follows:

[0095] in, Indicates the losses incurred during reconstruction. Indicates reconstruction features, This represents the frequency domain input characteristics.

[0096] It is understandable that the greater the difference between the reconstructed features and the frequency domain input features, the greater the reconstruction loss; the smaller the difference between the reconstructed features and the frequency domain input features, the smaller the reconstruction loss.

[0097] Figure 3 This is a training diagram of the pre-trained EEG decoding model provided by the present invention, as shown below. Figure 3 As shown, The target loss can be determined based on reconstruction loss, task classification loss, state classification loss, and orthogonality constraint loss. The formula for the target loss is as follows:

[0098] in, Indicates target loss. Indicates the losses incurred during reconstruction. Indicates task classification loss. Represents the state classification loss. This represents the loss due to orthogonality constraints. , , , These are all weighting coefficients, set according to the performance of the validation set.

[0099] Here, both the encoder and decoder use a parameter-sharing mechanism, meaning that the preprocessing and postprocessing branches use the same encoder and the same decoder.

[0100] Through the above-mentioned joint training strategy, the embodiments of the present invention effectively improve the task discrimination ability and structural decoupling ability of compressed feature z, providing a stable, controllable and interpretable feature foundation for subsequent feature enhancement and instruction recognition.

[0101] Understandably, the multi-task auxiliary module consists of three modules: the task classifier, the state discriminator, and the orthogonality constraint. Since the sample EEG signal set includes a first EEG signal set collected before neural modulation and a second EEG signal set collected after neural modulation, the compressed features correspondingly exist in two forms: one is the pre-neural modulation... One characteristic is after neural modulation. Features, and Features and The features are input into three modules: task classifier, state discriminator, and orthogonality constraint, which determine the task classification loss, state classification loss, and orthogonality constraint loss, respectively.

[0102] Figure 4 This is a training diagram of the EEG decoding model provided by the present invention, as shown below. Figure 4 As shown, the EEG decoding model includes an encoder, a feature enhancer, and a task classifier. The encoder is used to encode real-time EEG signals to obtain a compressed representation before neural modulation. The feature enhancer is used to enhance the features of the compressed representation, resulting in an enhanced representation. The task classifier is used to classify the enhanced representations to obtain the EEG decoding results.

[0103] Similar to the target loss during the training process of the pre-trained EEG decoding model, the training loss of the second-stage EEG decoding model is as follows:

[0104] in, Indicates training loss, Indicates the losses incurred during reconstruction. Indicates task classification loss. Indicates the loss in the state determination. This represents the loss due to orthogonality constraints. , , , These are all weighting coefficients, set according to the performance of the validation set.

[0105] It should be noted that the training loss of the EEG decoding model is only used for gradient backpropagation and is not used for parameter updates. That is, the parameters of the encoder, decoder, task classifier and state discriminator in the EEG decoding model are fixed.

[0106] In summary, the training of the EEG decoding model is divided into two stages: the first stage is the learning and optimization of the autoencoded compressed representation, and the second stage is the construction and training of the feature enhancer.

[0107] Phase 1 constructs a self-supervised autoencoder network with structural constraints and task orientation, using frequency domain EEG as input for compression and reconstruction. Through local masking and frequency domain modeling, the extraction capability of high-frequency and harmonic response features is effectively improved. Task classification constraints (distinguishing stimulus frequencies), state classification constraints (distinguishing signals before and after modulation), and orthogonality structural constraints are applied to the compressed features to make them discriminative, decoupled, and interpretable, providing stable features for subsequent modeling.

[0108] Phase two involves the construction and training of the feature enhancer. Building upon compressed features, a feature enhancer module is introduced to simulate the feature transfer relationship from the pre-modulation state to the post-modulation state. A nonlinear mapping is constructed using a residual network to achieve feature style transfer and task feature preservation, enabling the model to generate more discriminative virtual brain responses. This significantly improves the representation quality and decoding performance of signals under weak stimulation conditions.

[0109] Based on the above embodiments, the encoder is a harmonic alignment encoder; The harmonic alignment encoder includes a fundamental frequency feature extraction module, a harmonic frequency feature extraction module, and a self-attention feature mining module; the fundamental frequency feature extraction module is used to extract fundamental frequency features from the real-time EEG signal; the harmonic frequency feature extraction module is used to extract harmonic frequency features from the real-time EEG signal; the self-attention feature mining module is used to perform self-attention calculation based on the fundamental frequency features and the harmonic frequency features to obtain the compressed representation.

[0110] Specifically, the encoder can be a harmonic alignment encoder, which can include a fundamental frequency feature extraction module, an overtone feature extraction module, and a self-attention feature mining module. The fundamental frequency feature extraction module is used to extract fundamental frequency features from real-time EEG signals, the overtone feature extraction module is used to extract overtone features from real-time EEG signals, and the self-attention feature mining module is used to perform self-attention calculation based on fundamental frequency features and overtone features to obtain a compressed representation.

[0111] Here, the fundamental frequency feature is used to reflect the fundamental frequency information in the real-time EEG signal, and the harmonic frequency feature is used to reflect the harmonic frequency information in the real-time EEG signal.

[0112] The method provided in this invention, a harmonic alignment encoder, effectively addresses the problems of insufficient utilization of the harmonic structure and weak feature representation ability of traditional methods by setting a fundamental frequency feature extraction module, a harmonic frequency feature extraction module, and a self-attention feature mining module. The fundamental frequency feature extraction module and the harmonic frequency feature extraction module decouple the fundamental frequency and its harmonic components directly related to the visual stimulus frequency from the real-time EEG signal, ensuring the integrity of the core frequency information. The self-attention feature mining module adaptively mines and fuses the intrinsic correlations and weights among these harmonic components, thereby constructing a more discriminative compressed representation. This collaborative mechanism significantly improves the ability to capture and represent key patterns of SSVEP signals, laying a solid foundation for high-precision EEG decoding.

[0113] Based on the above embodiments, the fundamental frequency feature extraction module performs convolution calculation on the fundamental frequency part of the real-time EEG signal using a first convolution kernel with a first dilation rate to obtain the fundamental frequency feature; The frequency octave feature extraction module obtains the frequency octave features by performing convolution calculation on the frequency octave portion of the real-time EEG signal using a second convolution kernel with a second dilation rate. Wherein, the second convolution kernel has a larger size on the frequency axis than the first convolution kernel, and the second dilation rate is greater than the first dilation rate.

[0114] Specifically, in the frequency domain analysis of steady-state visual evoked potential signals, although the fundamental band and the octave band have the same spectral resolution, the number of sampling points between adjacent targets increases exponentially with the frequency order because the target stimulus interval is a multiple of each other in the octave band.

[0115] Therefore, if a uniform convolution kernel structure is used directly in different frequency bands, it will lead to an imbalance in the ability to capture local spectral patterns, making it difficult to meet the characterization requirements of the fundamental frequency and harmonics.

[0116] Accordingly, in this embodiment of the invention, the input signal is first segmented in the frequency domain into a fundamental frequency band and a second harmonic frequency band to ensure that different frequency bands have independent modeling paths during feature extraction. The fundamental frequency feature extraction module performs convolution calculation on the fundamental frequency portion of the real-time EEG signal using a first convolution kernel with a first dilation rate to obtain the fundamental frequency feature; the harmonic frequency feature extraction module performs convolution calculation on the harmonic frequency portion of the real-time EEG signal using a second convolution kernel with a second dilation rate to obtain the harmonic frequency feature.

[0117] The second convolution kernel has a larger size on the frequency axis than the first convolution kernel, and the second dilation rate is greater than the first dilation rate.

[0118] In an alternative embodiment, Figure 5 This is a schematic diagram of the harmonic alignment encoder provided by the present invention, as shown below. Figure 5 As shown, for the fundamental frequency band, a size of [size missing] is used. The convolutional kernel extracts the local correlation between adjacent sampling points, thus preserving the continuity of the narrow interval features of the fundamental frequency; while for the second octave band, a kernel with a size of [missing information] is used. The convolutional kernels are adapted to accommodate wider frequency intervals and cross-target relationships. Then, multi-dilated convolutional structures are further introduced in different frequency bands to achieve scale alignment. A convolutional kernel with a dilation rate of 1 is used for the fundamental band to preserve its local continuity; while for the second harmonic band, a convolutional kernel with a dilation rate of 2 is used to align the sampling points corresponding to the fundamental and harmonic frequencies within the receptive field. The convolutional kernels establish equivariance across harmonics on the frequency axis, effectively compensating for the inability of traditional methods to guarantee scale consistency in multi-band modeling. Subsequently, learnable parameters are introduced. and A harmonic weighted aggregation mechanism is implemented to weight and fuse the features of the fundamental frequency band and the octave band, and dynamically adjust the contribution ratio of different frequency band features in the global representation.

[0119] Finally, a three-layer stacked self-attention feature mining module is introduced, which can simultaneously model global dependencies in the frequency and lead dimensions, capture long-range correlations across frequency bands and spatial synergistic effects across brain regions, thereby generating compressed features with discriminative, robust, and interpretable characteristics. This provides a high-quality feature base for subsequent brain signal reconstruction and target classification tasks.

[0120] Here, frequency domain mask features can also be used. The input is fed into the harmonic alignment encoder to obtain the compression characteristics output by the harmonic alignment encoder.

[0121] Here, the self-attention feature mining module may include a normalization layer and a self-attention module.

[0122] Based on any of the above embodiments Figure 6 This is a schematic diagram of the overall architecture of the EEG signal processing system provided by the present invention, as shown below. Figure 6 As shown, the system first begins with the signal acquisition step. Figure 6 The top left side shows the head-mounted data acquisition device worn by the subject. This module is responsible for acquiring raw electroencephalogram (EEG) signals. Figure 6 The raw data is labeled in the image. This raw data appears as a chaotic waveform, representing the initial physiological signal state.

[0123] Subsequently, the system enters the signal processing stage. The acquired raw data is sent to the signal processing module, one of the core functions of which is to extract the effective signal segments related to steady-state visual evoked potentials, namely the SSVEP segment shown in the figure. The processed and normalized waveform lays the foundation for subsequent decoding and analysis.

[0124] Then, the system performs a weak implicit decoding process. The preprocessed and feature-enhanced SSVEP signal is input to the weak implicit decoding module, whose task is to accurately extract the user intent information hidden in the weak signal with a low signal-to-noise ratio.

[0125] The decoding results are ultimately sent to the target recognition stage. The system converts the decoded intent information into specific control commands, thereby enabling the recognition and selection of specific targets on external devices or interfaces, such as... Figure 6 The grid options displayed on the central display screen.

[0126] In addition, the system includes a result feedback path. The result of target recognition can be fed back to the user or other parts of the system, thus forming a closed-loop control loop from signal acquisition, processing, decoding, recognition to feedback, optimizing the interactive performance and stability of the entire system.

[0127] In one embodiment, firstly, a high-frequency transcranial magnetic stimulation (TSVEP) experimental paradigm oriented towards weak implicit stimulation conditions is designed, constructing a dual-domain dataset covering the states before and after neural response enhancement to collect EEG signals induced by real physical modulation. Subsequently, a neural network modeling framework is proposed to simulate the brain response enhancement effect. Specifically, this invention designs a harmonic alignment feature mining module to align the fundamental and harmonic components, and introduces a task classifier, state discriminator, and orthogonality constraints into the compressed representation to guide the encoder to learn the key features of SSVEP enhancement. Innovatively, the enhanced SSVEP signal is used as the decoder reconstruction target, thereby realizing the virtual enhancement and reconstruction of weak responses in the feature space. Finally, a weak implicit SSVEP brain-computer interface command issuing device is constructed to achieve accurate decoding and command output of weak brain responses. This device can be widely applied in scenarios such as brain-controlled typing, neurorehabilitation training, and intelligent interaction under zero-training conditions. Compared with existing methods, this invention can still significantly improve the recognition accuracy, signal-to-noise ratio, and robustness of brain-computer interfaces under weak implicit stimulation conditions, providing a novel technical solution for efficient and natural human-computer interaction under low-interference conditions.

[0128] The following describes the brain-computer interface command issuing device based on modulation enhancement simulation provided by the present invention. The brain-computer interface command issuing device based on modulation enhancement simulation described below and the brain-computer interface command issuing method based on modulation enhancement simulation described above can be referred to in correspondence with each other.

[0129] Based on any of the above embodiments, the present invention provides a brain-computer interface command issuing device based on modulation enhancement simulation. Figure 7 This is a schematic diagram of the brain-computer interface command issuing device based on modulation enhancement simulation provided by the present invention, as shown below. Figure 7 As shown, the device includes: Acquisition unit 710 is used to acquire the user's real-time EEG signals; The input unit 720 is used to input the real-time EEG signal into the EEG decoding model to obtain the EEG decoding result output by the EEG decoding model; The instruction issuing unit 730 is used to issue instructions based on the EEG decoding results; The EEG decoding model includes an encoder, a feature enhancer, and a task classifier; the encoder is used to encode the real-time EEG signal to obtain a compressed representation before neural modulation; the feature enhancer is used to enhance the compressed representation to obtain an enhanced representation; the task classifier is used to classify the enhanced representation to obtain the EEG decoding result. The feature enhancer is obtained by training a state discriminator based on sample EEG signals collected before neural modulation and real state labels after neural modulation.

[0130] The device provided in this invention includes an EEG decoding model comprising an encoder, a feature enhancer, and a task classifier. The encoder encodes real-time EEG signals to obtain a compressed representation before neural modulation. The feature enhancer enhances the compressed representation to obtain an enhanced representation. The task classifier classifies the enhanced representation to obtain the EEG decoding result. In this method, the feature enhancer is trained using sample EEG signals acquired before neural modulation and real-state labels after neural modulation, in conjunction with a state discriminator. This training method drives the feature enhancer to learn the feature transfer relationship between the compressed features before neural modulation and the features after neural modulation, simulating the enhancement effect of neural modulation. This significantly improves the EEG decoding model's ability to represent EEG signals by features, thereby enhancing the system's robustness in decoding weak stimulus signals without relying on high-intensity external stimuli. Ultimately, while ensuring user comfort, it improves the accuracy of brain-computer interface command recognition and the overall stability of the system.

[0131] Based on any of the above embodiments, a first training unit is further included, wherein the first training unit is specifically used for: An initial EEG decoding model including an initial feature enhancer is obtained, and the parameters of the initial encoder in the initial EEG decoding model are initialized based on the parameters of the pre-trained encoder in the pre-trained EEG decoding model; the parameters of the state discriminator are initialized based on the parameters of the pre-trained state discriminator in the pre-trained EEG decoding model. The sample EEG signal collected before the neural modulation is input into the initial EEG decoding model. The initial encoder encodes the sample EEG signal to obtain a compressed representation of the sample before neural modulation. The initial feature enhancer enhances the compressed representation of the sample to obtain an enhanced representation of the sample. The state discriminator discriminates the enhanced representation of the sample to obtain a state prediction result. Based on the difference between the state prediction result and the true state label, a state discrimination loss is determined, and the initial feature enhancer is trained based on the state discrimination loss to obtain the feature enhancer.

[0132] Based on any of the above embodiments, a second training unit is further included, the second training unit specifically including: The sample acquisition unit is used to acquire a sample EEG signal set including a first EEG signal set acquired before neural modulation and a second EEG signal set acquired after neural modulation, as well as an original EEG decoding model; the original EEG decoding model includes an original encoder, an original decoder, an original task classifier, and an original state discriminator; The prediction unit is used to input the sample EEG signal set into the original EEG decoding model, where the original encoder encodes the sample EEG signal set to obtain compressed features, the original task classifier classifies the compressed features to obtain a predicted category distribution, and the original state discriminator performs state prediction on the compressed features to obtain a state prediction probability. A task classification loss unit is determined, which is used to determine the task classification loss based on the difference between the predicted category distribution and the label category corresponding to the sample EEG signal set; A state classification loss unit is defined to determine the state classification loss based on the difference between the state prediction probability and the true state label corresponding to the sample EEG signal set. A target loss unit is defined to determine a target loss based on the task classification loss and the state classification loss, and to train the original EEG decoding model based on the target loss to obtain the pre-trained EEG decoding model.

[0133] Based on any of the above embodiments, the target loss determination unit is specifically used for: Based on the correlation between the internal dimensions of the compressed features, the orthogonality constraint loss is determined; The target loss is determined based on the task classification loss, the state classification loss, and the orthogonality constraint loss.

[0134] Based on any of the above embodiments, a joint training unit is further included, wherein the joint training unit is specifically used for: The sample EEG signal set is frequency domain transformed to obtain frequency domain input features; The frequency domain input features are locally masked to obtain the frequency domain masked features; The compressed features are obtained by encoding the frequency domain mask features using the original encoder; The original decoder performs spectral reconstruction on the compressed features to obtain the reconstructed features; The reconstruction loss is determined based on the difference between the reconstructed features and the frequency domain input features; The target loss is determined based on the reconstruction loss, the task classification loss, the state classification loss, and the orthogonality constraint loss.

[0135] Based on any of the above embodiments, the encoder is a harmonic alignment encoder; The harmonic alignment encoder includes a fundamental frequency feature extraction module, a harmonic frequency feature extraction module, and a self-attention feature mining module; the fundamental frequency feature extraction module is used to extract fundamental frequency features from the real-time EEG signal; the harmonic frequency feature extraction module is used to extract harmonic frequency features from the real-time EEG signal; the self-attention feature mining module is used to perform self-attention calculation based on the fundamental frequency features and the harmonic frequency features to obtain the compressed representation.

[0136] Based on any of the above embodiments, the fundamental frequency feature extraction module performs convolution calculation on the fundamental frequency part of the real-time EEG signal using a first convolution kernel with a first dilation rate to obtain the fundamental frequency feature; The frequency octave feature extraction module obtains the frequency octave features by performing convolution calculation on the frequency octave portion of the real-time EEG signal using a second convolution kernel with a second dilation rate. Wherein, the second convolution kernel has a larger size on the frequency axis than the first convolution kernel, and the second dilation rate is greater than the first dilation rate.

[0137] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 8 As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communications bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other through the communications bus 840. The processor 810 can call logical instructions in the memory 830 to execute a brain-computer interface instruction issuance method based on modulation enhancement simulation. This method includes: acquiring the user's real-time EEG signal; inputting the real-time EEG signal into an EEG decoding model to obtain the EEG decoding result output by the EEG decoding model; and issuing instructions based on the EEG decoding result. The EEG decoding model includes an encoder, a feature enhancer, and a task classifier. The encoder encodes the real-time EEG signal to obtain a compressed representation before neural modulation. The feature enhancer enhances the compressed representation to obtain an enhanced representation. The task classifier classifies the enhanced representation to obtain the EEG decoding result. The feature enhancer is trained based on sample EEG signals acquired before neural modulation and real-state labels after neural modulation, combined with a state discriminator.

[0138] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0139] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the brain-computer interface instruction issuance method based on modulation enhancement simulation provided by the above methods. The method includes: acquiring the user's real-time EEG signal; inputting the real-time EEG signal into an EEG decoding model to obtain the EEG decoding result output by the EEG decoding model; and issuing instructions based on the EEG decoding result. The EEG decoding model includes an encoder, a feature enhancer, and a task classifier. The encoder is used to encode the real-time EEG signal to obtain a compressed representation before neural modulation. The feature enhancer is used to enhance the compressed representation to obtain an enhanced representation. The task classifier is used to classify the enhanced representation to obtain the EEG decoding result. The feature enhancer is trained based on sample EEG signals acquired before neural modulation and real state labels after neural modulation, in conjunction with a state discriminator.

[0140] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a brain-computer interface instruction issuing method based on modulation enhancement simulation provided by the above methods. The method includes: acquiring a user's real-time EEG signal; inputting the real-time EEG signal into an EEG decoding model to obtain an EEG decoding result output by the EEG decoding model; and issuing an instruction based on the EEG decoding result. The EEG decoding model includes an encoder, a feature enhancer, and a task classifier. The encoder is used to encode the real-time EEG signal to obtain a compressed representation before neural modulation. The feature enhancer is used to enhance the compressed representation to obtain an enhanced representation. The task classifier is used to classify the enhanced representation to obtain the EEG decoding result. The feature enhancer is obtained by training a state discriminator based on sample EEG signals acquired before neural modulation and real state labels after neural modulation.

[0141] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0142] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A brain-computer interface command issuance method based on modulated augmentation simulation, characterized in that, include: Acquire the user's real-time brainwave signals; The real-time EEG signal is input into the EEG decoding model to obtain the EEG decoding result output by the EEG decoding model; Based on the EEG decoding results, commands are issued; The EEG decoding model includes an encoder, a feature enhancer, and a task classifier; the encoder is used to encode the real-time EEG signal to obtain a compressed representation before neural modulation. The feature enhancer is used to enhance the features of the compressed representation to obtain an enhanced representation; The task classifier is used to classify the enhanced representation to obtain the EEG decoding result; The feature enhancer is obtained by training a state discriminator based on sample EEG signals collected before neural modulation and real state labels after neural modulation.

2. The brain-computer interface command issuance method based on enhanced modulation simulation according to claim 1, characterized in that, The training steps of the feature enhancer include: An initial EEG decoding model including an initial feature enhancer is obtained, and the parameters of the initial encoder in the initial EEG decoding model are initialized based on the parameters of the pre-trained encoder in the pre-trained EEG decoding model; the parameters of the state discriminator are initialized based on the parameters of the pre-trained state discriminator in the pre-trained EEG decoding model. The sample EEG signal collected before the neural modulation is input into the initial EEG decoding model. The initial encoder encodes the sample EEG signal to obtain a compressed representation of the sample before neural modulation. The initial feature enhancer enhances the compressed representation of the sample to obtain an enhanced representation of the sample. The state discriminator discriminates the enhanced representation of the sample to obtain a state prediction result. Based on the difference between the state prediction result and the true state label, a state discrimination loss is determined, and the initial feature enhancer is trained based on the state discrimination loss to obtain the feature enhancer.

3. The brain-computer interface command issuance method based on enhanced modulation simulation according to claim 2, characterized in that, The steps for obtaining the pre-trained EEG decoding model include: The sample EEG signal set is acquired, including a first EEG signal set collected before neural modulation and a second EEG signal set collected after neural modulation, as well as an original EEG decoding model; the original EEG decoding model includes an original encoder, an original decoder, an original task classifier, and an original state discriminator; The sample EEG signal set is input into the original EEG decoding model. The original encoder encodes the sample EEG signal set to obtain compressed features. The original task classifier classifies the compressed features to obtain the predicted category distribution. The original state discriminator performs state prediction on the compressed features to obtain the state prediction probability. Based on the difference between the predicted category distribution and the label categories corresponding to the sample EEG signal set, the task classification loss is determined; The state classification loss is determined based on the difference between the state prediction probability and the true state label corresponding to the sample EEG signal set. Based on the task classification loss and the state classification loss, a target loss is determined, and the original EEG decoding model is trained based on the target loss to obtain the pre-trained EEG decoding model.

4. The brain-computer interface command issuance method based on enhanced modulation simulation according to claim 3, characterized in that, The determination of the target loss based on the task classification loss and the state classification loss includes: Based on the correlation between the internal dimensions of the compressed features, the orthogonality constraint loss is determined; The target loss is determined based on the task classification loss, the state classification loss, and the orthogonality constraint loss.

5. The brain-computer interface command issuance method based on modulated augmentation simulation according to claim 4, characterized in that, The method further includes: The sample EEG signal set is frequency domain transformed to obtain frequency domain input features; The frequency domain input features are locally masked to obtain the frequency domain masked features; The compressed features are obtained by encoding the frequency domain mask features using the original encoder; The original decoder performs spectral reconstruction on the compressed features to obtain the reconstructed features; The reconstruction loss is determined based on the difference between the reconstructed features and the frequency domain input features; The target loss is determined based on the reconstruction loss, the task classification loss, the state classification loss, and the orthogonality constraint loss.

6. The brain-computer interface command issuance method based on modulated augmentation simulation according to any one of claims 1 to 5, characterized in that, The encoder is a harmonic alignment encoder; The harmonic alignment encoder includes a fundamental frequency feature extraction module, a harmonic frequency feature extraction module, and a self-attention feature mining module; the fundamental frequency feature extraction module is used to extract fundamental frequency features from the real-time EEG signal; the harmonic frequency feature extraction module is used to extract harmonic frequency features from the real-time EEG signal; the self-attention feature mining module is used to perform self-attention calculation based on the fundamental frequency features and the harmonic frequency features to obtain the compressed representation.

7. The brain-computer interface command issuance method based on enhanced modulation simulation according to claim 6, characterized in that, The fundamental frequency feature extraction module performs convolution calculation on the fundamental frequency part of the real-time EEG signal using a first convolution kernel with a first dilation rate to obtain the fundamental frequency feature. The frequency octave feature extraction module obtains the frequency octave features by performing convolution calculation on the frequency octave portion of the real-time EEG signal using a second convolution kernel with a second dilation rate. Wherein, the second convolution kernel has a larger size on the frequency axis than the first convolution kernel, and the second dilation rate is greater than the first dilation rate.

8. A brain-computer interface command issuing device based on modulated augmentation simulation, characterized in that, include: The acquisition unit is used to acquire the user's real-time EEG signals; The input unit is used to input the real-time EEG signal into the EEG decoding model to obtain the EEG decoding result output by the EEG decoding model; The instruction issuing unit is used to issue instructions based on the EEG decoding results; The EEG decoding model includes an encoder, a feature enhancer, and a task classifier; The encoder is used to encode the real-time EEG signal to obtain a compressed representation before neural modulation. The feature enhancer is used to enhance the features of the compressed representation to obtain an enhanced representation; The task classifier is used to classify the enhanced representation to obtain the EEG decoding result; The feature enhancer is obtained by training a state discriminator based on sample EEG signals collected before neural modulation and real state labels after neural modulation.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the brain-computer interface instruction issuing method based on modulation enhancement simulation as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the brain-computer interface instruction issuing method based on modulation enhancement simulation as described in any one of claims 1 to 7.

Images