A transition type non-invasive brain electrical signal decoding method, system and storage medium

Abstract

This application relates to the field of EEG signal decoding technology, and more particularly to a transitional non-invasive EEG signal decoding method, system, and storage medium. The method includes acquiring a multimodal dataset, constructing a student network, and freezing the parameters of a pre-trained teacher network. After warming up the student network, the multimodal dataset is input in parallel into the student network and the teacher network to train the student network. During training, the parameters of the student network are updated using an error backpropagation algorithm based on the error signal calculated by a composite loss function. When the error signal of the composite loss function is detected to be below a preset threshold for multiple consecutive training cycles, parameter updates are stopped, and the training of the student network is completed. The trained student network is then put into the online inference stage, where it outputs motion control commands.

Description

Technical Field

[0001] This application relates to the field of EEG signal decoding technology, and in particular to a transitional non-invasive EEG signal decoding method, system and storage medium. Background Technology

[0002] In the construction of motor brain-computer interface systems, the choice of signal acquisition method directly determines the system's performance ceiling and application scope. While traditional invasive techniques can directly record cortical neuronal firing activity with high signal-to-noise ratio and high spatiotemporal resolution through implanted microelectrode arrays, thus achieving precise motor decoding, the risks of craniotomy, postoperative infection, and glial cell encapsulation after long-term implantation significantly limit their widespread adoption and application in a broad population. In contrast, non-invasive techniques based on scalp electroencephalography (EEG) capture macroscopic electrical activity of the brain through electrodes attached to the scalp. With their advantages of zero trauma, low cost, high temporal resolution, and ease of wear, they have become the most ideal "invisible bridge" connecting the brain and the outside world.

[0003] However, despite the progress made by non-invasive EEG in simple, discrete control tasks, its performance remains significantly hampered by the gap with invasive techniques when attempting to resolve continuous, fine motor trajectories. The fundamental challenge lies in the "volume conduction effect." Neural electrical signals generated in the cerebral cortex must pass through multiple layers of media, including cerebrospinal fluid, meninges, skull, and scalp, to reach scalp electrodes. Due to the extremely high resistivity of the skull and the varying conductivity of different tissue layers, this transmission path constitutes a natural low-pass filter and spatial scatterer. This physical effect means that the potentials recorded on the scalp do not originate solely from the cortical region directly beneath it, but rather are a weighted superposition of neuronal activity across a wide range of brain regions. This causes motor commands such as those for the hand and arm, which were clearly separated in the primary motor cortex (M1 area), to become mixed up after reaching the scalp, resulting in a sharp decrease in spatial resolution. Meanwhile, high-frequency neural oscillations (such as the Gamma band) and neuronal action potentials, which carry rich information about movement details, are significantly attenuated when penetrating the skull, resulting in scalp EEG being dominated by low-frequency components and losing the key features required for fine motor control.

[0004] To attempt to derive the true cortical source signals from fuzzy scalp EEG, traditional techniques often employ source localization algorithms based on physical models, such as LORETA or dipole fitting. However, inferring the three-dimensional brain source distribution from two-dimensional scalp surface potentials is mathematically a classic ill-posed problem, meaning there are infinitely many possible solutions. Existing source localization methods often rely on idealized head geometry models and imposed prior assumptions, resulting in extremely high computational complexity, making it difficult to meet the millisecond-level response speed requirements of real-time motion control. Furthermore, their localization accuracy and robustness significantly decrease when faced with non-brain-derived noise interference such as electromyographic artifacts. Summary of the Invention

[0005] The main objective of this application is to propose a transitional non-invasive EEG signal decoding method, system, and storage medium to solve the aforementioned problems.

[0006] To achieve the above objectives, one aspect of the embodiments of this application proposes... A transitional non-invasive EEG signal decoding method, the method comprising: Obtain a multimodal dataset; Construct the student network and freeze the parameters of the pre-trained teacher network; After warming up the student network, the multimodal dataset is input into the student network and the teacher network in parallel. Based on the outputs of the student network and the teacher network, training of the student network begins. During training, the error signal calculated based on the composite loss function is used to update the parameters of the student network through the error backpropagation algorithm. When the error signal of the composite loss function is detected to be lower than a preset threshold for multiple consecutive training cycles, the parameter update is stopped and the training of the student network is completed. The trained student network is then put into the online reasoning stage, where it decodes the EEG signals.

[0007] In some embodiments, obtaining the multimodal dataset specifically includes: EEG signals from the scalp surface were obtained using an EEG cap. Signals from the motor cortex are acquired through an implanted microelectrode array; Real motion trajectories are obtained through motion capture models.

[0008] In some embodiments, the step of inputting the multimodal dataset into the student network and the teacher network in parallel includes: Spatial feature vectors are extracted from the EEG signals on the scalp surface through the spatial filtering layer of the student network; The encoder of the student network extracts student feature vectors, student attention maps, and predicted action trajectories from the spatial feature vectors.

[0009] In some embodiments, after inputting the multimodal dataset into the student network and the teacher network in parallel, the method further includes: The teacher network's dynamic graph convolutional architecture extracts graph manifold features and teacher attention maps from the motion cortex signals.

[0010] In some embodiments, the preheating of the student network specifically includes: The scalp surface EEG signal and the actual motion trajectory in the multimodal dataset are called, and the mean square of the coordinate differences between the predicted motion trajectory output by the student network and the actual motion trajectory at all time steps and coordinate dimensions is calculated as the error loss. Based on error loss, the spatial filtering layer of the student network is driven to adjust its internal parameters through several rounds of purely supervised training to adapt to the distribution pattern of the EEG signal on the scalp surface.

[0011] In some embodiments, the expression for the composite loss function is: ; in For compound loss, , as well as These are the weighting coefficients for each type of loss. For the loss of mission return, For the characteristic manifold reconstruction loss, This refers to the loss of attentional cognitive alignment.

[0012] In some embodiments, the feature manifold reconstruction loss is obtained by calculating the distance between the student feature vector and the graph manifold feature; The attentional cognitive alignment loss is obtained by calculating the KL divergence between the student's attention map and the teacher's attention map; The task regression loss is obtained by calculating the error between the predicted trajectory and the actual trajectory.

[0013] In some embodiments, the step of deploying the trained student network into the online reasoning stage and decoding the EEG signals through the student network specifically includes: Remove the teacher network and the device used to collect the multimodal dataset; The trained student network is deployed on a host computer or embedded computing platform, while the subjects wear non-invasive EEG caps; Multi-channel EEG signals were collected using a non-invasive EEG cap and input into the student network. The student network extracts spatial features from the multi-channel EEG signal and captures temporal dependencies, then outputs the decoded EEG signal.

[0014] To achieve the above objectives, another aspect of this application proposes a transitional non-invasive EEG signal decoding system, the system comprising: The data acquisition module is used to acquire multimodal datasets; The student network construction module is used to build the student network and freeze the parameters of the pre-trained teacher network. The student network warm-up module is used to warm up the student network, input the multimodal dataset in parallel into the student network and the teacher network, and start training the student network based on the output of the student network and the teacher network. The student network training module is used to update the parameters of the student network during the training process based on the error signal calculated by the composite loss function and through the error backpropagation algorithm. When the error signal of the composite loss function is detected to be lower than a preset threshold for multiple consecutive training cycles, the parameter update is stopped and the training of the student network is completed. The student network deployment module is used to deploy the trained student network into the online reasoning stage and decode EEG signals through the student network.

[0015] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0016] The embodiments of this application include at least the following beneficial effects: This application provides a transitional non-invasive EEG signal decoding method, system, and storage medium. It establishes a teacher-student network-based training mechanism, using the teacher network to guide the student network processing scalp EEG signals. This enhances the student network's ability to extract effective information from fuzzy EEG signals, compensating to some extent for signal ambiguity caused by the skull's volume conductor effect. This results in smoother, more accurate motion trajectory predictions with a higher degree of fit to the actual trajectory. The student network used in this invention is relatively simple. In practical applications, the student network has fast computing speed and low latency, and can quickly convert EEG signals into control commands, making it more suitable for driving real-time rehabilitation devices such as robotic arms or exoskeletons. Attached Figure Description

[0017] Figure 1 A flowchart of a transitional non-invasive EEG signal decoding method provided in an embodiment of this application; Figure 2 for Figure 1 The flowchart for generating motion control commands in step S104 is shown below. Figure 3 A block diagram of the transitional non-invasive EEG signal decoding system provided in the embodiments of this application; Figure 4 This is a schematic diagram of student online training provided in an embodiment of this application; Figure 5 This is a schematic diagram of the deployment of an implantable microelectrode array provided in an embodiment of this application; Figure 6 A schematic diagram of the EEG signal decoding process provided in this embodiment of the application is provided for reference. Figure 7 This is a schematic diagram of the training process for male SD rats provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0019] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0020] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0022] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0023] EEG: Electroencephalogram or EEG signal. EEG signals are weak voltage fluctuations generated by the electrical activity of brain nerve cell groups (mainly neurons) recorded using electrodes placed on the surface of the scalp.

[0024] DST-GCN: A neural network model designed to process graph-structured data with spatiotemporal dependencies. Its core objective is to simultaneously and effectively capture the dynamic spatial correlations and complex temporal evolution patterns in the data.

[0025] Bi-LSTM: Bidirectional Long Short-Term Memory Network. It stacks two independent LSTM layers together, processing the same input sequence in opposite directions, so that the model can simultaneously acquire and fuse complete past context and complete future context information at each time step.

[0026] MS-TCN: Multi-scale Temporal Convolutional Network. It is a model that extends the basic TCN architecture. Its core innovation lies in explicitly designing and fusing features from different time scales (or different dilation rates) to more comprehensively and robustly model the short-term, medium-term, and long-term dependencies that exist simultaneously in a sequence.

[0027] The following problems exist in existing EEG signal decoding technologies: 1. Low spatial resolution and severe loss of high-frequency information: Due to the high resistivity and scattering characteristics of the skull and scalp as volume conductors, local neural activity in the cortex is severely diffused and superimposed when transmitted to the scalp. This results in signals recorded by a single scalp electrode being mixed with activity from multiple brain regions. Simultaneously, the skull has a significant low-pass filtering effect on electrical signals, causing action potentials and high-frequency oscillations, which contain rich details of motion, to be extremely weak or even lost at the scalp. Consequently, the accuracy of non-invasive decoding relying solely on low-frequency bands is far lower than that of invasive systems.

[0028] 2. High-performance models are overly reliant on large-scale paired training data: Existing deep learning decoding methods typically require massive amounts of paired EEG-motor data to train model parameters and cover various possible signal variations. However, in practical applications, acquiring high-quality, long-term, and accurately labeled motion data is extremely costly, and subjects or animals are prone to fatigue, leading to difficulties in data collection. This over-reliance on large sample data often results in lengthy calibration times when facing new users or new tasks, hindering rapid deployment.

[0029] 3. Existing deep learning models lack supervision and guidance from high-fidelity signals: Most current non-invasive decoding methods directly establish a mapping from low-quality EEG to motion parameters. During training, the models never encounter clear ground truth cortical signals, thus failing to truly understand the deep neural coding patterns masked by noise. This "blind" data fitting approach allows the model to learn only superficial statistical correlations, easily confusing real neural activity with artifacts, making it difficult to improve the system's generalization ability and control stability across individuals or complex environments.

[0030] In view of this, this application provides a transitional non-invasive EEG signal decoding method, system, and storage medium.

[0031] See Figure 1 , Figures 5-6 As shown, one aspect of this application's embodiments proposes a transitional non-invasive EEG signal decoding method, including but not limited to steps S101-S104, specifically as follows: S101: Obtain the multimodal dataset; S102: Construct the student network and freeze the parameters of the pre-trained teacher network; S103: After warming up the student network, the multimodal dataset is input into the student network and the teacher network in parallel. Based on the output of the student network and the teacher network, training of the student network begins. S104: During the training process, the error signal calculated based on the composite loss function is used to update the parameters of the student network through the error backpropagation algorithm. When the error signal of the composite loss function is detected to be lower than a preset threshold for multiple consecutive training cycles, the parameter update is stopped and the training of the student network is completed. S105: The trained student network is put into the online reasoning stage, and the EEG signals are decoded through the student network.

[0032] Furthermore, in some embodiments, after decoding the EEG signal, motion control commands are output, and the decoded EEG signal is subjected to online adaptive calibration to combat signal drift.

[0033] A teacher-student network-based training mechanism is established through steps S101-S105, using the teacher network to guide the student network processing scalp EEG. This enhances the student network's ability to extract effective information from fuzzy EEG signals and, to some extent, compensates for signal ambiguity caused by the skull's volume conductor effect. This results in a smoother, more accurate, and better-fitting motion trajectory prediction in the final output.

[0034] Specifically, obtaining the multimodal dataset in step S101 includes: EEG signals from the scalp surface were obtained using an EEG cap. Signals from the motor cortex are acquired through an implanted microelectrode array; Real motion trajectories are obtained through motion capture models.

[0035] In some embodiments, scalp surface EEG signals and motor cortex signals are spatiotemporally aligned via hardware triggering.

[0036] Specifically, motion capture models can consist of sensor arrays that record the actual movement trajectories of the subject's limbs, such as coordinates and speed.

[0037] Specifically, after inputting the multimodal dataset into the student network and teacher network in parallel in step S103, the process includes: Spatial feature vectors are extracted from EEG signals on the scalp surface through the spatial filtering layer of the student network; The encoder of the student network extracts student feature vectors, student attention maps, and predicted action trajectories from spatial feature vectors.

[0038] Furthermore, after inputting the multimodal dataset into the student network and teacher network in parallel in step S103, the method further includes: Graph manifold features and teacher attention maps are extracted from the motor cortex signal using a dynamic graph convolutional architecture of the teacher network.

[0039] In some embodiments, the system includes an experimental platform with an integrated neural signal acquisition environment. When a subject (such as an animal model or clinical patient with an implanted microelectrode array) performs a preset motor task, the system simultaneously activates three sensing subsystems: 1. Non-invasive EEG acquisition subsystem: Records macroscopic EEG signals on the scalp surface using a high-density EEG cap, serving as input data for the subsequent student network; 2. Invasive cortical acquisition subsystem: Records local field potentials (LFP) or pulse signals in the motor cortex using an implanted microelectrode array. This data will be used as on-site teaching material input to the teacher network during the training phase to generate guidance features; 3. Motion capture subsystem: Records the subject's actual limb movement trajectory (such as coordinates and speed), serving as the actual movement trajectory.

[0040] During the subjects' motor task, the experimental system simultaneously acquired high-density scalp surface EEG signals as input to the student network and motor cortex signals recorded by an implanted microfilament array as input to the teacher network. Hardware triggering ensured millisecond-level alignment between the scalp surface EEG signals and the motor cortex signals. In the data preprocessing stage, for the motor cortex signals, the system constructed a dynamic graph structure using physical distance and phase-locked value (PLV).

[0041] It should be noted that the construction process of this dynamic graph structure and the subsequent DST-GCN processing are only used as a means to generate supervision signals, aiming to provide the student network with a high-dimensional target tensor containing the dynamic information flow of the brain.

[0042] Furthermore, in some embodiments, to ensure strict time correspondence among the three, the system employs a hardware-level synchronization triggering mechanism. The master controller sends a unified trigger pulse with millisecond-level precision, forcing all recording devices to timestamp at the same moment.

[0043] After data acquisition, the data underwent cleaning and alignment: EEG signals were filtered and artifact removed; cortical signals were preprocessed to construct a dynamic graph structure; and motion trajectories were smoothed. Finally, the data was encapsulated into thousands of time-aligned triplet samples, each containing: {the EEG tensor at time t, the cortical graph tensor at time t, and the true trajectory at time t}.

[0044] Furthermore, the teacher network, which serves as the knowledge transmitter in this invention, adopts a mature architecture (based on DST-GCN and Bi-LSTM). In this invention, the parameters of the teacher network are frozen or pre-trained, so that it only serves as a stable feature extractor in subsequent training without parameter updates.

[0045] It is only responsible for processing high-fidelity cortical signals, generating "graph manifold features" that contain the spatiotemporal dynamics of the whole brain, as well as attention weights that reflect key neural events. These outputs are no longer used as the final result, but rather as "privileged knowledge" or "soft labels" to guide the optimization direction of student networks.

[0046] Specifically, the student network includes a spatial filtering layer and an encoder. The spatial filtering layer generates spatial feature vectors, and the encoder generates predicted movement trajectories based on these spatial feature vectors. The core of this invention lies in constructing a lightweight student network specifically designed for multi-channel scalp EEG input. Considering that EEG signals combine complex scalp spatial distributions with long-term temporal dynamics, this invention designs a unique space-time joint processing architecture for the student network, specifically including: Spatial Feature Extraction: A spatial convolutional layer is set in the network front end. The spatial convolutional layer consists of a set of convolutional kernels that span all EEG electrode channels. The spatial convolutional layer performs weighted operations across all EEG channels. Unlike conventional processing, the spatial convolutional layer is trained to simulate the "inverse source localization" process. By learning the spatial filter weights, the multi-channel aliased signal on the scalp surface is deconstructed into spatial feature vectors that can reflect cortical source activity.

[0047] Temporal dynamics capture end: The extracted spatial features are then fed into a multi-scale temporal convolutional network. The student network uses a multi-scale temporal convolutional network as the core encoder to capture long-term predicted motion trajectories. This core encoder expands the receptive field exponentially while maintaining temporal resolution by stacking multiple layers of causal dilated convolutions. This design enables the student network to capture neurodynamic patterns up to hundreds of milliseconds long, providing a structural basis for understanding complex motion intentions.

[0048] Furthermore, in some embodiments, a linear projection adaptation layer is provided at the end of the encoder to map the feature dimensions extracted by the students to a dimension space that is completely consistent with the network features of the teacher, thereby establishing an interface for subsequent feature alignment.

[0049] This invention designs a student network that includes spatial convolutional layers and multi-scale temporal convolutional layers. This student network is specifically designed for multi-channel EEG and can first integrate the spatial distribution information of the scalp surface, and then use dilated convolution to capture long-term neural change patterns, thereby effectively extracting and predicting motion trajectories.

[0050] The student network structure used in this invention is relatively simple, with far fewer parameters than complex graph convolutional teacher networks. In practical applications, this network boasts fast computation speed and low latency, enabling it to quickly convert EEG signals into control commands, making it more suitable for driving real-time rehabilitation devices such as robotic arms or exoskeletons.

[0051] To prevent the student network from experiencing feature confusion due to random weights in the initial stage, which would prevent it from effectively aligning with the teacher's high-level features, a warm-up of the student network is required before step S102. Specifically: The scalp surface EEG signal and the actual motion trajectory in the multimodal dataset are called, and the mean square of the coordinate differences between the predicted motion trajectory output by the student network and the actual motion trajectory at all time steps and coordinate dimensions is calculated as the error loss. Based on error loss, the spatial filtering layer of the student network is driven to adjust its internal parameters through several rounds of purely supervised training to adapt to the distribution pattern of the EEG signal on the scalp surface.

[0052] In some embodiments, warming up the student network also includes teaching the multi-scale temporal convolutional network basic kinematic statistical trends to lay the parameter foundation for subsequent challenging distillation tasks.

[0053] See Figure 4 As shown, after the preheating is completed, the system officially enters the distillation training phase. In each training iteration step, the system reads a batch of complete "triple samples" from the multimodal dataset and distributes the three signals to two parallel computation streams to work together.

[0054] In the teacher stream, the motor cortex signals synchronously recorded in the samples are input into the frozen teacher network. Since this signal originates directly from the intracranial cavity and contains clear neural firing patterns and topological connectivity information, the teacher network can use its dynamic graph convolutional architecture to extract high-quality graph manifold features and teacher attention maps reflecting key moments of action. These two outputs do not participate in direct prediction, but are passed to the composite loss function constructed by the system as "intermediate standard answers" reflecting the deep thinking process of the brain.

[0055] Meanwhile, in the student flow, the corresponding scalp EEG signals from the samples are input into the student network to be trained. Since the EEG signals are blurred and attenuated after being transmitted through the skull, the student network needs to be demixed through a spatial filtering layer and then captured through a multi-scale temporal convolutional network to capture temporal patterns, finally outputting student feature vectors, student attention maps, and corresponding predicted motion trajectories.

[0056] Specifically, the expression for the composite loss function is: ; in For compound loss, , as well as These are the weighting coefficients for each type of loss. For the loss of mission return, For the characteristic manifold reconstruction loss, This refers to the loss of attentional cognitive alignment.

[0057] In this application, during training, the parameters of the student network are updated using an error backpropagation algorithm based on the error signal calculated by the composite loss function. The calculation process of the composite loss function includes: First, the feature imitation loss is calculated using information derived from cortical signals, which measures the distance between the student's feature vector and the teacher's graph manifold features. This distance serves as the feature manifold reconstruction loss, forcing the student network to adjust its parameters so that the representation extracted from the fuzzy EEG can approximate the teacher's representation extracted from the clear cortical signals as closely as possible in the mathematical space, thus enabling the student network to learn to see beyond the surface. Secondly, the KL divergence between the student attention map and the teacher attention map is calculated as the attention alignment loss, which forces the student network to focus on the time points that the teacher considers important, thereby ignoring the background noise in the EEG. Finally, the task regression loss is calculated using the actual motion trajectory, which measures the error between the predicted trajectory and the actual trajectory, ensuring that the final decoding result conforms to the laws of physical motion. In this way, EEG serves as the input material, cortical signals guide the intermediate process, and the motion trajectory constrains the final result; the combined effect of these three elements achieves comprehensive training of the student network.

[0058] Specifically, the feature manifold reconstruction loss is used to enable the student network to learn to deduce deeper information. This invention introduces a learnable weight matrix at the end of the encoder of the student network. and bias terms The linear projection adaptation layer acts as a dimensional bridge, mapping the EEG spatial features extracted by the student network to the same dimensional space as the graph manifold features of the teacher network. The system calculates this loss by minimizing the mean squared error between the two, as shown in the formula: ; in For the characteristic manifold reconstruction loss, For the characteristics of the graph manifold, For student feature vectors, For learnable weight matrix, This is a bias term.

[0059] Specifically, the attentional cognitive alignment loss uses the teacher attention map of the Bi-LSTM layer in the teacher network as the ground truth probability distribution to guide the student attention map in the student network. The system defines this loss by calculating the Kolb-Leibler divergence (KL Divergence) between the two, as shown in the formula: ; in For attentional cognitive alignment loss, For teachers' attention maps, A diagram showing student attention.

[0060] Specifically, the task regression loss is used to ensure the accuracy of the final decoding. The system also calculates the error between the student's network-predicted trajectory and the actual motion trajectory, using the following formula: ; in For the loss of mission return, For the actual motion trajectory, To predict motion trajectories.

[0061] In some embodiments, the system of this application determines whether the training of the student network has ended based on a preset convergence criterion: Specifically, when it is detected that the decrease of the composite loss function in multiple consecutive training cycles is lower than a preset small threshold (such as 1e-4), and at the same time the decoding correlation coefficient calculated on the independent validation set no longer increases significantly with the increase of training rounds (i.e., triggering the Early Stopping mechanism), or when the training reaches the preset maximum number of iteration rounds, the system determines that the model has converged and terminates the training.

[0062] Compared to traditional methods that directly use EEG training, this invention, by mimicking the graph manifold characteristics of the teacher network, allows the student network to compensate to some extent for the signal ambiguity caused by the skull volume conductor effect. This results in smoother and more accurate motion trajectory predictions with a higher degree of fit to the actual trajectory.

[0063] See Figure 2 As shown, in step S104, the trained student network is put into the online reasoning stage, and the EEG signals are decoded through the student network, specifically including: S201: Remove the teacher network and the equipment used to collect multimodal datasets; S202: The trained student network is deployed on a host computer or embedded computing platform, while the subjects wear non-invasive EEG caps; S203: Collect multi-channel EEG signals using a non-invasive EEG cap and input them into the student network; S204: By extracting spatial features from multi-channel EEG signals and capturing temporal dependencies through the student network, the decoded EEG signal is output.

[0064] Specifically, in step S204, the student network utilizes its internalized cortical dynamics knowledge to quickly extract spatial features and capture temporal dependencies, directly outputting high-precision motion control commands for the current moment with millisecond-level low latency, such as the three-dimensional coordinates of the robotic arm or the speed of the cursor. This achieves a control effect that is close to the decoding precision of invasive systems using only non-invasive signals.

[0065] After students complete their network training, the system no longer requires implanted electrodes or the involvement of a teacher network. Only a lightweight student network needs to be deployed to directly map the real-time acquired EEG signals into motion trajectory coordinates.

[0066] This invention avoids the infection risks and tissue damage associated with implantation surgery, making it suitable for long-term use. Furthermore, thanks to the feature enhancements resulting from distillation training, its decoding performance is significantly superior to conventional non-invasive brain-computer interface systems, giving it greater practical value.

[0067] See Figure 3 As shown, in another aspect of the embodiments of this application, a transitional non-invasive EEG signal decoding system is also proposed, wherein the system includes: The data acquisition module is used to acquire multimodal datasets; The student network construction module is used to build the student network and freeze the parameters of the pre-trained teacher network. The student network warm-up module is used to warm up the student network, input the multimodal dataset in parallel into the student network and the teacher network, and start training the student network based on the output of the student network and the teacher network. The student network training module is used to update the parameters of the student network during the training process based on the error signal calculated by the composite loss function and through the error backpropagation algorithm. When the error signal of the composite loss function is detected to be lower than a preset threshold for multiple consecutive training cycles, the parameter update is stopped and the training of the student network is completed. The student network deployment module is used to deploy the trained student network into the online reasoning stage and decode EEG signals through the student network.

[0068] In another aspect of this application, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the above-described method.

[0069] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0070] See Figure 7 As shown, the following description, using specific application examples, will illustrate this embodiment: First, a "ternary synchronous" acquisition system was constructed, comprising two independent recording channels: a flexible surface electrode and a deep microfilament electrode. Adult male SD rats (weighing 280-320g) were selected as experimental subjects for high-precision stereotactic surgery.

[0071] In terms of deep recording as input to the teacher network, a customized 32-channel tungsten wire microelectrode array (arranged in 4x8, with a single filament diameter of 35μm and a tip impedance of approximately 500kΩ) was vertically implanted into the forelimb representative area of ​​the right primary motor cortex (M1). The implantation depth was set to 1.2-1.5mm subcortical, targeting the dense area of ​​pyramidal neurons in layer V, thereby obtaining a high-fidelity "true value" signal containing abundant neural firing and local field potentials (LFP).

[0072] For surface recordings used as student network input, a custom-designed 16-channel flexible EEG electrode array was employed. During the procedure, connective tissue was removed from the surface of the rat parietal bone and kept dry. The flexible electrode array was then laid flat and attached to the skull surface (or epidural layer) centered on the microelectrode implantation point. Its excellent mechanical flexibility enabled conformal contact with the curved surface of the skull, thereby acquiring macroscopic EEG signals with high signal-to-noise ratio and high spatial resolution.

[0073] Meanwhile, for motion capture, which serves as the actual movement trajectory, reflective markers were affixed to the wrist of the rat's left forelimb, and its three-dimensional spatial coordinates were recorded at a frequency of 120Hz using a high-speed infrared motion capture system. All three systems were triggered by a unified TTL pulse to ensure strict alignment of neural signals and movement trajectories on a millisecond-level time axis.

[0074] During the data acquisition and preprocessing phase, rats were behaviorally trained to perform an automatic lever-operated water-collecting task. Differential streaming preprocessing strategies were implemented for the acquired raw data. For the flexible EEG signal used as student input, a 1-150Hz bandpass filter was applied, and Laplace spatial filtering was performed using the high-density characteristics of the array to further improve the signal-to-noise ratio. A sliding window with a length of 500ms was used to construct the student input tensor. For the microwire electrode signal used as the input to the teacher network, the acquired microwire electrode signal is first amplified to enhance the signal strength, and then converted into a digital signal through analog-to-digital conversion. The digital signal enters a broadband LFP filter to filter out high-frequency noise in the signal, retains the broadband LFP filtered signal of 1-300Hz, and constructs a dynamic graph signal reflecting the topological connections of the brain. The final processed dynamic graph signal is used as the input signal to the teacher network.

[0075] For the motion trajectory, Kalman smoothing, temporal alignment, and downsampling are performed to generate a realistic motion trajectory.

[0076] Subsequently, a teacher network based on a Dynamic Spatiotemporal Graph Convolutional Network (DST-GCN) is first trained using cortical signals and motion trajectories until convergence and parameters are frozen, entering the core cross-modal distillation training phase. The system employs a dual-stream parallel input strategy. In each training step, a time-aligned "triple sample" is read. Flexible EEG signals are input into the student network to be trained (based on the MS-TCN architecture), while synchronized microfilament electrode signals are input into the frozen teacher network. The student network not only outputs motion trajectory predictions but also generates feature vectors and attention maps in intermediate layers. The system updates the student network parameters by minimizing a composite loss function, which includes a "feature imitation loss" that calculates the distance between student features and teacher graph manifold features, an "attention alignment loss" that calculates the KL divergence of student and teacher temporal attention distributions, and a "task regression loss" that compares the student's predicted trajectory with the actual trajectory. After approximately 100 epochs of joint training, the student network learned to simulate the cortical analysis logic of the teacher network by inputting only macroscopic flexible EEG signals. Its trajectory decoding accuracy on the validation set significantly surpassed that of the benchmark model trained using only traditional rigid electrode signals.

[0077] In the long-term application phase following the completion of distillation training by the student network, the system switched to an online operating mode of "flexible sensing independent drive." In this mode, the physical connection to the deep microfilament electrode channels was completely disconnected or removed, leaving only the connection between the flexible EEG electrode array attached to the skull surface and the acquisition system. Thanks to the extremely low Young's modulus of the flexible electronics, this electrode array can dynamically adapt to minute deformations of the skull and brain tissue, avoiding stress concentration and tissue damage that can occur with traditional rigid screw electrodes, ensuring extremely high stability of the signal interface over several months. At this point, the system no longer relies on any deep cortical signals, nor does it require the participation of the teacher network; it relies entirely on the trained student network for real-time decoding.

[0078] The system acquires voltage data from a 16-channel flexible EEG array in real time and streams it to the student network inference engine deployed on a high-performance host computer. The engine utilizes internalized cortical dynamics knowledge to perform online computation on a 500ms surface waveform window of the current input. Because the student network employs a lightweight fully convolutional MS-TCN structure, the single inference latency is controlled within a short time, meeting real-time control requirements.

[0079] Although the input is a macroscopic surface signal, the flexible electrodes provide richer spatial distribution information than traditional screw electrodes. Combined with the "feature imitation" during the training phase, the student network can accurately reconstruct the spatiotemporal activity patterns of the motor cortex and directly output the predicted three-dimensional motion coordinates of the rat's left forelimb. To cope with potential sweat interference or slight changes in contact impedance during long-term operation, the system integrates an online adaptive calibration module at the output end, which uses high-confidence prediction fragments to fine-tune the spatial filter weights of the shallow layers of the network.

[0080] Experimental results show that, several weeks after implantation, the system driven solely by the flexible EEG electrode successfully controlled the external robotic arm to follow the rat's forelimb in synchronous motion. The correlation coefficient between its trajectory and the actual movement remained stable at a relatively high value, achieving high-performance long-term motion control that is close to the level of invasive decoding even when completely removed from the invasive deep electrode.

[0081] Furthermore, in some embodiments, the graph convolutional layers in the teacher network are replaced with a spatiotemporal Transformer architecture: as an alternative to the core architecture of the teacher network, the original DST-GCN module can be replaced by a spatiotemporal Transformer based on a self-attention mechanism.

[0082] The Transformer architecture can capture global dependencies between all electrode channels in parallel through a multi-head attention mechanism, without being limited by a predefined graph adjacency matrix, thus enabling more flexible mining of potential long-range functional connections in cortical signals. In this alternative, the teacher network still receives dynamically constructed cortical signal inputs, but its output features will contain richer global contextual information. This replacement can further enhance the teacher model's representational ability when dealing with complex neurodynamic patterns, thereby raising the upper limit of the student network through distillation.

[0083] Furthermore, in some embodiments, a discriminator network is introduced to construct a generative adversarial (GAN) distillation framework: To improve the realism of the features reconstructed by the student network, a feature discriminator can be added to the existing "teacher-student" training process. This discriminator forms a zero-sum game relationship with the student network: The discriminator aims to distinguish whether the input feature vector originates from the graph manifold features of the teacher network or the reconstructed features output by the student network through the adaptation layer, while the student network strives to generate realistic features sufficient to deceive the discriminator, specifically: The system employs an alternating iterative training strategy: First, the student network parameters are fixed, and the discriminator is updated by minimizing the discriminator loss (L_D) to ensure that it classifies teacher features as true (label 1) and student features as false (label 0) as much as possible. Then, the discriminator parameters are fixed again, and the adversarial loss is added to the total loss function of the student network for backpropagation. Specifically, the adversarial loss is: ; in To combat the losses, To reconstruct features, It is a feature discriminator.

[0084] This adversarial mechanism forces the student network to learn the complete probability distribution of the teacher network's features, rather than simply fitting the average value of the features. This results in the generated features being closer to the real high-fidelity cortical signal in terms of statistical distribution, texture detail, and high-frequency components, significantly enhancing the robustness and resolution accuracy of the decoding system.

[0085] Furthermore, in some embodiments, an online adaptive calibration module is added to overcome EEG non-stationarity: In response to the problem that EEG signals are prone to drift (non-stationarity) over time, an online adaptive calibration module can be added to the output end of the student network.

[0086] This module consists of a "window buffer" for temporarily storing real-time signal segments, a set of lightweight "learnable affine transformation parameters" (including scaling factors and bias terms) located at the network front end, and a fast gradient update unit based on self-supervised loss (such as minimizing information entropy). This module does not alter the main structure of the student network; instead, it utilizes a small amount of real-time EEG data collected during actual use to make minor dynamic adjustments to the input or output layer parameters of the student network through test-time enhancement or domain adaptation algorithms. The module's role is to compensate in real-time for signal differences caused by changes in electrode impedance, slight shifts in wearing position, or fluctuations in the subject's physiological state. By adding this module, the system can maintain a high level of decoding stability during long-term use without requiring complex invasive-noninvasive joint training, significantly extending the system's effective operating cycle.

[0087] The transitional non-invasive EEG signal decoding method provided in this application relates to the field of information technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the XXX method, but is not limited to the above forms.

[0088] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0089] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0090] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0091] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0092] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0093] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0094] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0095] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0096] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0097] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0098] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A transitional non-invasive EEG signal decoding method, characterized in that, The method includes: Obtain a multimodal dataset; Construct the student network and freeze the parameters of the pre-trained teacher network; After warming up the student network, the multimodal dataset is input into the student network and the teacher network in parallel. Based on the outputs of the student network and the teacher network, training of the student network begins. During training, the error signal calculated based on the composite loss function is used to update the parameters of the student network through the error backpropagation algorithm. When the error signal of the composite loss function is detected to be lower than a preset threshold for multiple consecutive training cycles, the parameter update is stopped and the training of the student network is completed. The trained student network is then put into the online reasoning stage, where it decodes the EEG signals.

2. The transitional non-invasive EEG signal decoding method according to claim 1, characterized in that, The acquisition of the multimodal dataset specifically includes: EEG signals from the scalp surface were obtained using an EEG cap. Signals from the motor cortex are acquired through an implanted microelectrode array; Real motion trajectories are obtained through motion capture models.

3. The transitional non-invasive EEG signal decoding method according to claim 2, characterized in that, After inputting the multimodal dataset into the student network and teacher network in parallel, the process includes: Spatial feature vectors are extracted from the EEG signals on the scalp surface through the spatial filtering layer of the student network; The encoder of the student network extracts student feature vectors, student attention maps, and predicted action trajectories from the spatial feature vectors.

4. The transitional non-invasive EEG signal decoding method according to claim 2, characterized in that, After inputting the multimodal dataset into the student network and teacher network in parallel, the method further includes: The teacher network's dynamic graph convolutional architecture extracts graph manifold features and teacher attention maps from the motion cortex signals.

5. The transitional non-invasive EEG signal decoding method according to claim 1, characterized in that, The preheating of the student network specifically includes: The scalp surface EEG signal and the actual motion trajectory in the multimodal dataset are called, and the mean square of the coordinate differences between the predicted motion trajectory output by the student network and the actual motion trajectory at all time steps and coordinate dimensions is calculated as the error loss. Based on error loss, the spatial filtering layer of the student network is driven to adjust its internal parameters through several rounds of purely supervised training to adapt to the distribution pattern of the EEG signal on the scalp surface.

6. The transitional non-invasive EEG signal decoding method according to claim 1, characterized in that, The expression for the composite loss function is: ； in For compound loss, , as well as These are the weighting coefficients for each type of loss. For the loss of mission return, For the characteristic manifold reconstruction loss, This refers to the loss of attentional cognitive alignment.

7. The transitional non-invasive EEG signal decoding method according to claim 6, characterized in that, The feature manifold reconstruction loss is obtained by calculating the distance between the student feature vector and the graph manifold feature; The attentional cognitive alignment loss is obtained by calculating the KL divergence between the student's attention map and the teacher's attention map; The task regression loss is obtained by calculating the error between the predicted trajectory and the actual trajectory.

8. The transitional non-invasive EEG signal decoding method according to claim 1, characterized in that, The step of deploying the trained student network into the online reasoning stage and decoding the EEG signals through the student network specifically includes: Remove the teacher network and the device used to collect the multimodal dataset; The trained student network is deployed on a host computer or embedded computing platform, while the subjects wear non-invasive EEG caps; Multi-channel EEG signals were collected using a non-invasive EEG cap and input into the student network. The student network extracts spatial features from the multi-channel EEG signal and captures temporal dependencies, then outputs the decoded EEG signal.

9. A transitional non-invasive EEG signal decoding system, characterized in that, The system includes: The data acquisition module is used to acquire multimodal datasets; The student network construction module is used to build the student network and freeze the parameters of the pre-trained teacher network. The student network warm-up module is used to warm up the student network, input the multimodal dataset in parallel into the student network and the teacher network, and start training the student network based on the output of the student network and the teacher network. The student network training module is used to update the parameters of the student network during the training process based on the error signal calculated by the composite loss function and through the error backpropagation algorithm. When the error signal of the composite loss function is detected to be lower than a preset threshold for multiple consecutive training cycles, the parameter update is stopped and the training of the student network is completed. The student network deployment module is used to deploy the trained student network into the online reasoning stage and decode EEG signals through the student network.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 8.

Images