Non-invasive electroencephal decoding model training method and system based on invasive guidance

Abstract

The application relates to the technical field of brain-computer interfaces, and provides a non-invasive electroencephalogram decoding model training method and system based on invasive guidance, which comprises the following steps: acquiring synchronously collected invasive and non-invasive electroencephalogram signals; inputting the two signals into a first model and a second model of a shared semantic codebook respectively to obtain a corresponding neural activity mode set and a probability distribution; constructing a de-sequencing alignment loss based on set alignment loss and distribution difference, and constructing a physical consistency loss based on head anatomical structure signal conduction constraints; pre-training the second model based on the de-sequencing alignment loss and the physical consistency loss, and performing downstream task migration training on the pre-trained second model to obtain a third model. The application effectively reduces the cross-modal time sequence alignment difficulty, ensures that the spatial mapping conforms to the physiological law, and improves the decoding performance and safety of the non-invasive brain-computer interface.

Description

Technical Field

[0001] This application relates to the field of brain-computer interface technology, and in particular to a non-invasive EEG decoding model training method and system based on invasive guidance. Background Technology

[0002] Currently, in the field of brain-computer interfaces, non-invasive EEG signal acquisition methods have broad application prospects due to their safety and non-invasiveness. However, due to attenuation and aliasing effects from tissues such as the skull and scalp, non-invasive EEG signals generally suffer from low signal-to-noise ratio, poor spatial resolution, and blurred temporal information, resulting in significantly lower decoding accuracy compared to invasive methods. In practice, some users may temporarily implant invasive electrodes to accurately decode brain neural activity, but once the invasive electrodes are removed, they can only rely on non-invasive devices to decode brain neural activity, facing a significant decline in decoding performance.

[0003] Therefore, there is an urgent need for a training scheme for a non-invasive EEG decoding model based on invasive guidance, which can improve the decoding accuracy of non-invasive EEG signals by using invasive EEG signals collected within a short implantation window. Summary of the Invention

[0004] To address the problems of existing non-invasive brain-computer interface decoding models being prone to generating false mappings that violate physiological laws and having great difficulty in temporal alignment learning, this application proposes a non-invasive EEG decoding model training method and system based on invasive guidance, as well as an EEG signal decoding method and system, to achieve high-performance and high-safety non-invasive EEG signal decoding based on short-term invasive signal calibration.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A non-invasive EEG decoding model training method based on invasive guidance includes: acquiring synchronously acquired invasive and non-invasive EEG signals; inputting the invasive EEG signals into a first model to obtain a first set of neural activity patterns and a first average probability distribution; inputting the non-invasive EEG signals into a second model that shares a semantic codebook with the first model to obtain a second set of neural activity patterns and a second average probability distribution; constructing a de-temporalized alignment loss based on the set alignment loss between the first and second neural activity pattern sets and the distribution difference loss between the first and second average probability distributions; constructing a physical consistency loss based on the signal conduction constraints of the head anatomical structure; pre-training the second model based on the de-temporalized alignment loss and the physical consistency loss, and performing downstream task transfer training on the pre-trained second model to obtain a third model.

[0007] In this way, by transforming the continuous temporal regression problem into a discrete pattern set recognition problem, and combining the physical prior constraints of the head anatomy, it is possible to reduce the difficulty of temporal alignment while ensuring that the spatial mapping conforms to physiological laws.

[0008] In some embodiments, inputting the invasive EEG signal into a first model to obtain a first set of neural activity patterns and a first average probability distribution includes: dividing the invasive EEG signal into multiple first short-term window signal segments; for each first short-term window signal segment, extracting features through a feature encoder of the first model and mapping them to a probability distribution on the shared semantic codebook through a projection classifier; sampling the probability distribution corresponding to each first short-term window signal segment to obtain a corresponding neural activity pattern identifier, and deduplicating all the neural activity pattern identifiers to generate the first set of neural activity patterns; calculating the mean of the probability distributions corresponding to all first short-term window signal segments to obtain the first average probability distribution; wherein, the process of mapping to a probability distribution on the shared semantic codebook includes: mapping the feature vector to a logical value vector and converting it into a probability distribution on multiple prototypes of the shared semantic codebook through a continuous relaxation sampling function.

[0009] In this way, by using differentiated signal processing strategies, the high spatiotemporal resolution characteristics of invasive signals and the fuzzy integral characteristics of non-invasive signals are respectively adapted, thereby supporting the effective operation of the de-temporalization alignment mechanism.

[0010] In some embodiments, the step of inputting the non-invasive EEG signal into the second model to obtain a second set of neural activity patterns and a second average probability distribution includes: taking the non-invasive EEG signal as a long time window signal segment, extracting features through the spatiotemporal encoder of the second model, and mapping it to a probability distribution on the shared semantic codebook through a projection classifier; sampling and deduplicating the probability distribution output by the second model to generate the second set of neural activity patterns; and calculating the mean of the probability distribution corresponding to each acquisition channel output by the second model to obtain the second average probability distribution.

[0011] In this way, by abandoning point-by-point alignment of micro-waveforms and instead using macro-level set overlap and statistical distribution consistency as supervision signals, the difficulty of cross-modal learning is reduced and the robustness of the model is improved.

[0012] In some embodiments, the set alignment loss is calculated as follows: precision and recall are calculated based on the number of intersections between the first set of neural activity patterns and the second set of neural activity patterns, the total number of the first set of neural activity patterns, and the total number of the second set of neural activity patterns; and the set alignment loss is calculated based on the harmonic mean of the precision and the recall.

[0013] In some embodiments, constructing the de-temporalization alignment loss includes: calculating the distribution difference loss between the first average probability distribution and the second average probability distribution based on the probability distribution divergence; and constructing the de-temporalization alignment loss based on the set alignment loss and the distribution difference loss.

[0014] In some embodiments, the process of constructing the head volume conduction model includes: acquiring multimodal medical images of the target object and performing fusion registration; performing tissue segmentation based on the fusion-registered multimodal medical images and assigning conductivity to different segmented tissues to construct the head volume conduction model; and calculating the theoretical influence weight of each cortical electrode contact on each scalp electrode based on the head volume conduction model to generate a spatial transfer matrix.

[0015] In this way, by introducing biophysical priors as hard constraints, the decoding results of the second model are ensured to have physiological interpretability and safety.

[0016] In some embodiments, the step of using elements in the spatial transfer matrix as attenuation influence factors to weight and constrain the feature distribution differences between the invasive EEG signal and the non-invasive EEG signal to construct a physical consistency loss includes: using elements in the spatial transfer matrix as the attenuation influence factors to weight and sum the feature distribution differences corresponding to physically coupled cortical electrodes and scalp electrodes to obtain the physical consistency loss.

[0017] In some embodiments, the step of using the elements in the spatial transfer matrix as the attenuation influence factor to perform a weighted summation of the feature distribution differences corresponding to physically coupled cortical electrodes and scalp electrodes to obtain the physical consistency loss includes: converting the elements in the spatial transfer matrix into weight coefficients through a normalization function, wherein the elements comprehensively reflect the influence of the conductivity and geometry of cerebrospinal fluid, skull, and scalp tissue on signal attenuation; using probability distribution divergence to measure the distance between the non-invasive feature probability distribution corresponding to the i-th scalp electrode and the invasive feature probability distribution corresponding to the j-th cortical electrode; performing element-wise multiplication of the weight coefficients and the distance and summing them over all spatial locations to obtain the physical consistency loss.

[0018] In some embodiments, the construction process of the shared semantic codebook includes: acquiring separately collected invasive EEG signals and segmenting them into short-time signal segments; inputting the short-time signal segments into an encoder to obtain feature vectors, and replacing them with the closest prototype vectors in the initialized codebook through a vector quantization layer to obtain a neural activity pattern identifier sequence; performing random masking on the neural activity pattern identifier sequence; reconstructing the neural activity pattern identifiers corresponding to the masked parts through a decoder based on the unmasked neural activity pattern identifiers; optimizing the parameters of the codebook based on the difference between the reconstructed neural activity pattern identifiers and the original neural activity pattern identifiers to obtain the frozen shared semantic codebook shared in the first model and the second model.

[0019] This ensures that the codebook can carry general, physiologically meaningful neural activity primitives.

[0020] In some embodiments, the method further includes the step of determining the optimal time window length during the downstream task migration training phase: starting from a preset initial time window length, predicting the time window adjustment amount through a policy network based on the decoding uncertainty calculated in real time and the current task type; applying constraints to the application direction of the time window adjustment amount according to the current task type, and updating the current time window length; calculating the task loss based on the updated current time window length, and calculating the comprehensive loss by combining the delay penalty term corresponding to the current time window length; determining the optimal fixed time window length corresponding to each type of task by minimizing the comprehensive loss, and generating a mapping table between task types and optimal fixed time window lengths for window invocation during the inference phase.

[0021] Furthermore, the present invention also provides a method for decoding electroencephalogram (EEG) signals, comprising: receiving a non-invasive EEG signal to be decoded; inputting the non-invasive EEG signal to be decoded into an EEG signal decoding model and outputting a second set of neural activity patterns; wherein the EEG signal decoding model is a third model trained by the training method described in any of the above embodiments; inputting the second set of neural activity patterns into a downstream task classifier corresponding to the current task type to generate control instructions.

[0022] In this way, by adaptively calling the optimal time window according to the task type during the inference stage, the best balance between decoding speed and accuracy is achieved in different application scenarios.

[0023] In some embodiments, the step of inputting the non-invasive EEG signal to be decoded into an EEG signal decoding model and outputting a second set of neural activity patterns includes: extracting the context feature vector of the non-invasive EEG signal to be decoded through the spatiotemporal encoder of the EEG signal decoding model; mapping the context feature vector to an average probability distribution on a shared semantic codebook through a projection classifier; and selecting the most likely neural activity pattern identifier based on the average probability distribution to form the second set of neural activity patterns.

[0024] In some embodiments, the method further includes a model fine-tuning step: when the decoding confidence is detected to be lower than a preset threshold or a preset period is reached, the backbone network and shared semantic codebook of the EEG signal decoding model are frozen; newly acquired calibrated non-invasive EEG signals with task labels are acquired, and the parameters of the downstream task classifier are fine-tuned only.

[0025] Furthermore, this invention also provides a non-invasive EEG decoding model training system based on invasive guidance, comprising: a signal acquisition module for acquiring synchronously acquired invasive and non-invasive EEG signals; a semantic mapping module for inputting the invasive EEG signals into a first model to obtain a first set of neural activity patterns and a first average probability distribution; inputting the non-invasive EEG signals into a second model contributing a semantic codebook to the first model to obtain a second set of neural activity patterns and a second average probability distribution; a loss construction module for constructing a de-temporalized alignment loss based on the set alignment loss between the first and second neural activity pattern sets and the distribution difference loss between the first and second average probability distributions; and constructing a physical consistency loss based on the signal conduction constraints of the head anatomical structure; and a model training module for pre-training the second model based on the de-temporalized alignment loss and the physical consistency loss, and performing downstream task transfer training on the pre-trained third model to obtain a third model.

[0026] Furthermore, the present invention also provides an EEG signal decoding system, comprising: a task identification module for acquiring the current task type; a window determination module for determining the corresponding target time window length from a preset mapping table according to the current task type, the mapping table containing the correspondence between task types and optimal fixed time window lengths; a signal decoding module for acquiring non-invasive EEG signals to be decoded within the target time window length, and inputting the non-invasive EEG signals to be decoded into an EEG signal decoding model to output a second neural activity pattern set; wherein, the EEG signal decoding model is a third model trained by the training method described in any of the above embodiments; and an instruction generation module for inputting the second neural activity pattern set into a downstream task classifier corresponding to the current task type to generate control instructions.

[0027] Furthermore, the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a non-invasive EEG decoding model training method based on invasive guidance as described in any of the above embodiments, or an EEG signal decoding method as described in any of the above embodiments.

[0028] In addition, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the non-invasive EEG decoding model training method based on invasive guidance as described in any of the above embodiments, or the EEG signal decoding method as described in any of the above embodiments.

[0029] In addition, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the non-invasive EEG decoding model training method based on invasive guidance as described in any of the above embodiments, or the EEG signal decoding method as described in any of the above embodiments.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. This invention transforms the difficult problem of continuous waveform temporal regression in traditional methods into the problem of recognizing a set of disordered neural activity patterns by constructing a de-temporalized alignment loss. It uses the overlap of sets and the difference in probability distribution as supervision signals, effectively avoiding the interference of temporal ambiguity and phase distortion caused by the volume conduction effect of non-invasive EEG signals on model training, significantly reducing the learning difficulty of cross-modal alignment, and improving the convergence speed and robustness of the decoding model.

[0032] 2. This invention introduces a physical consistency loss based on head anatomy, using a spatial transfer matrix generated by an individualized head volume conduction model as a biophysical prior to weighted constraints on the spatial mapping of cross-modal features. This mechanism forces the model to learn spatial correspondences that conform to the laws of real electrical signal conduction, fundamentally suppressing spurious associations that may arise from purely data-driven models that violate anatomical common sense, thus ensuring the physiological interpretability and clinical safety of non-invasive brain-computer interface decoding results.

[0033] 3. This invention resolves the inherent contradiction between decoding speed and accuracy in brain-computer interface systems by offline searching and solidifying the mapping relationship between task types and optimal fixed time window lengths. During the inference phase, it adaptively calls preset windows based on the current task requirements. For tasks requiring fine control, it automatically matches longer windows to ensure accuracy, and for simple interactive tasks, it matches shorter windows to improve response speed. Thus, without increasing the online computing burden, it achieves a globally optimal balance of system performance across multiple scenarios. Attached Figure Description

[0034] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0035] Figure 1 This is a schematic diagram of a non-invasive EEG decoding model training method based on invasive guidance provided in one embodiment of this application;

[0036] Figure 2 This is a flowchart illustrating the determination of the optimal time window provided in one embodiment of this application;

[0037] Figure 3 This is a flowchart illustrating the construction process of a shared semantic codebook provided in one embodiment of this application;

[0038] Figure 4 This is a schematic flowchart of an EEG signal decoding method provided in one embodiment of this application;

[0039] Figure 5 This is a flowchart of the output of a neural activity pattern prediction set provided in one embodiment of this application;

[0040] Figure 6 This is a schematic diagram of the structure of a non-invasive EEG decoding model training system based on invasive guidance, provided in one embodiment of this application;

[0041] Figure 7 This is a schematic diagram of the structure of an EEG signal decoding system provided in one embodiment of this application. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0043] The technical solutions of the various embodiments of this application can be combined with each other, but only if they are based on the ability of a person skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by this application.

[0044] This solution does not aim to obtain disease diagnosis results or health status. Instead, it processes the target user's EEG data to implement a non-invasive EEG decoding model training method and system based on invasive guidance. All steps are information processing methods implemented by computers and other devices.

[0045] It should be fully understood that the target user information involved in this application (including but not limited to invasive EEG signals and non-invasive EEG signals) is information and data authorized by the target user or fully authorized by all parties. The use of the target user information shall comply with the privacy policies and practices of the industry that are generally considered to meet or exceed the privacy of the target user. The collection, use and processing of related data shall comply with relevant laws, regulations and standards, and provide corresponding operation access points for the target user to choose to authorize or refuse.

[0046] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, specific embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0047] Example 1:

[0048] like Figure 1 As shown, this embodiment provides a non-invasive EEG decoding model training method based on invasive guidance. This method aims to calibrate the EEG signal decoding model using high-quality invasive signals acquired synchronously over a short period, to address the temporal ambiguity and unreliable spatial mapping issues caused by the volume conduction effect of non-invasive signals. The method includes the following steps:

[0049] S100 acquires synchronously collected invasive and non-invasive EEG signals.

[0050] Specifically, the synchronous acquisition in this step refers to the recording of bimodal physiological data triggered by a unified clock within a time window when both invasive and non-invasive sensors are present on the head of the target subject (e.g., a subject undergoing rehabilitation research or clinical monitoring). Invasive EEG signals (such as cortical EEG) have high signal-to-noise ratio and high spatial resolution due to their direct contact with or proximity to cortical neurons, and are used as the benchmark or gold standard for supervised training in this embodiment. Non-invasive EEG signals (such as scalp EEG), although having a lower signal-to-noise ratio and exhibiting signal aliasing, offer the safety and convenience of long-term wear, and are the target signal source ultimately optimized in this embodiment. By acquiring these strictly time-aligned signal pairs, a direct data foundation is established between clear neural activity and fuzzy surface observations, providing the necessary physical prerequisites for subsequent cross-modal knowledge transfer. It should be understood that although this embodiment uses ECoG and EEG as examples, in other embodiments, invasive EEG signals can also be derived from deep brainwave (DBS) or Utah arrays, and non-invasive EEG signals can also be derived from functional near-infrared spectroscopy (fNIRS), as long as the conditions for synchronous paired acquisition are met.

[0051] S200, the invasive EEG signal is input into the first model to obtain a first set of neural activity patterns and a first average probability distribution; the non-invasive EEG signal is input into a second model that shares a semantic codebook with the first model to obtain a second set of neural activity patterns and a second average probability distribution.

[0052] Specifically, this embodiment constructs a dual-branch model architecture comprising a first model and a second model. The first model processes high-quality invasive EEG signals, extracting a set of discrete patterns representing real neural activity and their statistical distribution, serving as the teacher signal. The second model processes non-invasive EEG signals, attempting to predict a set of neural activity patterns and their distribution consistent with the first model, serving as the student signal. The core here is that both share the same semantic codebook, which defines a universal, discrete neurosemantic space that maps continuous, heterogeneous EEG waveforms to unified semantic primitives (i.e., neural activity patterns). By sharing the semantic codebook, two signals that were originally difficult to align directly in the temporal and spatial domains are transformed into interacting within the same comparable discrete semantic space. This design avoids the significant optimization difficulties associated with direct regression at the original waveform level, allowing the model to focus on learning semantic correspondences. It should be noted that the sets and distributions obtained in this step represent the overall neural activity within a time window, rather than the instantaneous state at a single moment, which lays the foundation for subsequent de-temporal alignment.

[0053] S300, construct a de-temporalized alignment loss based on the set alignment loss between the first neural activity pattern set and the second neural activity pattern set and the distribution difference loss between the first average probability distribution and the second average probability distribution; construct a physical consistency loss based on the signal transmission constraints of the head anatomy structure.

[0054] Specifically, to effectively guide the learning of the second model, this embodiment designs a dual loss constraint mechanism. First, addressing the inherent temporal ambiguity and phase distortion problems of non-invasive EEG signals, this invention abandons the traditional point-by-point waveform alignment strategy and instead constructs a de-temporally-oriented alignment loss. This loss function only requires that the set of neural activity patterns predicted by the second model be consistent with the first model in terms of type (measured by set alignment loss) and in terms of the statistical intensity of their overall occurrence (measured by distribution difference), completely disregarding the precise order of these patterns on the microscopic time axis. This approach transforms the extremely difficult continuous temporal regression problem into a relatively simple problem of unordered set recognition and distribution matching, significantly lowering the threshold for the model to extract effective information from ambiguous signals. Second, to prevent the purely data-driven model from learning false associations that violate physiological common sense (e.g., incorrectly mapping occipital visual signals to the frontal lobe motor control area), this embodiment introduces signal transmission constraints based on head anatomy to construct a physical consistency loss. This loss mechanism utilizes biophysical priors quantified by an individualized head volume conduction model, mandating that the model's learned cross-modal mappings conform to the actual attenuation and propagation patterns of electrical signals in cranial tissue. This physical guidance ensures that the non-invasive decoding results are not only statistically correct but also physiologically plausible, thereby enhancing the system's security and interpretability.

[0055] S400, based on the de-temporalization alignment loss and the physical consistency loss, the second model is pre-trained, and the pre-trained second model is transferred to downstream tasks for training to obtain the third model.

[0056] Specifically, during the training process, the total loss function in the pre-training phase... It can be represented as:

[0057]

[0058] In the formula, To eliminate temporal alignment loss, This represents a loss of physical consistency.

[0059] After joint pre-training in this step, the second model initially possesses the ability to extract general neurosemantic representations from non-invasive EEG signals. Its output is aligned with the first model in a shared neurosemantic space. Subsequent downstream task transfer training is then required to achieve decoding for specific tasks, ultimately resulting in the third model. This process essentially transfers and solidifies the high-quality neurosemantic knowledge obtained during the short-term invasive calibration period into the second model, laying the foundation for subsequent downstream task adaptation and independent decoding.

[0060] In some embodiments, the process of inputting the invasive EEG signal into a first model to obtain a first set of neural activity patterns and a first average probability distribution specifically includes the following steps:

[0061] The invasive EEG signal is divided into multiple first short time window signal segments. Specifically, since invasive EEG signals (such as ECoG) have extremely high temporal resolution and can clearly reflect the transient synchronous oscillatory activity of cortical neuronal populations, this embodiment uses a shorter time window for segmentation. For example, the length T0 of the first short time window can be set to any value between 100ms and 300ms, preferably 200ms. This short time window segmentation can precisely capture the dynamic changes in neural activity and avoid the aliasing of different neural patterns in the temporal dimension due to an excessively long window, thereby ensuring that the subsequently extracted features have clear transient physiological semantics.

[0062] For each of the first short-time-window signal segments, features are extracted by the feature encoder of the first model and mapped to a probability distribution on the shared semantic codebook by a projection classifier. Specifically, the feature encoder can employ a one-dimensional convolutional neural network or a recurrent neural network to convert the original multi-channel voltage time-series signal into a high-dimensional latent space feature vector. Subsequently, the projection classifier maps this feature vector into a K-dimensional logit vector, where K is the number of prototype vectors in the shared semantic codebook. To achieve a differentiable mapping from a continuous feature space to a discrete semantic space, the process of mapping to a probability distribution on the shared semantic codebook in this embodiment includes: mapping the feature vector to a logit vector and converting it into a probability distribution on multiple prototypes of the shared semantic codebook through a continuous relaxation sampling function. Here, the continuous relaxation sampling function is preferably the Gumbel-Softmax function. Since the shared semantic codebook is essentially a discrete lookup table, traditional hard sampling operations (such as Argmax) are not differentiable, which would prevent gradient backpropagation. Gumbel-Softmax, by introducing Gumbel noise and a Softmax temperature parameter, provides a continuous, differentiable probability distribution to approximate discrete class sampling. In the early stages of training, a higher temperature parameter smooths the distribution, facilitating exploration. As training progresses, the temperature gradually decreases, causing the distribution to become sharper, eventually approximating the true discrete one-hot distribution. This design enables the entire dual-branch model to achieve end-to-end gradient optimization training while maintaining discrete semantic representations.

[0063] The probability distribution corresponding to each of the first short-time window signal segments is sampled to obtain the corresponding neural activity pattern identifiers. All neural activity pattern identifiers are then deduplicated to generate the first neural activity pattern set. Specifically, based on the above probability distribution, sampling (using Gumbel-Softmax sampling during training and Argmax during inference) yields the discrete neural activity pattern identifier (Token ID) corresponding to each short-time window segment. Over a longer analysis period, a chronologically ordered Token sequence is obtained. The key to this embodiment is performing a deduplication operation on this sequence, i.e., removing duplicate identifiers and retaining only unique pattern types. For example, if the Token sequence in a certain time period is [A, B, A, C, B], the first neural activity pattern set obtained after deduplication is {A, B, C}. This step is the core means of achieving "de-temporalization": it transforms the sequence matching problem, which is originally sensitive to precise temporal order, into an element matching problem for an unordered set. Because non-invasive signals suffer from severe phase distortion and timing ambiguity, it is unrealistic to force them to align with invasive signals at the millisecond level. The deduplicated set representation ignores the order and specific timing of mode occurrences, thus greatly reducing the difficulty of cross-modal alignment and improving the model's robustness to timing jitter of non-invasive signals.

[0064] The mean of the probability distributions corresponding to all the first short-time window signal segments is calculated to obtain the first average probability distribution. Specifically, although set representation solves the problem of pattern type identification, it loses the frequency or intensity information of each pattern occurrence. Therefore, this embodiment further calculates the arithmetic mean of the probability distributions of all short-time windows within this time period as the first average probability distribution. The calculation of this distribution can be expressed as:

[0065]

[0066] In the formula, This indicates the number of signal segments within the first short time window contained in the invasive EEG signal. Indicates the number of ECoG channels. Let represent the probability distribution vector corresponding to the c-th ECoG channel within the t-th first short-time window signal segment. This formula takes the arithmetic mean of the probability distributions of all first short-time window signal segments and all ECoG channels within the invasive EEG signal to obtain the global average probability distribution. This distribution reflects the overall activation level or statistical proportion of various neural activity patterns in the current time period. Through the dual representation of "set + distribution", the model's ability to qualitatively judge the existence of patterns is ensured, while retaining the ability to quantitatively describe the intensity of patterns, providing a sufficient information foundation for the subsequent construction of a comprehensive alignment loss.

[0067] In some embodiments, the process of inputting the non-invasive EEG signal into a second model to obtain a second set of neural activity patterns and a second average probability distribution forms an asymmetric complementary design with the first model, specifically including the following steps:

[0068] The non-invasive EEG signal is treated as a long-time-window signal segment. Features are extracted using the spatiotemporal encoder of the second model, and mapped onto the shared semantic codebook using a projection classifier. Specifically, unlike the first model, non-invasive EEG signals (such as EEG) undergo low-pass filtering and spatial smoothing effects on the skull and scalp, resulting in extremely low instantaneous signal-to-noise ratios, and signals at individual moments often lack clear independent semantics. Therefore, this embodiment directly processes the non-invasive EEG signal as a whole long-time-window signal segment. The length of the long-time window T1 is typically much larger than the first short-time window T0, for example, it can be set to 1 to 5 seconds, preferably 3 seconds. The longer integration window helps accumulate sufficient signal energy to overcome noise interference and capture the complete shape of slow-wave components or event-related potentials. Meanwhile, the second model employs a spatiotemporal encoder (such as a spatiotemporal convolutional network ST-Conv or Transformer), which can simultaneously model the spatial topological relationships and long-range dependencies in the temporal dimension between multiple channels, extracting a comprehensive feature vector with global context from the ambiguous mixed signal. Subsequently, the feature vector is also mapped to a probability distribution on the shared semantic codebook through a projection classifier. The mapping mechanism is the same as that of the first model, ensuring the comparability of the two in the same semantic space.

[0069] The probability distribution output by the second model is sampled and deduplicated to generate the second neural activity pattern set. It's important to emphasize that for the second model, since its input itself represents the overall features of a long time window, the output probability distribution already contains comprehensive semantic information for that time period. The second neural activity pattern set generated by sampling and deduplicating this set represents the possible combinations of neural patterns that the model infers from the fuzzy EEG signal within that time period. This unordered set prediction target precisely avoids the inherent limitation of EEG signals not accurately corresponding to the micro-temporal sequence of ECoG, allowing the model to focus only on the consistency of macro-level patterns.

[0070] The mean of the probability distributions corresponding to each acquisition channel output by the second model is calculated to obtain the second average probability distribution. Specifically, considering the spatial heterogeneity of non-invasive signals, some channels may contain richer effective information. In this embodiment, the mean of the probability distributions corresponding to all acquisition channels (or channels weighted by an attention mechanism) is calculated as the second average probability distribution. The calculation of this distribution can be expressed as:

[0071]

[0072] In the formula, Indicates the number of EEG channels. Let represent the probability distribution vector corresponding to the c-th EEG channel. This formula takes the average probability distribution across all acquisition channels to obtain the predicted average probability distribution. This distribution represents the confidence estimate of the second model for the overall composition of neural activity patterns within the current time period.

[0073] In summary, this embodiment cleverly bridges the significant gap in spatiotemporal resolution between the two modalities by combining the "short-window fine slicing + discretization" of the first model with the "long-window global integration + semantic inference" of the second model through an asymmetric architecture. In particular, the unordered set representation constructed through deduplication fundamentally eliminates the dependence on strict temporal alignment; while the introduction of continuous relaxation sampling opens up the gradient path between the discrete semantic space and the continuous neural network. These two techniques work together to enable the second model to stably learn physiologically meaningful neural pattern representations from ambiguous signals even in the absence of precise temporal supervision, laying a solid data foundation for subsequent calculation of de-temporal alignment loss. It should be understood that although this embodiment uses Gumbel-Softmax as a preferred example of continuous relaxation sampling, similar functions can also be achieved in other implementations using Straight-Through Estimator (STE) or other differentiable quantization methods. Similarly, the specific durations of the short and long windows can be adaptively adjusted according to the sampling rate of the actual acquisition device and the time scale of the target task, as long as the relative relationship that the invasive window is significantly smaller than the non-invasive window is satisfied.

[0074] In some embodiments, the set alignment loss is calculated as follows: precision and recall are calculated based on the number of intersections between the first neural activity pattern set and the second neural activity pattern set, the total number of the first neural activity pattern set, and the total number of the second neural activity pattern set; the set alignment loss is then calculated based on the harmonic mean of the precision and the recall. Specifically, this step aims to measure the consistency in the types of neural activity patterns output by the two models. Precision reflects how many of the patterns predicted by the second model actually exist in the first model, used to suppress the generation of false patterns; recall reflects how many of the true patterns in the first model are successfully captured by the second model, used to reduce the omission of effective information. The formulas for calculating precision and recall are as follows:

[0075]

[0076] In the formula, This represents the number of intersections between the second set of neural activity patterns and the first set of neural activity patterns. The total number of the second neural activity patterns set. This represents the total number of the first neural activity patterns set.

[0077] The two are combined to form the harmonic mean (i.e., the F1 score) as a comprehensive evaluation index. The formula for calculating the F1 score is as follows:

[0078] .

[0079] Ultimately, set alignment loss It can be defined as 1 minus the F1 score:

[0080] .

[0081] The loss is constructed based on the F1 score, with smaller loss values ​​indicating higher set matching. This loss design based on set overlap fundamentally avoids the shortcomings of traditional methods, such as mean squared error (MSE) or dynamic time warping (DTW), which are extremely sensitive to temporal phase. In cases where non-invasive EEG signals have severe phase distortion and delay, forced waveform alignment often leads to model convergence difficulties or the learning of incorrect time-shift compensations; while set alignment loss only focuses on "whether the same neural pattern appears within this time window," allowing non-invasive signals to have some jitter or recombination on the time axis, as long as the pattern combination it contains is consistent with that of invasive signals. This greatly improves the model's adaptability to non-stationary physiological signals.

[0082] In some embodiments, constructing the de-temporalized alignment loss includes: calculating the distribution difference loss between the first average probability distribution and the second average probability distribution based on the probability distribution divergence; and constructing the de-temporalized alignment loss based on the set alignment loss and the distribution difference loss. Specifically, while the set alignment loss ensures the qualitative accuracy of the pattern types, it cannot reflect the quantitative differences in the activation intensity of each pattern. For example, if an invasive signal indicates that a certain motion pattern is in a high-intensity activation state, and the second model identifies the pattern but with extremely low confidence, the set loss alone cannot effectively punish this "intensity mismatch." Therefore, this embodiment introduces a distribution difference loss as a supplement. The first average probability distribution and the second average probability distribution represent the average activation level or statistical proportion of various neural activity patterns within the analysis window, respectively. In this embodiment, the probability distribution divergence is preferably measured using Kullback-Leibler (KL) divergence, and the distribution difference loss... The calculation formula is as follows:

[0083]

[0084] In the formula, This represents the probability value corresponding to the k-th neural activity pattern in the second average probability distribution. This represents the probability value corresponding to the k-th neural activity pattern in the first average probability distribution. The KL divergence is zero when the first and second average probability distributions are completely identical; the greater the difference, the higher the loss value. By minimizing this distributional difference loss, the second model is forced to align not only with the first model in terms of pattern types but also in the statistical relative intensity of each pattern, thus achieving a refined representation of neural activity states.

[0085] Combining the above two parts, the de-temporalization alignment loss is obtained by weighted summation of set alignment loss and distribution difference loss. The calculation formula can be:

[0086]

[0087] In the formula, Set alignment loss The weighting coefficients, To balance the loss of distribution differences The weighting coefficients of the two are determined by the task requirements. In actual training, the weighting coefficients of both can be dynamically adjusted according to the task requirements. For example, in the early stage of training, the weight of the set alignment loss can be appropriately increased to guide the model to quickly learn to identify the correct pattern type; in the later stage of training, the weight of the distribution difference loss can be increased to further improve the precision of decoding and confidence calibration capabilities. This dual constraint mechanism of "set + distribution" constitutes the core defense of the de-temporalization alignment strategy of this invention: set loss ensures the connectivity of cross-modal semantic space; distribution loss guarantees the fidelity of semantic representation. The two complement each other and jointly overcome the learning obstacles caused by the temporal ambiguity of non-invasive EEG signals, enabling the model to stably recover neural activity representations with clear physiological meaning from ambiguous signals even in the absence of precise temporal supervision. It should be understood that although this embodiment uses F1 score and KL divergence as preferred examples, in other embodiments, the set alignment loss can also adopt set similarity measurement methods such as Jaccard similarity coefficient or Dice coefficient, and the distribution difference loss can also adopt probability distribution distance measurement methods such as Wasserstein distance or JS divergence. As long as the function of unordered set matching and statistical distribution alignment can be achieved, they should be covered within the protection scope of this invention.

[0088] In some embodiments, the process of constructing the head volume conduction model includes: acquiring multimodal medical images of the target object and performing fusion registration; performing tissue segmentation based on the fused and registered multimodal medical images and assigning conductivity to different segmented tissues to construct the head volume conduction model; and calculating the theoretical influence weight of each cortical electrode contact on each scalp electrode based on the head volume conduction model to generate a spatial transfer matrix. Specifically, this step aims to establish a digital twin head model that faithfully reflects the individual anatomical structure of the target object, providing a quantitative basis for subsequent physical constraints. First, multimodal medical images of the target object during implantation are acquired, typically including preoperative high-resolution structural magnetic resonance (MRI) images and postoperative computed tomography (CT) images. MRI images can provide detailed three-dimensional anatomical structures of soft tissues such as brain tissue, cerebrospinal fluid, skull, and scalp, but they do not show the metal electrode contacts well; while CT images can clearly show the high-brightness positions of the metal electrode contacts and the skull contour, but the soft tissue contrast is low. Therefore, rigid or nonlinear fusion registration is necessary to map the precise electrode coordinates from CT to the fine anatomical framework provided by MRI, thereby obtaining the precise three-dimensional position of each invasive and non-invasive sensor in the individual brain space. Subsequently, tissue segmentation is performed based on the registered images, dividing the head structure into key regions such as scalp, skull, cerebrospinal fluid, gray matter, and white matter, and assigning corresponding conductivity values ​​to each region. For example, cerebrospinal fluid and scalp have high conductivity, while skull conductivity is extremely low; this heterogeneous conductivity distribution determines the complex propagation path of electrical signals within the skull. Finally, based on the segmented tissue mesh and assigned conductivity values, the quasi-static Maxwell's equations are solved using the boundary element method (BEM) or finite element method (FEM) to calculate the potential transfer relationship from each cortical electrode contact (as an equivalent current source) to each scalp electrode recording point, generating a spatial transfer matrix H. The dimensions of this matrix are:

[0089] ,

[0090] In the formula, Indicates the number of scalp electrodes. This represents the number of cortical electrode contacts. Each element H(i,j) is a scalar that quantitatively reflects the combined attenuation effect of the geometry, thickness, and conductivity of all tissues traversed during the transmission of the j-th cortical source signal to the i-th scalp recording point. It should be understood that although this embodiment uses MRI and CT as the preferred imaging modalities, in other embodiments, other imaging technologies such as ultrasound and PET can be used to assist in modeling, as long as sufficient anatomical and electrode location information can be obtained.

[0091] In some embodiments, the step of using elements of the spatial transfer matrix as attenuation factors to weight and constrain the feature distribution differences between the invasive and non-invasive EEG signals to construct a physical consistency loss includes: using elements of the spatial transfer matrix as attenuation factors to weight and sum the feature distribution differences corresponding to physically coupled cortical and scalp electrodes to obtain the physical consistency loss. Specifically, the spatial transfer matrix H not only describes the forward propagation law of the signal but also implicitly contains the physical coupling strength between cross-modal electrode pairs. For electrode pairs with a large H(i,j) value, it means that the j-th cortical source has a significant theoretical contribution to the i-th scalp electrode, and the two are physiologically strongly correlated; conversely, if H(i,j) is close to zero, it indicates that the two are almost physically decoupled. This embodiment utilizes this physical prior, directly using H(i,j) as an attenuation factor to weight the feature distribution differences of the dual-branch model output. The core logic of this design is as follows: if there is a significant difference between the non-invasive feature distribution predicted by the model and the invasive feature distribution on physically strongly coupled electrode pairs, it indicates that the model has failed to correctly capture the true signal transmission relationship and should be penalized more severely. Conversely, on physically weakly coupled or decoupled electrode pairs, even if there are differences in feature distribution, it may be caused by other far-field sources or noise, and should not be over-penalized. Through this selective constraint based on the strength of physical coupling, the cross-modal mapping learned by the model is forced to conform to the true laws of electrical signal propagation, fundamentally suppressing spurious associations that violate anatomical common sense, such as incorrectly mapping occipital visual signals to the frontal motor cortex, thereby improving the safety and reliability of the system.

[0092] In some embodiments, the step of using the elements of the spatial transfer matrix as the attenuation influence factor to weight and sum the differences in feature distributions corresponding to physically coupled cortical electrodes and scalp electrodes to obtain the physical consistency loss includes: converting the elements of the spatial transfer matrix into weight coefficients using a normalization function, wherein the elements comprehensively reflect the influence of the conductivity and geometry of cerebrospinal fluid, skull, and scalp tissue on signal attenuation; using probability distribution divergence to measure the distance between the non-invasive feature probability distribution corresponding to the i-th scalp electrode and the invasive feature probability distribution corresponding to the j-th cortical electrode; and performing element-wise multiplication of the weight coefficients with the distance and summing the results across all spatial locations to obtain the physical consistency loss. Specifically, since the numerical range of the elements in the original spatial transfer matrix H may be large and contain physical units, directly using them for weighting may lead to gradient instability or numerical overflow. Therefore, this embodiment first converts H(i,j) into dimensionless weight coefficients w(i,j) using a normalization function (such as Softmax or Min-Max normalization) to characterize the relative coupling strength rather than the absolute potential value. Subsequently, for each pair of electrodes (i,j), the corresponding second model probability distribution is calculated. Probability distribution of the first model The distance D between them. This distance metric preferably uses the Kullback-Leibler (KL) divergence to measure the statistical difference between the two probability distributions, which is consistent with the distribution difference metric used in the aforementioned de-temporalization alignment loss, facilitating joint optimization. Finally, the physical consistency loss... The calculation formula is expressed as:

[0093]

[0094] In the formula, This represents a normalization function that transforms the elements of the transfer matrix into weight coefficients. This represents the distance metric between the second model probability distribution corresponding to the i-th scalp electrode and the first model probability distribution corresponding to the j-th cortical electrode. This is achieved by minimizing the physical consistency loss. The second model is mandated to output neural pattern distributions highly consistent with corresponding cortical sources on scalp channels where strong signals should theoretically be received; while on channels with theoretically weak signals, greater degrees of freedom are allowed to accommodate background noise or other interference. This refined spatial constraint mechanism compensates for the inability of global set alignment loss alone to capture local spatial topological relationships, ensuring that the non-invasive decoding results are correct not only in terms of macroscopic pattern types but also in terms of microscopic spatial distribution conforming to biophysical laws. It should be understood that although this embodiment uses KL divergence as a preferred example of distance metric, other metrics such as Wasserstein distance, JS divergence, or Euclidean distance can also be used in other embodiments; similarly, the choice of normalization function can also be adjusted according to the characteristics of matrix numerical distribution. As long as the differentiated weighted constraint function based on physical priors can be achieved, it should be covered within the scope of protection of this invention.

[0095] like Figure 3 As shown, in some embodiments, the construction process of the shared semantic codebook includes: acquiring separately collected invasive EEG signals and segmenting them into short-time signal segments; inputting the short-time signal segments into an encoder to obtain feature vectors, and replacing them with the closest prototype vectors in the initialized codebook through a vector quantization layer to obtain a neural activity pattern identifier sequence; performing random masking on the neural activity pattern identifier sequence; reconstructing the neural activity pattern identifiers corresponding to the masked parts through a decoder based on the unmasked neural activity pattern identifiers; optimizing the parameters of the codebook based on the difference between the reconstructed neural activity pattern identifiers and the original neural activity pattern identifiers to obtain the frozen shared semantic codebook shared in the first model and the second model.

[0096] Specifically, the data source for this step is exclusive, using only high-quality invasive EEG signals (such as ECoG) acquired separately during implantation in the target subject, excluding simultaneously acquired non-invasive EEG signals. This is because invasive signals have extremely high signal-to-noise ratios and spatial resolution, best reflecting the true electrophysiological activity of cortical neural populations. Pre-training with such signals ensures that each prototype vector in the codebook corresponds to a genuine neural firing pattern or oscillatory rhythm, rather than non-neurogenic artifacts such as scalp electromyography or power line interference. During the preprocessing stage, continuous invasive signals are segmented into short signal fragments (e.g., 200ms) consistent with subsequent training to maintain semantic granularity.

[0097] Furthermore, this embodiment introduces a masked modeling mechanism as the core driving force for self-supervised learning. Random masking of the neural activity pattern identifier sequence refers to randomly selecting a portion of the neural activity pattern identifiers in the sequence according to a preset ratio (e.g., 30% to 50%) and replacing them with learnable special mask tokens. The underlying technical logic of this design is that the neural activity pattern identifier sequence is not randomly arranged, but reflects the inherent temporal structure and dynamic laws of cortical neural group activity. By forcing the model to infer and reconstruct the neural activity pattern identifiers corresponding to the masked parts based on the unmasked identifiers, it effectively forces the prototype vectors in the codebook to deeply capture the contextual dependencies and semantic structure of neural activity. Only when the prototype vectors in the codebook truly represent meaningful neural pattern primitives can the decoder accurately complete the identifier completion task. Conversely, if the codebook only remembers the surface statistical distribution without grasping the underlying structure, high-precision reconstruction under masked conditions will not be possible. Therefore, the mask reconstruction task based on the identifier sequence acts as a strict semantic filter, screening and solidifying those neural representations with generalization capabilities.

[0098] In the quantization stage, the key step in discretizing continuous neural features is to replace the nearest prototype vector in the initialized codebook with the vector quantization layer. Specifically, the Euclidean distance between the encoder's output feature vector and all prototype vectors in the codebook is calculated, and the prototype index with the smallest distance is selected as the neural activity pattern identifier for that location. Since the nearest neighbor search operation is itself non-differentiable, in actual training, a straight-through estimator (STE) or other gradient approximation techniques are typically used to directly copy the decoder's gradient to the encoder, thereby achieving end-to-end parameter updates. As the reconstruction loss is continuously minimized, the prototype vectors in the codebook gradually evolve from a randomly initialized state into cluster centers that can accurately represent various typical neural activities.

[0099] Finally, regarding the codebook freezing and sharing strategy. In this embodiment, after pre-training is completed and the reconstruction loss is confirmed to have converged, the parameters of the shared semantic codebook are immediately frozen. This means that in subsequent dual-branch joint training phases, the codebook parameters will no longer participate in gradient updates. This strategy is crucial, as it locks in the baseline coordinate system of the semantic space. If the codebook is allowed to continue updating during cross-modal alignment training, due to the low signal-to-noise ratio of non-intrusive signals, the codebook may drift to adapt to noise or non-specific components in the EEG, causing the originally clear neurosemantic definition to become blurred, or even destroying the accurate representation established by the first model. By freezing the codebook, it is equivalent to providing the dual-branch model with a fixed and authoritative neural dictionary. The training objective of the first and second models becomes learning how to translate the input signal into words in this dictionary, thereby achieving robust cross-modal knowledge transfer while ensuring semantic consistency. It should be understood that although this embodiment uses VQ-VAE combined with mask reconstruction as the preferred solution, in other embodiments, variant architectures such as VQ-GAN and DALL-E can also be used, or other self-supervised paradigms such as contrastive learning can be used to initialize the codebook. As long as the purpose of extracting discrete neural semantic primitives from invasive signals and sharing them between the two branches can be achieved, they should all be covered within the scope of protection of this invention.

[0100] like Figure 2 As shown, in some embodiments, the method further includes the step of determining the optimal time window length during the downstream task transfer training phase: starting from a preset initial time window length, the time window adjustment amount is predicted by the policy network based on the decoding uncertainty calculated in real time and the current task type; constraints are imposed on the application direction of the time window adjustment amount according to the current task type, and the current time window length is updated; the task loss is calculated based on the updated current time window length, and a comprehensive loss is calculated by combining the delay penalty term corresponding to the current time window length; by minimizing the comprehensive loss, the optimal fixed time window length corresponding to each type of task is determined, and a mapping table between task type and optimal fixed time window length is generated for window invocation during the inference phase.

[0101] Specifically, the core of this step lies in transforming the traditional "real-time adaptation" of the inference phase into "offline search and fixation" of the training phase. In existing technologies, dynamically adjusting the window during inference not only increases computational overhead and leads to response latency, but may also cause frequent jumps in window length due to signal fluctuations, affecting user experience and control stability. This embodiment simulates performance under different window lengths using a policy network during training, pre-selects a globally optimal fixed window length for each task type, and directly calls it through a lookup table during the inference phase. This design retains the flexibility of configuring windows differently for different tasks while ensuring determinism, low latency, and low power consumption in the inference process, perfectly adapting to the deployment requirements of embedded or mobile brain-computer interface devices.

[0102] During the search process, decoding uncertainty U is a key input to the policy network's decision-making. In this embodiment, decoding uncertainty U is calculated by combining two complementary metrics: set entropy and channel consistency. Set entropy The calculation formula is as follows:

[0103]

[0104] In the formula, This represents the total number of different neural activity pattern identifiers in the second set of neural activity patterns. Indicates the first Each pattern identifier represents the frequency of its occurrence in the second set of neural activity patterns.

[0105] Set entropy measures the degree of disorder in the set of second neural activity patterns output by the second model. If the set contains a wide variety of patterns with a uniform probability distribution, it means that the model cannot be sure of the current dominant neural state, and the entropy value is high.

[0106] Channel consistency ratio The calculation formula is as follows:

[0107]

[0108] In the formula, This indicates the number of EEG channels. Channel consistency measures whether the prediction results of all acquired channels converge. If the most probable mode identifiers predicted by each channel differ significantly, it indicates that the spatial pattern is chaotic and the consistency ratio is low. The difference is relatively low. Combining the two, we obtain the real-time decoding uncertainty U, calculated as follows:

[0109]

[0110] In the formula, This represents an adjustable weighting coefficient used to balance the influence of the two indicators. The U-value accurately reflects the perplexity of the model at the current moment: a high U-value means that the current signal quality is poor or the features are not obvious, and the model needs a longer time window to accumulate more contextual evidence to improve confidence; conversely, a low U-value indicates that the signal is clear and the pattern is well-defined, and a shorter window can meet the decoding requirements, so response speed should be prioritized. This uncertainty quantification method based on information theory and spatial statistics is more robust than simply relying on the maximum output probability and can effectively avoid misjudgments caused by noise in a single channel.

[0111] The policy network outputs a time window adjustment amount Δt based on the input decoding uncertainty U and the current task type label. To ensure that the adjustment direction conforms to the task characteristics, this embodiment applies explicit constraints on the application direction of the adjustment amount according to the current task type. For fine control tasks (such as single-finger motion imagination and precise cursor positioning), the requirement for decoding accuracy is much higher than speed; therefore, the constraint strategy is to increase the adjustment amount based on the current window (i.e., ...). The system tends to extend the integration time to achieve a higher signal-to-noise ratio and accuracy. However, for simple classification tasks (such as distinguishing between left and right hands) or natural state monitoring (such as switching between resting and moving states), it is more sensitive to real-time performance and has a lower decoding difficulty. Therefore, the constraint strategy is to subtract the adjustment amount from the current window (i.e., ...). This approach tends to shorten the window to reduce system latency. During this process, constraints on the output layer activation function are used to ensure... > >0. This task-aware bidirectional adjustment mechanism enables the model to automatically adapt to diverse application scenarios, ranging from high precision to low latency, within the same parameter system.

[0112] After determining the current time window length for the candidates, the system calculates the comprehensive loss to evaluate the merits of that window length. The formula for calculating the comprehensive loss is as follows:

[0113]

[0114] In the formula, This represents the average validation loss (e.g., cross-entropy loss) of the task under window length T1. This represents a trade-off with hyperparameters, used to penalize latency caused by excessively long windows. The overall loss consists of two parts: task loss. With delay penalty By minimizing this overall loss, the model is forced to find the optimal balance between "improving accuracy" and "reducing latency": While excessively extending the window might slightly improve accuracy, it leads to a significant increase in latency, thus raising the overall loss; conversely, over-compressing the window causes accuracy to collapse, and the task loss term also worsens the overall loss. Only when the window length is precisely at the "speed-accuracy Pareto optimal front" of the task does the overall loss reach its minimum. The weight coefficient λ can be preset according to the tolerance for real-time performance in specific application scenarios. For example, in rehabilitation training scenarios, λ can be appropriately increased to encourage faster feedback, while in offline data analysis scenarios, λ can be decreased to pursue maximum accuracy.

[0115] The optimal fixed time window length can be determined as follows:

[0116]

[0117] In the formula, This represents the value of the variable that minimizes the subsequent function. Indicates the length of the time window and task type The overall loss function.

[0118] Finally, the comprehensive loss is achieved by traversing the search on the validation set. The minimized window length is determined as the optimal fixed-time window length for this type of task. The mapping table is then written to correspond to the task type. This mapping table is persistently stored as part of the training output. It should be understood that although this embodiment describes a gradient optimization search process based on a policy network, in other embodiments, discrete optimization strategies such as grid search, Bayesian optimization, or genetic algorithms can also be used to find the optimal window. Similarly, the form of the delay penalty term is not limited to linear weighting; it can also adopt nonlinear forms such as exponential decay or step functions. As long as the joint optimization objective of accuracy and speed can be achieved, it should be covered within the scope of protection of this invention.

[0119] Example 2:

[0120] like Figure 4 As shown, this embodiment provides a method for decoding electroencephalogram (EEG) signals. This method is applied in the stage of purely non-invasive brain-computer interface applications after the removal of invasive electrodes. It aims to achieve efficient and real-time decoding of neural activity and device control using the model trained and solidified in Embodiment 1. The method includes the following steps:

[0121] S701, Get the current task type.

[0122] Specifically, the current task type refers to the category of brain-computer interface interaction function that the user expects to perform at the current moment, such as fine finger motor imagery, left- or right-hand differentiation, character spelling, or resting state monitoring. This task type can be obtained in various ways: a common method is through external cue-based triggering, such as when a specific visual icon appears on the screen or a specific cue sound is heard, the system automatically identifies the corresponding task category; another method is for the user to actively switch the current application mode through a preset interactive interface (such as buttons, voice commands, or eye-tracking selection). Regardless of the triggering method, clearly defining the current task type is a necessary prerequisite for initiating the subsequent adaptive decoding process, because different types of tasks have fundamentally different requirements for decoding accuracy and response speed. This information will directly determine the system's subsequent signal analysis strategy.

[0123] S702, based on the current task type, determine the corresponding target time window length from a preset mapping table, wherein the mapping table contains the correspondence between task types and optimal fixed time window lengths.

[0124] Specifically, the preset mapping table in this step is the product generated through the offline search and solidification process in the aforementioned embodiment 1. During the inference phase, the system no longer performs any time-consuming online window search or dynamic adjustment calculations. Instead, it directly uses the current task type as an index to perform a simple lookup operation in the mapping table, immediately obtaining the pre-optimized target time window length for that task. For example, for a fine finger movement imagery task, the optimal window length recorded in the mapping table might be 4 seconds to ensure sufficient signal integration time to distinguish subtle differences in neural patterns; while for a simple left- or right-hand distinction task, the optimal window length might be only 1.5 seconds to prioritize smooth interaction and low latency. This "offline optimization, online table lookup" design mechanism fundamentally decouples the computational load of window adaptation and real-time decoding, enabling the system to complete parameter configuration within milliseconds. It retains the flexibility to configure windows differently for different tasks while ensuring determinism, low latency, and low power consumption in the inference process, perfectly adapting to the deployment requirements of embedded or mobile brain-computer interface devices. It should be understood that although this embodiment uses a table lookup method as the preferred implementation, in other embodiments, if hardware computing power allows, lightweight function fitting or rule engines can be used instead of discrete mapping tables, as long as the target window length can be quickly determined according to the task type.

[0125] S703, within the target time window length, acquire the non-invasive EEG signal to be decoded, input the non-invasive EEG signal to be decoded into the EEG signal decoding model, and output a set of neural activity pattern predictions; wherein, the EEG signal decoding model is a third model trained by the training method described in the aforementioned embodiment 1.

[0126] Specifically, once the target time window length is determined, the system continuously acquires multi-channel non-invasive EEG signals (such as EEG) within that time frame, starting from the task trigger moment. The acquired raw signals undergo necessary online preprocessing (such as bandpass filtering, power line interference suppression, and artifact removal) before being fed into the EEG signal decoding model loaded into the local computing unit. This model is the third model described in Example 1, after dual-branch joint training, physical constraint optimization, and parameter solidification. It contains a frozen shared semantic codebook, an optimized spatiotemporal encoder, and a projection classifier. The forward propagation process of the model fully follows the processing logic of the second model described in Example 1: the spatiotemporal encoder extracts long-term contextual features, the projection classifier maps them to a probability distribution in the shared semantic space, and a neural activity pattern prediction set is formed by sampling or selecting the most probable identifier. It is important to emphasize that this process relies solely on non-invasive signals, without the assistance of any invasive signals; and since the window length has been optimally matched to the current task, the neural activity pattern prediction set output by the model achieves the best balance between signal-to-noise ratio and information integrity, laying a solid foundation for subsequent accurate classification.

[0127] S704, the neural activity pattern prediction set is input into the downstream task classifier corresponding to the current task type to generate control instructions.

[0128] Specifically, the downstream task classifier is a lightweight network (such as a linear layer or a small multilayer perceptron) specifically trained and solidified for a particular task type during the transfer training phase of Example 1. Different task types correspond to different downstream task classifier instances, and the system simultaneously loads or activates the corresponding classifier when acquiring the task type in S701. The neural activity pattern prediction set, as a high-order, discrete feature representation rich in physiological semantics, is input into this classifier, mapped to a specific control intention probability distribution through its nonlinear transformation, and finally generates executable control instructions through argmax or other decision rules. For example, for a prosthetic limb control task, the instruction might be "bend index finger" or "abduct thumb"; for a cursor control task, the instruction might be "move right" or "click to confirm". Since the upstream EEG decoding model training system has already completed the cross-modal translation from fuzzy signals to clear neural patterns, the downstream classifier only needs to focus on the simple mapping from pattern to instruction. This not only significantly reduces the design complexity of the classifier and the training data requirements, but also significantly improves the accuracy and robustness of instruction generation. Through the above four-step closed-loop process, this embodiment achieves efficient decoding of the entire chain from user intent generation to external device response, seamlessly transforming the high-performance capabilities obtained from short-term invasive calibration into a long-term, safe, and practical non-invasive brain-computer interface application experience.

[0129] like Figure 4 and Figure 5 As shown, in some embodiments, the process of inputting the non-invasive EEG signal to be decoded into the EEG decoding model training system and outputting a set of neural activity pattern predictions specifically includes the following steps:

[0130] S7031, the spatiotemporal encoder of the EEG decoding model training system extracts the contextual feature vector of the non-invasive EEG signal to be decoded. Specifically, during the inference phase, the spatiotemporal encoder receives the pre-processed multi-channel non-invasive EEG signal to be decoded within the target time window, and converts the original voltage time-series data into a high-dimensional latent space feature representation through its internal convolutional layers or attention mechanism layers. Unlike the training phase, the forward propagation during inference is completely deterministic, without any random sampling or noise injection. This contextual feature vector not only encodes the instantaneous signal morphology within the current window, but also integrates the spatial topological information and long-range dependencies in the temporal dimension between multiple channels, providing a sufficient information foundation for subsequent semantic mapping.

[0131] S7032, the context feature vector is mapped to a probability distribution on the shared semantic codebook using a projection classifier. Specifically, the projection classifier, as a lightweight linear or nonlinear transformation layer, projects the high-dimensional context feature vector onto the K-dimensional semantic space defined by the shared semantic codebook, outputting a K-dimensional probability distribution vector. Each element in this vector represents the probability that the current input signal belongs to the corresponding neural activity pattern prototype. Since the shared semantic codebook is frozen and solidified during the training phase, this mapping process is essentially positioning within a stable, physiologically meaningful discrete semantic coordinate system, ensuring the consistency and comparability of decoding results at different time points and in different sessions.

[0132] S7033, based on the probability distribution, the most likely neural activity pattern identifiers are selected to form the neural activity pattern prediction set. Specifically, unlike the training phase which uses continuous relaxation sampling such as Gumbel-Softmax to support gradient backpropagation, the inference phase directly adopts a deterministic decision-making strategy, such as selecting one or more indices with the highest probability values ​​as the final neural activity pattern identifiers (Token IDs). For single-channel or multi-channel integrated output, all selected unique identifiers are aggregated to form the neural activity pattern prediction set. This set, as a highly abstract, denoised, and semantically rich intermediate representation, is directly fed into the downstream task classifier for instruction generation. This decoding paradigm based on discrete semantic sets significantly reduces the learning difficulty of the downstream classifier compared to directly using the original waveform or continuous features for classification, and effectively shields the non-specific noise interference remaining in the non-invasive signal, improving the accuracy and robustness of instruction generation.

[0133] In some embodiments, to address the issue of gradually declining decoding performance caused by factors such as changes in user neural state over time, fluctuations in electrode contact impedance, or environmental interference, this embodiment also provides a model fine-tuning step. This step aims to adapt to the user's current state through extremely low-cost local parameter updates without compromising the model's core cross-modal mapping capabilities.

[0134] S7034: When the decoding confidence level is detected to be lower than a preset threshold or a preset period is reached, the fine-tuning process is triggered. Specifically, the system has a built-in dual triggering mechanism to ensure the timeliness and necessity of fine-tuning. The first is a performance-based passive trigger: the system monitors in real time the maximum probability value output by the downstream task classifier (i.e., decoding confidence level) or the decoding uncertainty index U defined in the aforementioned embodiment 1. When its average value within the sliding time window is continuously lower than a preset safety threshold, it indicates that the current model's ability to discern user intent has significantly deteriorated, and fine-tuning is automatically triggered. The second is a time-based active trigger: regardless of whether the current performance has significantly deteriorated, the system initiates fine-tuning periodically according to a preset fixed period (e.g., once a week or every 20 hours of cumulative use) as a preventative maintenance measure to compensate for the slowly accumulating physiological or hardware drift. This dual-insurance mechanism ensures that the system is always in optimal working condition, while avoiding the waste of computing resources and user experience interruption caused by frequent fine-tuning.

[0135] S7035, freeze the backbone network and shared semantic codebook of the EEG decoding model training system. Specifically, this is the core of the catastrophic forgetting prevention strategy in this embodiment. The backbone network (i.e., the spatiotemporal encoder) and the shared semantic codebook carry valuable cross-modal mapping knowledge transferred from the short-term invasive calibration phase. This knowledge was learned under high signal-to-noise ratio supervision signals and cannot be relearned or corrected using only low-quality non-invasive signals. If these parameters are allowed to be updated during fine-tuning, the model is prone to overfitting to the current small amount of calibration data or noise, causing the originally correct neurosemantic representation to collapse or drift. Therefore, during fine-tuning, these two parameters must be strictly locked so that they are used only as fixed feature extractors and semantic dictionaries.

[0136] S7036, Acquire newly acquired, task-labeled, calibration-graded non-invasive EEG signals and fine-tune only the parameters of the downstream task classifier. Specifically, the data required for fine-tuning is acquired through a short, system-guided calibration session. During the calibration session, the user interface prompts the user to perform a series of standardized target tasks (e.g., "Imagine grasping with your left hand 5 times"). Since the task instructions are issued by the system, the corresponding labels are known and precise, requiring no manual annotation. Using this fresh, labeled, small sample data, only the weights of the downstream task classifier (typically a lightweight fully connected layer or MLP) are updated. Due to the extremely small number of classifier parameters, the fine-tuning process can be completed within seconds to tens of seconds, almost imperceptibly to the user. More importantly, this local update strategy allows the model to quickly adapt to the current signal distribution characteristics (achieved by the updated classifier) ​​while preserving general neurosemantic understanding capabilities (guaranteed by the frozen backbone), thus achieving an optimal balance between stability and adaptability.

[0137] Furthermore, to prevent drastic changes in classifier parameters during fine-tuning from damaging historical knowledge, this embodiment introduces an Elastic Weight Consolidation (EWC) regularization term into the loss function of fine-tuning. This technique evaluates the importance of each parameter to the old task by calculating the Fisher Information Matrix, imposing stronger penalty constraints on changes in important parameters, while allowing larger adjustments to unimportant parameters. This is equivalent to defining a safety boundary for learning new knowledge, ensuring that the fine-tuned classifier can accurately respond to the current neural pattern distribution without forgetting the decision boundaries obtained from training on a large amount of previous data. It should be understood that although this embodiment uses EWC as the preferred anti-forgetting method, other continuous learning strategies such as knowledge distillation, replay buffer, or L2 regularization can also be used in other embodiments. Similarly, the trigger threshold, cycle length, and task paradigm of the calibration session can all be customized according to specific application scenarios and user habits. As long as the basic principle of "freezing the core and fine-tuning the end" is followed, they should all be covered within the protection scope of this invention.

[0138] Example 3:

[0139] like Figure 6 and Figure 7 As shown, this embodiment provides a non-invasive EEG decoding model training system and an EEG signal decoding system based on invasive guidance. The system architecture aims to encapsulate the method flow described in the aforementioned embodiments into a modular virtual device to support the implementation and deployment of software products. It should be understood that each module in this embodiment can be a software functional unit running on a processor, a logic unit embedded in hardware circuitry, or a combination of software and hardware components. The division method is only used for a clear description of logical functions and does not constitute a limitation on physical implementation.

[0140] In some embodiments, such as Figure 6 As shown, the non-invasive EEG decoding model training system based on invasive guidance includes a signal acquisition module, a semantic mapping module, a loss construction module, and a model training module.

[0141] Specifically, the signal acquisition module is used to acquire synchronously acquired invasive and non-invasive EEG signals. This module typically includes a data reading interface and a time alignment subunit, responsible for loading bimodal data from storage media or real-time acquisition devices, and ensuring strict synchronization of the two sets of signals on the time axis through unified clock stamp verification, providing a basis for paired samples for subsequent cross-modal learning. The semantic mapping module is used to input the invasive EEG signals into a first model to obtain a first set of neural activity patterns and a first average probability distribution; and to input the non-invasive EEG signals into a second model to obtain a second set of neural activity patterns and a second average probability distribution; wherein the first model and the second model share a semantic codebook. This module internally encapsulates the forward propagation logic of a dual-branch neural network and the lookup and quantization operations of the shared semantic codebook, and its outputs are respectively connected to the loss construction module, providing intermediate representations for calculating the supervision signal. The loss construction module is used to construct a de-temporalized alignment loss based on the set alignment loss of the first set of neural activity patterns and the second set of neural activity patterns and the distribution difference loss of the first average probability distribution and the second average probability distribution; and to construct a physical consistency loss based on the signal conduction constraints of the head anatomy. This module integrates an ensemble operation unit, a probability distribution divergence calculation unit, and a physical prior weighting unit, and can automatically generate the scalar loss value required for gradient backpropagation based on a preset loss function formula. The preset loss function formula can be:

[0142]

[0143] In the formula, The de-temporalization alignment loss described in Example 1; The physical consistency loss described in Example 1 is used. The model training module is used to pre-train the second model based on the de-temporalization alignment loss and the physical consistency loss, and then perform downstream task transfer training on the pre-trained second model to obtain the third model. This module typically uses stochastic gradient descent or its variants, receives the gradient information output by the loss construction module, and iteratively updates the network weights of the second model until the pre-training convergence condition is met. Subsequently, downstream task transfer training is performed using training data with task labels to train the downstream task classifier. During the training process, the optimal fixed time window length for each task type is adaptively determined based on the decoding uncertainty, generating a mapping table between task types and optimal fixed time window lengths. The specific implementation logic of the downstream transfer task is described in Example 1. Through the above modular design, the training process is decoupled into independent data flow processing units, which facilitates migration between different computing platforms and allows for algorithm upgrades for specific modules without affecting the overall system architecture.

[0144] In some embodiments, such as Figure 7As shown, the EEG signal decoding system includes a task recognition module, a window determination module, a signal decoding module, and an instruction generation module.

[0145] Specifically, the task identification module is used to obtain the current task type. This module can determine the type of brain-computer interface function that the user expects to execute at the current moment by parsing external trigger signals, user interface interaction events, or host computer instructions, and pass the task type identifier to the downstream module. The window determination module is used to determine the corresponding target time window length from a preset mapping table based on the current task type. The mapping table contains the correspondence between task types and optimal fixed time window lengths. This module maintains a lightweight lookup table, which is the product of offline search optimization during the training phase. During inference, the module only needs to perform a hash lookup or array indexing operation once to obtain the optimal window parameters, avoiding the latency caused by complex online calculations. The signal decoding module is used to collect non-invasive EEG signals to be decoded within the target time window length, input the non-invasive EEG signals to be decoded into the EEG decoding model training system, and output a set of neural activity pattern predictions. The EEG decoding model training system is a third model trained by the training method described in Embodiment 1 above. This module is the core execution unit of the decoding system. It loads the model file produced by the training system and completes signal buffering, preprocessing, and forward inference within a defined time window, outputting discrete neural semantic representations. The instruction generation module is used to input the neural activity pattern prediction set into the downstream task classifier corresponding to the current task type to generate control instructions. This module dynamically loads or activates the corresponding classifier weights according to the task type, mapping high-order neural semantics to specific device control commands, such as prosthetic joint angles, cursor movement vectors, or character selection signals. Through this lookup-style window adaptation and modular decoding architecture, the system exhibits extremely high determinism and low resource consumption during runtime, making it particularly suitable for deployment in computationally limited embedded brain-computer interface terminals or wearable rehabilitation devices, achieving effective transformation from laboratory algorithms to clinically applicable products.

[0146] Example 4:

[0147] This embodiment provides an electronic device, a computer-readable storage medium, and a computer program product, aiming to provide comprehensive protection for the technical solution of the present invention from three dimensions: hardware entity, data carrier, and software distribution form, ensuring that in practical applications, regardless of whether the infringer appears as a terminal equipment manufacturer, software service provider, or cloud platform provider, it will fall within the protection scope of the present invention.

[0148] In some embodiments, the present invention also provides an electronic device. The electronic device includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the non-invasive EEG decoding model training method based on invasive guidance as described in Embodiment 1 above, or the EEG signal decoding method as described in Embodiment 2 above. Specifically, the processor here can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., or a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The memory can include volatile memory, such as random access memory (RAM); it can also include non-volatile memory, such as flash memory, hard disk, or solid-state drive. In practical application scenarios of the present invention, the specific form of the electronic device is diverse. For example, during the model training phase, the electronic device can be a high-performance workstation or cloud server cluster deployed in a hospital or research institution, utilizing its powerful parallel computing capabilities to process massive amounts of synchronously acquired bimodal EEG data and perform complex dual-branch model joint training and physical consistency loss optimization. During the model inference and application phase, the electronic device can be a low-power microcontroller embedded in a portable brain-computer interface headband, a real-time control unit integrated into a rehabilitation robot, or a smartphone or tablet carried by the user. When the electronic device is used to execute the decoding method, its processor, by calling the lookup table logic and lightweight decoding model embedded in memory, can complete the entire process from task recognition to control command generation within milliseconds, meeting the stringent real-time requirements of brain-computer interface systems. It should be understood that although various specific hardware forms have been listed above, any computing device that possesses a processor and memory and is capable of running the methods described in this invention should be included within the scope of protection of this invention.

[0149] In some embodiments, the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the non-invasive EEG decoding model training method based on invasive guidance as described in Embodiment 1, or the EEG signal decoding method as described in Embodiment 2. Specifically, a computer-readable storage medium refers to any tangible medium capable of carrying computer program code and readable by a computer system, excluding transient propagation signals themselves. Common computer-readable storage media include, but are not limited to, disks (such as floppy disks, hard disks), optical disks (such as CD-ROMs, DVD-ROMs), and semiconductor memories (such as ROMs, EEPROMs, USB flash drives, SD cards), etc. In the technical solution of the present invention, the storage medium is not only the physical carrier of the code, but also a key node for the delivery and circulation of technological achievements. For example, in the scenario of pre-installation on medical devices, the decoding algorithm of the present invention can be pre-programmed into the Flash storage chip inside the device and sold as part of the firmware along with the hardware; in the scenario of software updates or remote deployment, a server storing the program of the present invention can transmit the program code to the local storage medium of the terminal device via a network to achieve online upgrades of functions. This independent protection of storage media allows rights holders to assert their rights even if the infringement only involves the copying, sale, or offer to sell of the storage media, without directly manufacturing the complete device, thereby effectively combating piracy and illegal distribution.

[0150] In some embodiments, the present invention also provides a computer program product. This computer program product includes a computer program that, when executed by a processor, implements the non-invasive EEG decoding model training method based on invasive guidance as described in Embodiment 1 above, or the EEG signal decoding method as described in Embodiment 2 above. Specifically, the computer program product emphasizes the commercial circulation attributes of the software itself, and its form can be an application installation package (APK / IPA), a software development kit (SDK), a dynamic link library (DLL), a cloud service image file, or an API interface call package, etc. Unlike storage media, computer program products emphasize the functional set of program code and its transaction value as an independent commodity. For example, third-party rehabilitation device manufacturers can purchase an SDK containing the decoding method of the present invention and integrate it into their self-developed prosthetic control system without having to develop the underlying EEG decoding algorithm themselves; similarly, cloud service platforms can encapsulate the training method of the present invention as a PaaS service, allowing research institutions to call it on demand to build personalized models. By including computer program products within the scope of protection, the present invention constructs a complete intellectual property protection chain from the underlying hardware and physical carrier to the upper-level software product, ensuring that the core technical solutions can receive full legal protection under different business models and technological ecosystems.

[0151] Example 5:

[0152] This embodiment provides a specific application scenario verification of a non-invasive brain-computer interface performance enhancement method based on short-term invasive signal calibration. It aims to demonstrate, through real clinical rehabilitation cases, how this invention addresses the capability gap between the short-term invasive high-performance window and the long-term non-invasive application requirement period in existing technologies, and verifies the effectiveness and safety of the technical solutions described in the foregoing embodiments in practical applications. It should be noted that the application scenarios described in this embodiment are merely illustrative examples and are not intended to limit the scope of protection of this invention. Any clinical or non-clinical applications that conform to the technical concept of this invention should be included.

[0153] In this application scenario, the target user is a user suffering from severe upper limb motor dysfunction due to stroke. This user has voluntarily joined a prospective clinical study evaluating the effects of invasive brain-computer interfaces on motor function rehabilitation. According to the established medical protocol of this study, the user will undergo approximately one month of subdural cortical EEG electrode implantation to record motor cortex signals and attempt to control an external neuroprosthetic limb for rehabilitation training. It should be understood that the implementation of this invention strictly adheres to medical research ethics. All data collection is based on the user's independent and necessary clinical diagnosis or scientific research plan, does not constitute additional treatment or surgical intervention, does not change the original implantation and removal surgical plan, and does not prolong the implantation time or increase any additional invasive risks. The execution of the protocol is considered a non-interventional observational data collection and research activity added to the main clinical procedures, and all user information (including physiological sensor data and personal information) is fully authorized by the user or legally authorized by all parties. Data processing complies with relevant laws, regulations, and privacy protection standards throughout the process. Furthermore, this solution does not aim to obtain disease diagnosis results or health status; it is merely an information processing method for processing physiological sensor data, and all steps are implemented by computer devices.

[0154] During the training phase, specifically within the window period after the user's electrodes are implanted as planned, the system initiates a synchronous acquisition process. Specifically, a unified clock trigger is used to achieve millisecond-level synchronous acquisition of invasive EEG signals (such as cortical EEG) and non-invasive EEG signals (such as scalp EEG). During this period, several task-guided synchronous acquisition sessions are scheduled daily to record the user's bimodal signals and corresponding behavioral labels under paradigms such as motor imagery and resting state. Simultaneously, based on the user's preoperative high-resolution structural MRI images and postoperative computed tomography images, a personalized head volume conduction model is constructed through fusion registration and tissue segmentation, and a spatial transfer matrix is ​​calculated to provide anatomical priors for the subsequent construction of physical consistency loss. Subsequently, using the training method described in Example 1, the bi-branch model is jointly pre-trained: the first model extracts a high-precision first set of neural activity patterns and a first average probability distribution as supervised ground truth; the second model learns to predict a second set of neural activity patterns and a second average probability distribution from fuzzy signals; by minimizing de-temporalization alignment loss and physical consistency loss, the high-quality neurosemantic knowledge contained in the invasive signals is transferred to the second model. Furthermore, based on the mechanism described in Example 1, during the downstream task transfer training phase, for different training tasks such as fine finger grasping and simple hand opening and closing, the optimal fixed time window length for each task is searched offline and fixed, generating a mapping table between task type and target time window length. After training is completed, the first model parameters are discarded or archived, retaining only the optimized EEG decoding model training system, the frozen shared semantic codebook, the downstream task classifiers for each task, and the window mapping table, which are then deployed as personalized decoding assets to edge computing devices.

[0155] In the application phase, after the user removes the intracranial electrodes according to the original clinical plan, they enter a long-term community and home training phase. At this time, the user only needs to wear a lightweight, non-invasive EEG acquisition cap to call up the previously trained personalized model for training. The specific decoding process follows the methods described in Examples 1 and 2 above: when the user receives the "fine grasping" visual cue on the screen, the system identifies the current task type and immediately looks up the corresponding target time window length (e.g., 4 seconds) from the preset mapping table; within this window, the non-invasive EEG signal to be decoded is acquired, input into the EEG decoding model training system, and after mapping by the spatiotemporal encoder and projection classifier, a second neural activity pattern set is output; this set is input into the downstream task classifier corresponding to the "fine grasping" task to generate control commands to drive home-use neural prostheses or functional electrical stimulation devices. Throughout the process, the system continuously monitors the decoding confidence in the background. If performance drift is detected, a lightweight fine-tuning mechanism is automatically triggered. Under the premise of freezing the backbone network and the shared semantic codebook, only the parameters of the downstream task classifier are updated to adapt to the long-term changes in the user's neural state.

[0156] Through the above-described full-process application verification, this invention demonstrates significant technical advantages. Compared with traditional purely data-driven non-invasive brain-computer interface decoding schemes, the core advantage of the model calibrated using the scheme of this invention lies in its construction of a more robust and generalizable neural representation based on short-term synchronous high signal-to-noise ratio ECoG signals, thereby systematically improving the robustness and usability of non-invasive decoding. The microscopic mechanism of this performance improvement is as follows: on the one hand, the de-temporalization alignment loss successfully avoids the temporal ambiguity problem caused by the volume conduction effect of non-invasive signals, enabling the model to stably capture neural intent at the macroscopic pattern set level, rather than falling into the local optimum trap of microscopic waveform alignment; on the other hand, the physical consistency loss forces the model to learn signal conduction patterns that conform to individual anatomical structures, effectively suppressing false spatial associations that violate physiological common sense, making the decoding results not only statistically accurate but also physiologically reliable. More importantly, this invention, by embedding the high-performance decoding capabilities obtained through short-term invasive calibration into a purely non-invasive system, completely solves the pain point of users losing high-quality training methods due to electrode removal. It achieves a seamless transition from short-term invasive enhancement to long-term non-invasive benefits, significantly improving the clinical practical value and accessibility of brain-computer interface technology in the neuroscience field. It should be understood that although this embodiment uses stroke rehabilitation as an example, the technical solution of this invention is also applicable to other clinical or non-clinical scenarios involving short-term invasive electrode implantation, such as epilepsy focus localization monitoring, deep brain stimulation adjustment for Parkinson's disease, and spinal cord injury-assisted communication. As long as the conditions of synchronous acquisition of bimodal signals and cross-modal knowledge transfer are met, they should be covered within the scope of protection of this invention.

[0157] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0158] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A non-invasive EEG decoding model training method based on invasive guidance, characterized in that, include: Acquire synchronously collected invasive and non-invasive EEG signals; The invasive EEG signal is input into a first model to obtain a first set of neural activity patterns and a first average probability distribution; the non-invasive EEG signal is input into a second model that shares a semantic codebook with the first model to obtain a second set of neural activity patterns and a second average probability distribution. Based on the set alignment loss between the first neural activity pattern set and the second neural activity pattern set, and the distribution difference loss between the first average probability distribution and the second average probability distribution, a de-temporalized alignment loss is constructed. Constructing a physical consistency loss based on signal transduction constraints of head anatomy; The second model is pre-trained based on the de-temporalization alignment loss and the physical consistency loss, and the pre-trained second model is then transferred to downstream tasks for training to obtain the third model.

2. The method according to claim 1, characterized in that, The step of inputting the invasive EEG signal into the first model to obtain a first set of neural activity patterns and a first average probability distribution includes: The invasive EEG signal is divided into multiple first short time window signal segments; For each of the first short time window signal segments, features are extracted by the feature encoder of the first model and mapped to a probability distribution on the shared semantic codebook by a projection classifier. The probability distribution corresponding to each of the first short time window signal segments is sampled to obtain the corresponding neural activity pattern identifier, and all the neural activity pattern identifiers are deduplicated to generate the first neural activity pattern set. Calculate the mean of the probability distributions corresponding to all the first short time window signal segments to obtain the first average probability distribution; The process of mapping to a probability distribution on the shared semantic codebook includes: mapping a feature vector to a logical value vector and converting it into a probability distribution on multiple prototypes of the shared semantic codebook through a continuous relaxation sampling function.

3. The method according to claim 1, characterized in that, The step of inputting the non-invasive EEG signal into the second model to obtain a second set of neural activity patterns and a second average probability distribution includes: The non-invasive EEG signal is used as a long time window signal segment. Features are extracted by the spatiotemporal encoder of the second model and mapped to a probability distribution on the shared semantic codebook by a projection classifier. The probability distribution output by the second model is sampled and deduplicated to generate the second set of neural activity patterns; Calculate the mean of the probability distributions corresponding to each acquisition channel output by the second model to obtain the second average probability distribution.

4. The method according to claim 1, characterized in that, The set alignment loss is calculated using the following method: Precision and recall are calculated based on the number of intersections between the first set of neural activity patterns and the second set of neural activity patterns, the total number of the first set of neural activity patterns, and the total number of the second set of neural activity patterns. The set alignment loss is calculated based on the harmonic mean of the precision and the recall.

5. The method according to claim 4, characterized in that, The construction of the de-temporalized alignment loss includes: Calculate the distribution difference loss between the first average probability distribution and the second average probability distribution based on the probability distribution divergence; Based on the set alignment loss and the distribution difference loss, the de-temporalization alignment loss is constructed.

6. The method according to claim 1, characterized in that, The construction process of the head volume conduction model includes: Acquire multimodal medical images of the target object and perform fusion and registration; Based on the fused and registered multimodal medical images, tissue segmentation is performed, and electrical conductivity is assigned to different segmented tissues to construct the head volume conduction model. Based on the head volume conduction model, the theoretical influence weight of each cortical electrode contact on each scalp electrode is calculated, and a spatial transfer matrix is ​​generated.

7. The method according to claim 6, characterized in that, The physical consistency loss constructed based on the signal transduction constraints of the head anatomy includes: Using the elements in the spatial transfer matrix as attenuation influence factors, the difference in the characteristic distributions corresponding to the physically coupled cortical electrodes and scalp electrodes is weighted and summed to obtain the physical consistency loss.

8. The method according to claim 7, characterized in that, The step of using elements in the spatial transfer matrix as the attenuation influence factor to perform a weighted summation of the feature distribution differences corresponding to physically coupled cortical electrodes and scalp electrodes to obtain the physical consistency loss includes: The elements in the spatial transfer matrix are transformed into weighting coefficients by a normalization function. These elements comprehensively reflect the influence of the conductivity and geometry of cerebrospinal fluid, skull, and scalp tissue on signal attenuation. The distance between the non-invasive feature probability distribution corresponding to the i-th scalp electrode and the invasive feature probability distribution corresponding to the j-th dermal electrode is measured using the probability distribution divergence metric. The physical consistency loss is obtained by performing element-wise multiplication of the weight coefficients and the distance, and summing the results across all spatial locations.

9. The method according to claim 1, characterized in that, The process of constructing the shared semantic codebook includes: Acquire individually collected invasive EEG signals and segment them into short-time signal fragments; The short-time signal segment is input into the encoder to obtain a feature vector, and the vector quantization layer searches for the closest original vector in the initialized codebook and replaces it to obtain a neural activity pattern identifier sequence. The neural activity pattern identifier sequence is randomly masked; The neural activity pattern identifiers corresponding to the masked part are reconstructed by the decoder based on the unmasked neural activity pattern identifiers. Based on the difference between the reconstructed neural activity pattern identifier and the original neural activity pattern identifier, the parameters of the codebook are optimized to obtain the shared semantic codebook that is frozen and shared in the first model and the second model.

10. The method according to claim 1, characterized in that, The method also includes the step of determining the optimal time window length during the downstream task transfer training phase: Starting with a preset initial time window length, the policy network predicts the time window adjustment amount based on the decoding uncertainty calculated in real time and the current task type. The application direction of the time window adjustment is constrained according to the current task type, and the current time window length is updated. The task loss is calculated based on the updated current time window length, and the overall loss is calculated by combining the delay penalty term corresponding to the current time window length. By minimizing the comprehensive loss, the optimal fixed time window length for each type of task is determined, and a mapping table between task type and optimal fixed time window length is generated for window invocation during the inference phase.

11. A method for decoding electroencephalogram (EEG) signals, characterized in that, include: Receive non-invasive EEG signals to be decoded; The non-invasive EEG signal to be decoded is input into the EEG signal decoding model, and a set of predicted neural activity patterns is output; wherein, the EEG signal decoding model is a third model trained by the training method as described in any one of claims 1 to 10; The neural activity pattern prediction set is input into the downstream task classifier corresponding to the current task type to generate control instructions.

12. The method according to claim 11, characterized in that, The step of inputting the non-invasive EEG signal to be decoded into the EEG signal decoding model and outputting a set of predicted neural activity patterns includes: The contextual feature vector of the non-invasive EEG signal to be decoded is extracted by the spatiotemporal encoder of the EEG signal decoding model. The context feature vector is mapped to an average probability distribution on a shared semantic codebook using a projection classifier; The most likely neural activity pattern identifier is selected based on the average probability distribution to form the neural activity pattern prediction set.

13. The method according to claim 11, characterized in that, The method also includes a model fine-tuning step: When the decoding confidence level is detected to be lower than a preset threshold or to reach a preset period, the backbone network and shared semantic codebook of the EEG signal decoding model are frozen. Newly acquired, task-tagged, calibrated non-invasive EEG signals are acquired, and the parameters of the downstream task classifier are fine-tuned only.

14. A non-invasive EEG decoding model training system based on invasive guidance, characterized in that, include: The signal acquisition module is used to acquire synchronously collected invasive and non-invasive EEG signals. The semantic mapping module is used to input the invasive EEG signal into a first model to obtain a first set of neural activity patterns and a first average probability distribution; and to input the non-invasive EEG signal into a second model that shares a semantic codebook with the first model to obtain a second set of neural activity patterns and a second average probability distribution. The loss construction module is used to construct a de-temporalized alignment loss based on the set alignment loss of the first neural activity pattern set and the second neural activity pattern set and the distribution difference loss of the first average probability distribution and the second average probability distribution. And a physical consistency loss is constructed based on the signal transduction constraints of the head anatomy. The model training module is used to pre-train the second model based on the de-temporalization alignment loss and the physical consistency loss, and to perform downstream task transfer training on the pre-trained second model to obtain the third model.

15. A brainwave signal decoding system, characterized in that, include: The task identification module is used to obtain the current task type; The window determination module is used to determine the corresponding target time window length from a preset mapping table based on the current task type. The mapping table contains the correspondence between task types and optimal fixed time window lengths. A signal decoding module is used to acquire non-invasive EEG signals to be decoded within the target time window length, input the non-invasive EEG signals to be decoded into an EEG signal decoding model, and output a set of neural activity pattern predictions; wherein, the EEG signal decoding model is a third model trained by the training method as described in any one of claims 1 to 10. The instruction generation module is used to input the neural activity pattern prediction set into a downstream task classifier corresponding to the current task type to generate control instructions.

16. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which enable the at least one processor to perform the non-invasive EEG decoding model training method based on invasive guidance as described in any one of claims 1 to 10, or the EEG signal decoding method as described in any one of claims 11 to 13.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the non-invasive EEG decoding model training method based on invasive guidance as described in any one of claims 1 to 10, or the EEG signal decoding method as described in any one of claims 11 to 13.

18. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the non-invasive EEG decoding model training method based on invasive guidance as described in any one of claims 1 to 10, or the EEG signal decoding method as described in any one of claims 11 to 13.

Images