An electroencephalogram signal acquisition and processing system based on deep learning and digital twinning

Abstract

The application relates to the technical field of electroencephalogram signal acquisition and processing, in particular to an electroencephalogram signal acquisition and processing system based on deep learning and digital twinning, which comprises a physical entity layer, a digital twinning layer and an application service layer; the physical entity layer collects and pre-processes original electroencephalogram signals; the digital twinning layer constructs an initial twinning model based on prior parameters and pre-processed signals, extracts dynamic space-time features by using a first deep learning network to adaptively update the model, makes the model real-time synchronized with the physical entity, and then executes processing tasks such as noise suppression, artifact removal, event detection or intention classification by using a second deep learning network; the application service layer realizes three-dimensional visual display and closed-loop feedback regulation; the system can also accelerate personalized modeling of new users and improve processing accuracy through cross-subject transfer learning; the application deeply integrates deep learning and digital twinning, and improves the precision and adaptive ability of electroencephalogram signal processing.

Description

Technical Field

[0001] This invention relates to the field of electroencephalogram (EEG) signal acquisition and processing technology, and in particular to an EEG signal acquisition and processing system based on deep learning and digital twins. Background Technology

[0002] Electroencephalography (EEG) signals, as important biological signals reflecting the electrophysiological activity of the brain, have been widely used in neuroscience, clinical medicine, brain-computer interfaces, and rehabilitation engineering. While EEG signals offer advantages such as non-invasiveness and high temporal resolution, they also present technical challenges, including weak signals (microvolt levels), susceptibility to various types of noise (power frequency interference, electromyography, electrooculography, motion artifacts, etc.), significant inter-individual variability, and strong non-stationarity.

[0003] Traditional EEG signal processing methods mainly include: denoising techniques based on digital filters (such as bandpass filtering and notch filtering), independent component analysis (ICA), wavelet transform, and artificial classifiers based on feature engineering (such as support vector machines and linear discriminant analysis). These methods typically rely on fixed signal models or manually designed features, making it difficult to adaptively track the dynamic characteristics of EEG signals changing over time and in different states. Furthermore, their generalization ability is limited, especially under different individual and task conditions. For example, while ICA can separate some artifacts, it requires manual identification of components and has low efficiency for real-time online processing; the accuracy of traditional classifiers for tasks such as motor imagery and emotion recognition is limited by the effectiveness of feature representation.

[0004] In recent years, deep learning methods (such as convolutional neural networks, recurrent neural networks, graph convolutional networks, and generative adversarial networks) have been introduced into the field of EEG signal analysis, which has improved the automation of feature extraction and the accuracy of pattern recognition to some extent. However, most existing deep learning-based EEG processing systems are open-loop structures: deep learning models are usually only used to directly infer labels (such as event type and intention category) from the acquired EEG signals, lacking physical modeling and dynamic bidirectional mapping of the signal generation process. These systems suffer from the following shortcomings: Purely data-driven deep learning models struggle to incorporate prior physical information such as the subject's head anatomy and conductivity distribution, resulting in poor transfer performance between individuals. They often require extensive retraining with labeled individual data and lack individualized physical benchmarks. EEG signals are highly non-stationary and susceptible to factors like fatigue, changes in attention, and impedance drift. Statically trained models cannot update synchronously with changes in physical entities, leading to performance degradation over long-term use and weak real-time adaptive capabilities. Existing deep learning methods for artifact removal often rely solely on statistical learning, lacking "reference signals" or "twin benchmarks" driven by real physiological processes. This can easily lead to over-removal or over-retention of artifacts, and noise and artifact processing lack physical constraints. Signal acquisition, preprocessing, feature extraction, and pattern recognition are typically composed of independent modules connected in series, failing to form a closed-loop collaborative feedback between the acquisition and processing ends. This hinders system-level real-time optimization, resulting in a disconnect between the acquisition and processing processes.

[0005] In view of this, we propose a brainwave signal acquisition and processing system based on deep learning and digital twins to solve the existing problems. Summary of the Invention

[0006] The purpose of this invention is to provide a brainwave signal acquisition and processing system based on deep learning and digital twins to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a brainwave signal acquisition and processing system based on deep learning and digital twins, comprising a physical entity layer, a digital twin layer, and an application service layer; the physical entity layer includes brainwave signal acquisition equipment and a signal preprocessing module; the digital twin layer performs bidirectional data mapping with the physical entity layer, and the digital twin layer includes an initial twin model construction module, a model adaptive update module, and a signal intelligent processing module; the application service layer is used to visualize the processing results and generate control commands or feedback signals to the physical entity layer based on the processing results; The EEG signal acquisition device is used to acquire the raw EEG signals of the subject; the signal preprocessing module is connected to the EEG signal acquisition device and is used to perform preliminary filtering and amplification on the raw EEG signals to obtain preprocessed EEG signals; the initial twin model construction module is used to construct an initial digital twin model reflecting the characteristics of the subject's brain activity based on the subject's prior physiological parameters and preprocessed EEG signals; the model adaptive update module has a first deep learning network model embedded in it, which is used to receive the preprocessed EEG signals in real time, extract the dynamic spatiotemporal features in the preprocessed EEG signals through the first deep learning network model, and iteratively update the digital twin model based on the dynamic spatiotemporal features to keep the digital twin model synchronized with the real-time state of the physical entity layer; the signal intelligent processing module has a second deep learning network model embedded in it, which is used to input the real-time synchronized digital twin model and the preprocessed EEG signals into the second deep learning network model to perform at least one processing task, including noise suppression, artifact removal, event detection or intent classification, and generate processing results.

[0008] Furthermore, the EEG signal acquisition device is a multi-channel dry electrode EEG cap or wet electrode EEG cap, and integrates multimodal physiological sensors for synchronous acquisition of electrooculography, electromyography or electrocardiogram signals.

[0009] Furthermore, the signal preprocessing module includes a notch filter, a bandpass filter, and a programmable gain amplifier; the notch filter is used to remove power frequency interference; the bandpass filter is used to extract the effective components of EEG in the 0.5Hz to 50Hz frequency band; and the programmable gain amplifier is used to adaptively adjust the signal amplitude.

[0010] Furthermore, the initial twin model construction module is specifically used to: construct a three-dimensional head volume conductor model based on the subject's head magnetic resonance imaging data or a standard head model; and generate the theoretical potential distribution on the surface of the three-dimensional head volume conductor model by solving the electromagnetic field forward problem using preprocessed EEG signals as source signals, thereby forming the initial digital twin model.

[0011] Furthermore, the first deep learning network model is a spatiotemporal graph convolutional network based on the attention mechanism. The spatiotemporal graph convolutional network uses EEG acquisition channels as graph nodes and spatial distances or functional connectivity strengths between channels as graph edges. It is used to simultaneously learn the topological relationships of preprocessed EEG signals in the spatial dimension and the dynamic evolution patterns in the temporal dimension.

[0012] Furthermore, the model adaptive update module iteratively updates the digital twin model in the following way: it calculates the residual between the dynamic spatiotemporal features extracted by the first deep learning network model and the current state variables of the digital twin model; it uses the residual as a feedback signal to drive the model parameters of the digital twin model to be updated in the direction of minimizing the residual, so that the simulated EEG signal output by the updated digital twin model approaches the preprocessed EEG signal collected at the next moment.

[0013] Furthermore, the second deep learning network model is a generative adversarial network or a variational autoencoder; when performing noise suppression or artifact removal tasks, the second deep learning network model is used to guide the separation of clean EEG signals from the preprocessed EEG signals, using the noise-free simulated EEG signals output by the digital twin model as a reference standard.

[0014] Furthermore, the signal intelligent processing module also includes an event detection unit and an intention classification unit; the event detection unit is used to identify spikes, sharp waves or sleep spindles in the preprocessed EEG signal using a second deep learning network model, and output the event type and the time of occurrence; the intention classification unit is used to classify motor imagery or emotion-related EEG patterns using a second deep learning network model, and output the corresponding control intention category.

[0015] Furthermore, the application service layer includes an augmented reality display unit and a closed-loop feedback unit; the augmented reality display unit is used to overlay the digital twin model and processing results onto the subject's real head image in a three-dimensional visualization form; the closed-loop feedback unit is used to generate tactile, visual or electrical stimulation signals based on the processing results and feed them back to the subject to form a closed-loop control.

[0016] Furthermore, the digital twin layer also includes a cross-subject transfer learning module, which stores a general deep learning feature extractor pre-trained based on EEG data from multiple historical subjects. When the system is used on a new subject, the cross-subject transfer learning module uses the pre-processed EEG signals of the new subject to adjust the last few layers of the general deep learning feature extractor, generating a personalized feature extraction network adapted to the new subject. The personalized feature extraction network is used to accelerate the modeling process of the initial twin model construction module and to improve the accuracy of noise suppression and intent classification of the signal intelligent processing module on the new subject.

[0017] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention establishes an initial twin model construction module and a model adaptive update module within the digital twin layer. The initial twin model is constructed based on the subject's prior physiological parameters and initial EEG signals, providing a physically interpretable benchmark for subsequent processing. Building upon this, the model adaptive update module utilizes an embedded first deep learning network model to extract dynamic spatiotemporal features from the EEG signals in real time and drives iterative updates of the digital twin model parameters through residual feedback. This enables the virtual twin model to synchronize with the EEG signals of the physical entity layer in real time, overcoming the limitation of traditional static models in tracking non-stationary EEG changes. It enhances the individual adaptability and real-time dynamic tracking capability of EEG signal processing, significantly improving the adaptive tracking ability to individual differences, fatigue, attention drift, and other state changes.

[0018] 2. This invention employs a second deep learning network model in the signal intelligent processing module and uses the noise-free simulated EEG signal output by the real-time synchronized digital twin model as a reference standard. Compared to existing deep learning denoising methods that rely solely on statistical learning, this invention introduces a "twin benchmark" generated by a physical model, making the signal separation process both data-driven and physically constrained. This effectively removes artifacts such as power frequency, electrooculography (EOG), and electromyography (EMG) while avoiding excessive attenuation of real EEG components, significantly improving the purity and fidelity of noise suppression and artifact removal, thereby obtaining a clean EEG signal with a higher signal-to-noise ratio and more complete feature preservation.

[0019] 3. The event detection unit and intent classification unit of this invention can identify pathological or physiological events such as spikes, sharp waves, and sleep spindles, as well as classify EEG patterns such as motor imagery and emotions. Since all the above processing is performed within a real-time synchronized digital twin model framework, the state estimation provided by the twin model can serve as a dynamic prior information, effectively reducing the uncertainty space of classification / detection and achieving multi-task intelligent processing and high-precision pattern recognition. Compared to traditional feature engineering or single deep learning models, this invention significantly improves both event detection rate and intent classification accuracy under conditions of few samples and high noise.

[0020] 4. The application service layer of this invention generates control commands or feedback signals to the physical entity layer based on the processing results, introducing a closed-loop feedback unit and an augmented reality display unit. By feeding back the processing results of the digital twin layer to the subject or acquisition device, a closed-loop link of "acquisition → twin update → intelligent processing → feedback adjustment" is formed. On the one hand, it can realize real-time neurofeedback training or rehabilitation stimulation based on EEG status; on the other hand, it can dynamically adjust the parameters of the acquisition end to achieve adaptive optimization of the acquisition process. The augmented reality display unit overlays the three-dimensional visualization of the twin model and processing results onto the real head, constructing a two-way mapping between virtual and real and a closed-loop feedback control mechanism, which facilitates doctors, researchers, or subjects to intuitively understand the dynamic changes of EEG.

[0021] 5. This invention introduces a cross-subject transfer learning module, which stores a general deep learning feature extractor pre-trained based on data from multiple historical subjects. When the system is used on new subjects, only a small amount of EEG data from the new subjects is needed to fine-tune the last few layers of the feature extractor, generating a personalized feature extraction network. This mechanism also brings the following effects: shortens the modeling time of the initial twin model, reduces the dependence on the large amount of source data or time-consuming simulation calculations required for accurate individual modeling; improves the noise suppression and intent classification accuracy of the signal intelligent processing module on new subjects, enabling the system to have rapid adaptability across individuals and tasks, significantly improving its practicality and generalization ability for clinical or daily use; accelerates model construction and task transfer for new subjects, and reduces data dependence.

[0022] 6. This invention is not limited to independent improvements of individual modules. By providing both physical baseline and dynamic state prediction through a digital twin layer, the spatiotemporal features extracted by the first deep learning network are used both to update the twin model and as supplementary information for subsequent intelligent signal processing, reducing redundant feature calculations. A spatiotemporal graph convolutional network is used to model multi-channel EEG as a graph signal, effectively utilizing the spatial topological relationships between channels. This is more consistent with the physiological nature of EEG than traditional two-dimensional matrix input, and the computational parameters are optimized. Simultaneously, generative adversarial networks or variational autoencoders, in conjunction with the twin model, can achieve high-precision denoising and classification while reducing dependence on large-scale labeled data, resulting in improved overall system synergistic gain and computational efficiency. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the structure of an electroencephalogram (EEG) signal acquisition and processing system based on deep learning and digital twins according to the present invention. Detailed Implementation

[0024] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments. Example 1

[0025] like Figure 1 As shown, this embodiment provides a brainwave signal acquisition and processing system based on deep learning and digital twins. The system includes a physical entity layer, a digital twin layer, and an application service layer.

[0026] The physical layer comprises an EEG signal acquisition device and a signal preprocessing module. The EEG signal acquisition device uses a multi-channel dry electrode EEG cap (e.g., 64 channels), worn on the subject's head, to acquire raw EEG signals in real time. The signal preprocessing module is connected to the EEG cap via wires and integrates an analog front-end (AFE) to perform preliminary filtering (0.5Hz high-pass, 50Hz notch filtering) and programmable gain amplification (2000x gain) on the raw signal, outputting a digitized preprocessed EEG signal with a sampling rate set to 500Hz.

[0027] The digital twin layer is deployed on a high-performance computing server (equipped with a GPU accelerator card) and performs bidirectional data mapping with the physical entity layer via the TCP / IP protocol. The digital twin layer includes an initial twin model construction module, a model adaptive update module, and a signal intelligent processing module. These modules work collaboratively to achieve real-time virtual mapping and intelligent analysis of physical EEG signals.

[0028] The application service layer runs on the workstations of doctors or laboratory personnel and includes a visual interface (developed based on Unity3D) and a feedback control unit. The visual interface is used to display the digital twin model and processing results in real time; the feedback control unit generates control commands for the physical entity layer based on the processing results (such as adjusting the acquisition gain and sending electrical stimulation parameters).

[0029] The EEG signal acquisition device employs a multi-channel wet electrode EEG cap (Ag / AgCl electrodes, 32 channels), and integrates a triaxial accelerometer and surface electromyography electrodes for simultaneous acquisition of electrooculography (EOG), electromyography (EMG), and head motion signals. Multimodal sensor data and EEG signals are synchronously sampled using the same data acquisition card (NI USB-6363), with a time synchronization accuracy better than 1ms.

[0030] The signal preprocessing module consists of a notch filter, a bandpass filter, and a programmable gain amplifier. The notch filter uses dual second-order IIR notch filters with a center frequency of 50Hz (or 60Hz, selected according to the local power grid frequency) and a quality factor Q=30, used to remove power frequency interference. The bandpass filter uses a fourth-order Butterworth bandpass filter with a passband frequency of 0.5Hz~50Hz and a stopband attenuation ≥40dB, used to extract the effective frequency band of the EEG. The programmable gain amplifier is based on the AD8424 instrumentation amplifier, with the gain controlled by a digital potentiometer (AD5292) ranging from 1000 to 50000 times, adaptively adjusted according to the signal peak amplitude to prevent saturation. The preprocessed EEG signal is transmitted to the digital twin layer via a USB interface.

[0031] The specific implementation steps of the initial twin model construction module include: First, acquiring the subject's head magnetic resonance imaging (MRI) data (T1-weighted, 1mm resolution). 3 The Freesurfer software was used to extract the boundaries of the scalp, skull, cerebrospinal fluid, gray matter, and white matter, generating a three-dimensional finite element mesh (tetrahedral elements, approximately 2 × 10⁻⁶ nodes). 5If no MRI data is available, a standard head model (such as the Montreal Neurological Institute, MNI152 template) is used for affine registration to the subject's head size. Next, the conductivity of each tissue is set as follows: scalp 0.33 S / m, skull 0.008 S / m, cerebrospinal fluid 1.79 S / m, and brain tissue 0.33 S / m. Using preprocessed EEG signals (selecting initial 5 seconds of resting-state data) as the source signal, the Maxwell's electromagnetic field forward problem is solved using the boundary element method (BEM) or finite element method (FEM) to calculate the theoretical potential value at each electrode location, obtaining the surface potential distribution of the three-dimensional head volume conductor model, which is the initial digital twin model. To accelerate computation, the transfer matrix can be pre-calculated for solving the forward problem, and subsequent matrix multiplication is sufficient to quickly obtain the potential distribution.

[0032] The model adaptive update module embeds the first deep learning network model, and the specific implementation is as follows: The first deep learning network model employs an attention-based spatiotemporal graph convolutional network (AST-GCN). The network structure includes an input layer, graph construction layer, spatial graph convolutional layer, temporal convolutional layer, attention module, and output layer. The input layer receives 32 channels of preprocessed EEG signals from the most recent second (500 sampling points), with a data shape of (32, 500). The graph construction layer uses 32 electrodes as nodes, with node feature vectors representing temporal signal segments from the corresponding channels; edge weights are calculated based on the Euclidean distance (or Pearson correlation coefficient) between channels, forming an adjacency matrix. The spatial graph convolutional layer uses Chebyshev polynomial approximation (order K=3) to implement graph convolution and extract spatial topological features. The temporal convolutional layer uses one-dimensional dilated causal convolution (kernel size 3, dilation factor 2, 4, 8) to capture temporal dependencies. For the attention module, self-attention mechanisms are introduced in both the spatial and temporal dimensions to dynamically adjust graph edge weights and temporal window weights. For the output layer, the fully connected layer outputs a 128-dimensional dynamic spatiotemporal feature vector.

[0033] During the iterative update process, let the current time t be S(t) (including model parameters θ(t) and internal state variables) of the digital twin model. The model adaptive update module performs the following steps: input the preprocessed EEG signal x(t) at time t into AST-GCN and output the feature vector f(t). Calculate the residual between f(t) and the simulated signal y(t) = G(θ(t)) output by the twin model: r(t) = f(t) - Φ(y(t)), where Φ is the mapping function. Use the residual r(t) as a feedback signal to update the parameters of the twin model using the gradient descent method: θ(t+Δt) = θ(t) - η·▽ θ ||r(t)|| 2η is the learning rate, set to 0.001. The updated twin model generates the simulation signal y(t+Δt) for the next time step, making it approximate the sampled x(t+Δt) for the next time step. The above update is performed every 50ms (i.e., 25 sampling points) to achieve real-time synchronization between the virtual and real models.

[0034] For y(t) = G(θ(t)): θ(t) is the set of all adjustable parameters of the digital twin model at time t; these parameters determine the physical behavior and dynamic characteristics of the twin model, including the conductivity tensor of each tissue (scalp, skull, cerebrospinal fluid, brain tissue) in the three-dimensional head volume conductor model, as well as the position, orientation, and intensity (or distribution parameters of multiple dipoles) of the equivalent current dipole; if the twin model adopts a hybrid modeling approach of data-driven and physical models, it also includes the weights and biases of the neural network submodule; the initial value of θ(t) is set by the initial twin model building module, and then iteratively optimized through the model adaptive update module. G is the forward mapping function of the digital twin model. It receives the current parameter θ(t) and calculates and outputs the theoretical potential values ​​at each EEG electrode location based on the physical field equations (such as Maxwell's forward electromagnetic problem) or a hybrid physics-data driven model. In specific implementations, G can be a pre-packaged numerical simulation function (such as a finite element solver or boundary element transfer matrix multiplication) or a differentiable neural network surrogate model (e.g., mapping parameters to time series of electrode potentials). y(t) is the simulated EEG signal output by the digital twin model at time t; it is usually a vector with a length equal to the number of electrode channels (e.g., 32-dimensional), representing the noise-free, idealized EEG potential values ​​predicted by each electrode given the current model parameter θ(t).

[0035] For r(t) = f(t) - Φ(y(t)): f(t) is the dynamic spatiotemporal feature vector extracted from the preprocessed EEG signal x(t) at time t by the first deep learning network model (such as the attention mechanism spatiotemporal graph convolutional network, AST-GCN); this vector has a dimension of 128, comprehensively reflecting the high-dimensional features of the EEG signal at the current time in terms of spatial electrode topology and temporal evolution. Φ is a mapping function (which can be a linear transformation or a shallow fully connected network) used to map the simulated EEG signal y(t) (dimension is the number of channels, such as 32) to the same feature space as the dynamic spatiotemporal feature f(t) (128 dimensions) for subtraction. r(t) is the residual vector (128 dimensions), representing the difference between the features extracted from the real EEG signal and the features mapped from the simulated EEG signal; this residual quantifies the synchronization error between the current digital twin model and the physical entity.

[0036] For θ(t+Δt) = θ(t) - η·▽ θ ||r(t)|| 2Δt represents the time step of a single update. ▽ θ Let ||r(t)|| be the gradient operator (the gradient of the function with respect to the parameter θ). 2 A vector consisting of the partial derivatives with respect to each parameter. ||r(t)|| 2 is the square of the L2 norm of the residual vector r(t) (i.e., the sum of squares of each element); this is a scalar loss function used to measure the degree of mismatch between the simulated signal generated by the current twin model and the real EEG signal; by minimizing this loss, the twin model parameters are adjusted in the direction of reducing residuals.

[0037] The signal intelligent processing module has a second deep learning network model embedded in it, and includes an event detection unit and an intent classification unit.

[0038] For the second deep learning network model: This embodiment uses a Conditional Generative Adversarial Network (cGAN) to achieve noise suppression and artifact removal. The cGAN consists of a generator G and a discriminator D. The generator G takes as input a dual-channel data set, consisting of preprocessed EEG signals (containing noise) and noise-free simulated EEG signals output from a digital twin model. It employs a U-Net structure (4 downsampling layers, 4 upsampling layers, skip connections) and outputs clean EEG signals. The discriminator D takes as input clean EEG signals (generator output or real noise-free data) and outputs a scalar value to distinguish between real and fake data. During training, the noise-free simulated EEG signals output from the digital twin model are used as the reference standard (real label), and the loss function is adversarial loss + L1 reconstruction loss. In practice, only the generator G is used, taking noisy EEG and twin model signals as input, and outputting the separated clean EEG signals.

[0039] The event detection unit utilizes the feature extraction part of the discriminator D in the cGAN (with the last layer removed), followed by a temporal convolutional layer and a softmax classifier to identify spikes (sharp waves) and sleep spindles. The discriminator feature extraction part is pre-trained on a public dataset (such as the TUHEEG anomaly dataset) and then fine-tuned on target subject data. During detection, the sliding window is 500ms long with a step size of 100ms, outputting the event category (spike / sharp wave / spindle) and the center time.

[0040] The intention classification unit, designed for motor imagery tasks, inputs clean EEG signals generated by cGAN into a lightweight EEGNet classifier (convolutional layers + depthwise convolutions + separable convolutions + fully connected layers), outputting the probability distribution of motor imagery for the left hand, right hand, foot, or tongue. With the assistance of a Siamese model, the classifier can utilize the Siamese model to generate a large number of simulated training samples, enhancing its classification robustness.

[0041] The application service layer includes augmented reality display units and closed-loop feedback units.

[0042] The augmented reality display unit utilizes the Microsoft HoloLens 2 head-mounted display. First, real-time images of the subject's head are acquired using the HoloLens' depth camera, and a head coordinate system is established using a SLAM algorithm. Then, a three-dimensional head volumetric conductor model from the digital twin layer, along with its surface potential distribution map (using pseudo-color to represent potential intensity), is overlaid in a semi-transparent manner at the corresponding location on the real head. Simultaneously, event detection results (such as spike locations) are marked with flashing red balls near the corresponding electrodes. Doctors can interact by rotating or zooming the model using gestures.

[0043] The closed-loop feedback unit generates feedback signals based on the intent classification results. For example, in motor imagery brain-computer interface rehabilitation training, if the classifier identifies that the subject imagines "right-hand movement," the closed-loop feedback unit generates a 200Hz vibration through a vibration motor (tactile feedback) attached to the subject's right forearm, while simultaneously displaying a "right hand" icon on the screen (visual feedback). For transcranial electrical stimulation (tDCS) applications, the feedback unit can automatically adjust the stimulation current intensity (0.5mA~2mA) based on real-time EEG characteristics (such as the theta / beta ratio), forming a "collection-processing-stimulation" closed loop to regulate abnormal brain activity.

[0044] The digital twin layer also includes a cross-subject transfer learning module. This module pre-stores a general deep learning feature extractor, which is a multi-layer spatiotemporal graph convolutional network (similar to the structure of the first deep learning network). It has been pre-trained on an EEG dataset containing 100 healthy subjects and 50 patients with epilepsy. The pre-training task is multi-task learning (simultaneously reconstructing signals, classifying events, and estimating source locations).

[0045] When the system is used on a new subject, the operation is as follows: One minute of resting-state EEG data (approximately 30,000 sampling points) is collected from the new subject and preprocessed before being input into the module. The parameters of the first 80% of the general feature extractor layers are frozen, and only the last two layers (fully connected layers) are fine-tuned. The learning rate is set to 0.0001, and the iteration is performed for 50 epochs. Due to the small number of parameters, the fine-tuning process can be completed within 30 seconds. After generating the personalized feature extraction network, its parameters are copied to: the initial Siamese model building module, which utilizes the intermediate layer features of this network to replace the original cumbersome MRI-based modeling process, quickly estimating the equivalent dipole source parameters, reducing the initial model building time from several minutes to less than 10 seconds; and the signal intelligent processing module, which uses the personalized feature extraction network as a front-end feature extractor and connects it to the event detection and intent classification networks, improving the artifact removal accuracy of the new subject by approximately 15% and the intent classification accuracy by approximately 20% (compared to the no-transfer-learning approach).

[0046] Taking a subject undergoing motor imagery brain-computer interface rehabilitation training as an example, the complete workflow of the system includes the preparation stage, twin initialization, real-time synchronization, intelligent processing, application feedback, and cross-subject transfer.

[0047] During the preparation phase, the subject wore a 32-channel wet electrode EEG cap, and the system was activated. The physical layer acquired 30 seconds of resting-state EEG data, which was then transmitted to the digital twin layer.

[0048] In the twin initialization, the initial twin model building module establishes the initial digital twin model based on the standard head model and resting state data (takes about 5 seconds).

[0049] During real-time synchronization, the model adaptive update module starts running, updating the twin model parameters every 50ms to synchronize the virtual model with the real EEG in real time.

[0050] In the intelligent processing, the signal intelligent processing module receives real-time preprocessed EEG and twin model outputs, and performs motor imagery intention classification (left hand / right hand / foot / tongue). The classification results (e.g., "right hand" probability 0.85) are output 10 times per second.

[0051] In the application feedback, the application service layer displays the classification results on the virtual head model of the subject through augmented reality, while the closed-loop feedback unit drives the vibration motor to provide tactile feedback; in addition, the results are also used to control the external robotic arm to realize the "imagination-movement" substitution.

[0052] In cross-subject transfer learning, when a new subject is introduced, the system automatically calls the cross-subject transfer learning module to fine-tune the feature extractor using one minute of data from the new subject, thereby accelerating initialization and improving classification performance.

[0053] The above specific embodiments are merely several preferred embodiments of the present invention. Based on the technical solutions of the present invention and the relevant teachings of the above embodiments, those skilled in the art can make various alternative improvements and combinations to the above specific embodiments.

Claims

1. A brainwave signal acquisition and processing system based on deep learning and digital twins, characterized in that: It includes a physical entity layer, a digital twin layer, and an application service layer. The physical entity layer includes an EEG signal acquisition device and a signal preprocessing module. The digital twin layer performs bidirectional data mapping with the physical entity layer and includes an initial twin model construction module, a model adaptive update module, and a signal intelligent processing module. The application service layer is used to visualize the processing results and generate control commands or feedback signals to the physical entity layer based on the processing results. Electroencephalogram (EEG) signal acquisition equipment is used to acquire raw EEG signals from subjects; The signal preprocessing module is connected to the EEG signal acquisition device and is used to perform preliminary filtering and amplification on the raw EEG signal to obtain the preprocessed EEG signal. The initial twin model construction module is used to construct an initial digital twin model reflecting the characteristics of the subject's brain electrical activity based on the subject's prior physiological parameters and preprocessed EEG signals. The model adaptive update module embeds a first deep learning network model, which is used to receive the preprocessed EEG signals in real time and extract dynamic spatiotemporal features from the preprocessed EEG signals through the first deep learning network model. Based on the dynamic spatiotemporal features, the digital twin model is iteratively updated to keep the digital twin model synchronized with the real-time state of the physical entity layer. The signal intelligent processing module embeds a second deep learning network model, which is used to input the real-time synchronized digital twin model and the preprocessed EEG signals into the second deep learning network model to perform at least one processing task, including noise suppression, artifact removal, event detection or intent classification, and generate processing results. The first deep learning network model is a spatiotemporal graph convolutional network based on the attention mechanism. The spatiotemporal graph convolutional network uses EEG acquisition channels as graph nodes and spatial distances or functional connectivity strengths between channels as graph edges. It is used to simultaneously learn the topological relationships of preprocessed EEG signals in the spatial dimension and the dynamic evolution patterns in the temporal dimension. The second deep learning network model is a generative adversarial network or variational autoencoder; when performing noise suppression or artifact removal tasks, the second deep learning network model is used to guide the separation of clean EEG signals from the preprocessed EEG signals, using the noise-free simulated EEG signals output by the digital twin model as a reference standard.

2. The EEG signal acquisition and processing system based on deep learning and digital twins according to claim 1, characterized in that: The EEG signal acquisition device is a multi-channel dry electrode EEG cap or wet electrode EEG cap, and integrates multimodal physiological sensors for synchronous acquisition of electrooculography, electromyography or electrocardiogram signals.

3. The EEG signal acquisition and processing system based on deep learning and digital twins according to claim 1, characterized in that: The signal preprocessing module includes a notch filter, a bandpass filter, and a programmable gain amplifier; the notch filter is used to remove power frequency interference; the bandpass filter is used to extract the effective components of EEG in the 0.5Hz to 50Hz frequency band; and the programmable gain amplifier is used to adaptively adjust the signal amplitude.

4. The EEG signal acquisition and processing system based on deep learning and digital twins according to claim 1, characterized in that, The initial twin model construction module is specifically used to: construct a three-dimensional head volume conductor model based on the subject's head magnetic resonance imaging data or a standard head model; and generate the theoretical potential distribution on the surface of the three-dimensional head volume conductor model by solving the electromagnetic field forward problem using preprocessed EEG signals as source signals, thereby forming the initial digital twin model.

5. The EEG signal acquisition and processing system based on deep learning and digital twins according to claim 1, characterized in that, The model adaptive update module iteratively updates the digital twin model in the following way: it calculates the residual between the dynamic spatiotemporal features extracted by the first deep learning network model and the current state variables of the digital twin model; it uses the residual as a feedback signal to drive the model parameters of the digital twin model to be updated in the direction of minimizing the residual, so that the simulated EEG signal output by the updated digital twin model approaches the preprocessed EEG signal collected at the next moment.

6. The EEG signal acquisition and processing system based on deep learning and digital twins according to claim 1, characterized in that: The signal intelligent processing module also includes an event detection unit and an intention classification unit. The event detection unit is used to identify spikes, sharp waves, or sleep spindles in the preprocessed EEG signals using a second deep learning network model, and outputs the event type and the time of occurrence. The intention classification unit is used to classify motor imagery or emotion-related EEG patterns using a second deep learning network model, and outputs the corresponding control intention category.

7. The EEG signal acquisition and processing system based on deep learning and digital twins according to claim 1, characterized in that: The application service layer includes an augmented reality display unit and a closed-loop feedback unit. The augmented reality display unit is used to overlay the digital twin model and processing results onto the subject's real head image in a three-dimensional visualization. The closed-loop feedback unit is used to generate tactile, visual, or electrical stimulation signals based on the processing results and feed them back to the subject to form a closed-loop control.

8. The EEG signal acquisition and processing system based on deep learning and digital twins according to claim 1, characterized in that: The digital twin layer also includes a cross-subject transfer learning module, which stores a general deep learning feature extractor pre-trained based on EEG data from multiple historical subjects. When the system is used on a new subject, the cross-subject transfer learning module uses the new subject's pre-processed EEG signals to adjust the last few layers of the general deep learning feature extractor, generating a personalized feature extraction network adapted to the new subject. The personalized feature extraction network is used to accelerate the modeling process of the initial twin model construction module and to improve the accuracy of noise suppression and intent classification of the signal intelligent processing module on the new subject.

Images