A long-term stable brain-computer interface system and method based on unsupervised domain adaptation
Patent Information
- Application Number
- CN202211542205.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-02
- Publication Date
- 2026-08-28
- Estimated Expiration
- 2042-12-02
AI Technical Summary
这些方法得到的脑机借口算法仍然会面临随着时间长久算法鲁棒性下降的问题
[0031]与现有技术相比,本发明具有的有益效果至少包括:
Smart Images

Figure CN116186530B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of invasive action potential neural signal analysis, specifically relating to a long-term stable brain-computer interface system and method based on unsupervised domain adaptation. Background Technology
[0002] Brain-computer interfaces (BCIs) enable direct control of external devices from the brain by decoding motor intentions in neural activity. BCIs have the potential to achieve motor rehabilitation and neuroprosthetics. Currently, motor-related neural signals, including those acquired from the dorsal anterior motor cortex (PMd) and primary motor cortex (M1), have been shown to be closely related to the preparation and execution of muscle movements. They have been widely used in mature BCI experiments and have even been used in neuroprosthetics to successfully simulate limb movements in subjects.
[0003] Dynamics of neural signals. Numerous studies have utilized observed neuronal data to model the brain's internal dynamic systems, extracting denoised low-dimensional hidden states as dynamic features of neural signals, and attempting to estimate motion-related parameters and infer motion intentions based on these dynamic features. This method, based on Linear Dynamics System (LDS), assumes that the current hidden state can be modeled as a linear transformation of past hidden states. Using this LDS-based approach, the intrinsic characteristics of neural signals are described as smooth hidden states, and some methods have achieved more accurate and robust brain-computer interface algorithms.
[0004] Unsupervised domain adaptation (UDA) is a promising problem with broad applications and can be used to achieve long-term robustness of brain-computer interfaces (BCIs). However, BCIs exhibit instability over time due to unavoidable issues such as random flow of brain tissue, neuronal death, and electrode array displacement. The longer a BCI is used, the worse its motor prediction capabilities become, limiting its adoption from laboratory settings to practical applications.
[0005] Popular UDA methods can be broadly categorized into distance minimization methods and adversarial domain adaptation methods. Distance-based methods employ various domain distance metrics and minimize inter-domain discrepancies by reducing inter-domain distances. Adversarial domain adaptation methods, based on deep neural networks, introduce an additional discriminator to obfuscate features from different domains. For domain shifts in neural signals, some related works address brain-computer interface recalibration through supervised retraining or retraining with different types of auxiliary information. However, these methods still face the challenge of declining robustness over time in resulting brain-computer interface algorithms.
[0006] Currently, there is no brain-computer interface system that can achieve long-term stability through unsupervised domain adaptation based on the dynamic characteristics of neural signals. Summary of the Invention
[0007] In view of the above, the purpose of this invention is to provide a long-term stable brain-computer interface system and method based on unsupervised domain adaptation, so as to achieve long-term stable motor control based on neural signals.
[0008] To achieve the above-mentioned objectives, an embodiment provides a long-term stable brain-computer interface system based on unsupervised domain adaptation, wherein the brain-computer interface system includes a motion control model capable of performing motion control based on neural signals;
[0009] The motion control model includes a domain adaptation observation module, a classification module, and a motion decoding module. The domain adaptation observation module is used to encode neural signals in the target domain to extract hidden features. The classification module is used to predict the motion direction based on the hidden features. The motion decoding module is used to predict the motion trajectory based on the hidden features after decoding.
[0010] The domain adaptation observation module can optimize and update parameters using unsupervised domain adaptation learning based on the neural signals of the target domain, so as to ensure the long-term stable motion control of the motion control model.
[0011] Preferably, the process of constructing the motion control model is as follows:
[0012] A pre-trained model is constructed, which includes a source domain observation module, a classification module, and a motion decoding module. The source domain observation module is used to encode source domain neural signals to extract hidden features.
[0013] The parameters of the pre-trained model are optimized based on source domain neural signal samples and using supervised learning.
[0014] The source domain observation module with optimized parameters is used to assist the training of the domain adaptation observation module. The parameters of the pre-trained model are kept unchanged. Based on the source domain neural signal samples and the target domain neural signal samples, the parameters of the domain adaptation observation module are optimized by cross-domain aligned feature space probability division using an unsupervised pre-adaptive learning method.
[0015] The optimized domain-adaptive observation module, classification module, and motion decoding module are used to form a motion control model.
[0016] Preferably, the source domain observation module and the domain adaptation observation module adopt the same structure, including a multilayer perceptron.
[0017] Preferably, the classification module includes a multilayer perceptron and a multi-class activation layer.
[0018] Preferably, the motion decoding module includes a linear dynamic system and a decoding unit. The linear dynamic system extracts continuous kinematic features based on hidden features, and the decoding unit decodes the kinematic features to obtain the motion trajectory prediction result.
[0019] Preferably, the decoding unit includes a recurrent neural network, a fully connected layer, and a Tanh activation function.
[0020] Preferably, when using supervised learning to optimize the parameters of the pre-trained model, the loss function used includes the cross-entropy loss between the motion direction prediction result output by the classification module and the real motion direction label, the reconstruction loss between the motion trajectory prediction result output by the motion decoding module and the real motion trajectory label, and the regularization loss based on the parameter framework of the linear dynamic system.
[0021] Preferably, when using unsupervised pre-adaptive learning to optimize the parameters of the domain adaptation observation module in the training architecture by cross-domain aligned feature space probability scores, the loss functions used include marginal distribution alignment constraint loss and conditional distribution alignment constraint loss.
[0022] Marginal distribution alignment constraint loss L m Represented as:
[0023] L m =||E[φ(h) s )]-E[φ(h t )]||
[0024] Conditional distribution alignment constraint loss L CD Represented as:
[0025]
[0026] Among them, h s This indicates that the hidden features extracted by the source domain observation module based on the source domain neural signal samples, h t The domain-adaptive observation module extracts hidden features from neural signal samples in the target domain. φ(·) represents the mapping operation that maps the hidden features to the regenerating kernel Hilbert space using a kernel method. E(·) represents the expectation, ||·|| represents the distance, k represents the classification index of the motion direction, and l s Indicates the direction of motion of the source domain neural signal sample, l t ′ Indicates the direction of motion of the neural signal sample in the target domain. This represents the hidden features corresponding to the source domain neural signal sample with motion direction k. This represents the hidden features corresponding to the target domain neural signal sample with motion direction k.
[0027] Preferably, the motion direction l of the target domain neural signal samplet ′ =k is the motion direction prediction result output by the classification module. By evaluating the confidence of the motion direction prediction results of the classification module, high-confidence motion direction prediction results and their corresponding target domain neural signal samples are selected to optimize the parameters of the domain adaptation observation module.
[0028] To achieve the above-mentioned objectives, an embodiment provides a long-term stable brain-computer interface method based on unsupervised domain adaptation. The method applies the aforementioned brain-computer interface system and includes the following steps:
[0029] Collect neural signals in the target domain and use a motion control model to control the motion of the neural signals in the target domain, including: using a domain adaptation observation module to encode the neural signals in the target domain to extract hidden features, using a classification module to predict the motion direction based on the hidden features, and using a motion decoding module to predict the motion trajectory based on the hidden features after decoding.
[0030] Based on the neural signals of the target domain, the parameters of the domain adaptation observation module are optimized and updated using an unsupervised domain adaptation learning method to ensure long-term stable motion control of the motion control model.
[0031] Compared with the prior art, the beneficial effects of the present invention include at least the following:
[0032] By aligning the potential dynamic features of the motor cortex with domain adaptation and fixing them in the manifold space, long-term stable neural signal dynamic features can be obtained by fine-tuning the pre-adaptation observation module. This enables unsupervised calibration of the motion control model on long-term unlabeled neural signals. Furthermore, joint distribution alignment is employed, considering both marginal and conditional distribution alignment. Semantic information is explicitly considered during the alignment process, ensuring the accuracy of the motion control model in motion control prediction. Attached Figure Description
[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0034] Figure 1 This is a schematic diagram of the motion control model provided in the embodiment;
[0035] Figure 2 This is a schematic diagram of the structure of the pre-trained model provided in the embodiment;
[0036] Figure 3 This is a structural diagram of each module in the model provided in the embodiment. Detailed Implementation
[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.
[0038] There is a certain correlation between neural signals in the brain's motor cortex and arm movement parameters, which can be applied to neuroprosthetics to realize brain-computer interfaces. However, research shows that due to unavoidable problems such as random flow of brain tissue, neuronal death, and electrode array displacement, the predictive ability of brain-computer interfaces for motor control becomes unstable over time, and the stability decreases with longer usage. To address this technical problem, this embodiment provides a long-term stable brain-computer interface system and method based on unsupervised domain adaptation. Based on long-term (multi-day) collection of neural signals from the brain's motor cortex, a long-term stable semantic-dynamic information of the neural signals is extracted using an unsupervised domain adaptation method to construct a long-term stable motor control model, thereby improving the accuracy of long-term motor prediction.
[0039] The embodiment provides a long-term stable brain-computer interface system based on unsupervised domain adaptation, which includes a motion control model to perform long-term stable motion control based on neural signals. For example... Figure 1 As shown, the motion control model provided in this embodiment includes a domain adaptation observation module, a classification module, and a motion decoding module. The domain adaptation observation module is used to encode the target domain neural signals to extract hidden features h. s The classification module is used to classify the hidden features h of the target domain neural signals. s Predicting motion direction, the motion decoding module is used to predict the hidden features h of the target domain neural signals. s The motion trajectory is predicted through decoding.
[0040] In this embodiment, the motion control model requires parameter optimization before it is applied. The construction process is as follows:
[0041] (a) such as Figure 2 As shown, a pre-trained model is constructed that includes a source domain observation module, a classification module, and a motion decoding module.
[0042] The pre-trained model is also used for motor control based on neural signals. However, unlike the motor control model, the input data is different; the input data for the pre-trained model is the source domain neural signal x. s The input data for the motion control model is the target domain neural signal x. t Source domain neural signal x s and target domain neural signal x tThese are collectively referred to as neural signals x.
[0043] In the pre-trained model, the source domain observation module is used to encode the source domain neural signals to extract hidden features h. t The classification module is used to classify the hidden features h of the source domain neural signals. t Predicting motion direction, the motion decoding module is used to predict the hidden features h of the source domain neural signals. t The motion trajectory is predicted through decoding.
[0044] The source domain observation module and the domain adaptation observation module adopt the same structure and are collectively referred to as the observation module. They are responsible for encoding high-dimensional, noisy neural signals into low-dimensional hidden features h. The observation module can be composed of multilayer sensing mechanisms, such as... Figure 3 As shown, the input neural signal x is first reduced in dimensionality using fully connected neural network layers FC, BN, and PReLU. Then, a flattening layer is introduced to concatenate the neural signal sequence along the time dimension. After that, another fully connected neural network layer FC is introduced to reduce the dimensionality to a fixed-length hidden feature h. This hidden feature h retains the spatial and temporal information of the original neural signal. At the same time, the obtained hidden feature also includes semantic and dynamic information of the motion direction, which is convenient for subsequent prediction of motion direction and trajectory.
[0045] It should be noted that the neural signal sample input to the observation module each time is a neural signal sequence x of duration T. i ={x i,j ,j∈[1,T]}, where i represents the index of the sample, j represents the index of the time step, and x i,j This represents the neural signal collected at time j in the i-th sample. This neural signal sequence corresponds to a motion sequence that forms a motion trajectory.
[0046] To ensure that the hidden feature h has a clear semantic representation (such as direction of motion) and facilitates subsequent feature space alignment, an additional classification task is introduced for the hidden feature h to predict the direction of motion from the latent space (an eight-direction classification task), making the latent space more clustering-oriented. In this embodiment, the classification module can employ a multilayer perceptron and a multi-class activation layer. Figure 3 As shown, this invention employs a single fully connected (FC) neural network, followed by an activation function used to infer the posterior probability of each motion direction category.
[0047] The motion decoding module is used to constrain the hidden feature h and simultaneously characterize the initial point of the motion dynamic trajectory. It includes a linear dynamic system LDS and a decoding unit M. The linear dynamic system LDS is based on the hidden feature h. i Extracting continuous kinematic features Zi ={z i,j ,j∈[1,T]}, this kinematic feature Z i The hidden feature h, representing the reduced dimensionality, already contains complete motion intent information. The decoding unit M decodes the motion based on the kinematic feature Z to obtain the motion trajectory prediction result.
[0048] Treating the hidden feature h as the initial point of the dynamic trajectory, the decoded trajectory is derived from the autonomous linear dynamical system (LDS) through self-evolution starting from the initial point:
[0049]
[0050] in It is the first-order partial derivative of Z, and its physical meaning includes encoding the velocity of motion. For the parameter W of LDS, it can be estimated by using the parameter W as part of the end-to-end model during training.
[0051] like Figure 3 As shown, the decoding unit M comprises a recurrent neural network (RNN), a fully connected layer (FC), and a Tanh activation function connected in sequence. The RNN is a single-layer nonlinear RNN (RNN Unit), and the Tanh activation function is linearly selected. Figure 3 (Not shown in the image).
[0052] (b) Optimize the parameters of the pre-trained model based on source domain neural signal samples and using supervised learning.
[0053] In this embodiment, in order to simultaneously extract hidden features containing semantic information (such as the direction of movement intention) and dynamic information (such as the speed and trajectory of movement) from the neural signals of motor cortex such as PMd and M1 into the hidden feature space, the neural signals collected on the first day are regarded as source domain data and supervised learning is performed on the source domain neural signals to optimize the parameters of the pre-trained model. During supervised learning, the category labels and actual movement trajectories in the source domain are used as supervision signals.
[0054] When using supervised learning to optimize the parameters of the pre-trained model, the loss functions used include the cross-entropy loss between the motion direction prediction results output by the classification module and the real motion direction labels, the reconstruction loss between the motion trajectory prediction results output by the motion decoding module and the real motion trajectory labels, and the regularization loss based on the parameters of the linear dynamic system.
[0055] Specifically, reconstruction loss L rec Represented as:
[0056]
[0057] Regularization loss L reg Represented as:
[0058] L reg =||W||
[0059] Where ||·|| represents the distance, W represents the parameters of the linear dynamical system LDS, and the regularization loss can prevent the kinematic feature Z estimated by the linear dynamical system LDS from producing maximal results.
[0060] Reconstruction loss L rec The weighted derivative of regularization loss and cross-entropy loss is used as the first total loss. By reducing this first total loss, the parameters of the pre-trained model are optimized end-to-end to jointly supervise the training of the parameters of the source domain observation module, classification module, and motion decoding module. During training, we use the Adam optimizer with a fixed learning rate of 0.002.
[0061] (c) Extract the optimized source domain observation module to assist the training of the domain adaptation observation module. Keep the pre-trained model parameters unchanged. Based on the source domain neural signal samples and the target domain neural signal samples, and using unsupervised pre-adaptive learning, optimize the parameters of the domain adaptation observation module by cross-domain aligned feature space probability scores.
[0062] A pre-trained model has been trained using source domain neural signals (day 1 neural signals) through steps (a) and (b). However, the performance of the pre-trained model significantly decreases when directly applied to predict target domain neural signals (neural signals from other days after day 1). This domain drift problem of neural signals is considered to be caused by factors such as neural tissue drift, neuronal death, and electrode array drift in brain-computer interface algorithms over time. To address this issue, an unsupervised domain adaptation method is employed to align the joint probability distribution of the feature space before and after drift, thereby eliminating the domain drift problem in brain-computer interfaces.
[0063] In this embodiment, the domain adaptation problem of joint probability distribution alignment is divided into two sub-problems based on Bayes' theorem: marginal distribution alignment and conditional distribution alignment. Marginal distribution alignment only considers the marginal distribution similarity of the hidden features of the observation module in the source domain neural signals and the target domain neural signals, i.e., minimizing |P(h s )-P(h t )|,h s To input source domain neural signal samples into the source domain observation module for encoding to obtain hidden features h s h tTo input neural signal samples from the target domain into the domain adaptation observation module for encoding hidden features, the example calculates the domain drift of the feature space based on the classic Kernel Maximum Mean Discrepancy (KMMD) algorithm, specifically introducing a marginal distribution alignment constraint loss L. m Represented as:
[0064] L m =||E[φ(h) s )]-E[φ(h t )]||
[0065] Where φ(·) represents the mapping operation that uses the kernel trick to map the hidden features to the regenerated kernel Hilbert space, thereby achieving alignment of the feature space after nonlinear transformation. E(·) represents the expectation, and ||·|| represents the distance. During training, by minimizing L... m Able to reduce h t Relative to h s The amount of drift.
[0066] Marginal distribution alignment only aligns the feature distribution statistics between domains, but not the conditional probability distribution, i.e., it does not align semantic information. To further improve the model's decoding accuracy in the target domain, a conditional distribution alignment constraint loss L is introduced. CD Align the feature distributions in the same category subspaces of the source and target domains respectively, and apply conditional distribution alignment constraint loss L. CD Represented as:
[0067]
[0068] Where k represents the classification index of the direction of motion, l s Indicates the direction of motion of the source domain neural signal sample, l t ′ This indicates the direction of motion of the neural signal sample in the target domain (i.e., the pseudo-label). This represents the hidden features corresponding to the source domain neural signal sample with motion direction k. This represents the hidden features corresponding to the target domain neural signal sample with motion direction k.
[0069] Conditional distribution alignment loss L CD It measures the distance between source and target domain features that have the same labels and pseudo-labels. This is achieved by minimizing L... CD By making the feature distributions of neural signals with the same reverse motion but in different domains (different days) similar, the domain drift of the feature space on the conditional distribution is reduced.
[0070] It should be noted that the target domain only contains neural signals and no motion labels. Therefore, pseudo-labels predicted by the classification module are introduced to replace the true labels in order to align the subspace. However, directly using the pseudo-labels obtained from the classification module C trained on the source domain and performing feature space alignment based on such unreliable pseudo-labels is unlikely to improve performance and may even lead to negative transfer. Therefore, before alignment, the pseudo-labels predicted by the classification module are first... t ′ Confidence assessments are performed, and samples with excessively low confidence are removed to prevent negative impacts on transfer learning. In this example, posterior probabilities are ranked, and only a few target domain neural signal samples with high posterior probabilities (motion direction prediction results) are selected to calculate the conditional alignment distribution constraint loss L. CD .
[0071] The implementation example is based on L m and L CD A weighted summation is used to construct a second total loss. The parameters of the domain adaptation observation module are optimized by minimizing this second total loss, thus achieving unsupervised domain adaptation. In actual training, the pre-trained model is kept constant, and only the domain adaptation observation module is trained. The Adam optimizer and a fixed learning rate of 0.002 are selected during optimization. In confidence selection, the 75% highest posterior probabilities of target domain neural signal samples in each batch are selected for alignment, while the same number of source domain neural signal samples are randomly selected for calculating L. CD This avoids overfitting to source domain neural signal samples.
[0072] (d) The optimized domain adaptation observation module, classification module and motion decoding module are used to form a motion control model.
[0073] After parameter optimization of the domain adaptation observation module, the optimized domain adaptation observation module, along with the classification module and motion decoding module in the pre-trained model, are used to form a motion control model. This motion control model can achieve motion control prediction of neural signals in the target domain.
[0074] In this embodiment, during application, the parameters of the domain adaptation observation module are optimized and updated using an unsupervised domain adaptation learning method based on the target domain neural signals to ensure long-term stable motion control of the motion control model.
[0075] Since the underlying dynamic characteristics of the motor cortex remain unchanged in a long-term stable manifold, by aligning with domain adaptation and fixing this manifold space, only fine-tuning the pre-adaptive observation module is needed to obtain the long-term stable neural signal dynamic characteristics, thereby achieving unsupervised calibration of the motion control model on long-term unlabeled neural signals. Furthermore, joint distribution alignment is employed, simultaneously considering marginal distribution alignment and conditional distribution alignment. Semantic information is explicitly considered during the alignment process, ensuring the accuracy of the motion control model in motion control prediction.
[0076] Experimental Example
[0077] In the experiment, two adult rhesus monkeys (named Jango and Chewie, hereinafter abbreviated as J and C) were trained to perform an eight-directional movement task (center-out task) using their upper limbs. During the process, the monkeys were fixed to chairs with a monitor placed directly in front of them. The monkeys controlled a joystick to move a cursor on a two-dimensional plane from an initial position to a destination. A 96-channel microelectrode array (10×10 matrix arrangement, 4.2×4.2 mm in size) was used. The array was implanted into the motor cortex PMd and M1 areas of the monkeys' brains to conduct the above experiment and collect neural signal data.
[0078] In the experiment, multi-day data was used, specifically three days of neural signal data from C monkeys (D1–D3) and four days of neural signal data from J monkeys (D1–D4). The original neural signal data underwent preprocessing, including amplitude amplification, bandpass filtering, and thresholding to detect signal spikes. The number of spikes was counted every 20ms, and then the 1200ms signal was recorded as a single complete trial. Finally, both the neural and trial signals were standardized to ensure that the neural signal data followed a standard distribution with zero mean and one variance.
[0079] In the experiment, the labeled data from the first day (D1) was used for supervised training of the pre-trained model, and the unlabeled data from the subsequent days (D2 to Dn) was used for unsupervised domain adaptation alignment to directly predict motion information.
[0080] The example measures the performance of the motor control model on real brain signal data from two perspectives: accuracy in predicting motion trajectory (regression task) and accuracy in predicting motion direction (classification task). After training on neural signals on the first day, the pre-trained model undergoes unsupervised domain adaptation and alignment. In the regression task, it predicts the limb motion trajectories of subjects over subsequent days and calculates the coefficient of determination (R²) between the predicted and actual motion trajectories. 2 The higher the coefficient of determination, the more accurate the prediction. In classification tasks, the intended movement direction of the subject is predicted each time (there are eight movement directions in total), and the accuracy (acc) of the predicted direction matching the actual movement direction is calculated.
[0081] The motor control model of this invention was compared with various neural signal decoding methods on the same dataset. The results are shown in Table 1 (c and j are abbreviations for two subjects, Chewie and Jango). The methods compared included linear decoders, Long Short-Term Memory (LSTM) networks, the method of Dyer et al., and the Wild-Gen Adversarial Network (WGAN). The final results show that using the pre-trained model (Ours(before)) and the motor control model (Ours(aligned)) significantly improves the accuracy of regression and classification in the target domain, and the improvement is most significant compared to various classic brain-computer interface algorithms.
[0082] Table 1
[0083]
[0084] The embodiment also provides a long-term stable brain-computer interface method based on unsupervised domain adaptation. This method applies the above-mentioned brain-computer interface system and includes the following steps:
[0085] The target domain neural signals are collected, and the motion control model is used to control the motion of the target domain neural signals, including: encoding the target domain neural signals using the domain adaptation observation module to extract hidden features, predicting the motion direction based on the hidden features using the classification module, and predicting the motion trajectory based on the hidden features after decoding using the motion decoding module.
[0086] Based on the neural signals of the target domain, the parameters of the domain adaptation observation module are optimized and updated using an unsupervised domain adaptation learning method to ensure long-term stable motion control of the motion control model.
[0087] The advantage of this method is that it enables a motion control model trained on only one day's data to achieve long-term stable decoding on unstable neural signals over multiple days.
[0088] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A long-term stable brain-computer interface system based on unsupervised domain adaptation, characterized in that, The brain-computer interface system includes a motion control model capable of performing motion control based on neural signals; The motion control model includes a domain adaptation observation module, a classification module, and a motion decoding module. The domain adaptation observation module is used to encode neural signals in the target domain to extract hidden features. The classification module is used to predict the motion direction based on the hidden features. The motion decoding module is used to predict the motion trajectory based on the hidden features after decoding. The process of constructing the motion control model is as follows: A pre-trained model is constructed, comprising a source domain observation module, a classification module, and a motion decoding module. The source domain observation module encodes source domain neural signals to extract hidden features. The pre-trained model's parameters are optimized using supervised learning based on source domain neural signal samples. The loss functions used in supervised learning for parameter optimization include the cross-entropy loss between the motion direction prediction results from the classification module and the actual motion direction labels, the reconstruction loss between the motion trajectory prediction results from the motion decoding module and the actual motion trajectory labels, and a regularization loss based on the linear dynamic system parameters. The regularization loss prevents the kinematic features estimated by LDS of the linear dynamic system from being compromised. It produces enormous results; The source domain observation module, which extracts and optimizes parameters, assists in the training of the domain adaptation observation module. The parameters of the pre-trained model are kept fixed. Based on the source domain neural signal samples and the target domain neural signal samples, the parameters of the domain adaptation observation module are optimized by cross-domain alignment feature space probability division using an unsupervised domain adaptation learning method to ensure the long-term stable motion control of the motion control model. The loss functions used include marginal distribution alignment constraint loss and conditional distribution alignment constraint loss. Marginal distribution alignment constraint loss Represented as: Conditional distribution alignment constraint loss Represented as: in, This indicates that the hidden features extracted by the source domain observation module based on the source domain neural signal samples, The representation domain adaptation observation module extracts hidden features from neural signal samples in the target domain. This indicates a mapping operation that uses a kernel method to map hidden features to a reproducing kernel Hilbert space. Expressing expectations, Indicates distance, A classification index indicating the direction of motion. Indicates the direction of motion of the source domain neural signal sample. Indicates the direction of motion of the neural signal sample in the target domain. Indicates the direction of motion as k The hidden features corresponding to the source domain neural signal samples, Indicates the direction of motion as k The hidden features corresponding to the target domain neural signal samples; The direction of motion of the target domain neural signal sample The parameters of the domain adaptation observation module are optimized by evaluating the confidence of the motion direction prediction results output by the classification module and selecting high-confidence motion direction prediction results and their corresponding target domain neural signal samples. The source domain observation module and the domain adaptation observation module adopt the same structure, including a multilayer perceptron.
2. The long-term stable brain-computer interface system based on unsupervised domain adaptation according to claim 1, characterized in that, The classification module includes a multilayer perceptron and a multi-class activation layer.
3. The long-term stable brain-computer interface system based on unsupervised domain adaptation according to claim 1, characterized in that, The motion decoding module includes a linear dynamic system and a decoding unit. The linear dynamic system extracts continuous kinematic features based on hidden features, and the decoding unit decodes the kinematic features to obtain the motion trajectory prediction result.
4. The long-term stable brain-computer interface system based on unsupervised domain adaptation according to claim 3, characterized in that, The decoding unit includes a recurrent neural network, a fully connected layer, and a Tanh activation function.
5. A long-term stable brain-computer interface method based on unsupervised domain adaptation, characterized in that, The method, which applies the brain-computer interface system according to any one of claims 1-4, includes the following steps: The target domain neural signals are collected, and the motion control model is used to control the motion of the target domain neural signals, including: encoding the target domain neural signals using the domain adaptation observation module to extract hidden features, predicting the motion direction based on the hidden features using the classification module, and predicting the motion trajectory based on the hidden features after decoding using the motion decoding module. Based on the neural signals of the target domain, the parameters of the domain adaptation observation module are optimized and updated using an unsupervised domain adaptation learning method to ensure long-term stable motion control of the motion control model.
Citation Information
Patent Citations
Motor imagery electroencephalogram transfer learning method based on Wasserstein distance
CN113010013A
Domain-adaptive cross-subject motor imagery electroencephalogram signal recognition system and method
CN113408397A