Multi-nerve regulation and control system based on digital twinning and multiple modes

Abstract

The embodiment of the invention provides a multi-nerve regulation and control system based on digital twinning and multiple modes, and relates to the field of neuroscience and artificial intelligence. The system comprises a data acquisition unit used for acquiring multi-modal neural signal data of a patient; the model construction unit is used for constructing a nervous system state prediction model by using the improved incremental transfer learning neural network; the illness state prediction unit is used for inputting the multi-mode neural signal data into the nervous system state prediction model to obtain the nervous system illness state change state of the patient; the closed-loop modulation unit is used for performing closed-loop nerve modulation on the patient based on the nervous system state; and the man-machine interaction unit is used for constructing a personalized virtual 3DCAD human brain model by using a digital twinning technology, displaying the patient condition of the patient in real time and performing a closed-loop neural modulation process on the patient. Through the scheme, the technical problems of complex training and low efficiency in the prior art are relieved.

Description

Technical Field

[0001] This disclosure relates to the fields of neuroscience and artificial intelligence, and in particular to a multimodal neural modulation system based on digital twins and multimodality. Background Technology

[0002] In the fields of neuroscience and medical technology, the precise modulation of the brain and nervous system has always been a research hotspot and challenge. Traditional neuromodulation methods, such as deep brain stimulation (DBS) via surgical electrode implantation, have achieved certain therapeutic effects, but they suffer from limitations such as high invasiveness, high risk, high cost, and irreversibility. With the rapid development of non-invasive medical technologies, non-invasive neuromodulation techniques are gradually becoming a new research direction. These techniques can stimulate and modulate the nervous system through external devices without damaging human structures, and have broad application prospects.

[0003] However, the realization of non-invasive neuromodulation technology faces many challenges, one of which is the issue of data acquisition and processing. Medical data is often subject to strict access restrictions, and due to ethical and privacy factors, it is difficult to acquire and label large amounts of sample data, making it particularly difficult to train complex and efficient deep learning models. In addition, the nervous system states of different patients vary, and their responses to stimuli also differ significantly. How to achieve personalized neuromodulation strategies is also an urgent problem to be solved. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this disclosure provides a multi-modal neuromodulation system based on digital twins. This addresses the challenges of acquiring and labeling large amounts of sample data, which makes training complex and efficient deep learning models particularly difficult. Furthermore, it addresses the technical problems of varying neurological states and significantly different responses to stimuli among different patients.

[0005] This disclosure provides a multi-modal neural modulation system based on digital twins and multimodality, including: The data acquisition unit is used to collect multimodal neural signal data from patients using a variety of non-invasive biosignal monitoring devices; The model building unit is used to build and train a neural system state prediction model using an improved incremental transfer learning neural network. The disease prediction unit is used to input the multimodal neural signal data into the nervous system state prediction model to obtain the patient's nervous system disease status changes. A closed-loop modulation unit is used to perform closed-loop neural modulation on the patient based on the nervous system state. The human-computer interaction unit is used to construct a personalized virtual 3D CAD human brain model using digital twin technology, display the patient's condition in real time, and perform closed-loop neural modulation on the patient.

[0006] In one possible implementation, the plurality of non-invasive biosignal monitoring devices include: an electroencephalogram (EEG) monitoring device, an electrocardiogram (ECG) monitoring device, and an electromyogram (EMG) monitoring device. The multimodal neural signal data includes: deep brain electrical stimulation data, spinal cord electrical stimulation data, vagus nerve stimulation data, gastric electrical stimulation data, and sacral nerve stimulation data.

[0007] In one possible implementation, the step of constructing and training a neural system state prediction model using an improved incremental transfer learning neural network includes: By injecting small samples of neural signal data into a pre-trained speech signal model based on existing training, a transfer learning model is obtained. Based on the transfer learning model, incremental learning technology is introduced, and new neural signal data obtained subsequently is injected and trained through incremental learning to construct a neural system state prediction model.

[0008] In one possible implementation, the neural system state prediction model includes: a data preprocessing layer, a pre-trained model, a feature transfer mapping layer, a neural signal classifier, an incremental input layer, an incremental learning layer, and an incremental classifier; wherein, The data preprocessing layer is used to preprocess the multimodal neural signal data to obtain preprocessed data. The pre-trained model is a convolutional neural network, used to extract feature vectors from the preprocessed data; The feature transfer mapping layer is used to transfer and map the feature vector to a transferred feature vector in the neural signal feature space using a fully connected layer. The neural signal classifier is used to classify the transfer feature vector using a fully connected layer to obtain a classified feature vector. The incremental input layer is used to preprocess the incremental neural signal data to obtain preprocessed incremental neural signal data. The incremental learning layer is used to incrementally learn and update the classified feature vector using the preprocessed incremental neural signal data to obtain the updated feature vector. The incremental classifier is used to classify the updated feature vector using a fully connected layer to obtain the classification result of the neural signal.

[0009] In one possible implementation, the data preprocessing layer is used to preprocess the multimodal neural signal data to obtain preprocessed data, including: The input raw multimodal neural signal data Perform moving average noise reduction; Time-shift data augmentation is used to enhance the denoised original multimodal neural signal data. Various transformations are performed to generate new training samples; The newly generated training sample data is scaled to a fixed range and normalized using min-max normalization to obtain the preprocessed data. ; where the input raw data X and the output preprocessed data are... The dimension is .

[0010] In one possible implementation, the pre-trained model is a convolutional neural network, used to extract feature vectors from the preprocessed data, including: The preprocessed data The input is fed into a pre-trained speech signal model to extract a feature vector F. The dimension is .

[0011] In one possible implementation, the feature transfer mapping layer is used to transfer and map the feature vector to a transferred feature vector in the neural signal feature space using a fully connected layer, including: In the formula, W1 is the first weight matrix with dimension 1. b1 is the first bias vector, with dimension . σ is the activation function; F is the feature vector input to the feature transfer mapping layer; The mapped feature vector has a dimension of .

[0012] In one possible implementation, the neural signal classifier is used to classify the transfer feature vector using a fully connected layer to obtain a classified feature vector, including: In the formula, W2 is the second weight matrix with dimension 1. b2 is the second bias vector, with dimension . Y is the feature vector after classification, with dimension . ; In one possible implementation, the incremental learning layer is used to incrementally learn and update the classified feature vector using the preprocessed incremental neural signal data to obtain the updated feature vector, including: In the formula, W3 is the third weight matrix with dimension 1. b3 is the third bias vector, with dimension . α is the learning rate. To increase the amount of new neural signal data The data output after being input to the incremental input layer has the following dimensions: the dimensions of the input features and the output features of the incremental input layer are... ; The updated feature vector has a dimension of .

[0013] In one possible implementation, the incremental classifier is used to classify the updated feature vector using a fully connected layer to obtain a classification result for the neural signal, including: In the formula, W4 is the fourth weight matrix with dimension 1. b4 is the fourth bias vector, with dimension . ;Y new The classification result of the neural signal, with dimension . .

[0014] The aforementioned multimodal neuromodulation system based on digital twins combines multimodal biosignal monitoring with an improved incremental transfer learning neural network. Multimodal biosignal monitoring comprehensively captures the patient's neural activity information, providing rich data support for neuromodulation. Meanwhile, machine learning techniques such as transfer learning and incremental learning, with limited data samples, continuously improve model performance through knowledge transfer and continuous learning, achieving precise neuromodulation. This alleviates the technical problems of existing technologies being limited by sample data, resulting in complex training and low efficiency. By constructing personalized virtual human brain models and combining multimodal biosignal monitoring with advanced machine learning algorithms, precise, non-invasive, and automated modulation of the patient's nervous system can be achieved, providing new ideas and methods for neuroscience research and clinical applications. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of a multi-neural modulation system based on digital twins and multimodality, provided as an embodiment of this disclosure.

[0017] Figure 2 This is a schematic diagram of an improved incremental transfer learning neural network provided in an embodiment of the present disclosure. Detailed Implementation

[0018] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0019] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them. It should also be understood that in the embodiments of this disclosure, "multiple" can refer to two or more, and "at least one" can refer to one, two, or more. It should also be understood that any component, data, or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless explicitly limited or given a contrary indication in the context. Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this disclosure generally indicates that the related objects before and after are in an "or" relationship. It should also be understood that the descriptions of the various embodiments in this disclosure emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be elaborated upon one by one.

[0020] Furthermore, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn to actual scale. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use. Techniques, methods, and apparatus known to those skilled in the art will not be discussed in detail, but where appropriate, such techniques, methods, and apparatus should be considered part of the specification. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0022] This disclosure proposes a multi-modal neuromodulation system based on digital twins and multimodality. This technology aims to achieve precise, non-invasive, and automated modulation of the patient's nervous system by constructing a personalized virtual human brain model and combining multimodal biosignal monitoring and advanced machine learning algorithms, providing new ideas and methods for neuroscience research and clinical applications.

[0023] By combining various non-invasive biosignal monitoring methods (such as electroencephalography, electrocardiography, electromyography, etc.) to collect multimodal neural signal data (such as deep brain stimulation (DBS), spinal cord stimulation (SCS), vagus nerve stimulation (VNS), gastric stimulation (GES), sacral nerve stimulation (SNS), etc.), and using the improved incremental transfer learning neural network developed in the research to construct a neural system state prediction model, the patient's condition changes can be predicted in real time based on feedback.

[0024] Closed-loop neuromodulation is a neuromodulation method that automatically adjusts intervention parameters in real time based on changes in biomarkers and through data processing algorithms. It establishes feedback and stimulation loops to achieve automated and precise control through simultaneous monitoring and non-invasive intervention. This treatment method is low-risk, reversible, adjustable, and non-invasive. It not only provides controllable stimulation to neurons but also collects and analyzes real-time signals from the target and adjusts the stimulation pattern according to the target's state.

[0025] Due to limited access to medical data and the inability to acquire massive amounts of samples, the improved incremental transfer learning neural network introduces transfer learning techniques to address these issues. Since speech and neural signal waveforms are similar, the algorithm uses a pre-trained speech signal model based on existing training, injects a small sample of neural signal data, and trains it through transfer learning to obtain a transfer learning model. Simultaneously, the improved incremental transfer learning neural network also incorporates incremental learning techniques. Based on the transfer learning model, it injects newly acquired neural signal data and trains it through incremental learning to obtain an incremental learning model, which serves as a neural system state prediction model, reducing the time cost of repeatedly training with old data.

[0026] Figure 1 A schematic diagram of a multi-modal neural modulation system based on digital twins and multimodality is provided for embodiments of this disclosure; Figure 2 This is a schematic diagram illustrating the structure of an improved incremental transfer learning neural network provided in an embodiment of this disclosure. Figure 1 and 2 As shown, a multi-neural modulation system 100 based on digital twins and multimodality includes: The data acquisition unit 110 is used to acquire multimodal neural signal data of patients using a variety of non-invasive biosignal monitoring devices; Among them are various non-invasive biosignal monitoring devices, including: electroencephalogram (EEG) monitoring devices, electrocardiogram (ECG) monitoring devices, electromyogram (EMG) monitoring devices, etc. The multimodal neural signal data includes: deep brain stimulation (DBS) data, spinal cord stimulation (SCS) data, vagus nerve stimulation (VNS) data, gastric electrical stimulation (GES) data, and sacral nerve stimulation (SNS) data.

[0027] Model building unit 120 is used to build and train a neural system state prediction model using an improved incremental transfer learning neural network; Specifically, the model building unit 120 constructs a neural system state prediction model by: injecting a small sample of neural signal data into a pre-trained speech signal model based on existing training, and training it through transfer learning to obtain a transfer learning model; and based on the transfer learning model, simultaneously introducing incremental learning technology, injecting newly obtained neural signal data, and training it through incremental learning to construct a neural system state prediction model.

[0028] The condition prediction unit 130 is used to input the multimodal neural signal data into the nervous system state prediction model to obtain the patient's nervous system condition change status. The closed-loop modulation unit 140 is used to perform closed-loop neural modulation on the patient based on the nervous system state. The human-computer interaction unit 150 is used to construct a personalized virtual 3D CAD human brain model using digital twin technology, and to display the patient's condition and the closed-loop neural modulation process for the patient in real time.

[0029] In this context, a digital twin refers to a virtual copy or model of a physical entity, interconnected through real-time data exchange. A digital twin can simulate the state of its physical counterpart in real time, and vice versa. This disclosure utilizes digital twin technology to construct a personalized virtual 3D CAD model of the human brain, which can improve stimulation effects and individual adaptability, and display the patient's condition in real time through a human-computer interaction interface.

[0030] like Figure 2 As shown, the neural system state prediction model includes: a data preprocessing layer, a pre-trained model, a feature transfer mapping layer, a neural signal classifier, an incremental input layer, an incremental learning layer, and an incremental classifier; among which, A data preprocessing layer is used to preprocess the multimodal neural signal data to obtain preprocessed data. The pre-trained model is a convolutional neural network (CNN) used to extract feature vectors from the preprocessed data; A feature transfer mapping layer is used to transfer and map the feature vector to a transferred feature vector in the neural signal feature space using a fully connected layer. A neural signal classifier is used to classify the transferred feature vector using a fully connected layer to obtain a classified feature vector. The incremental input layer is used to preprocess the incremental neural signal data to obtain preprocessed incremental neural signal data. The incremental learning layer is used to incrementally learn and update the classified feature vector using the preprocessed incremental neural signal data to obtain the updated feature vector. An incremental classifier is used to classify the updated feature vector using a fully connected layer to obtain the classification result of the neural signal.

[0031] Furthermore, in some embodiments, the data preprocessing layer is used to preprocess the multimodal neural signal data to obtain preprocessed data, specifically including: The input raw multimodal neural signal data Perform moving average denoising; the moving average denoising formula is: in, The original signal is at the location The value; The denoised signal is located at... The value; The size of the sliding window, with a range of positive integers, typically set to [value missing]. wait.

[0032] Time-shift data augmentation is used to enhance the denoised original multimodal neural signal data. Various transformations are performed to generate new training samples; the time shift formula is as follows: For the original signal at position value Time shift The translated characteristic quantity This is the time shift value, which can be either a positive or negative integer, and is typically set to a value of [value missing]. wait.

[0033] The newly generated training sample data is scaled to a fixed range and normalized using min-max normalization to obtain the preprocessed data. ; where the input raw data X and the output preprocessed data are... The dimension is .

[0034] The minimum-maximum normalization formula is as follows: In the formula, The minimum value of the original signal; : The maximum value of the original signal.

[0035] Furthermore, in some embodiments, the pre-trained model is a convolutional neural network used to extract feature vectors from preprocessed data, including: Preprocessed data The input is fed into a pre-trained speech signal model to extract a feature vector F. The dimension is .

[0036] In this disclosure, since the waveforms of speech signals and neural signals are similar, the algorithm injects small samples of neural signal data into a pre-trained speech signal model based on existing training, and trains it through transfer learning to obtain a transfer learning model. Simultaneously, the improved incremental transfer learning neural network also introduces incremental learning technology. Based on the transfer learning model, it injects subsequently obtained new neural signal data and trains it through incremental learning to obtain an incremental learning model, reducing the time cost of repeatedly training with old data.

[0037] Furthermore, in some embodiments, the feature transfer mapping layer, used to transfer and map the feature vector to a transferred feature vector in the neural signal feature space using a fully connected layer, includes: In the formula, W1 is the first weight matrix with dimension 1. Typically initialized as normally distributed random values; b1 is the first bias vector with dimension . σ is typically initialized as a zero vector; σ is the activation function, such as ReLU or Sigmoid; F is the feature vector input to the feature transfer mapping layer. The mapped feature vector has a dimension of .

[0038] Furthermore, in some embodiments, the neural signal classifier is used to classify the neural signal transfer feature vector using a fully connected layer to obtain a classified feature vector, including: In the formula, W2 is the second weight matrix with dimension 1. b1 is typically initialized with normally distributed random values; b2 is the second bias vector with dimension 1. Typically initialized as a zero vector; Y is the feature vector after classification, with dimension 1. (here (This refers to the number of categories). Furthermore, in some embodiments, for the incremental input layer, new neural signal data is preprocessed and passed to the transfer learning model: wherein, Input: New incremental neural signal data , dimension ; Output: Data passed to the incremental learning model , dimension .

[0039] Furthermore, in some embodiments, the incremental learning layer is used to incrementally learn and update the classified feature vector using the preprocessed incremental neural signal data to obtain an updated feature vector, including: In the formula, W3 is the third weight matrix with dimension 1. b1 is typically initialized with normally distributed random values; b2 is the third bias vector with dimension 1. α is usually initialized as a zero vector; α is the learning rate, a scalar and usually between 0 and 1, with common values ​​being 0.001, 0.01, 0.1, etc. To increase the amount of new neural signal data The data output after being input to the incremental input layer has the following dimensions: the dimensions of the input features and the output features of the incremental input layer are... ; The updated feature vector has a dimension of .

[0040] Furthermore, in some embodiments, the incremental classifier is used to classify the updated feature vector using a fully connected layer to obtain a classification result for the neural signal, including: In the formula, W4 is the fourth weight matrix with dimension 1. b4 is typically initialized with normally distributed random values; b4 is the fourth bias vector with dimension 1. Y is usually initialized as a zero vector; new The classification result of the neural signal, with dimension . .

[0041] To verify the effectiveness of the improved incremental transfer learning neural network provided in this embodiment, a comparative experiment with a pure convolutional neural network was designed. The experimental parameters are shown in Table 1, and the comparative experimental results are shown in Table 2. Table 1 Experimental parameters Table 2 Comparison of experimental results index Improved Incremental Transfer Learning Neural Network Pure convolutional neural networks accuracy 92.5% 87.2% Accuracy 91.8% 86.0% Recall rate 93.0% 88.0% F1 score 92.4% 87.0% Training time 30 minutes 45 minutes As shown in Table 2, the improved incremental transfer learning neural network provided in this disclosure has the following advantages compared with the pure convolutional neural network in the prior art: Accuracy: The improved incremental transfer learning neural network achieves an accuracy approximately 5.3 percentage points higher than the pure CNN model, indicating that the transfer incremental learning model performs better on classification tasks; Accuracy: The improved incremental transfer learning neural network achieves approximately 5.8 percentage points higher accuracy than the pure CNN model, indicating that the former is more accurate in predicting positive class samples; Recall: The improved incremental transfer learning neural network has a recall rate that is about 5 percentage points higher than that of the pure CNN model, indicating that the former is more effective in identifying all positive samples. F1-score: The improved incremental transfer learning neural network achieves an F1-score approximately 5.4 percentage points higher than the pure CNN model, combining performance in both precision and recall. Training time: The training time of the improved incremental transfer learning neural network is 15 minutes shorter than that of the pure CNN model, indicating that the former is more efficient in terms of data processing and model updates.

[0042] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this disclosure, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in the various embodiments of this disclosure or some parts of the embodiments.

[0043] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0044] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0045] The above description of the disclosed embodiments enables those skilled in the art to make or use this disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-neural modulation system based on digital twins and multimodality, characterized in that, include: The data acquisition unit is used to collect multimodal neural signal data from patients using a variety of non-invasive biosignal monitoring devices; The model building unit is used to build and train a neural system state prediction model using an improved incremental transfer learning neural network. The disease prediction unit is used to input the multimodal neural signal data into the nervous system state prediction model to obtain the patient's nervous system disease status changes. A closed-loop modulation unit is used to perform closed-loop neural modulation on the patient based on the state of the nervous system. The human-computer interaction unit is used to construct a personalized virtual 3D CAD human brain model using digital twin technology, display the patient's condition in real time, and perform closed-loop neural modulation on the patient.

2. The system according to claim 1, characterized in that: in, The various non-invasive biosignal monitoring devices include: electroencephalogram (EEG) monitoring devices, electrocardiogram (ECG) monitoring devices, and electromyogram (EMG) monitoring devices; The multimodal neural signal data includes: deep brain electrical stimulation data, spinal cord electrical stimulation data, vagus nerve stimulation data, gastric electrical stimulation data, and sacral nerve stimulation data.

3. The system according to claim 2, characterized in that: The method of constructing and training a neural system state prediction model using an improved incremental transfer learning neural network includes: By injecting small samples of neural signal data into a pre-trained speech signal model based on existing training, a transfer learning model is obtained. Based on the transfer learning model, incremental learning technology is introduced, and new neural signal data obtained subsequently is injected and trained through incremental learning to construct a neural system state prediction model.

4. The system according to claim 3, characterized in that: The neural system state prediction model includes: a data preprocessing layer, a pre-trained model, a feature transfer mapping layer, a neural signal classifier, an incremental input layer, an incremental learning layer, and an incremental classifier; wherein, The data preprocessing layer is used to preprocess the multimodal neural signal data to obtain preprocessed data. The pre-trained model is a convolutional neural network, used to extract feature vectors from the preprocessed data; The feature transfer mapping layer is used to transfer and map the feature vector to a transferred feature vector in the neural signal feature space using a fully connected layer. The neural signal classifier is used to classify the transfer feature vector using a fully connected layer to obtain a classified feature vector. The incremental input layer is used to preprocess the incremental neural signal data to obtain preprocessed incremental neural signal data. The incremental learning layer is used to incrementally learn and update the classified feature vector using the preprocessed incremental neural signal data to obtain the updated feature vector. The incremental classifier is used to classify the updated feature vector using a fully connected layer to obtain the classification result of the neural signal.

5. The system according to claim 4, characterized in that: The data preprocessing layer is used to preprocess the multimodal neural signal data to obtain preprocessed data, including: The input raw multimodal neural signal data Perform moving average noise reduction; Time-shift data augmentation is used to enhance the denoised original multimodal neural signal data. Various transformations are performed to generate new training samples; The newly generated training sample data is scaled to a fixed range and normalized using min-max normalization to obtain the preprocessed data. ; where the input raw data X and the output preprocessed data are... The dimension is .

6. The system according to claim 5, characterized in that: in, The pre-trained model is a convolutional neural network, used to extract feature vectors from the preprocessed data, including: The preprocessed data The input is fed into a pre-trained speech signal model to extract a feature vector F. The dimension is .

7. The system according to claim 6, characterized in that: The feature transfer mapping layer is used to transfer and map the feature vector to a transferred feature vector in the neural signal feature space using a fully connected layer, including: In the formula, W1 is the first weight matrix with dimension 1. b1 is the first bias vector, with dimension . σ is the activation function; F is the feature vector input to the feature transfer mapping layer; The mapped feature vector has a dimension of .

8. The system according to claim 7, characterized in that: The neural signal classifier is used to classify the transfer feature vector using a fully connected layer to obtain a classified feature vector, including: In the formula, W2 is the second weight matrix with dimension 1. b2 is the second bias vector, with dimension . Y is the feature vector after classification, with dimension . .

9. The system according to claim 8, characterized in that: The incremental learning layer is used to incrementally learn and update the classified feature vector using the preprocessed incremental neural signal data, to obtain the updated feature vector, including: In the formula, W3 is the third weight matrix with dimension 1. b3 is the third bias vector, with dimension . α is the learning rate. To increase the amount of new neural signal data The data output after being input to the incremental input layer has the following dimensions: the dimensions of the input features and the output features of the incremental input layer are... ; The updated feature vector has a dimension of .

10. The system according to claim 9, characterized in that: The incremental classifier is used to classify the updated feature vector using a fully connected layer to obtain the classification result of the neural signal, including: In the formula, W4 is the fourth weight matrix with dimension 1. b4 is the fourth bias vector, with dimension . ;Y new The classification result of the neural signal, with dimension . .

Images