A method and system for predicting the efficacy of vagus nerve electrical stimulation based on deep learning and multi-modal data

Abstract

The present application relates to a kind of vagus nerve electric stimulation efficacy prediction method based on deep learning and multi-modal data, belong to medical artificial intelligence cross technical field.The method of the present application includes: collecting the electroencephalogram of epilepsy patient before vagus nerve electric stimulation and multi-dimensional clinical features;The electroencephalogram is preprocessed, and the multi-dimensional electroencephalogram static timing feature is obtained by using electroencephalogram feature extraction model automatic feature extraction;Based on the multi-dimensional electroencephalogram static timing feature, corresponding dynamic brain network timing feature is constructed, and the multi-dimensional clinical features of patient are fused to obtain multi-modal timing feature;Based on multi-modal timing feature, the seizure reduction rate of patient after vagus nerve electric stimulation is predicted by using deep learning regression model.The present application constructs multi-modal timing feature including dynamic brain network timing feature and multi-dimensional clinical feature, realizes the prediction of curative effect by using sample level and patient level two-stage modeling, improves the prediction accuracy, improves the clinical reference value of prediction result.

Description

Technical Field

[0001] This invention relates to the field of medical artificial intelligence, and in particular to a method and system for predicting the efficacy of vagus nerve electrical stimulation based on deep learning and multimodal data. Background Technology

[0002] Vagus nerve stimulation (VNS) is a neuromodulation method that stimulates the afferent branches of the vagus nerve. It is primarily used to treat drug-resistant epilepsy and has advantages such as high safety and flexible postoperative control. However, this treatment is invasive, and clinical evidence shows significant heterogeneity in VNS efficacy, along with high treatment costs. Therefore, clinical practice aims to predict VNS efficacy before implantation. Existing techniques mainly rely on clinical experience or single-modal data analysis. This involves collecting preoperative resting-state EEG signals, manually extracting relevant EEG features, and using traditional linear models to predict VNS efficacy; or using functional magnetic resonance imaging (fMRI) data to analyze changes in the functional connectivity strength of the patient's default mode network to predict efficacy.

[0003] The main shortcomings of existing technologies are reflected in four aspects: First, the manual feature extraction process relies on expert experience, resulting in poor model reproducibility; second, single-modal data cannot fully reflect the physiological mechanisms of vagal nerve regulation. For example, EEG can only capture cortical electrical activity while ignoring the patient's clinical characteristics, and although fMRI can show brain region activation, its single temporal resolution is insufficient; third, traditional linear models have difficulty handling the nonlinear correlation between neural signals and therapeutic effects, resulting in limited model generalization ability; fourth, existing technologies use a binary classification approach to predict therapeutic effects, dividing data into effective and ineffective based on whether the seizure reduction rate is greater than 50%. This classification method is too general and makes it difficult to distinguish differences for patients with seizure reduction rates around 50%. Summary of the Invention

[0004] Based on the above analysis, this invention aims to disclose a method and system for predicting the efficacy of vagus nerve electrical stimulation based on deep learning and multimodal data. By collecting the electroencephalogram (EEG) signals of epilepsy patients before vagus nerve electrical stimulation, the corresponding dynamic brain network temporal features are obtained, and multi-dimensional clinical features of the patients are fused to obtain multimodal temporal features. Using the multimodal temporal features, a deep learning regression model is used to predict the reduction rate of epileptic seizures after vagus nerve electrical stimulation, thereby achieving the prediction of the efficacy of vagus nerve electrical stimulation.

[0005] On the one hand, this invention provides a method for predicting the therapeutic effect of vagus nerve electrical stimulation based on deep learning and multimodal data, specifically including the following steps: Collecting electroencephalogram (EEG) signals and multidimensional clinical characteristics of epilepsy patients before vagus nerve electrical stimulation; The EEG signals are preprocessed, and the corresponding multidimensional EEG static temporal features are obtained by automatically extracting features using a pre-trained EEG feature extraction model. Based on the aforementioned multidimensional EEG static temporal features, a corresponding dynamic brain network temporal feature is constructed. The dynamic brain network temporal features are fused with the corresponding patient's multidimensional clinical features to obtain multimodal temporal features; Based on the aforementioned multimodal temporal features, a trained deep learning regression model is used to predict the reduction rate of epileptic seizures after vagus nerve electrical stimulation, thereby predicting the efficacy of vagus nerve electrical stimulation.

[0006] Furthermore, the construction of corresponding dynamic brain network temporal features based on the multidimensional EEG static temporal features includes: The multidimensional EEG static temporal features are traversed using a sliding window of a preset time length; Within each time window, a weighted functional connectivity matrix corresponding to that time window is generated based on the static features corresponding to all time points in that time window. For each of the weighted functional connectivity matrices, calculate the corresponding graph theory topology index data; Based on all graph theory topology index data, dynamic brain network temporal features corresponding to the multidimensional EEG static temporal features are formed.

[0007] Furthermore, the graph theory topology metrics include: Local efficiency, global efficiency, modularity, degree centrality, betweenness centrality, clustering coefficient, characteristic path length, and small-world index.

[0008] Furthermore, the process of fusing the dynamic brain network temporal features with the corresponding patient's multidimensional clinical features to obtain multimodal temporal features includes: The patient's multidimensional clinical features are mapped using an independent fully connected layer to obtain the corresponding clinical feature vector; The clinical feature vector is copied in the time dimension and concatenated with the dynamic brain network temporal features in the channel dimension to obtain multimodal temporal features.

[0009] Furthermore, training the deep learning regression model includes: The multimodal temporal features corresponding to multiple epilepsy patients were used as training samples, with each patient corresponding to multiple training samples; the postoperative seizure reduction rate of each patient was used as its corresponding training label. Patient-level grouping is used to ensure that all samples from the same patient belong to the same dataset; the training set, validation set, and test set are divided according to a set proportion of patients. The training includes a sample-level modeling stage and a patient-level modeling stage. In the sample-level modeling stage, all the multimodal temporal features of each patient are used as input to learn an end-to-end mapping from the multimodal temporal features to the postoperative seizure reduction rate of each patient. In the patient-level modeling stage, the statistical features of all sample-level prediction results of each patient are calculated to obtain a comprehensive feature vector of the patient. Based on the comprehensive feature vector of each patient, the mapping relationship from the comprehensive feature vector of the patient to the postoperative seizure reduction rate of each patient is learned.

[0010] Furthermore, the deep learning regression model consists of an attention-enhanced temporal prediction model based on a one-dimensional convolutional network and a bidirectional long short-term memory network, and a gradient boosting regression tree model. The attention-enhanced temporal prediction model based on a one-dimensional convolutional network and a bidirectional long short-term memory network is used for sample-level prediction, taking the multimodal temporal features as input and outputting the first postoperative seizure reduction rate as the sample-level prediction result. The gradient boosting regression tree model is used for patient-level prediction, taking the patient's comprehensive feature vector as input and outputting the second postoperative seizure reduction rate as the final prediction result of the efficacy prediction model.

[0011] Furthermore, the pre-trained EEG feature extraction model employs a variational autoencoder structure; the automated feature extraction using the pre-trained EEG feature extraction model to obtain the corresponding multidimensional EEG static temporal features includes: The signal of each channel of the EEG signal is decomposed into a single-channel signal segment of a preset duration; Each of the single-channel signal segments is separated into multiple frequency band signals with different frequency ranges by filtering; For each frequency band signal, an independent pre-trained variational autoencoder is used to process the corresponding latent variables; The latent variables corresponding to each channel and frequency band signal are combined in chronological order to form the multidimensional static temporal features of the EEG signal.

[0012] Furthermore, the variational autoencoder adopts a structure combining a one-dimensional convolutional neural network with residual blocks.

[0013] Furthermore, the training loss function of the variational autoencoder is composed of the reconstruction error loss of the frequency band signal and the KL divergence regularization loss of the latent variable relative to the standard Gaussian distribution.

[0014] On the other hand, the present invention also discloses a vagus nerve electrical stimulation efficacy prediction system based on deep learning and multimodal data, comprising: The data acquisition module is used to collect electroencephalogram (EEG) signals and multidimensional clinical characteristics of epilepsy patients before vagus nerve electrical stimulation. The data preprocessing module preprocesses the EEG signals and uses a pre-trained EEG feature extraction model to automatically extract the corresponding multidimensional static temporal features of the EEG. A multimodal temporal feature fusion module is used to construct corresponding dynamic brain network temporal features based on the multidimensional EEG static temporal features; and to fuse the dynamic brain network temporal features with the multidimensional clinical features of the corresponding patient to obtain multimodal temporal features; The efficacy prediction module is used to predict the reduction rate of epileptic seizures after vagus nerve electrical stimulation based on the multimodal temporal features and a trained deep learning regression model, thereby predicting the efficacy of vagus nerve electrical stimulation.

[0015] The present invention can achieve at least one of the following beneficial effects: By using brain network nonstationarity as the core biomarker for predicting VNS efficacy, this study overcomes the neglect of dynamic brain states by traditional static features. Technically, it replaces traditional static aggregated statistics with dynamic functional connectivity network topological features to construct dynamic brain network temporal features and integrates multidimensional clinical features. This comprehensively captures key synchronous dynamic information of brain neurons related to VNS efficacy, greatly enhancing the clinical reference value for predicting VNS efficacy.

[0016] By integrating the dynamic brain network temporal features and multidimensional clinical features of patients, multimodal temporal features are obtained, which more comprehensively and robustly characterize the biomarker features of patients, realize end-to-end joint optimization of "dynamic brain network + static clinical", and significantly improve the model's interpretability and generalization ability.

[0017] By using a pre-trained EEG feature extraction model, multi-dimensional EEG features across frequency bands and brain regions are automatically extracted, eliminating reliance on expert experience. By constructing corresponding dynamic brain network temporal features based on multi-dimensional static EEG temporal features using graph theory topological index data, the therapeutic effect prediction results can be directly linked to brain physiological mechanisms such as neural circuits, comprehensively reflecting the physiological mechanisms of vagal nerve regulation and improving the interpretability of the prediction model.

[0018] By using the statistical features of sample-level prediction results as feature inputs for patient-level prediction, the problem of inconsistent input dimensions caused by the varying number of original EEG signal segments can be solved, thereby enhancing the model's generalization ability and making it robust to differences in equipment and changes in acquisition conditions. Furthermore, combining patient-level prediction with secondary modeling of the patient's comprehensive feature vector to predict the final postoperative seizure reduction rate enables a dimensional transformation from short-term EEG segments to individual therapeutic effects, effectively addressing the data heterogeneity problem caused by uneven EEG segment lengths. Simultaneously, employing a patient-level data isolation strategy ensures the integrity of patient samples, effectively preventing overfitting during training and prediction, and improving prediction accuracy.

[0019] By predicting the reduction rate of postoperative seizures, the gray area caused by the binary classification (effective / ineffective) of traditional prediction methods is avoided, and the reduction rate of postoperative seizures has more clinical reference value.

[0020] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0021] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0022] Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a specific representation of the 24 clinical features in the embodiments of the present invention. Detailed Implementation

[0023] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0024] Method Implementation Examples One embodiment of the present invention discloses a method for predicting the efficacy of vagus nerve electrical stimulation based on deep learning and multimodal data. By constructing multimodal temporal features based on the electroencephalogram (EEG) signals and multidimensional clinical characteristics of epilepsy patients before vagus nerve electrical stimulation, and using a deep learning regression model to predict the reduction rate of epileptic seizures after vagus nerve electrical stimulation, the efficacy of vagus nerve electrical stimulation can be predicted.

[0025] like Figure 1 As shown, the specific steps include S1 to S5.

[0026] S1. Collect EEG signals and multidimensional clinical characteristics of epilepsy patients before vagus nerve electrical stimulation.

[0027] During acquisition, EEG signals are acquired using a clinically standard EEG amplifier with a sampling rate ranging from 200 to 1000 Hz. In this embodiment, a sampling frequency of 500 Hz is preferred.

[0028] Furthermore, in clinical practice, EEG signal monitoring typically lasts for 2 hours or more, and the EEG signal should be acquired as continuously as possible during acquisition. In this embodiment, the EEG signal must include at least 6 minutes of acquisition data.

[0029] During data acquisition, the system strictly follows the international 10-20 standard configuration, consisting of 19 scalp electrodes (FP1 / FP2, F3 / F4, etc.) and bilateral ear electrodes (A1 / A2). The bilateral ear electrodes serve as reference electrodes, and the 19 scalp electrodes correspond to 19 data acquisition channels.

[0030] S2. The EEG signal is preprocessed, and the corresponding multidimensional EEG static temporal features are obtained by automatically extracting features using a pre-trained EEG feature extraction model, specifically including S21-S22.

[0031] S21. Preprocess the EEG signal to obtain the preprocessed EEG signal.

[0032] Specifically, the preprocessing of EEG data from all channels includes the following operations in sequence: power frequency removal, base drift removal, removal of invalid high-frequency components, resampling, removal of motion artifacts and bad segments, and segmentation into standard segments.

[0033] Specifically, a notch filter (e.g., 50Hz) is used to eliminate power frequency noise interference, resulting in an EEG signal after power frequency noise removal.

[0034] Furthermore, removing baseline drift refers to performing a high-pass filter (typically, the cutoff frequency is set to 0.1Hz-1Hz) on the EEG signal after removing power frequency noise to obtain the EEG signal after removing baseline drift.

[0035] Furthermore, the common frequency range of epileptic abnormal discharges in EEG signals is 40-100Hz. Therefore, it is necessary to remove invalid high-frequency components after removing baseline drift. A low-pass filter (exemplarily 100Hz) is used to remove signal components with frequencies higher than 100Hz, resulting in a low-pass filtered EEG signal.

[0036] Furthermore, to reduce the amount of data and improve the calculation speed, the low-pass filtered EEG signal needs to be downsampled. The EEG signal is resampled at a sampling frequency of 200Hz to obtain the resampled EEG signal.

[0037] Furthermore, motion artifacts are removed from the resampled EEG signal, specifically including steps a, b, c, and d: a. Calculate the line length of the EEG signal, which is used to measure the intensity of local fluctuations in the signal. The calculation formula is: ; in, After resampling EEG signal at any given moment; Indicates the length of the EEG signal; This is the window width used in online length calculation, with a value ranging from 0.5 to 1 millisecond; It is an absolute value function; Indicates the time sampling point.

[0038] b. Calculate the line length threshold : ; in, This represents the median length of the line. This indicates a multiple, with a range of [2, 10].

[0039] c. Mark all lines whose length exceeds the threshold. At each time point, adjacent time points exceeding the threshold (interval less than 500 milliseconds) are merged into a continuous pseudo-event, and its start and end times are recorded.

[0040] d. Remove artifacts and retain valid long segments: Based on the markers of artifact events, remove these data segments contaminated by duration from the resampled EEG signal, and retain only continuous long segments of EEG signal with a duration greater than the specified duration, preferably 6 minutes.

[0041] Furthermore, according to the set requirements, the long EEG signal obtained in the previous step is divided into standard segments (in this embodiment, the preferred standard segment length is 6 minutes), thus obtaining the preprocessed EEG signal.

[0042] S22. Automated feature extraction is performed using a pre-trained EEG feature extraction model to obtain the corresponding multidimensional EEG static temporal features.

[0043] The pre-trained EEG feature extraction model employs a variational autoencoder (VAE) structure, which includes an encoder and a decoder. The encoder can be a Transformer, CNN, or LSTM.

[0044] In one embodiment, the variational autoencoder employs a structure combining a one-dimensional convolutional neural network (1D-CNN) with residual blocks. The encoder uses a 1D-CNN as its base module. Specifically, the encoder comprises multiple parallel 1D-CNNs followed by four deep neural network modules, each consisting of a 1D-CNN connecting two residual blocks. It should be noted that, to obtain more stable feature results, each CNN layer in the model is followed by a normalization layer and a ReLU activation layer.

[0045] Furthermore, the decoder, being a structure symmetrical to the encoder, can also be composed of simple linear or convolutional layers combined with normalization and activation layers.

[0046] Specifically, step S22 includes S22-1 to S22-4.

[0047] S22-1. The signal of each channel of the preprocessed EEG signal is decomposed into single-channel signal segments of a preset duration. The preprocessed EEG signal includes EEG signals from 19 channels.

[0048] The preset duration is preferably 1 second.

[0049] S22-2. Each single-channel signal segment is separated into multiple frequency band signals with different frequency ranges by filtering.

[0050] Preferably, each single-channel signal segment is filtered and separated into five frequency bands with different frequency ranges. The five frequency ranges are intervals divided into segments from 1 to 30 Hz, preferably divided into... Wave (1-4Hz), Wave (4-8 Hz), Wave (8-13 Hz), low Wave (13-20 Hz) and high Wave (20-30 Hz).

[0051] S22-3. For each frequency band signal, an independent pre-trained variational autoencoder is used to process the corresponding latent variables.

[0052] Specifically, the variational autoencoder described in this embodiment encodes the input samples into a distribution in the latent space. Then, the decoder uses the data points sampled from the latent space to perform the decompression process. Furthermore, regularization of the latent space is achieved by transforming the encoded latent variables into a Gaussian distribution.

[0053] in, x For input samples; For encoder functions; This represents an approximate posterior distribution; It is a latent variable; For decoder functions; Represents a given latent variable Data generated in time The conditional probability distribution; This indicates the reconstruction of data.

[0054] Specifically, during the pre-training of the EEG feature extraction model, the training loss function for each independent variational autoencoder... It consists of the reconstruction error loss of the frequency band signal and the KL divergence regularization loss of the latent variable relative to the standard Gaussian distribution, specifically expressed as: ； in, x For the input sample, Reconstruct the corresponding input data samples; It reflects the error between the reconstructed data and the input; It is a latent variable. These are latent variables generated by the encoder. It is the prior distribution in the latent space that follows a Gaussian distribution. express KL Divergence reflects the difference between the latent spatial distribution of features and the hypothesized prior distribution (usually a Gaussian distribution).

[0055] Furthermore, during pre-training, reparameterization is used to address the stochastic sampling gradient problem. During training, the distribution of the encoder output needs to be analyzed. (from mean) and standard deviation Random sampling is obtained from the definition. The latent variables are then fed into the decoder. However, the sampling operation is random, and its gradient cannot be directly calculated, which would prevent the gradient from propagating back from the decoder to the encoder. Therefore, the latent variables are recalculated using reparameterization as shown in the following formula: ;in and Let represent the mean and standard deviation of the Gaussian distribution, respectively. It is random noise used to maintain the randomness of z; This represents element-wise multiplication. After reparameterization, the gradient can be bypassed by noise variables. The data is then propagated to the encoder network, thereby enabling training.

[0056] Furthermore, during pre-training, for each frequency band, a hyperparameter traversal method can be used to determine the dimension of the final model's latent space. For each frequency band, there are 1-20 candidate parameters for the latent space dimension. The number of model iterations is set, and the loss value at the end of model training is saved. The final model is selected by balancing the model dimension and the loss value. Preferably, the latent space dimension is 10 dimensions.

[0057] S22-4. Combine the latent variables corresponding to each channel and frequency band signal in chronological order to form the multidimensional static temporal features of the EEG signal.

[0058] Specifically, the latent spatial dimension of each frequency band is 10, that is, each frequency band corresponds to a latent variable with a dimension of 10, and each channel contains five frequency bands. After combination, the latent variable dimension is 50. The EEG signal has a total of 19 channels, which correspond to a multidimensional EEG static temporal feature with a time interval of 1 second and a dimension of 950.

[0059] S3. Construct corresponding dynamic brain network temporal features based on the multidimensional EEG static temporal features. Specifically, this includes S31-S34.

[0060] S31. Traverse the multidimensional EEG static temporal features using a sliding window of a preset time length.

[0061] Specifically, the preferred preset time length is 5 seconds, that is, a 5-second sliding window (step size 1 second) is used to traverse the multidimensional EEG static temporal features. For the multidimensional EEG static temporal features corresponding to the standard segment of EEG data with a duration of 6 minutes (360 seconds) in this embodiment, a total of 356 continuous windows are generated (360 - 5 + 1 = 356).

[0062] S32. Within each time window, generate a weighted functional connection matrix based on the static features corresponding to all time points in that time window.

[0063] Specifically, the functional connection strength is calculated using two nonlinear synchronization indices, Phase Lock Value (PLV) and Coherence, to generate a 19×19 weighted functional connection matrix for each time window.

[0064] S33. Calculate the corresponding graph theory topology index data for each of the weighted functional connection matrices.

[0065] Specifically, each weighted functional connectivity matrix is ​​treated as a node in the graph, and the corresponding graph theory topology metrics are calculated, including: local efficiency, global efficiency, modularity, degree centrality, betweenness centrality, clustering coefficient, feature path length, and small-world index.

[0066] Local efficiency is a node The average inverse shortest path length between all node pairs in the neighbor subgraph. Local efficiency reflects the plasticity and robustness of local neural circuits, i.e., the level of local neural modulation. The calculation formula is: ; in, For local efficiency; For nodes The number of neighbors; For nodes The neighbor subgraph; For nodes and The shortest path length between.

[0067] Global efficiency is the average of the reciprocals of the shortest path lengths between all node pairs, reflecting the brain's information integration capacity. The calculation formula is: ; in, For overall efficiency; This represents the total number of network nodes. For nodes and The shortest path length between, if no path exists. Its reciprocal is 0.

[0068] Modularity measures the strength of a network's modularity, assessing the state of its modular structure (too dispersed or too integrated). The formula is as follows: ; in, For modularity; The connection weight between the two nodes; , They are nodes , The weighted degree; m= This represents the total network weight. , They are nodes , Belongs to module; This is the Kronecker delta indicator function (1 if the two nodes are in the same module, 0 otherwise).

[0069] Degree centrality is a node The weighted degree is the sum of its connection weights, representing the breadth of a node's connections and measuring whether a node is a key hub in a brain region. The calculation formula is: ; in, For nodes Degree centrality.

[0070] Betweenness centrality measures the importance of a node as an information hub, and is calculated using the following formula: ; in, For betweenness centrality; For nodes arrive The total number of shortest paths, For the nodes The number of shortest paths.

[0071] The clustering coefficient reflects the tendency for local triangular closure, indicating coordinated activity in local brain regions. The calculation formula is as follows: ; in, Clustering coefficient; For nodes The number of neighbors; , These represent nodes respectively.

[0072] The characteristic path length is the geometric mean of the shortest path lengths between all node pairs, representing the speed of information transmission in the whole brain. The calculation formula is: ; in, The length of the feature path; This represents the total number of network nodes. For nodes and The shortest path length between.

[0073] The Small World Index quantifies the "high clustering + short path" structural characteristics of a network by comparing the clustering coefficient and feature path length of an actual network with their corresponding values ​​for random and regular networks. It characterizes the overall optimization level of the brain network, and the calculation formula is as follows: ; in, is the small-world index; C is the average clustering coefficient of the actual network; and L is the feature path length of the actual network. The average clustering coefficient of a random network with the same number of nodes and edges as the actual network. This represents the characteristic path length of the corresponding random network.

[0074] S34. Based on all graph theory topology indices, form the dynamic brain network temporal features corresponding to the multidimensional EEG static temporal features.

[0075] Specifically, 356 sets of graph theory topology indicators are calculated from the weighted functional connectivity matrices corresponding to all 356 time windows. Based on these graph theory topology indicators, corresponding dynamic brain network temporal features are formed.

[0076] In steps S2 and S3, multi-dimensional EEG features across frequency bands and brain regions are automatically extracted using a pre-trained EEG feature extraction model, eliminating reliance on expert experience. By constructing corresponding dynamic brain network temporal features based on multi-dimensional static EEG temporal features using graph theory topological index data, the efficacy prediction results can be directly linked to brain physiological mechanisms such as neural circuits, comprehensively reflecting the physiological mechanisms of vagal nerve regulation and improving the interpretability of the prediction model.

[0077] S4. The dynamic brain network temporal features are fused with the corresponding patient's multidimensional clinical features to obtain multimodal temporal features.

[0078] Specifically, there are 24 clinical features, including: gender, age of onset, age of VNS implantation, duration of epilepsy before VNS implantation, presence of multiple seizure types, presence of generalized seizures, presence of epileptiform seizures, presence of partial seizures, presence of tonic seizures, presence of tonic-clonic seizures, presence of myoclonic seizures, presence of absence seizures, presence of atypical absence seizures, presence of atonic seizures, presence of negative myoclonic seizures, presence of a history of syndromes, presence of current syndromes, number of medications previously used, number of medications at baseline, history of status epilepticus, EEG findings, MRI findings, signal quality index grading, and etiology. Figure 2 The image shows a detailed representation of the 24 clinical features.

[0079] S41. Map the patient's multidimensional clinical features using an independent fully connected layer to obtain the corresponding clinical feature vector.

[0080] Specifically, the multidimensional clinical features (24-dimensional static clinical features) consist of 24 clinical features, which are non-linearly encoded through an independent fully connected layer (8 units, ReLU activation) and mapped to an 8-dimensional clinical feature vector.

[0081] S42. Copy the clinical feature vector in the time dimension and concatenate it with the dynamic brain network temporal features in the channel dimension to obtain multimodal temporal features.

[0082] The clinical features are broadcast and replicated in the time dimension, and then concatenated with the dynamic brain network temporal features of 356 time steps to form a 356×(8+8) multimodal temporal feature.

[0083] In step S4, by fusing the temporal features of dynamic brain networks with static clinical features, the patient's biomarker features are characterized more comprehensively and robustly, improving the model's predictive performance and generalization ability.

[0084] S5. Based on the multimodal temporal features, a trained deep learning regression model is used to predict the reduction rate of epileptic seizures after vagus nerve electrical stimulation, thereby predicting the efficacy of vagus nerve electrical stimulation.

[0085] Specifically, the deep learning regression model consists of an attention-enhanced temporal prediction model based on one-dimensional convolutional and bidirectional long short-term memory networks and a gradient boosting regression tree model. The prediction of the deep learning regression model includes two stages: sample-level prediction and patient-level prediction.

[0086] The attention-enhanced temporal prediction model based on a one-dimensional convolutional network and a bidirectional long short-term memory network is used for sample-level prediction. It takes the multimodal temporal features as input and outputs the first postoperative seizure reduction rate as the sample-level prediction result. The gradient boosting regression tree model is used for patient-level prediction. It takes the patient's comprehensive feature vector as input and outputs the second postoperative seizure reduction rate as the final prediction result.

[0087] The attention-enhanced temporal prediction model of a one-dimensional convolutional and bidirectional long short-term memory network consists of a 1D-CNN layer (using 32 convolutional kernels, kernel size = 5, output channels 64), connected to a two-layer bidirectional LSTM (128 units per layer), and dynamically weights the contributions of different time steps through an attention mechanism; finally, the predicted value of the first postoperative seizure reduction rate is output through Dropout (0.5) and a fully connected layer.

[0088] The gradient boosting regression tree model is an ensemble of multiple decision trees, exemplified by an ensemble of 100 decision trees of depth 3. It performs secondary modeling on the comprehensive patient feature vector obtained from the statistical features of all sample-level predictions for each patient, outputting the second postoperative seizure reduction rate as the most accurate prediction. The statistical features include mean, standard deviation, and quartiles.

[0089] Specifically, before practical application, the efficacy prediction model needs to be trained to obtain a trained efficacy prediction model. The training process includes: The multimodal temporal features corresponding to multiple epilepsy patients were used as training samples, with each patient corresponding to multiple training samples; the postoperative seizure reduction rate of each patient was used as its corresponding training label. Patient-level grouping is used to ensure that all samples from the same patient belong to the same dataset; the training set, validation set, and test set are divided according to a set proportion of patients. The training process includes a sample-level modeling stage and a patient-level modeling stage. In the sample-level modeling stage, the multimodal temporal features are used as input to learn the end-to-end mapping from the multimodal temporal features to the postoperative seizure reduction rate. In the patient-level modeling stage, the statistical features of all sample-level prediction results for each patient are calculated to obtain the patient comprehensive feature vector. Based on the patient comprehensive feature vector, the mapping relationship from the patient to the postoperative seizure reduction rate is learned.

[0090] Furthermore, each patient has multiple samples, and since the duration of EEG data collection varies for each patient, the number of samples may also differ between patients. All samples were obtained through steps S1-S4, and there is no overlap between samples.

[0091] Furthermore, when partitioning the training dataset, the GroupShuffleSplit method is used to divide the samples, ensuring that all samples from the same patient (i.e., each patient's sample is an independent group) belong to the same dataset to prevent intra-group data leakage. When partitioning the training, validation, and test sets, the distribution of postoperative seizure reduction rates in each dataset is ensured to be similar to the overall dataset, guaranteeing that the model is fully learned and fairly evaluated across all categories. The preferred ratios for the training, validation, and test sets are 60%, 20%, and 20% of the number of patients, respectively.

[0092] Based on the above training process, a well-trained deep learning regression model is obtained. In specific applications, multiple samples (multimodal temporal features) of the patient to be predicted are collected. Based on all the patient's samples, sample-level prediction results are obtained through sample-level prediction. Based on the statistical features of the sample-level prediction results, the patient's comprehensive feature vector is obtained. Based on the patient's comprehensive feature vector, patient-level prediction is then performed to obtain the final prediction result of the epileptic seizure reduction rate.

[0093] In step S5, statistical features of the sample-level prediction results are selected as feature inputs for patient-level prediction. This addresses the inconsistency in input dimensions caused by the varying number of original EEG signal segments, thereby enhancing the model's generalization ability and making it robust to differences in equipment and acquisition conditions. Combining this with patient-level prediction to perform secondary modeling of the patient's comprehensive feature vector predicts the final postoperative seizure reduction rate. This achieves a dimensional transformation from short-term EEG segments to individual therapeutic effects, effectively solving the data heterogeneity problem caused by uneven EEG segment lengths. Simultaneously, a patient-level data isolation strategy is employed to ensure the integrity of patient samples, effectively preventing overfitting during training and prediction, and improving prediction accuracy. By predicting the postoperative seizure reduction rate, the gray area caused by the binary classification (effective / ineffective) of traditional prediction methods is avoided, making the postoperative seizure reduction rate more clinically valuable.

[0094] This embodiment discloses a method and system for predicting the efficacy of vagus nerve electrical stimulation (VNS) based on deep learning and multimodal data. For the first time, it uses the non-stationarity of brain networks as a core biomarker for VNS efficacy prediction, breaking through the traditional static features' neglect of dynamic brain states. Technically, it replaces traditional static aggregated statistics with dynamic functional connectivity network topological features to construct dynamic brain network temporal features and integrates multidimensional clinical features, comprehensively capturing key synchronous dynamic information of brain nerves related to VNS efficacy, greatly improving the clinical reference value of VNS efficacy prediction.

[0095] This embodiment overcomes the limitations of traditional prediction methods that only focus on single-modal data. It uses clinical features as a global prior and integrates the patient's dynamic brain network temporal features with multidimensional clinical features to obtain multimodal temporal features. This achieves end-to-end joint optimization of "dynamic brain network + static clinical data," significantly improving the model's interpretability and generalization ability. Test results show that without considering only EEG features and excluding clinical features, the Pearson correlation coefficient (r) on the test set is 0.5495, p-value is 9.508e-5, and mean squared error is 0.321. With clinical features included, the Pearson correlation coefficient (r) on the test set is 0.586, p-value is 2.373e-5, and mean squared error is 0.322. It is evident that adding clinical features increases r, indicating a significantly enhanced consistency between the model's predictive trend and the actual therapeutic effect; the smaller p-value indicates stronger statistical significance of the correlation and more reliable results; and the mean squared error remains almost unchanged, indicating that the model improves the consistency of predictive trends without sacrificing overall accuracy. This demonstrates that the vagus nerve electrical stimulation therapeutic method described in this embodiment is scientific and effective.

[0096] System Implementation Examples Another specific embodiment of the present invention discloses a vagus nerve electrical stimulation efficacy prediction system based on deep learning and multimodal data, including a data acquisition module, a data preprocessing module, a multimodal temporal feature fusion module, and an efficacy prediction module.

[0097] The data acquisition module is used to collect electroencephalogram (EEG) signals and multidimensional clinical characteristics of epilepsy patients before vagus nerve electrical stimulation. The data preprocessing module preprocesses the EEG signals and uses a pre-trained EEG feature extraction model to automatically extract the corresponding multidimensional static temporal features of the EEG. A multimodal temporal feature fusion module is used to construct corresponding dynamic brain network temporal features based on the multidimensional EEG static temporal features; and to fuse the dynamic brain network temporal features with the multidimensional clinical features of the corresponding patient to obtain multimodal temporal features; The efficacy prediction module is used to predict the reduction rate of epileptic seizures after vagus nerve electrical stimulation based on the multimodal temporal features and a trained deep learning regression model, thereby predicting the efficacy of vagus nerve electrical stimulation.

[0098] Compared with existing technologies, the beneficial effects of the vagus nerve electrical stimulation efficacy prediction system based on deep learning and multimodal data provided in this embodiment are basically the same as those provided in the method embodiment, and will not be elaborated here.

[0099] It should be noted that the above embodiments are based on the same inventive concept, and any parts not described repeatedly can be referenced from each other.

[0100] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for predicting the therapeutic effect of vagus nerve electrical stimulation based on deep learning and multimodal data, characterized in that, Includes the following steps: Collecting electroencephalogram (EEG) signals and multidimensional clinical characteristics of epilepsy patients before vagus nerve electrical stimulation; The EEG signal is preprocessed, and the corresponding multidimensional EEG static temporal features are obtained by automatic feature extraction using a pre-trained EEG feature extraction model; the pre-trained EEG feature extraction model adopts a variational autoencoder structure. The automated feature extraction using a pre-trained EEG feature extraction model to obtain corresponding multidimensional EEG static temporal features includes: decomposing the signal of each channel of the EEG signal into single-channel signal segments of a preset duration; separating each single-channel signal segment into multiple frequency band signals of different frequency ranges through filtering; processing each frequency band signal using an independent pre-trained variational autoencoder to obtain corresponding latent variables; and combining the latent variables corresponding to each channel and each frequency band signal in chronological order to form the multidimensional EEG static temporal features corresponding to the EEG signal. Constructing corresponding dynamic brain network temporal features based on the multidimensional static EEG temporal features includes: traversing the multidimensional static EEG temporal features using a sliding window of preset time length; within each time window, generating a weighted functional connectivity matrix corresponding to the time window based on the static features corresponding to all time points in the time window; calculating corresponding graph theory topology index data for each weighted functional connectivity matrix; and forming dynamic brain network temporal features corresponding to the multidimensional static EEG temporal features based on all graph theory topology index data. The dynamic brain network temporal features are fused with the corresponding patient's multidimensional clinical features to obtain multimodal temporal features; Based on the aforementioned multimodal temporal features, a trained deep learning regression model is used to predict the reduction rate of epileptic seizures after vagus nerve electrical stimulation, thereby predicting the efficacy of vagus nerve electrical stimulation.

2. The method for predicting the therapeutic effect of vagus nerve electrical stimulation according to claim 1, characterized in that, The graph theory topology metrics include: Local efficiency, global efficiency, modularity, degree centrality, betweenness centrality, clustering coefficient, characteristic path length, and small-world index.

3. The method for predicting the therapeutic effect of vagus nerve electrical stimulation according to claim 2, characterized in that, The process of fusing the dynamic brain network temporal features with the corresponding patient's multidimensional clinical features to obtain multimodal temporal features includes: The patient's multidimensional clinical features are mapped using an independent fully connected layer to obtain the corresponding clinical feature vector; The clinical feature vector is copied in the time dimension and concatenated with the dynamic brain network temporal features in the channel dimension to obtain multimodal temporal features.

4. The method for predicting the therapeutic effect of vagus nerve electrical stimulation according to claim 3, characterized in that, Training the deep learning regression model includes: The multimodal temporal features corresponding to multiple epilepsy patients were used as training samples, with each patient corresponding to multiple training samples; the postoperative seizure reduction rate of each patient was used as its corresponding training label. Patient-level grouping is used to ensure that all samples from the same patient belong to the same dataset; the training set, validation set, and test set are divided according to a set proportion of patients. The training includes a sample-level modeling stage and a patient-level modeling stage. In the sample-level modeling stage, all the multimodal temporal features of each patient are used as input to learn an end-to-end mapping from the multimodal temporal features to the postoperative seizure reduction rate of each patient. In the patient-level modeling stage, the statistical features of all sample-level prediction results of each patient are calculated to obtain a comprehensive feature vector of the patient. Based on the comprehensive feature vector of each patient, the mapping relationship from the comprehensive feature vector of the patient to the postoperative seizure reduction rate of each patient is learned.

5. The method for predicting the therapeutic effect of vagus nerve electrical stimulation according to claim 4, characterized in that, The deep learning regression model consists of an attention-enhanced temporal prediction model based on a one-dimensional convolutional network and a bidirectional long short-term memory network, and a gradient boosting regression tree model. The attention-enhanced temporal prediction model based on a one-dimensional convolutional network and a bidirectional long short-term memory network is used for sample-level prediction, taking the multimodal temporal features as input and outputting the first postoperative seizure reduction rate as the sample-level prediction result. The gradient boosting regression tree model is used for patient-level prediction, taking the patient's comprehensive feature vector as input and outputting the second postoperative seizure reduction rate as the final prediction result of the efficacy prediction model.

6. The method for predicting the therapeutic effect of vagus nerve electrical stimulation according to claim 5, characterized in that, The variational autoencoder adopts a structure combining a one-dimensional convolutional neural network with residual blocks.

7. The method for predicting the therapeutic effect of vagus nerve electrical stimulation according to claim 6, characterized in that, The training loss function of the variational autoencoder consists of the reconstruction error loss of the frequency band signal and the KL divergence regularization loss of the latent variable relative to the standard Gaussian distribution.

8. A vagus nerve electrical stimulation efficacy prediction system based on deep learning and multimodal data, characterized in that, include: The data acquisition module is used to collect electroencephalogram (EEG) signals and multidimensional clinical characteristics of epilepsy patients before vagus nerve electrical stimulation. The data preprocessing module preprocesses the EEG signal and uses a pre-trained EEG feature extraction model to automatically extract the corresponding multidimensional EEG static temporal features; the pre-trained EEG feature extraction model adopts a variational autoencoder structure. The automated feature extraction using a pre-trained EEG feature extraction model to obtain corresponding multidimensional EEG static temporal features includes: decomposing the signal of each channel of the EEG signal into single-channel signal segments of a preset duration; separating each single-channel signal segment into multiple frequency band signals of different frequency ranges through filtering; processing each frequency band signal using an independent pre-trained variational autoencoder to obtain corresponding latent variables; and combining the latent variables corresponding to each channel and each frequency band signal in chronological order to form the multidimensional EEG static temporal features corresponding to the EEG signal. A multimodal temporal feature fusion module is used to construct corresponding dynamic brain network temporal features based on the multidimensional static EEG temporal features; and to fuse the dynamic brain network temporal features with the multidimensional clinical features of the corresponding patient to obtain multimodal temporal features; the construction of corresponding dynamic brain network temporal features based on the multidimensional static EEG temporal features includes: traversing the multidimensional static EEG temporal features with a sliding window of a preset time length; generating a weighted functional connectivity matrix corresponding to the time window based on the static features corresponding to all time points in the time window within each time window; calculating the corresponding graph theory topology index data for each weighted functional connectivity matrix; and forming the dynamic brain network temporal features corresponding to the multidimensional static EEG temporal features based on all graph theory topology index data. The efficacy prediction module is used to predict the reduction rate of epileptic seizures after vagus nerve electrical stimulation based on the multimodal temporal features and a trained deep learning regression model, thereby predicting the efficacy of vagus nerve electrical stimulation.

Images