Epileptic seizure prediction method and system based on deep learning

Abstract

The invention provides an epileptic seizure prediction method and system based on deep learning, and the method comprises the steps: obtaining and preprocessing an electroencephalogram data set of an epileptic, and then dividing the electroencephalogram data set into a pooling data set, a training set, a verification set and a test set; constructing an epileptic seizure prediction model based on a deep learning algorithm; a semi-supervised active learning strategy is adopted, the pooling data set and the training set are utilized to iteratively train the epileptic seizure prediction model, training parameters are saved, the verification set is utilized to verify the prediction effect of the trained model, and an optimal epileptic seizure prediction model is obtained; inputting the test set into the optimal epileptic seizure prediction model for prediction, and obtaining a prediction result; according to the invention, epilepsy prediction is carried out by constructing the time-space coupled neural network, and the prediction result is more accurate; meanwhile, a semi-supervised active learning strategy is adopted, so that the method can adapt to the specificity of a patient; in addition, a model distillation scheme is adopted, the training speed is increased, and the parameter quantity is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electroencephalogram signal processing, in particular to a seizure prediction method and system based on deep learning. BACKGROUND

[0002] Epilepsy is a chronic neurological disease, mainly caused by transient brain dysfunction of sudden super-synchronous abnormality of cerebral nerve cells, with the characteristics of transience, suddenness and repeatability. According to the statistics of WHO, by 2019, nearly 50 million people in the world suffer from epilepsy, about 1%~2% of the world population, accounting for more than 0.5% of the global disease burden. It is estimated that 25% of epilepsy cases can be intervened in the early stage. Therefore, early diagnosis helps to prevent seizures and prevent brain damage, and plays a key role in improving the quality of life of patients with epilepsy.

[0003] Electroencephalogram (EEG) technology uses scalp to record the electrical activity of brain neurons, and records the electroencephalogram by placing a number of electrodes (scalp surface) or probes (intracranial) on the head of the subject according to certain rules. Different electroencephalogram signals reflect the electrical activity of the brain in different states. Epilepsy diagnosis is mainly based on the patient's electroencephalogram record and medical history. In the early stage of epilepsy, a small amount of brain neurons begin to release weak electricity, and the electroencephalogram of epilepsy often shows abnormal waves such as sharp waves, spiky waves and spiky slow complex waves. In clinical practice, neurosurgeons infer whether characteristic waveforms appear in electroencephalogram by extracting the shape, frequency, amplitude and other characteristics of electroencephalogram under the condition of epilepsy activity. However, it is a very time-consuming and boring work for medical experts to diagnose whether there is a seizure by visual inspection based on clinical experience. According to statistics, the misdiagnosis rate is about 30%. Therefore, designing an efficient and accurate automatic classification system plays an important role in diagnosing seizures.

[0004] With the development of artificial intelligence and the improvement of computer processing technology, more and more neural network algorithms are applied to the processing of electroencephalogram signals. From the traditional two-layer neural network structure to the current multi-layer structure, more and more machine learning algorithms are combined with this field, giving birth to one after another new achievements, and also promoting the continuous advancement of this field. At present, clustering (Cluster), support vector machine (SVM), RBF neural network, convolutional neural network (CNN) and other neural network methods have been widely used; CNN has achieved good results in image recognition, language recognition, object detection and other aspects, and researchers have combined CNN with the field of epilepsy research, and have achieved great results.

[0005] In order to realize the automatic prediction of EEG seizures, many deep learning-based methods have been studied in the prior art, most of which aim to utilize the spatio-temporal evolution in electroencephalogram to predict seizures, because the occurrence of seizures is a double result of space and time. For example, the patent document with publication number "CN109674469A" discloses a seizure warning algorithm based on a CNN model, which is based on a convolutional neural network for seizure prediction, but most traditional deep learning networks including this prior art can only capture short-range spatial correlations between EEG channels, because the receptive field of the convolution operator is regular and local. However, the abnormalities in the brain of a patient with epilepsy involve not only the focus of seizures, but also the spatio-temporal correlation. Therefore, the prediction effect in the prior art still needs to be improved. SUMMARY

[0006] In order to overcome the problems of poor effect and low accuracy of traditional deep learning in predicting epilepsy in the prior art, the present application provides a deep learning-based seizure prediction method and system, which predicts epilepsy by constructing a spatio-temporal coupled neural network, and the prediction result is more accurate. At the same time, the present application adopts a semi-supervised active learning strategy, which can adapt to the specificity of patients. In addition, the present application also adopts a model distillation scheme, which improves the training speed and reduces the parameter amount.

[0007] To solve the above technical problems, the technical scheme of the present application is as follows: A deep learning-based seizure prediction method, comprising the following steps: S1: obtaining an electroencephalogram data set of a patient with epilepsy and preprocessing it, and dividing the preprocessed electroencephalogram data set into a pooling data set, a training set, a validation set and a test set; S2: constructing a seizure prediction model based on a deep learning algorithm; first, constructing a student model and a residual neural network teacher model, and based on a model distillation framework, using the residual neural network teacher model to pretrain the student model; The seizure prediction model comprises a gating neural network and a pre-trained student model arranged side by side, and a graph neural network; the outputs of the pre-trained student model and the gating neural network are input into the graph neural network respectively, and the graph neural network is used to output the prediction result of the pre-seizure period of the patient; S3: using a semi-supervised active learning strategy, iteratively training the seizure prediction model using the pooling data set and the training set and saving the training parameters, verifying the prediction effect of the trained seizure prediction model using the validation set, and obtaining the optimal seizure prediction model; S4: inputting the test set into the optimal seizure prediction model for prediction to obtain a prediction result of the preictal time of the patients in the test set.

[0008] Preferably, in the step S1, The preprocessing comprises: performing sliding window processing on the electroencephalogram data set of the epilepsy patient by using a preset length sliding window, and then performing FastICA processing on the data in each window to obtain independent components ICs corresponding to the electroencephalogram data of each patient, the dimension being 1×E×T, wherein E is the number of ICs, and T is the number of time points.

[0009] Preferably, in the step S1, for the preprocessed electroencephalogram data set, the data a minutes before the seizure and the data during the seizure are jointly saved as a main data set; the data from b minutes before the seizure to a minutes before the seizure is saved as the pooling data set; and the data more than b minutes before the seizure is discarded; wherein a and b are respectively a first time parameter and a second time parameter, and a < b. The main data set is further divided at a certain ratio to obtain the training set, the validation set and the test set.

[0010] Preferably, in the step S2, the student model comprises, connected in sequence: a first convolutional layer, an SE attention module and a second convolutional layer.

[0011] Preferably, in the step S2, the gating neural network comprises, connected in sequence: a scale transformation layer, a first nonlinear conversion layer, a second nonlinear conversion layer, a first ReLU activation layer. The first nonlinear conversion layer is composed of a first full connection layer and a first ELU activation layer. The second nonlinear conversion layer is composed of a second full connection layer and a Tanh activation layer. The input of the gating neural network is an independent component coupling matrix of the patient, the dimension being n×n, wherein the element in the ith row and the jth column represents the coupling degree between the ith independent component and the jth independent component; n is a preset positive integer; and the output of the gating neural network is a sparse adjacency matrix, the dimension being n×n.

[0012] Preferably, in the step S2, the graph neural network comprises, connected in sequence: a rhythm attention module, a residual graph convolution module, a third full connection layer and a fourth full connection layer. The input of the rhythm attention module is first subjected to 1×1 convolution to generate a frequency band weight score for the features of each channel of the input data, then the data after the weight scoring is activated by a Softmax activation function to obtain an attention score of each channel, and finally the attention scores of each channel are weighted and summed to obtain the output of the rhythm attention module. The input of the residual graph convolution module sequentially passes through a first graph convolution layer, a first batch normalization layer, a second ReLU activation layer, a second graph convolution layer and a second batch normalization layer; the output of the second batch normalization layer is added to the input of the residual graph convolution module, and then passes through a third ReLU activation layer to obtain the output of the residual graph convolution module.

[0013] Preferably, in the step S2, The residual neural network teacher model comprises a time convolution module, a multi-level spectral analysis module, a multi-level time analysis module, a gcSE module, a first time convolution layer, a third batch normalization layer and a second ELU activation layer. The time convolution module comprises two cascaded branches, one of which comprises a second time convolution layer, a third time convolution layer and a fourth time convolution layer connected in sequence, and the other of which comprises a fifth time convolution layer; the outputs of the two branches are added to obtain the output of the time convolution module; The input of the residual neural network teacher model is the independent components ICs of the patient; the output of the time convolution module is concatenated with the input of the teacher model, and then is input into the multi-level spectral analysis module and the multi-level time analysis module respectively; the outputs of the multi-level spectral analysis module and the multi-level time analysis module are concatenated and then input into the gcSE module, the first time convolution layer, the batch normalization layer and the second ELU activation layer in sequence to obtain the output of the teacher model.

[0014] Preferably, in the residual neural network teacher model, the multi-level spectral analysis module comprises sixth to eighth time convolution layers connected in sequence, wherein the receptive fields are 3, 5 and 7 respectively; The multi-level time analysis module comprises ninth to eleventh time convolution layers connected in sequence, wherein the convolution kernel interval parameters are 1, 2 and 4 respectively; The gcSE module comprises a Squeeze submodule, an Excitation submodule and a Scale layer connected in sequence; The Squeeze submodule is used for generating feature vectors of different channels, and comprises a global average pooling layer and a group convolution layer connected in sequence; The Excitation submodule is used for generating weight vectors of different channels, and comprises a fifth full connection layer, a third ELU activation layer, a sixth full connection layer and a Sigmoid activation layer connected in sequence; The Scale layer is used for calculating dot products of the feature vectors and the weight vectors by channel to obtain attention weights of each channel as the output of the gcSE module.

[0015] Preferably, in the step S3, a semi-supervised active learning strategy is adopted, the seizure prediction model is iteratively trained using the pooled dataset and the training set, and the training parameters are saved, including: In the qth iteration training, the training parameters of the seizure prediction model are denoted as , the current training set is trained, and the training parameters are updated to ; The current pooled dataset is input into the seizure prediction model with the training parameters for prediction labeling, and the labeling results are obtained; the Shannon entropy of each data in the current pooled dataset is calculated based on the labeling results, the data with a Shannon entropy greater than a preset threshold are screened out and added to the current training set , to obtain the training set of the next round ; at the same time, the screened-out data are deleted from the current pooled dataset , to obtain the pooled dataset of the next round ; The above steps are repeated to iteratively train the seizure prediction model until the training parameters converge, the training parameters at this time are saved, and the trained seizure prediction model is obtained.

[0016] The application also provides a seizure prediction system based on deep learning, comprising: A data preprocessing unit is configured to obtain an electroencephalogram dataset of an epilepsy patient and perform preprocessing, and divide the preprocessed electroencephalogram dataset into a pooled dataset, a training set, a validation set and a test set; A model construction unit is configured to construct a seizure prediction model based on a deep learning algorithm; first, a student model and a residual neural network teacher model are constructed, and the student model is pre-trained using the residual neural network teacher model based on a model distillation framework; the seizure prediction model comprises a residual neural network student model and a gated neural network arranged side by side, and a graph neural network; the outputs of the residual neural network student model and the gated neural network are connected to the inputs of the graph neural network, and the graph neural network is configured to output a prediction result of a preictal period of the patient; A model training unit is configured to adopt a semi-supervised active learning strategy, iteratively train the seizure prediction model using the pooled dataset and the training set, save the training parameters, and verify the prediction effect of the trained seizure prediction model using the validation set, to obtain an optimal seizure prediction model; Seizure prediction unit: for inputting the test set into the optimal seizure prediction model for prediction, obtaining the prediction result of the preictal time of the patient in the test set.

[0017] Compared with the prior art, the beneficial effects of the technical scheme of the present application are: The present application builds a time-space coupled graph convolutional neural network, which takes independent components (ICs) obtained by independent component analysis (ICA) as input, and is parallelly passed through a residual neural network containing time convolution and a neural network containing a gating mechanism, the residual neural network adopts multi-scale time analysis to extract the time-frequency response representing the five characteristic rhythms of electroencephalogram, the gating neural network takes an adjacency matrix composed of ICs numbers as input to model the directional dependence relationship between ICs, the outputs of the two neural networks are concatenated, and through a node graph neural network, considering the specificity of the preictal time of different patients, a semi-supervised active learning strategy is interspersed, and at the same time, a model distillation scheme is used to simplify the network structure, finally realizing the prediction task of epilepsy.

[0018] Through the time-space coupled graph convolutional neural network, combined with multi-scale time analysis, the present application can more accurately extract the time-frequency response of the electroencephalogram characteristic rhythm, thereby significantly improving the accuracy of epilepsy prediction; the semi-supervised active learning strategy is also adopted, which can effectively adapt to the specificity of the preictal time of different patients, improve the adaptability and prediction accuracy of the model to individual differences; finally, through the model distillation scheme, the network structure is simplified, the number of convolution layers and the level of spectral decomposition are reduced, thereby significantly improving the training speed, and reducing the computational complexity and parameter amount. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 A deep learning-based epilepsy seizure prediction method flowchart provided in embodiment 1; Figure 2 A deep learning-based epilepsy seizure prediction method flowchart provided in embodiment 2; Figure 3 A residual neural network teacher model structure diagram of a deep learning-based epilepsy seizure prediction method provided in embodiment 2; Figure 4 A residual neural network student model structure diagram of a deep learning-based epilepsy seizure prediction method provided in embodiment 2; Figure 5 A gating neural network structure diagram of a deep learning-based epilepsy seizure prediction method provided in embodiment 2; Figure 6 A graph neural network structure diagram of a deep learning-based epilepsy seizure prediction method provided in embodiment 2; Figure 7A deep learning-based seizure prediction system structure diagram provided in Embodiment 3. DETAILED DESCRIPTION

[0020] The drawings are only used for illustrative description and cannot be understood as a limitation on the present application; In order to better illustrate the present embodiment, some components in the drawings may be omitted, enlarged or reduced, and do not represent the actual product size; It is understandable for those skilled in the art that some well-known structures in the drawings and their descriptions may be omitted.

[0021] The technical solutions of the present application will be further described below in combination with the drawings and embodiments.

[0022] Embodiment 1 As shown in Figure 1 The present embodiment provides a deep learning-based seizure prediction method, comprising the following steps: S1: obtaining an electroencephalogram data set of a patient with epilepsy and preprocessing, dividing the preprocessed electroencephalogram data set into a pooling data set, a training set, a validation set and a test set; S2: constructing a seizure prediction model based on a deep learning algorithm, first constructing a student model and a residual neural network teacher model, and based on a model distillation framework, using the residual neural network teacher model to pretrain the student model; The seizure prediction model comprises: a gating neural network and a pre-trained student model arranged side by side, and a graph neural network; the outputs of the pre-trained student model and the gating neural network are input into the graph neural network respectively, and the graph neural network is used to output the prediction result of the preictal period of the patient; S3: using a semi-supervised active learning strategy, using the pooling data set and the training set to iteratively train the seizure prediction model and save the training parameters, using the validation set to verify the prediction effect of the trained seizure prediction model, and obtaining the optimal seizure prediction model; S4: inputting the test set into the optimal seizure prediction model for prediction to obtain the prediction result of the preictal period of the patient in the test set.

[0023] In the specific implementation process, first, the electroencephalogram data set of the patient with epilepsy is preprocessed, and independent components (ICs) are obtained by the Fast Independent Component Analysis (FastICA) method. These independent components can represent different source signals in the electroencephalogram signal, which is helpful for subsequent feature extraction and model training. Then, the preprocessed electroencephalogram data set is divided into a pooling data set, a training set, a validation set and a test set.

[0024] Then, a seizure prediction model is constructed based on a deep learning algorithm. First, a student model and a residual neural network teacher model are constructed. Then, the student model is pre-trained by using the residual neural network teacher model based on a model distillation framework. The student model is a final model used for prediction, and the residual neural network teacher model is used for training the student model.

[0025] The seizure prediction model comprises a gating neural network and a pre-trained student model arranged side by side, and a graph neural network. The outputs of the pre-trained student model and the gating neural network are input into the graph neural network, respectively. The graph neural network is used to output a prediction result of a pre-seizure time of a patient.

[0026] Subsequently, a semi-supervised active learning strategy is used to iteratively train the seizure prediction model by using the pooled data set and the training set, save training parameters, verify the prediction effect of the trained seizure prediction model by using the validation set, and obtain an optimal seizure prediction model. Finally, the test set is input into the optimal seizure prediction model for prediction to obtain a prediction result of a pre-seizure time of a patient in the test set.

[0027] The method can predict epilepsy by constructing a neural network coupled in time and space, and the prediction result is more accurate. Meanwhile, the method can adapt to the specificity of patients by using a semi-supervised active learning strategy. In addition, the method can improve the training speed and reduce the parameter amount by using a model distillation scheme.

[0028] Embodiment 2 As shown in Figure 2 The embodiment provides a seizure prediction method based on deep learning, which comprises the following steps: S1: Obtain an electroencephalogram data set of a patient with epilepsy and perform preprocessing. The preprocessed electroencephalogram data set is divided into a pooled data set, a training set, a validation set and a test set. S2: Construct a seizure prediction model based on a deep learning algorithm. First, a student model and a residual neural network teacher model are constructed. Then, the student model is pre-trained by using the residual neural network teacher model based on a model distillation framework. The seizure prediction model comprises a gating neural network and a pre-trained student model arranged side by side, and a graph neural network. The outputs of the pre-trained student model and the gating neural network are input into the graph neural network, respectively. The graph neural network is used to output a prediction result of a pre-seizure time of a patient. S3: Employ a semi-supervised active learning strategy, iteratively train the epileptic seizure prediction model using the pooled dataset and training set, save the training parameters, and use the validation set to validate the prediction effect of the trained epileptic seizure prediction model to obtain the optimal epileptic seizure prediction model. S4: Input the test set into the optimal epileptic seizure prediction model for prediction, and obtain the prediction results of the pre-seizure time of patients in the test set.

[0029] In step S1 Preprocessing includes: using a sliding window of preset duration to process the EEG dataset of epilepsy patients, and then performing FastICA processing on the data in each window to obtain the independent components (ICs) corresponding to the EEG data of each patient, with a dimension of 1×E×T, where E is the number of ICs and T is the number of time points.

[0030] In step S1, for the preprocessed EEG dataset, the data from a minutes before the epileptic seizure and the data during the epileptic seizure are jointly saved as the main dataset; the data from b minutes to a minutes before the epileptic seizure are saved as the pooled dataset; data with a time greater than b minutes before the epileptic seizure are discarded; where a and b are the first time parameter and the second time parameter, respectively, and a <b; The main dataset is further divided according to a certain ratio to obtain the training set, validation set, and test set.

[0031] In step S2, the student model includes, in sequence: a first convolutional layer, an SE attention module, and a second convolutional layer.

[0032] In step S2, the gated neural network includes the following layers connected in sequence: a scaling layer, a first nonlinear transformation layer, a second nonlinear transformation layer, and a first ReLU activation layer. The first nonlinear conversion layer consists of a first fully connected layer and a first ELU activation layer; The second nonlinear conversion layer consists of a second fully connected layer and a Tanh activation layer; The input to the gated neural network is the patient's independent component coupling matrix, which has an n×n dimension. The element in the i-th row and j-th column represents the degree of coupling between the i-th and j-th independent components. n is a preset positive integer. The output of the gated neural network is a sparse adjacency matrix with an n×n dimension.

[0033] In step S2, the graph neural network includes the following sequentially connected components: a rhythm attention module, a residual graph convolution module, a third fully connected layer, and a fourth fully connected layer. The input of the rhythm attention module is first subjected to 1*1 convolution to generate a frequency band weight score for the features of each channel of the input data, then the data after weight scoring is activated by a Softmax activation function to obtain an attention score of each channel, and finally the attention scores of each channel are weighted and summed to obtain the output of the rhythm attention module. The input of the residual graph convolution module sequentially passes through a first graph convolution layer, a first batch normalization layer, a second ReLU activation layer, a second graph convolution layer and a second batch normalization layer; the output of the second batch normalization layer is added to the input of the residual graph convolution module, and then passes through a third ReLU activation layer to obtain the output of the residual graph convolution module.

[0034] In the step S2, The residual neural network teacher model comprises a time convolution module, a multi-level spectral analysis module, a multi-level time analysis module, a gcSE module, a first time convolution layer, a third batch normalization layer and a second ELU activation layer. The time convolution module comprises two cascaded branches, one of which comprises a second time convolution layer, a third time convolution layer and a fourth time convolution layer connected in sequence, and the other of which comprises a fifth time convolution layer; the outputs of the two branches are added to obtain the output of the time convolution module; The input of the residual neural network teacher model is the independent components ICs of the patient, the output of the time convolution module is concatenated with the input of the teacher model, and then is input into the multi-level spectral analysis module and the multi-level time analysis module respectively; the outputs of the multi-level spectral analysis module and the multi-level time analysis module are concatenated and sequentially input into the gcSE module, the first time convolution layer, the batch normalization layer and the second ELU activation layer to obtain the output of the teacher model.

[0035] In the residual neural network teacher model, the multi-level spectral analysis module comprises sixth to eighth time convolution layers connected in sequence, wherein the receptive fields are 3, 5 and 7 respectively; The multi-level time analysis module comprises ninth to eleventh time convolution layers connected in sequence, wherein the convolution kernel interval parameters are 1, 2 and 4 respectively; The gcSE module comprises a Squeeze submodule, an Excitation submodule and a Scale layer connected in sequence; The Squeeze submodule is used to generate feature vectors of different channels, and comprises a global average pooling layer and a group convolution layer connected in sequence; The Excitation submodule is used to generate weight vectors of different channels, and comprises a fifth fully connected layer, a third ELU activation layer, a sixth fully connected layer and a Sigmoid activation layer connected in sequence; The Scale layer is used to calculate the dot product of the feature vector and the weight vector by channel to obtain the attention weight of each channel as the output of the gcSE module.

[0036] In the step S3, the semi-supervised active learning strategy is adopted to iteratively train the seizure prediction model using the pooled dataset and the training set and save the training parameters, including: In the qth iteration training, the training parameters of the seizure prediction model are denoted as , the current training set is trained, and the training parameters are updated to ; The current pooled dataset is input into the seizure prediction model with the training parameters for prediction and labeling to obtain the labeling result; the Shannon entropy of each data in the current pooled dataset is calculated based on the labeling result, the data with the Shannon entropy greater than the preset threshold are screened out and added to the current training set to obtain the training set of the next round ; at the same time, the screened data are deleted from the current pooled dataset to obtain the pooled dataset of the next round ; The above steps are repeated to iteratively train the seizure prediction model until the training parameters converge, the training parameters at this time are saved, and the trained seizure prediction model is obtained.

[0037] In the specific implementation process, first, the input EEG dataset in the embodiment adopts the EEG signals (CHB-MIT) from the Boston Children's Hospital in the MIT EEG database, as shown in Table 1, and Table 1 is a part of sample examples in the EEG dataset; Table 1 Part of sample examples in the EEG dataset

[0038] The acquired electroencephalogram data set of the epilepsy patient is preprocessed. The preprocessing is specifically processing the electroencephalogram data set through a preset length of sliding window, and then performing FastICA processing on the data in each window to obtain independent components ICs corresponding to the electroencephalogram data of each patient, with a dimension of 1×E×T, where E is the number of ICs, and T is the number of time points. Then the data a minutes before the seizure and the data during the seizure are saved together as the main data set; the data from b minutes before the seizure to a minutes before the seizure is saved as the pooling data set; the data more than b minutes before the seizure is discarded. Wherein a and b are respectively a first time parameter and a second time parameter, and a < b. Finally, the main data set is further divided according to a certain proportion to obtain a training set, a verification set and a test set. In this embodiment, a is 15 min, and b is 90 min, that is, less than 15 min is the pre-seizure period, more than 90 min is the interictal period, and the interval from 15 min to 90 min is classified into the pooling data set. Then a seizure prediction model is constructed based on a deep learning algorithm, and a student model and a residual neural network teacher model are first constructed. The structure diagram of the residual neural network teacher model is as shown in Figure 3 The residual neural network teacher model is a complex deep learning structure, which aims to process the independent components (ICs) data of the epilepsy patient and generate high-quality feature representation for seizure prediction. The model is composed of a time convolution module, a multi-level spectral analysis module, a multi-level time analysis module, a gcSE module, a first time convolution layer, a third batch normalization layer and a second ELU activation layer; the time convolution module captures time series data through two cascaded branches, one branch contains the second, third and fourth time convolution layers, and the other branch contains the fifth time convolution layer, the outputs of the two branches are added to form the final output of the time convolution module; the input of the residual neural network teacher model is the independent components (ICs) of the electroencephalogram signal of the epilepsy patient, which are first cascaded with the output of the time convolution module, and then sent into the multi-level spectral analysis module and the multi-level time analysis module respectively; the multi-level spectral analysis module is composed of the sixth to eighth time convolution layers, which have different receptive fields (3, 5 and 7 respectively) for analyzing features of different frequencies; the multi-level time analysis module is composed of the ninth to eleventh time convolution layers, which have different convolution kernel spacing parameters (1, 2 and 4 respectively) for capturing long-term dependencies in time series.

[0039] The gcSE module comprises a Squeeze sub-module, an Excitation sub-module and a Scale layer; the Squeeze sub-module generates feature vectors of different channels through a global average pooling layer and a group convolution (Gconv); the Excitation sub-module generates weight vectors of different channels through a fifth full connection layer, a third ELU activation layer, a sixth full connection layer and a Sigmoid activation layer; the Scale layer calculates the attention weights of each channel by performing dot product calculation on the feature vectors and the weight vectors by channel, and the weights are used as the output of the gcSE module to adjust the contribution degree of different feature channels. The residual neural network student model structure diagram is as shown in Figure 4 The student model replaces the original multi-level wavelet decomposition with a single time convolution, learns the frequency band separation through a 1*1 convolution, replaces the original filter bank, and retains the existing attention mechanism module, and the output structure still retains the 5 feature frequency band splicing of the teacher model, and the parameter amount is reduced by about 95% compared with the original.

[0040] Then, based on the model distillation framework, the residual neural network teacher model is used to pre-train the student model. The seizure prediction model comprises a parallelly arranged gated neural network and a pre-trained student model and a graph neural network.

[0041] The structure diagram of the gated neural network is as shown in Figure 5 The gated neural network comprises a scale transformation layer, a first nonlinear conversion layer, a second nonlinear conversion layer and a first ReLU activation layer connected in sequence. The first nonlinear conversion layer is composed of a first full connection layer and a first ELU activation layer. The second nonlinear conversion layer is composed of a second full connection layer and a Tanh activation layer. The input matrix A of the gated neural network represents the spatial mixing matrix of each IC component after FastICA, and first performs a Reshape operation on the matrix, that is, performs vectorization processing on the matrix, and flattens the two-dimensional matrix into a dimensional vector, and the elements from top to bottom are ,..., ,..., , Then, the first layer neural network conversion (Fully Connected + ELU activation) is performed on each , the input similarity is nonlinearly enhanced, higher-order feature representation is extracted, and then the second layer nonlinear conversion (Fully Connected + Tanh activation) is performed, so as to facilitate subsequent sparsification operation. The obtained matrix is restored to a two-dimensional structure by row, and a new matrix The dimension is still nxn, and the matrix learns the dynamic connection strength between ICAs through two layers of MLP. The dimension is nxn.

[0042] The structure diagram of the graph neural network is as shown in Figure 6 The graph neural network comprises, in sequence, a rhythm attention module, a residual graph convolution module, a third full connection layer, and a fourth full connection layer. The rhythm attention module inputs are the stacked feature results after graph convolution in the five feature bands in the layered cortical graph. The input is first subjected to 1x1 convolution to generate a frequency band weight score from each channel feature. Then, the weighted data after the weight scoring is activated by a softmax activation function to obtain the attention score of each channel. Finally, the attention scores of each channel are weighted and summed to obtain the final fused features, thereby realizing the independent learning of the relative importance of the five EEG feature rhythm bands and the fusion of the graph convolution features of different frequency bands in an attention manner. The residual graph convolution module is composed of a graph convolution layer (Graph Convolution, gc), a batch normalization function (BatchNorm2d, bn), and a relu activation function. The input of the residual graph convolution network is sequentially subjected to a first graph convolution layer, a first batch normalization layer, a second ReLU activation layer, a second graph convolution layer, and a second batch normalization layer. The output of the second batch normalization layer is added to the input of the residual graph convolution network, and then the output is obtained after passing through a third ReLU activation layer. That is, the output after passing through two layers of resGCN is added to the original input to obtain the final output. The input signal is the original EEG time domain signal, the graph convolution operation performs graph convolution calculation on the input signal and the adjacency matrix output by the gating neural network, so that the original signal fuses the graph structure information, and the residual structure is used to improve the depth and stability of the training, and the graph convolution features in each frequency band are extracted by resGCN.

[0043] Then, the pre-trained student model and the output of the gating neural network are respectively input into the graph neural network, and the graph neural network is used to output the prediction result of the pre-ictal period of the patient. In the model training process of this embodiment, a semi-supervised active learning strategy is adopted, and the epilepsy seizure prediction model is iteratively trained and the training parameters are saved using the pooled data set and the training set, including: In the qth iteration training, the training parameters of the epilepsy seizure prediction model are denoted as , and the training parameters are updated as ; The current pooled data set is input, and the training parameters are​​ the epilepsy seizure prediction model to obtain a tagging result; based on the tagging result, calculating the current pooled data set the Shannon entropy of each data in the current pooled data set , and adding the data with the Shannon entropy greater than the preset threshold to the current training set to obtain the next round of training set ; at the same time, deleting the screened data in the current pooled data set to obtain the next round of pooled data set ;

[0044] The pseudo code of the training process of the semi-supervised active learning of the embodiment is as follows:

[0045] Finally, the test set is input into the optimal epilepsy seizure prediction model for prediction to obtain the prediction result of the pre-ictal time of the patients in the test set.

[0046] The method can predict epilepsy by constructing a neural network coupled in time and space, and the prediction result is more accurate. Meanwhile, the method adopts a semi-supervised active learning strategy, which can adapt to the specificity of patients. In addition, the method adopts a model distillation scheme, which improves the training speed and reduces the parameter amount.

[0047] Embodiment 3 As shown in Figure 7 , the embodiment provides an epilepsy seizure prediction system based on deep learning, which comprises a data processing unit 301, a model construction unit 302, a model training unit 303, and an epilepsy seizure prediction unit 304. The data preprocessing unit 301 is used to obtain and preprocess the electroencephalogram data set of epilepsy patients, and divide the preprocessed electroencephalogram data set into a pooled data set, a training set, a validation set and a test set. The model construction unit 302 is used to construct an epilepsy seizure prediction model based on a deep learning algorithm. First, a student model and a residual neural network teacher model are constructed, and the student model is pre-trained by using the residual neural network teacher model based on a model distillation framework. The epilepsy seizure prediction model comprises a residual neural network student model and a gated neural network arranged side by side, and a graph neural network. The outputs of the residual neural network student model and the gated neural network are connected to the inputs of the graph neural network, respectively, and the graph neural network is used to output the prediction result of the pre-ictal time of the patient. The model training unit 303 is configured to iteratively train the seizure prediction model using the pooled dataset and the training set based on a semi-supervised active learning strategy, and save the training parameters. The prediction effect of the trained seizure prediction model is verified using the validation set, and the optimal seizure prediction model is obtained. The seizure prediction unit 304 is configured to input the test set into the optimal seizure prediction model for prediction, and obtain the prediction result of the pre-seizure time of the patient in the test set.

[0048] In the specific implementation process, The data processing unit 301 is configured to obtain the electroencephalogram dataset of the epilepsy patient and pre-process the same. The pre-processing step includes extracting independent components (ICs) from the original electroencephalogram signal using the FastICA method. These independent components can effectively represent different source signals in the electroencephalogram signal, providing a basis for subsequent feature extraction and model training. The pre-processed electroencephalogram dataset is further divided into four parts: a pooled dataset, a training set, a validation set, and a test set. Dividing the electroencephalogram dataset helps to reasonably allocate data resources in model training and seizure prediction, ensuring the generalization ability and prediction accuracy of the model. The model construction unit 302 is configured to construct a seizure prediction model based on a deep learning algorithm. First, a residual neural network teacher model and a student model are constructed, and then the student model is trained based on a model distillation scheme using the residual neural network teacher model, so that the student model can learn effective feature representation and knowledge from the teacher model. The student model is the final model used for prediction, while the teacher model is used to guide the learning process of the student model. The seizure prediction model includes a residual neural network student model and a gated neural network arranged side by side, as well as a graph neural network. The outputs of the residual neural network student model and the gated neural network are connected to the inputs of the graph neural network, respectively. The graph neural network is used to integrate the spatio-temporal features and brain region connection patterns, and output the prediction result of the pre-seizure time of the patient. The model training unit 303 is configured to iteratively train the seizure prediction model using the pooled dataset and the training set based on a semi-supervised active learning strategy, and save the training parameters. This strategy can effectively utilize unlabeled data for model training in the case of limited labeled data, improving the generalization ability of the model. At the same time, the prediction effect of the trained seizure prediction model is verified using the validation set, and the optimal seizure prediction model is obtained by comparing the prediction performance of different models. The seizure prediction unit 304 inputs the test set into the optimal seizure prediction model for prediction, and obtains the prediction result of the pre-seizure time of the patient in the test set. This step is a key link for evaluating the final performance of the model. By comparing with the actual seizure time, the prediction accuracy and reliability of the model can be evaluated.

[0049] The deep learning-based seizure prediction system in the embodiment builds a spatio-temporal coupled graph convolutional neural network, which takes independent components (ICs) obtained by independent component analysis (ICA) as input, and is parallelly passed through a residual neural network containing temporal convolution and a neural network containing a gating mechanism, the residual neural network adopts and multi-scale time analysis to extract time-frequency responses representing five characteristic rhythms of brain electrical signals, the gating neural network takes an adjacency matrix composed of IC numbers as input to model the directional dependence relationship between ICs, the outputs of the two neural networks are concatenated, and through a node graph neural network, the specificity of the pre-seizure period of different patients is considered, a semi-supervised active learning strategy is interspersed, and a model distillation scheme is used to simplify the network structure, and finally the seizure prediction task is realized.

[0050] The deep learning-based seizure prediction system in the embodiment innovatively builds a spatio-temporal coupled neural network, which captures spatio-temporal features through the spatio-temporal coupled neural network, significantly improves the accuracy of seizure prediction, and solves the problem of poor effect of traditional methods; secondly, semi-supervised active learning is introduced, and a semi-supervised active learning strategy is used to effectively solve the problem of patient specificity in the pre-seizure period and improve the recognition ability of the model in the pre-seizure period of different patients; finally, the model distillation technology is used to reduce the training time and memory requirement, so that the model is easier to deploy, and the problem that the model is difficult to apply on hardware is solved.

[0051] The same or similar reference signs correspond to the same or similar components; The positional relationship described in the drawings is only used for illustrative description, and should not be understood as a limitation on the application; Obviously, the above embodiments of the application are only examples for clearly illustrating the application, and are not intended to limit the implementation modes of the application. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, all the implementation modes are not required or can not be exhausted. Any modification, equivalent replacement and improvement made within the spirit and principle of the application should be included in the protection scope of the claims of the application.

Claims

1. A deep learning-based seizure prediction method, characterized by, The method comprises the following steps: S1: obtaining an electroencephalogram data set of an epilepsy patient and preprocessing the electroencephalogram data set, and dividing the preprocessed electroencephalogram data set into a pooling data set, a training set, a verification set and a test set; S2: constructing an epilepsy seizure prediction model based on a deep learning algorithm; first, constructing a student model and a residual neural network teacher model, and pretraining the student model based on the residual neural network teacher model using a model distillation framework; The epilepsy seizure prediction model comprises: a gating neural network and a pre-trained student model arranged side by side, and a graph neural network; The outputs of the pre-trained student model and the gating neural network are input into the graph neural network, respectively, and the graph neural network is used to output a prediction result of a pre-ictal period of the patient; S3: using a semi-supervised active learning strategy, iteratively training the epilepsy seizure prediction model using the pooling data set and the training set, saving the training parameters, verifying the prediction effect of the trained epilepsy seizure prediction model using the verification set, and obtaining an optimal epilepsy seizure prediction model; S4: inputting the test set into the optimal epilepsy seizure prediction model for prediction to obtain a prediction result of a pre-ictal period of the patient in the test set.

2. The deep learning-based seizure prediction method of claim 1, wherein, In the step S1, The preprocessing comprises: using a preset time length sliding window to perform sliding window processing on the electroencephalogram data set of the epilepsy patient, and then performing FastICA processing on the data in each window to obtain independent components ICs corresponding to the electroencephalogram data of each patient, with a dimension of 1×E×T, wherein E is the number of ICs, and T is the number of time points. 3.The deep learning-based seizure prediction method of claim 1, wherein, In the step S1, for the preprocessed electroencephalogram data set, data a minutes before a seizure and data during a seizure are saved together as a main data set; data from b minutes before a seizure to a minutes before a seizure is saved as the pooling data set; data more than b minutes before a seizure is discarded; wherein a and b are a first time parameter and a second time parameter respectively, and a < b; The main data set is further divided in a certain proportion to obtain the training set, the verification set and the test set.

4. The deep learning-based seizure prediction method of claim 1, wherein, In the step S2, the student model comprises, connected in sequence: a first convolutional layer, an SE attention module and a second convolutional layer.

5. The deep learning-based seizure prediction method of claim 1, wherein, In the step S2, the gating neural network comprises, connected in sequence: a scale transformation layer, a first nonlinear conversion layer, a second nonlinear conversion layer, a first ReLU activation layer; The first nonlinear conversion layer is composed of a first full connection layer and a first ELU activation layer; The second nonlinear conversion layer is composed of a second full connection layer and a Tanh activation layer; The input of the gating neural network is an independent component coupling matrix of the patient, with a dimension of n×n, wherein the element in the ith row and the jth column represents the coupling degree between the ith independent component and the jth independent component; n is a preset positive integer; and the output of the gating neural network is a sparse adjacency matrix with a dimension of n×n.

6. The deep learning-based seizure prediction method of claim 1, wherein, In the step S2, the graph neural network comprises, connected in sequence: a rhythm attention module, a residual graph convolution module, a third full connection layer and a fourth full connection layer; The input of the rhythm attention module is first subjected to 1*1 convolution to generate a frequency band weight score for the features of each channel of the input data, then the data after weight scoring is activated by a Softmax activation function to obtain an attention score of each channel, and finally the attention scores of each channel are weighted and summed to obtain the output of the rhythm attention module. The input of the residual graph convolution module sequentially passes through a first graph convolution layer, a first batch normalization layer, a second ReLU activation layer, a second graph convolution layer, and a second batch normalization layer; the output of the second batch normalization layer is added to the input of the residual graph convolution module, and then passes through a third ReLU activation layer to obtain the output of the residual graph convolution module.

7. The deep learning-based seizure prediction method of claim 1, wherein, In the step S2, The residual neural network teacher model comprises a time convolution module, a multi-level spectral analysis module, a multi-level time analysis module, a gcSE module, a first time convolution layer, a third batch normalization layer, and a second ELU activation layer; The time convolution module comprises two concatenated branches, one of which comprises a second time convolution layer, a third time convolution layer, and a fourth time convolution layer connected in sequence, and the other of which comprises a fifth time convolution layer; the outputs of the two branches are added to obtain the output of the time convolution module; The input of the residual neural network teacher model is the independent component ICs of the patient, the output of the time convolution module is concatenated with the input of the teacher model, and then is input into the multi-level spectral analysis module and the multi-level time analysis module, respectively; the outputs of the multi-level spectral analysis module and the multi-level time analysis module are concatenated and then input into the gcSE module, the first time convolution layer, the batch normalization layer, and the second ELU activation layer in sequence to obtain the output of the teacher model.

8. The deep learning-based seizure prediction method of claim 7, wherein, In the residual neural network teacher model, the multi-level spectral analysis module comprises sixth to eighth time convolution layers connected in sequence, wherein the receptive fields are 3, 5, and 7, respectively; The multi-level time analysis module comprises ninth to eleventh time convolution layers connected in sequence, wherein the convolution kernel interval parameters are 1, 2, and 4, respectively; The gcSE module comprises a Squeeze submodule, an Excitation submodule, and a Scale layer connected in sequence; The Squeeze submodule is used to generate feature vectors of different channels and comprises a global average pooling layer and a group convolution layer connected in sequence; The Excitation submodule is used to generate weight vectors of different channels and comprises a fifth fully connected layer, a third ELU activation layer, a sixth fully connected layer, and a Sigmoid activation layer connected in sequence; The Scale layer is used to perform dot product calculation on the feature vectors and the weight vectors by channel to obtain an attention weight of each channel as the output of the gcSE module. 9.The deep learning based seizure prediction method of claim 1, wherein, In the step S3, a semi-supervised active learning strategy is adopted to iteratively train the seizure prediction model using the pooled data set and the training set and save the training parameters, comprising: In the qth round of iterative training, the training parameters of the seizure prediction model are denoted as , and the training parameters are updated as ;​​ Use the current pooling dataset Input training parameters are The epilepsy seizure prediction model is used to predict and label the seizures, and the labeling results are obtained; the current pooling dataset is calculated based on the labeling results. The Shannon entropy of each data point is used to filter data whose Shannon entropy is greater than a preset threshold and add them to the current training set. To obtain the training set for the next round Meanwhile, in the current pooling dataset Delete the filtered data to obtain the next round of pooled dataset. ; The seizure prediction model is iteratively trained by repeating the above steps until the training parameters converge, the training parameters at this time are saved, and the trained seizure prediction model is obtained.

10. A deep learning based seizure prediction system, applying the method of any one of claims 1-8, characterized in that, Comprise: A data preprocessing unit is configured to obtain and preprocess an electroencephalogram data set of an epilepsy patient, and divide the preprocessed electroencephalogram data set into a pooled data set, a training set, a validation set and a test set; A model construction unit is configured to construct a seizure prediction model based on a deep learning algorithm; first, a student model and a residual neural network teacher model are constructed, and the student model is pre-trained using the residual neural network teacher model based on a model distillation framework; the seizure prediction model comprises a residual neural network student model and a gated neural network arranged side by side, and a graph neural network; the outputs of the residual neural network student model and the gated neural network are respectively connected to the inputs of the graph neural network, and the graph neural network is configured to output a prediction result of a pre-seizure time of a patient; A model training unit is configured to use a semi-supervised active learning strategy to iteratively train the seizure prediction model using the pooled data set and the training set, save the training parameters, use the validation set to verify the prediction effect of the trained seizure prediction model, and obtain an optimal seizure prediction model; A seizure prediction unit is configured to input the test set into the optimal seizure prediction model for prediction, and obtain a prediction result of a pre-seizure time of a patient in the test set.

Images