A multitask eeg automatic detection and prediction system for epilepsy

Abstract

This design is a multi-task automatic EEG detection and prediction system for epilepsy. The system first receives the EEG signals from epilepsy patients to be identified or the collected patient EEG signals used for training the module through a preprocessing module. The preprocessing module performs noise reduction, filtering, segmentation, and normalization on the signals, and outputs the processed patient EEG signals to the training module. In the training module, the collected patient EEG signals are used for training and testing. First, feature extraction is performed using a multi-scale convolutional network. Then, the detection and prediction branches incorporating a dual attention mechanism are trained, and the trained model parameters are saved for use in the detection and prediction module. Finally, the detection and prediction module provides detection and prediction results for the EEG signals from epilepsy patients to be identified. This invention employs a modular design, achieving real-time detection and prediction of noisy, unbalanced EEG signals, providing a reliable system for practical applications.

Description

Technical Field

[0001] This invention relates to the field of electroencephalogram (EEG) signal processing technology, and more particularly to a deep learning-based EEG detection and prediction system for epilepsy, specifically a multi-task automatic EEG detection and prediction system for epilepsy. Background Technology

[0002] Epilepsy is a neurological disorder caused by sudden abnormal electrical discharges in the brain, resulting in temporary impairment of brain function. It manifests as recurrent seizures without any apparent trigger. There are approximately 9 million epilepsy patients in my country, with about 400,000 new cases each year, two-thirds of whom are under 18 years old. With proper and cost-effective medication, about 70% of patients can avoid seizures. Therefore, accurate diagnosis is crucial in the prevention and treatment of epilepsy. Currently, diagnosis relies primarily on experienced doctors observing the waveforms and amplitudes of the patient's electroencephalogram (EEG). However, relying solely on doctors is inefficient and prone to misdiagnosis over time. To enable rapid diagnosis and timely treatment to reduce the risk of seizures, neural networks are needed to detect and predict epileptic EEG signals, automatically identifying the current stage of the seizure and predicting future seizures. Therefore, the designed network must be faster and more accurate in its detection and prediction capabilities.

[0003] Convolutional neural networks (CNNs) can automatically extract features compared to traditional neural networks and operate faster. Currently, CNNs perform well in the detection and prediction of electroencephalogram (EEG) signals. The attention mechanism allows the model to retain more key feature information in space, time, and channels, thus improving the model's accuracy. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-task automatic EEG detection and prediction system for epilepsy, which solves the problems of existing epilepsy EEG signal systems, such as noise, power frequency interference, unbalanced EEG signal data, slow detection speed of epileptic phases, and low accuracy in predicting epileptic seizures. This system can provide a reliable reference for practical engineering applications.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a multi-task epilepsy EEG signal detection and prediction system, which has high detection and prediction accuracy and short time, and includes a signal preprocessing module, a training module, and a detection and prediction module.

[0006] The model preprocessing module is used to receive relevant EEG signals input by the user and process the signals, including: receiving data, dividing the EEG signal into periods, denoising the signal, filtering the signal, normalizing the signal, segmenting the signal, extracting the signal's spectrogram, and outputting the processed information to the training module. The preprocessing module includes the following steps.

[0007] S1.1: Obtain electroencephalogram (EEG) data of epilepsy patients under sEEG monitoring. The EEG data is sourced from Boston Children’s Hospital and includes 24 EEGs of 23 pediatric patients with refractory seizures.

[0008] S1.2: The EEG signals of epilepsy are divided into five types: epileptic seizure period, epileptic pre-seizure period 1, epileptic pre-seizure period 2, epileptic pre-seizure period 3, and epileptic interictal period.

[0009] S1.3: Wavelet function is used to process EEG to remove noise interference from electrooculography, electrocardiography, and electromyography. Band-stop filter is used to filter out power frequency interference of 57-63Hz and 117-123Hz in EEG data. Deviation standardization is used to normalize the data.

[0010] S1.4: Divide the epileptic EEG signal into 2-second segments. The segments during the epileptic seizure have 75% overlap, while the segments divided in other periods have no overlap.

[0011] S1.5: The time-frequency features of each data segment are extracted by fusing continuous wavelet transform and short-time Fourier transform. The time-frequency matrices generated by the two methods are resampled and then spliced ​​and fused to obtain richer image features. The feature map is then passed to the prediction network in the training module.

[0012] The training module is used to train the EEG signal after the preprocessing module through a neural network, including signal feature extraction, dual attention module, detection network in multi-task network, prediction network in multi-task network, detection and prediction branch loss function and total loss function, and saves the trained model parameters for the detection and prediction modules. The training module includes the following steps.

[0013] S2.1: Feature extraction is performed using a multi-scale convolutional network composed of multi-scale convolutional blocks. A K-max pooling layer is used instead of a max pooling layer for pooling operations. A dual attention module is used after each multi-scale convolutional block to pass the extracted information to the detection network.

[0014] S2.2: Dual attention module, which uses max pooling and average pooling operations to aggregate the input features along the horizontal and vertical directions into two separate feature maps. The two feature maps capture the relationship between the input feature map and time and the relationship between each channel along the temporal and spatial directions, respectively, and then the feature maps are fused.

[0015] S2.3: Detection of epileptic EEG signals using stacked residual networks. Each stacked residual network consists of three residual blocks. Each residual block sequentially includes pointwise convolution (Pointwise Conv2d), batch normalization (BN) processing, depthwise separable convolution (Device Conv2d), BN processing, pointwise conv2d, BN processing, and a modified linear unit.

[0016] S2.4: Predict epileptic EEG signals using a bidirectional long short-term memory network (Bi-LSTM), and add a dual attention module to the last layer of the network. The dual attention module consists of spatial attention and channel attention, and is connected in parallel.

[0017] S2.5: Train the detection branch network and the prediction branch network respectively to obtain the loss function (GHM Loss) of the detection branch network and the loss function (Cross Entropy Loss) of the prediction branch network; calculate the total loss function through the two branch loss functions; and perform training for a preset number of iterations to obtain the optimal multi-task epilepsy EEG detection and prediction model, thus completing the model establishment.

[0018] The detection and prediction module is used to provide detection and prediction results for the collected patient's electroencephalogram (EEG) signals.

[0019] The present invention has the following beneficial effects:

[0020] This invention adopts a modular design and discloses a multi-task epilepsy detection and prediction system. To address the problem of noise such as electrooculography (EOG) and power frequency interference in the collected EEG data, wavelet transform and band-stop filtering are used to solve the problem of incorrect prediction of seizure occurrence time, which leads to untimely treatment.

[0021] In the feature extraction part of this invention, a method of fusing continuous wavelet transform and short-time Fourier transform is adopted and combined with a multi-scale convolutional network for feature extraction. The cooperation between these two parts enables the system to extract more comprehensive information and more important information.

[0022] This invention designs a dual attention module, which enables the features extracted in each step of the feature extraction part of the multi-scale convolutional network to be utilized, thereby improving the network detection performance. Furthermore, the dual attention module of the prediction branch takes into account the relationship between a certain moment and the preceding and following time periods, as well as the relationship between various channels, making the consideration more comprehensive and improving the prediction accuracy.

[0023] The detection section of this invention utilizes a stacked residual network to retain useful information, improve model running speed, and reduce the number of parameters. The prediction section introduces a Bi-LSTM module, effectively preventing gradient explosion and vanishing gradients, and better capturing temporal relationships. It also incorporates a dual attention module, enabling the network to focus on temporal relationships and relationships between different channels, thereby strengthening the region of interest from both channel and temporal perspectives. This multi-task detection and prediction method can simultaneously learn detection and prediction tasks and share feature information between tasks, thus improving the performance of each task.

[0024] This invention inputs the EEG signals of the epilepsy patient to be identified into the system model. The data is processed through a preprocessing module and a training module. The cooperation of these two modules enables the system to perform more comprehensive and rapid detection of patients under various noise interference conditions and provide a more accurate result. This system can provide a reference for the engineering application of epilepsy EEG. Attached Figure Description

[0025] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of a multi-task epilepsy detection and prediction system;

[0027] Figure 2 This is a schematic diagram of a multi-scale convolutional module structure;

[0028] Figure 3 This is a schematic diagram of the dual attention module structure;

[0029] Figure 4 This is a schematic diagram of the residual block and the stacked residual network structure;

[0030] Figure 5 This is a schematic diagram of the structure of a multi-task epilepsy detection and prediction model; Detailed Implementation

[0031] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the present invention and not all of the embodiments. Any embodiments of the present invention that are obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] This system includes a signal preprocessing module, a training module, and a detection and prediction module.

[0033] The model preprocessing module is used to receive relevant EEG signals input by the user and process the signals, including: receiving data, dividing the EEG signal into periods, denoising the signal, filtering the signal, normalizing the signal, segmenting the signal, extracting the signal's spectrogram, and outputting the processed information to the training module. The preprocessing module includes the following steps.

[0034] S1.1: Electroencephalogram (EEG) data of epilepsy patients under sEEG monitoring were acquired. The EEG data originated from Boston Children's Hospital and included 24 EEGs from 23 pediatric patients with refractory seizures. During the data acquisition process for the 23 patients, a 10-20 lead system was used, and the sampling frequency was 256 Hz. The Boston dataset totaled 844 hours and included 196 seizures. In actual seizures, the interval between two seizures may be very short, which makes it difficult to extract pre-seizure data for each seizure. In the dataset, if the interval between two seizures is no more than 30 minutes, it is considered one seizure. The predicted number of seizures in the experiment was less than the actual number of seizures.

[0035] S1.2: EEG signals in epilepsy are divided into five types: the ictal phase, preictal phase 1, preictal phase 2, preictal phase 3, and interictal phase, labeled 0, 1, 2, 3, and 4, respectively. The preictal phase is the first 30 minutes of the ictal phase, preictal phase 3 is the first 10 minutes of the ictal phase, preictal phase 2 is the first 10 minutes of preictal phase 3, preictal phase 1 is the first 10 minutes of preictal phase 2, and the interictal phase includes all other periods.

[0036] S1.3: The raw EEG signal is collected from the cerebral cortex and contains some noise interference, such as electrooculogram (EOG) artifacts, electromyogram (EMG) interference, and power frequency interference. To better extract EEG features, the signal needs to be denoised. Wavelet functions are used to process the EEG to remove noise interference from EOG, ECG, and EMG, eliminating a large amount of noise that affects detection and prediction accuracy. Band-stop filters are used to filter out power frequency interference in the 57-63Hz and 117-123Hz ranges from the EEG data, resulting in less noise and a smoother signal. Deviation standardization is used to normalize the data; the expression for deviation standardization is as follows:

[0037]

[0038] In the formula, max is the maximum value of the sample data, min is the minimum value of the sample data, x is the initial value, and x* is the result of the deviation standardization of the initial value.

[0039] S1.4: The epileptic EEG signal is divided into 2-second segments. The segments during the epileptic seizure have 75% overlap, while the segments from other periods have no overlap. Dividing the data into time windows reduces the amount of data input into the neural network. Using overlapping windows during the seizure period is to amplify the data, as there is less data during the seizure period than in other periods. To make the training results as effective as possible, the data needs to be amplified to meet the detection requirements.

[0040] S1.5: The time-frequency features of each data segment are extracted by fusing continuous wavelet transform and short-time Fourier transform. The time-frequency matrices generated by the two methods are resampled and then spliced ​​and fused to obtain richer image features. The feature map is then passed to the prediction network in the training module. The resampling measure is adopted because the feature matrix after feature extraction by the two methods contains a large amount of redundant data. Redundant data does not help with feature classification and increases the amount of data.

[0041] The training module is used to train the EEG signal after the preprocessing module through a neural network, including signal feature extraction, dual attention module, detection network in multi-task network, prediction network in multi-task network, detection and prediction branch loss function and total loss function, and saves the trained model parameters for the detection and prediction modules. The training module includes the following steps.

[0042] S2.1: Feature extraction is performed using a multi-scale convolutional network, improving the number of convolutional kernels and adjusting their size. The multi-scale convolutional network consists of three multi-scale convolutional modules and three K-max pooling layers, with each multi-scale convolutional module preceded by a K-max pooling layer. A dual attention module is used after each multi-scale convolutional module to pass the extracted information to the detection network. The multi-scale convolutional network extracts more comprehensive and important information. The K-max pooling layer primarily selects the K largest values ​​to prevent the loss of important information. Pooling layers compress features, reducing the size of the parameter matrix, simplifying the network model complexity, reducing the number of parameters, accelerating computation, and preventing overfitting. The dual attention module is mainly used to pass important information from the signal extracted by the backbone network to the branch networks, enabling the detection branch to obtain more necessary information and improving its accuracy.

[0043] Multi-scale convolutional modules such as Figure 2 As shown, in the multi-scale convolution module, three 1x1 convolutions are first used to reduce the number of channels to 1 / 2, 1 / 2, and 1 / 4 of the input channel number, respectively. After the second 1x1 convolution, a 1x32 convolution is added to restore the number of channels to the input channel number. After the third 1x1 convolution, a 1x64 convolution is added to restore the number of channels to 1 / 2 of the input channel number. Finally, the three convolutional blocks are combined to restore the number of channels to twice the input channel number.

[0044] S2.2: As Figure 3 This section describes the dual attention module, which includes a temporal attention module and a channel attention module. In the temporal attention module, the input feature map is first processed through parallel max-pooling and average-pooling layers in the temporal direction, reducing the feature size from channel number * feature map length * feature map width to channel number * 1 * 1. Then, it passes through a multilayer perceptron (MLP) module, where the number of channels is first compressed to 1 / r of the original number (r is the reduction rate), then expanded back to the original number of channels. The two features are then summed using the ReLU activation function, and finally, the output of the temporal attention module is obtained through the sigmoid activation function. The temporal attention formula is as follows:

[0045] M T (F)=σ(MLP(xAvgPool(F))+MLP(xMaxPool(F))) (2)

[0046] In the formula M T (F) represents the temporal attention module, σ is the sigmoid function, F is the input feature map, and xAvgPool and xMaxPool are the average pooling layer and the max pooling layer in the temporal direction, respectively.

[0047] In the channel attention module, the input feature map is first processed through parallel max pooling and average pooling layers along the channel direction, reducing the feature size from channel number * feature map length * feature map width to channel number * 1 * 1. Then, it passes through the MLP module, where the number of channels is first compressed to 1 / r of the original number (r is the reduction rate), then expanded back to the original number of channels. The two features are then added using the ReLU activation function, and finally, the output of the channel attention module is obtained through the sigmoid activation function. The following is the channel attention formula:

[0048] M C (F)=σ(MLP(yAvgPool(F))+MLP(yAvgPool(F))) (3)

[0049] In the formula M C (F) represents the channel attention module, σ is the sigmoid function, F is the input feature map, and yAvgPool and yMaxPool are the average pooling layer and the max pooling layer in the channel direction, respectively.

[0050] The dual attention module uses max pooling and average pooling operations to aggregate the input features along the horizontal and vertical directions into two separate feature maps. These two feature maps capture the temporal relationship and the relationships between different channels of the input feature map along the temporal and spatial directions, respectively. The feature maps are then fused. The following is the dual attention formula:

[0051]

[0052] In the formula F OUT For the output results, This represents element-wise multiplication.

[0053] S2.3: Detection of epileptic EEG signals using stacked residual networks. A schematic diagram of the residual blocks and stacked residual networks is shown below. Figure 4 As shown, each stacked residual network consists of three residual modules. Each residual module sequentially includes pointwise convolution (Pointwise Conv2d), batch normalization (BN), depthwise separable convolution (Dev2d), BN, and pointwise convolution (Dev2d), BN, and a rectified linear unit (RELU). Depthwise Conv2d is a channel-wise convolutional layer operation, with the number of convolution kernels being the same as the number of channels in the previous layer, so the number of channels in the output feature map remains unchanged. This significantly reduces the number of parameters in the convolutional layers, but this method cannot change the feature map and does not effectively utilize the feature information of different channels at the same spatial location. Pointwise Conv2d is a pointwise convolutional layer operation with a fixed kernel size of 1x1. It can change the number of feature maps and can also weight and combine the feature maps generated in the previous step in the spatial dimension, which makes up for the shortcomings of Depthwise Conv2d. The input feature tensor is added to itself after passing through three convolutional layers, and then passed through a ReLU as an activation function to finally obtain the output feature tensor. By directly passing the input information to the output via a shortcut, the integrity of the information can be protected, and the entire network only needs to learn the part that differs between the input and output, simplifying the learning objective and difficulty.

[0054] In the residual module, Pointwise Conv2d first uses 1x1 convolution, and the number of output channels becomes 1 / 4 of the number of input channels. Depthwise Conv2d uses 1x32 convolution to perform convolution operation, and the number of output channels is the same as the number of input channels. Pointwise Conv2d uses 1x1 convolution, and the number of output channels becomes 1 / 2 of the number of input channels.

[0055] S2.4: A Bi-LSTM network is used to predict epileptic EEG signals. A dual-attention module is added at the end of the Bi-LSTM network, consisting of temporal attention and channel attention, which are implemented in parallel. Max pooling and average pooling operations are used to aggregate the input features along the horizontal and vertical directions into two separate feature maps. These two feature maps capture the temporal relationship and the relationships between the individual channels of the input feature map along the temporal and spatial directions, respectively.

[0056] S2.5: Train the detection branch network and the prediction branch network separately, obtain the gradient equalization mechanism loss function (GHM Loss) of the detection branch network and the cross-entropy loss function of the prediction branch network, calculate the total loss function (Add Cross Entropy Loss) using the two branch loss functions, and perform multiple training runs to obtain the optimal multi-task epilepsy EEG detection and prediction model, thus completing the model establishment. The following are the expressions for the gradient equalization mechanism loss function, the cross-entropy loss function, and the total loss function, respectively:

[0057]

[0058]

[0059] ACELoss=αGHM Loss+βCross Entropy Loss (7)

[0060] (5) Where: β i Here, GD(g) is the density compatibility parameter, N is the sample size, and GD(g) is the density comp i Let L be the gradient density function. CE (p i ,p i *) is the binary cross-entropy function.

[0061] (6) In the formula: a is the predicted label, y is the real sample label, and n is the number of samples.

[0062] (7) In the formula: ACELOSs is the total loss function, and α and β are the weights of the loss function, both of which are taken as 0.5 here.

[0063] It's important to note that the detection and prediction networks are trained simultaneously. Since a single backbone network is used, parameters such as the input signal duration, number of channels, and learning rate are shared. The more shared parameters, the lower the risk of overfitting. To address the training difficulties and inaccurate signal detection caused by the inherent class imbalance in EEG signals, the loss function for the detection branch is changed to GHM Loss. GHM Loss does not consider samples that are difficult or easy to learn from, treating them as outliers, which improves detection accuracy.

[0064] The detection and prediction module is used to provide detection and prediction results for the acquired patient EEG signals. First, the acquired EEG signals to be identified are sent to the preprocessing module for noise reduction, filtering, and normalization to reduce noise interference and make the detection and prediction steps more accurate. Then, the data after preprocessing is sent to the model stored in the detection and prediction module, and the module provides the identification result: whether the current EEG signal segment is in the pre-seizure, interictal, or ictal phase, and predicts whether a seizure will occur in the next 30 minutes, 20 minutes, or 10 minutes.

[0065] In this implementation method, the backbone network is a multi-scale convolutional network, the stacked residual network is the detection branch network, and the Bi-LSTM network is the prediction branch network. The backbone network extracts features from the input EEG signal. Features extracted from each layer are fed into the detection network via a dual-attention mechanism. In the detection branch network, each stacked residual network is combined with the input features to reduce the loss of key signal features. A fusion method of continuous wavelet transform and short-time Fourier transform is used to extract the time-frequency features of each data segment, which are then combined with the features extracted from the backbone network and fed into the Bi-LSTM prediction branch network. A dual-attention module is added at the end of the prediction branch to extract more features and optimize prediction performance. Finally, this implementation method uses the loss functions of the two branch networks for joint training to optimize the shared parameters, enhancing their ability to learn features, reducing their individual losses, and thus reducing the total loss and improving the model's recognition accuracy. The structure diagram of the multi-task epilepsy detection and prediction model is shown below. Figure 5 As shown.

[0066] The above description clearly illustrates the technical solution, process, and advantages of the present invention. Those skilled in the art will obviously understand that the present invention is not limited by the above embodiments. The embodiments and specifications described above are only the technical solutions and principles of the present invention and do not represent the whole. Without departing from the spirit and content of the present invention, any improvements to the corresponding algorithms are within the scope of protection claimed by the present invention. The experimental results of the present invention are implemented in a unique form. The scope of protection of the present invention is defined by the appended claims and equivalents.

Claims

1. A multi-task automatic EEG detection and prediction system for epilepsy, characterized by including a signal preprocessing module, a training module, and a detection and prediction module; The model preprocessing module is used to receive relevant EEG signals input by the user, process the signals, and output the processed information to the training module, including the following steps: S1.1: EEG signals in epilepsy are classified into five types: epileptic seizure phase, pre-seizure phase 1, pre-seizure phase 2, pre-seizure phase 3, and interictal phase. S1.2: Wavelet function is used to process EEG to remove noise interference from electrooculography, electrocardiography, and electromyography. Band-stop filter is used to filter out power frequency interference of 57-63Hz and 117-123Hz in EEG data. Deviation standardization is used to normalize the data. S1.3: Divide the epileptic EEG signal into 2s segments. The segments during the epileptic seizure have 75% overlap, while the segments divided in other periods have no overlap. S1.4: The time-frequency features of each data segment are extracted by fusing continuous wavelet transform and short-time Fourier transform. The time-frequency matrices generated by the two methods are resampled and then spliced ​​and fused. The feature map is then passed to the prediction network in the training module. The training module is used to train the preprocessed EEG signals through a neural network and save the trained model parameters for use in the detection and prediction module. It includes the following steps: S2.1: Feature extraction is performed using a multi-scale convolutional network composed of multi-scale convolutional blocks. A K-max pooling layer is used to replace the max pooling layer for pooling operations. A dual attention module is used after each multi-scale convolutional block to pass the extracted information to the detection network. S2.2: Dual attention module, which uses max pooling and average pooling operations to aggregate the input features along the horizontal and vertical directions into two separate feature maps. The two feature maps capture the relationship between the input feature map and time and the relationship between each channel along the time and channel directions, respectively, and then the feature maps are fused. S2.3: Detection of epileptic EEG signals using stacked residual networks. Each stacked residual network consists of three residual blocks. Each residual block sequentially includes pointwise convolution (Pointwise Conv2d), batch normalization (BN) processing, depthwise separable convolution (Dev2d), BN processing, and pointwise conv2d, BN processing, as well as a corrected linear unit. S2.4: Predict epileptic EEG signals using a bidirectional long short-term memory network (Bi-LSTM), and add a dual attention module to the last layer of the network. The dual attention module consists of temporal attention and channel attention, and the temporal attention module and channel attention module are used in parallel. S2.5: Train the detection branch network and the prediction branch network respectively, obtain the loss function (GHMLoss) of the detection branch network and the loss function (CrossEntropyLoss) of the prediction branch network, calculate the total loss function through the two branch loss functions, and perform training for a preset number of iterations to complete the model building; The detection and prediction module is used to provide detection and prediction results for the collected patient's electroencephalogram (EEG) signals.

Images