A method for predicting and diagnosing epilepsy based on multi-domain features of EEG signals

By combining technologies such as CSP, STFT and SE modules in EEG signal prediction, using multi-domain feature extraction and capsule networks, the problem of failure to fully utilize the multi-dimensional characteristics of EEG data in the existing technology is solved, and epilepsy prediction is improved accuracy and model performance.

CN118680582BActive Publication Date: 2024-11-15NORTHEAST DIANLI UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411194588.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-15
Estimated Expiration
2044-08-29

AI Technical Summary

Technical Problem

When using EEG data, existing epilepsy prediction methods fail to fully utilize the multi-dimensional features of the time domain, frequency domain and spatial domain, resulting in insufficient information and insufficient accuracy in prediction.

Method used

Using multi-domain feature extraction method based on EEG signals, spatial features are enhanced through CSP algorithm, time-frequency features are extracted by STFT, and feature pre-extraction is performed by combining SE module and spatial attention mechanism ResNet18 network, and finally using capsule network prediction module to construct epilepsy Seismic prediction model.

Benefits of technology

It improves the accuracy and robustness of epilepsy prediction, can better capture the complex patterns of EEG data and the spatial and hierarchical relationships in the time-frequency graph, and enhances the accuracy of feature extraction and the performance of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118680582B_ABST
    Figure CN118680582B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of epilepsy diagnosis, and specifically discloses a method for predicting and diagnosing epilepsy based on multi-domain features of electroencephalogram (EEG) signals, including training a generative adversarial network in an unsupervised manner, and then using the trained discriminator as a feature extractor. The features generated by the feature extractor are used to predict labeled EEG signals by two fully connected layers. Liu et al. proposed a multi-view convolutional neural network framework, the purpose of which is to obtain a shared representation of time domain and frequency domain features to predict epileptic seizures. Then, we use 1D-CNN to extract additional features from the EEG signal, and use these two features to estimate the probability of an epileptic seizure event. The present invention uses a vectorized representation and a dynamic routing mechanism of a capsule network to make it more sensitive to complex and subtle changes. It not only pays attention to the presence or absence of features, but also pays attention to the relative position and posture changes between features, better captures the spatial and hierarchical relationships in the time-frequency diagram, and improves the accuracy of epileptic seizure prediction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of epilepsy diagnosis, and specifically to a method for predicting and diagnosing the onset of epilepsy based on multi-domain features of electroencephalogram (EEG) signals. Background Art

[0002] Epilepsy is a neurological disease caused by continuous abnormal discharges of brain neurons, which is characterized by suddenness and recurrence. The main characteristics of its attacks are loss of consciousness to varying degrees, as well as partial or whole-body involuntary convulsions, which can even lead to death in severe cases. According to statistics from the World Health Organization, about 70 million people suffer from epilepsy worldwide, and with the increasing social pressure, the prevalence of epilepsy is still increasing year by year. Since epileptic seizures are accidental, patients are very likely to suddenly develop the disease and lose their will during daily activities, causing accidents and injuries. Therefore, timely and effective prediction of epilepsy can predict the condition in advance, facilitate timely prevention and treatment measures, reduce patients' physical loss and mental stress, and improve the quality of life of epilepsy patients.

[0003] Epileptic seizures are divided into four stages according to the course of the disease, namely, interictal, preictal, ictal, and postictal. The interictal period refers to the period between two epileptic seizures. In this stage, intermittent sharp waves or spike wave discharges may occur in some cases, but brain activity is relatively normal and there are no obvious epileptic symptoms; the preictal period is the transitional stage between the interictal and ictal periods, and can also be regarded as the early warning stage before an epileptic seizure. During this stage, the patient's abnormal EEG activity gradually increases, but due to individual differences and other factors, the specific duration of the preictal period has no clear clinical definition. In recent studies, most of them define it as 30 minutes, which effectively warns patients while not causing excessive anxiety due to too long advance reminders; the ictal period refers to the period when epilepsy occurs, which lasts from a few seconds to a few minutes. At this stage, patients will have abnormally high-frequency, synchronized EEG activity, accompanied by obvious clinical symptoms such as convulsions and loss of consciousness; the postictal period refers to the recovery period after the end of an epileptic seizure, which usually lasts from a few minutes to a few hours. At this stage, the patient's body and brain gradually recover from the seizure. At present, a variety of analysis methods have been applied to epilepsy research, including clinical interviews, imaging examinations, electroencephalography (EEG), etc. Clinical interviews include detailed descriptions of symptoms, frequency of seizures, duration, and other information, which are important references for disease diagnosis. Imaging examinations mainly include magnetic resonance imaging (MRI) and computed tomography (CT), which generate brain images through magnetic fields or rays to study changes in brain function or structure caused by epilepsy. For example, Frederick et al. detected reversible abnormalities in the cerebral cortex or subcutaneous layer of patients with first epileptic seizures through magnetic resonance imaging, which usually returned to normal within days to weeks after the onset of epilepsy; Peter et al. found that among 103 children with epilepsy who underwent computed tomography, 51.5% showed abnormal results such as hydrocephalus and brain atrophy, verifying the importance of CT scanning in the screening and evaluation of childhood epilepsy. Epilepsy is a disease caused by abnormal brain discharges. Imaging examinations can only indirectly reflect brain structure and activity through images. Electroencephalography, as a method of recording brain neurophysiological activity, can detect the transmission of electrical signals between brain cells by applying electrodes to the subject's head. It is a more direct way to reflect brain structure and activity. EEG is non-invasive, low-cost, easy to collect, and contains rich physiological and pathological information.Li et al. found that the synchronization of EEG network connections in the pre-ictal period was enhanced compared with the interictal period; Park et al. extracted the spectral power features of 9 frequency bands in EEG, predicted them through cost-sensitive support vector machines and obtained a sensitivity of 92.5%; Zhang et al. proposed the concept of multidimensional sample entropy, and used a bidirectional long short-term memory network to verify that it had significant differences in different stages of epilepsy. It has been proven that imaging examinations are mainly used to identify epileptic lesions and evaluate structural abnormalities, but low temporal resolution and insufficient functional information have limited their application in predicting epileptic seizures. In contrast, EEG can capture the dynamic changes in different stages of epilepsy and is more suitable for predicting epileptic seizures.

[0004] Although some results have been achieved by combining time-frequency domain features to analyze and predict EEG data, EEG data also contains rich spatial information that has not been fully utilized.

[0005] EEG data contains three dimensions: time domain, frequency domain, and space domain, each of which provides different information. Time domain features reflect the changes of EEG signals over time, such as signal fluctuations, trends, and instantaneous activity patterns; frequency domain features reveal the frequency components of EEG signals and describe the energy distribution of signals at different frequencies; spatial domain features describe the distribution pattern of EEG signals in different brain regions and can reveal the spatial relationship between signals in different electrodes. Although the feature extraction method combining the time domain and frequency domain dimensions reveals some information of EEG signals to a certain extent, it is not enough to fully characterize the complexity of epileptic seizures. Therefore, how to combine the features of the time domain, frequency domain, and space domain, comprehensively consider information of different dimensions, and design a model that can fully capture the complex patterns of EEG data is an important issue. Summary of the invention

[0006] The purpose of the present invention is to provide a method for predicting and diagnosing the onset of epilepsy based on multi-domain features of EEG signals, so as to solve the problems raised in the above-mentioned background technology.

[0007] To achieve the above object, the present invention provides the following technical solution: a method for predicting and diagnosing epilepsy based on multi-domain features of EEG signals, comprising the following steps:

[0008] S1: Enhanced spatial features;

[0009] S2: perform feature extraction;

[0010] S3: Perform capsule classification and build a capsule network prediction module.

[0011] Preferably, a set of optimal spatial filters is selected for projection using a CSP algorithm to enhance the EEG spatial features.

[0012] Preferably, the covariance matrices of the two types of EEG data, pre-ictal and interictal, are calculated separately to capture the linear relationship and variation characteristics between the channels.

[0013] Preferably, the data is subjected to computational analysis;

[0014] The specific formula is as follows:

[0015] (1)

[0016] (2)

[0017] (3)

[0018] (4)

[0019] (5)

[0020] (6)

[0021] (7)

[0022] Among them, (1) is used to calculate the covariance matrix, is the covariance matrix, which is a matrix describing the linear relationship between EEG signals in various channels. It is EEG signal vector, Indicates the number of samples, that is, the total number of EEG signal samples; calculate The trace of , i.e. the sum of the diagonal elements, is used for normalization;

[0023] Among them, (2) is used to calculate the total covariance matrix, represents the total covariance matrix, which is the sum of the covariance matrices of the preictal and interictal periods; is the covariance matrix of the preictal period; is the covariance matrix of the interictal intervals;

[0024] Among them, (3) is the generalized eigenvalue decomposition, is the eigenvector matrix, is the total covariance matrix The characteristic vector of is the diagonal eigenvalue matrix, is the total covariance matrix The characteristic value of

[0025] Among them, (4) is used to calculate the whitening matrix, is the whitening matrix, the total covariance matrix Perform a whitening transformation to standardize the covariance matrix; represents a diagonal matrix containing The negative square root of each eigenvalue of is used for the whitening process; represents the transpose of the eigenvector matrix;

[0026] Among them, (5) represents the whitening transformation of the covariance matrix, represents the whitening covariance matrix of the pre-ictal period; represents the whitened covariance matrix of the interictal period, is the transpose of the whitening matrix;

[0027] Where (6) represents the generalized eigenvalue decomposition, Represents the CSP projection matrix, which is a matrix of eigenvectors used to map data to a new feature space; is a diagonal eigenvalue matrix, which represents the distribution of data in the new feature space;

[0028] Among them, (7) is used to calculate the CSP projection matrix, is the final CSP projection matrix, which is used to project the original EEG data into a new feature space to enhance the discrimination between preictal and interictal periods.

[0029] Preferably, the EEG data is converted into a time-frequency diagram that can provide both time and frequency information using a STFT method;

[0030] The definition of STFT is as follows:

[0031] (8)

[0032] in, is the original signal, expressed at time The purpose of STFT is to analyze the characteristics of the EEG signal in time and frequency. represents a moving window function, where Indicates the position of the center point of the window. is a window function used to Up signal Weighted, is a time index used to identify the position of the window sliding on the signal, is the sliding step length, is a complex exponential term, representing the sinusoidal component in the frequency domain, where is an imaginary unit, is time, ω is angular frequency, and the time domain signal is converted to frequency domain representation by integrating the entire time domain signal;

[0033] After extracting the time-frequency features of the EEG data, the ResNet18 network with the SE module and spatial attention mechanism is used to pre-extract features of the time-frequency graph.

[0034] Preferably, the input image is preliminarily processed using a 7×7 convolution kernel and a convolution operation with a stride of 2, followed by a 3×3 maximum pooling layer to reduce computational complexity and extract local features.

[0035] Preferably, the core of the feature pre-extraction module consists of four residual modules, each of which consists of two 3×3 convolutional layers;

[0036] Each convolutional layer is followed by batch normalization and ReLU activation function is applied to ensure the stability and nonlinearity of the network;

[0037] After the second convolution, the module is further batch normalized, and the SE module and spatial attention mechanism are introduced to enhance feature expression.

[0038] Preferably, global average pooling is performed, and the pooled feature , the calculation formula is:

[0039] (9)

[0040] Among them, by the feature map Global average pooling in the spatial dimension obtains the representation channel Compression feature , and are the height and width of the feature map, respectively. It is the first Channels in The value of .

[0041] Preferably, an Excitation operation is performed and a weight vector is generated;

[0042] The weight vector is normalized by the Sigmoid function to indicate the importance of each channel. The channel weighting process is as follows:

[0043] (10)

[0044] (11)

[0045] In formula (10), and is the weight matrix of the fully connected layer, ReLU is the activation function used to introduce nonlinearity, Is the Sigmoid activation function, used to generate channel weights , Indicates The weight of each channel;

[0046] In formula (11), Represents the weighted feature map value, through the channel weight Adjustment.

[0047] Preferably, generating a spatial attention map;

[0048] The calculation formula of the spatial attention map is:

[0049] (12)

[0050] in, is the output spatial attention map, is the Sigmoid activation function, It represents a 7×7 convolution operation, which is used to generate a spatial attention map. represents concatenation in the channel dimension, It is the input feature map The feature map obtained by average pooling is It is the input feature map The feature map obtained by performing maximum pooling.

[0051] Preferably, a capsule layer and a dynamic routing algorithm are introduced to construct a capsule network prediction module;

[0052] The capsule layer represents the features as the vector output of a group of neurons, namely the capsule form, and maps the feature map into a higher-dimensional capsule representation;

[0053] The capsule network prediction module consists of two parts: the primary capsule layer and the advanced capsule layer.

[0054] Preferably, the primary capsule layer converts the output of the feature pre-extraction module into a capsule form through 1×1 convolution, and each primary capsule contains multiple vectors representing different aspects of the input feature map;

[0055] The output of the primary capsule layer is normalized by the capsule scaling (Squash) function to ensure that the norm of the output vector is between 0 and 1. The formula is as follows, where is the input vector, is the scaled output vector;

[0056] (13)

[0057] Preferably, the advanced capsule layer further processes the output of the primary capsule through a dynamic routing algorithm to achieve the prediction of epilepsy;

[0058] Use the MarginLoss loss function to calculate the classification loss of the model;

[0059] MarginLoss optimizes the model by calculating the Euclidean norm of each capsule output vector to evaluate the activation value, maximizing the activation value of the correctly predicted capsule and minimizing the activation value of the incorrectly predicted capsule, thereby enhancing the model's sensitivity to the prediction boundary.

[0060] In order to improve the stability and efficiency of training, an exponential decay learning rate strategy is chosen; the Euclidean norm and MarginLoss loss function formulas are as follows:

[0061] (14)

[0062] (15)

[0063] In formula (14), It is The output of a capsule, It is The first capsule output vector Quantity, It is The Euclidean norm of the capsule output vector, which is the square root of the sum of the squares of all components;

[0064] In formula (15), It is The loss of capsules, is an indicator variable, when the category It is 1 if it is the target category, otherwise it is 0. is the target norm threshold of positive samples, is the target norm threshold of negative samples, Represents the weight of the negative sample loss and adjusts the loss impact of negative samples.

[0065] The present invention has at least the following beneficial effects:

[0066] 1. This scheme makes full use of the multi-dimensional characteristics of EEG data. First, CSP is used to enhance the spatial characteristics of the data. Then, STFT is used to convert the EEG data into a time-frequency diagram. After spatial domain analysis, time-frequency analysis of the data is realized. Multi-dimensional information is combined to realize data representation for epilepsy prediction.

[0067] 2. The fusion of SE module and spatial attention mechanism enhances the feature expression ability of the network. This improvement enables the model to adaptively adjust the importance of feature channels and further enhance the expression ability of spatial information. The SE module learns channel weights through global average pooling and fully connected layers, and can adaptively emphasize feature channels that are more useful for classification tasks; the spatial attention mechanism strengthens the features of important spatial positions by generating spatial attention maps, thereby improving the accuracy and robustness of feature extraction;

[0068] 3. Previous methods of epileptic seizure prediction based on time-frequency graphs have limitations such as information loss and high computational complexity. The vectorized representation and dynamic routing mechanism of capsule networks make them more sensitive to complex and subtle changes. They not only focus on the existence or absence of features, but also on the relative position and posture changes between features, and better capture the spatial and hierarchical relationships in time-frequency graphs, thereby improving the accuracy of epileptic seizure prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a schematic diagram of the overall architecture of the epilepsy onset prediction and diagnosis method based on EEG spatiotemporal and frequency multi-domain features of the present invention;

[0070] Figure 2 Schematic diagram of the architecture of adding SE module and spatial attention mechanism to the present invention. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] Embodiment 1

[0073] See also Figure 1 - Figure 2, a method for predicting and diagnosing epilepsy based on multi-domain features of EEG signals, including an epilepsy prediction model based on MSA-ResNet CapsNet, which mainly includes three modules, namely, a spatial feature enhancement module, a feature preprocessing module and a capsule network prediction module. The spatial feature enhancement module uses the CSP algorithm to select a set of optimal spatial filters for projection, enhance the EEG spatial features, and maximize the difference between the two types of EEG data; the feature preprocessing module first extracts the time-frequency features of the EEG data after spatial filtering through STFT, and converts it into a time-frequency graph, and then uses the ResNet18 network combined with the SE module and the spatial attention mechanism to perform feature preprocessing and enhancement on the time-frequency graph; the capsule network prediction module includes a primary capsule layer and a high-level capsule layer. The primary capsule layer converts the extracted feature representation into a capsule form, and each capsule is represented by a set of vectors, which can better capture the spatial information of the input feature; the high-level capsule layer uses a dynamic routing mechanism to establish a connection between the primary capsule layer and the high-level capsule layer, update the network parameters, avoid the loss caused by the pooling operation, and finally output a feature vector, whose length is the probability that the sample belongs to a certain class, thereby realizing the prediction of epilepsy;

[0074] Spatial feature enhancement module:

[0075] In order to fully consider the characteristics of EEG data containing multi-dimensional information, the CSP method is first applied to process the spatial dimension information and enhance the spatial features before the time-frequency analysis. CSP is an algorithm used to enhance the spatial differences of electroencephalogram (EEG) signals under different states. The original EEG signal is projected into a new space through linear transformation, so that the variance difference of the projected signal under different states is maximized. The CSP method is applied before the time-frequency graph conversion, which reduces the computational complexity of subsequent processing and analysis. This preprocessing step makes the features of the time-frequency graph conversion more significant, improves the feature recognition and the performance of the prediction model. This innovative method not only improves the accuracy and reliability of epileptic seizure prediction, but also provides new ideas for multi-dimensional feature fusion. Enhance spatial features.

[0076] First, the covariance matrices of the two types of EEG data, pre-ictal and interictal, are calculated respectively to capture the linear relationship and variation characteristics between each channel, so as to reflect the correlation between each channel in the EEG data; then, the covariance matrices of the pre-ictal and interictal periods are added to obtain the total covariance matrix for subsequent feature extraction; then, the total covariance matrix is ​​subjected to eigenvalue decomposition to obtain the eigenvalues ​​and eigenvectors; in order to standardize the data, the total covariance matrix is ​​subjected to whitening transformation; then, the EEG data is subjected to linear transformation to highlight the difference between the pre-ictal and interictal periods; then, the transformed covariance matrix is ​​subjected to generalized eigenvalue decomposition to find a set of optimal spatial filters that enhance the distinguishability of the two types of data, namely, the CSP projection matrix;

[0077] After obtaining the CSP projection matrix, it is applied to the original EEG data to perform a linear transformation on the data. Through this transformation, the various channels in the original EEG signal will be recombined so that the signals before and after the seizure have more significant differences in the transformed space. This process actually achieves the filtering of EEG data, retaining the signal components that are most useful for distinguishing different states, while suppressing noise and irrelevant components, thereby extracting clearer and more distinguishing spatial features. The specific operation formula is as follows:

[0078] (1)

[0079] (2)

[0080] (3)

[0081] (4)

[0082] (5)

[0083] (6)

[0084] (7)

[0085] Among them, (1) is used to calculate the covariance matrix, is the covariance matrix, which is a matrix that describes the linear relationship between EEG signals in each channel. It is EEG signal vector, Indicates the number of samples, that is, the total number of EEG signal samples; calculate The trace of , i.e. the sum of the diagonal elements, is used for normalization. The role of (2) is to calculate the total covariance matrix, represents the total covariance matrix, which is the sum of the covariance matrices of the preictal and interictal periods; is the covariance matrix of the preictal period; is the covariance matrix of the interictal period. (3) is the generalized eigenvalue decomposition, is the eigenvector matrix, is the total covariance matrix The characteristic vector of is the diagonal eigenvalue matrix, is the total covariance matrix The function of (4) is to calculate the whitening matrix, is the whitening matrix, the total covariance matrix Perform whitening transformation to standardize the covariance matrix; represents a diagonal matrix containing The negative square root of each eigenvalue of is used for the whitening process; represents the transpose of the eigenvector matrix. (5) represents the whitening transformation of the covariance matrix, represents the whitening covariance matrix of the pre-ictal period; represents the whitened covariance matrix of the interictal period, is the transpose of the whitening matrix. (6) represents the generalized eigenvalue decomposition, Represents the CSP projection matrix, which is a matrix of eigenvectors used to map data to a new feature space; is a diagonal eigenvalue matrix, which represents the distribution of data in the new feature space. The function of (7) is to calculate the CSP projection matrix, is the final CSP projection matrix, which is used to project the original EEG data into a new feature space to enhance the discrimination between preictal and interictal periods.

[0086] Feature pre-extraction module:

[0087] After processing the spatial domain information, the EEG data is subjected to time-frequency analysis, and the STFT method is used to convert the EEG data into a time-frequency diagram that can provide both time and frequency information. STFT applies a sliding window function to the signal, divides the signal into multiple short time periods and slides across the entire signal. In order to capture subtle changes in the signal over time while maintaining a high frequency resolution, the data is segmented with a window length of 10 seconds and a sliding step of 0.5 seconds. The Fourier transform of the EEG signal in each time period covered by the window function is then calculated to obtain the frequency domain components of the entire time period. The definition of STFT is as follows:

[0088] (8)

[0089] in, is the original signal, expressed at time The purpose of STFT is to analyze the characteristics of the EEG signal in time and frequency. represents a moving window function, where Indicates the position of the center point of the window. is a window function used to Up signal Weighted, is a time index used to identify the position of the window sliding on the signal, is the sliding step length, is a complex exponential term, representing the sinusoidal component in the frequency domain, where is an imaginary unit, is time, ω is angular frequency, and the time domain signal is converted to frequency domain representation by integrating the entire time domain signal.

[0090] After extracting the time-frequency features of the EEG data, the ResNet18 network with the SE module and spatial attention mechanism is used to pre-extract features from the time-frequency graph. The SE module can enhance important features by adaptively adjusting the weights of feature channels and suppress irrelevant or minor features to better capture the relationship between channels; then the spatial attention mechanism is introduced at the output of each residual block to focus on more important spatial areas and ignore unimportant areas by adjusting the weights of the feature map in the spatial dimension, so as to better capture spatial structural information and enhance the network's feature expression ability. Figure 2 The basic architecture with the addition of SE modules and spatial attention mechanism is shown.

[0091] In order to pre-extract features before prediction, the input image is first processed using a 7×7 convolution kernel and a convolution operation with a stride of 2, followed by a 3×3 maximum pooling layer to reduce computational complexity and extract local features. The core of the feature pre-extraction module consists of four residual blocks, each of which consists of two 3×3 convolutional layers. Batch normalization is performed after each convolution layer, and the ReLU activation function is applied to ensure the stability and nonlinearity of the network. After the second convolution, the module is further batch normalized, and the SE module and spatial attention mechanism are introduced to enhance feature expression.

[0092] The SE module improves network performance by adaptively adjusting the importance of feature channels. First, the input feature map is compressed in the spatial dimension through global average pooling (Squeeze operation), and the feature map of each channel is converted into a scalar value to obtain the global information of each channel. channels, each channel size is The input feature map, the pooled feature The calculation formula is as follows (9), where the feature map Global average pooling in the spatial dimension obtains the representation channel Compression feature , and are the height and width of the feature map, respectively. It is the first Channels in The value of .

[0093] (9)

[0094] Then, the Excitation operation is performed to learn the relationship between feature channels through two fully connected layers and generate a weight vector. This weight vector is normalized by the Sigmoid function to indicate the importance of each channel. The weighting process of the channel is as follows:

[0095] (10)

[0096] (11)

[0097] In formula (10), and is the weight matrix of the fully connected layer, ReLU is the activation function used to introduce nonlinearity, Is the Sigmoid activation function, used to generate channel weights , Indicates The weight of each channel. In formula (11), Represents the weighted feature map value, through the channel weight Adjustment.

[0098] The spatial attention mechanism further improves the expressiveness of feature maps. First, a spatial attention map is generated, and the input feature map is processed in the channel dimension through average pooling and maximum pooling to obtain two feature maps. Then, the two feature maps are concatenated in the channel dimension and a spatial attention map is generated through a 7×7 convolution layer. Finally, the generated spatial attention map is element-wise multiplied with the input feature map to enhance important spatial locations. The calculation formula of the spatial attention map is:

[0099] (12)

[0100] in, is the output spatial attention map, is the Sigmoid activation function, It represents a 7×7 convolution operation, which is used to generate a spatial attention map. represents concatenation in the channel dimension, It is the input feature map The feature map obtained by average pooling is It is the input feature map The feature map obtained by performing maximum pooling.

[0101] Capsule classification module:

[0102] In the prediction of epilepsy, although traditional convolutional neural networks perform well in feature extraction, they still have certain limitations in capturing complex spatial relationships and feature hierarchical expressions. The capsule network introduces capsule layers and dynamic routing algorithms. The capsule layer represents features as a vector output of a set of neurons, namely capsule form, and maps feature maps to higher-dimensional capsule representations. The dynamic routing algorithm enables the capsule network to dynamically adjust the information transmission path between different layers and more effectively process the complex relationships of data, thus showing significant advantages in epilepsy prediction based on time-frequency graphs. The capsule network prediction module consists of two parts, namely the primary capsule layer and the advanced capsule layer.

[0103] The primary capsule layer includes multiple primary capsules. The primary capsule layer converts the output of the feature pre-extraction module into a capsule form through 1×1 convolution. Each primary capsule contains multiple vectors representing different aspects of the input feature map. The output of the primary capsule layer is normalized by the capsule scaling (Squash) function to ensure that the norm of the output vector is between 0 and 1. The formula is as follows, where is the input vector, is the scaled output vector.

[0104] (13)

[0105] The advanced capsule layer includes multiple advanced capsules, which receive the output of the primary capsules. The advanced capsule layer further processes the output of the primary capsule layer through a dynamic routing algorithm to achieve the prediction of epilepsy. In the dynamic routing algorithm, the output vector of the primary capsule is gradually assigned to the advanced capsules in the advanced capsule layer through multiple iterations. In this process, the coupling coefficient is updated according to the relationship between the primary capsule and the advanced capsule, so as to effectively transmit and aggregate information and finally obtain the output capsule. Initialize the coupling coefficient is 0, and then the transformation matrix updated by back propagation is designed , each primary capsule Generate the initial prediction vector by transforming the matrix ; Use the softmax function to calculate the coupling coefficient , the input of the advanced capsule is obtained by weighting the initial prediction vector according to the coupling coefficient , calculate the output vector through the Squash function ,according to and Update coupling coefficients Repeat the above process until the coupling coefficient and output vector converge. The final advanced capsule output vector A high-level capsule representation containing input features, reflecting the category or other attributes of the input data.

[0106] In order to enhance the prediction ability of the model, the MarginLoss loss function is used to calculate the classification loss of the model. MarginLoss optimizes the model by calculating the Euclidean norm of each capsule output vector to evaluate the activation value, maximize the activation value of the correctly predicted capsule, and minimize the activation value of the incorrectly predicted capsule to enhance the model's sensitivity to the prediction boundary. In order to improve the stability and efficiency of training, an exponential decay learning rate strategy is chosen. The formulas for the Euclidean norm and MarginLoss loss function are as follows:

[0107] (14)

[0108] (15)

[0109] In formula (14), It is The output of a capsule, It is The first capsule output vector Quantity, It is The Euclidean norm of the capsule output vector is the square root of the sum of the squares of all components. In formula (15), It is The loss of capsules, is an indicator variable, when the category It is 1 if it is the target category, otherwise it is 0. is the target norm threshold of positive samples, is the target norm threshold of negative samples, Represents the weight of the negative sample loss and adjusts the loss impact of negative samples.

[0110] Experimental dataset:

[0111] We conducted epileptic seizure prediction experiments on the CHB-MIT dataset publicly available from Boston Children's Hospital. The following are the dataset details:

[0112] The dataset has continuous and non-continuous EEG recordings from 24 cases, each case contains 9 to 42 multi-channel EEG recordings, with a total of 664 recording files. The recordings were made at a sampling frequency of 256 Hz with 16-bit resolution. Although most of the data contains 23 channels, cases with EEG or vagus nerve stimulation signals are also included.

[0113] Specific implementation:

[0114] The experimental hardware platform is configured with a 3.6GHz Intel Core i7 12700K processor, 32G memory, and a GeForce RTX 3090 graphics card with 24G video memory. The programming environment is Windows 10 operating system, and Python 3.8 and TensorFlow framework are used to build the model. All experimental methods are run 5 times, and the average is taken as the final result.

[0115] We take the thirty minutes before the onset as the preictal data. Since continuous onset will affect the data when intercepting the preictal EEG data, if the interval between consecutive onsets of the same patient is less than two hours, it will be regarded as one epileptic onset, that is, the second onset of consecutive onsets (interval less than 120 minutes) will be ignored. By eliminating data with a preictal time of less than half an hour and continuous onset, the final epilepsy data set information is shown in Table 1. In order to unify the position and number of sampling electrodes, remove duplicate electrodes and electrodes with unclear electrode positions, and finally select 18 sampling electrodes, including: FP1-F7, F7-T7, T7-P7, P7-O1, FP1-F3, F3-C3, C3-P3, P3-O1, FP2-F4, F4-C4, C4-P4, P4-O2, FP2-F8, F8-T8, T8-P8, P8-O2, FZ-CZ, CZ-PZ. Semi-overlapping 10-second time windows were selected for data segmentation to ensure data continuity, and then a 0-50 Hz bandpass filter was used for filtering, and artifacts such as electrooculography and electromyography were removed by ICA to reduce manual data processing.

[0116] To reduce the subjective influence of the EEG data, ICLabel, a tool that can automatically identify and label artifact components in EEG data, was used to remove artifacts and retain the true EEG activity. Finally, to ensure the balance of sample size, the interictal samples were processed in equal amounts according to the number of preictal samples of each subject.

[0117] Table 1 Epilepsy dataset information

[0118]

[0119] Experimental comparison results:

[0120] The experimental results are compared with several state-of-the-art methods on the CHB-MIT dataset.

[0121] Since the evaluation indicators used in the existing methods are not uniform, Tables 2 to 4 show the comparison of the results of the methods for a single indicator, namely accuracy, sensitivity and AUC (area under the curve). Truong et al. used short-time Fourier transform to extract information in the time domain and frequency domain, and designed a convolutional neural network to optimize features for each patient. Theekshana et al. designed two epilepsy prediction methods. The first one used the Siamese model to learn the variation patterns between patients so as to adapt to the data of different patients when predicting epileptic seizures; the second one applied the feature extraction method MFCC (Mel-Frequency Cepstral Coefficients) commonly used in audio and speech processing to EEG data. By performing Mel filter groups and discrete cosine transform (DCT) on the EEG signal, the MFCC feature map was obtained to represent the EEG activity in different time periods, realizing patient-independent epileptic seizure prediction. Xu et al. proposed a pre-ictal artificial signal synthesis algorithm based on generative adversarial networks to generate synthetic multi-channel EEG pre-ictal samples, and then used convolutional neural networks to achieve epilepsy prediction. Wang et al. proposed a novel convolutional neural network to analyze the time, frequency and channel information of EEG signals, and applied multi-scale dilated convolution to obtain a more flexible receptive field, thereby realizing epilepsy prediction. Deng et al. [1] proposed a new consistency-based semi-supervised epilepsy seizure prediction model (CSSPM), in which only a small part of the training data is labeled. By using random enhancement and dropout, the entire neural network is regarded as a random model, and consistency constraints are applied to penalize the difference between the current prediction and the previous prediction. Unlabeled data can be fully utilized to improve the decision boundary and improve the prediction performance. Deng et al. [2] proposed a consistency-based training strategy, which uses random enhancement to make the input vary between iterations during training, and regards the output as a random variable. Then, consistency constraints are constructed to penalize the difference between the current output and the previous output, thereby enhancing the generalization of the model. Nhan et al. used short-time Fourier transform on a 28-second EEG window as a preprocessing step, and trained the generative adversarial network in an unsupervised manner. The trained discriminator is then used as a feature extractor. The features generated by the feature extractor are used by two fully connected layers to predict the labeled EEG signals.

[0122] Liu et al. proposed a multi-view convolutional neural network framework to analyze frequency and time domain features to provide two different views of the same data source, with the goal of obtaining a shared representation of time and frequency domain features to predict epileptic seizures.

[0123] Bahareh et al. proposed a convolutional neural network auxiliary factor graph to evaluate the correlation between different EEG channels as features using neural mutual information estimation. We then used 1D-CNN to extract additional features from the EEG signal and used these two features to estimate the probability of epileptic seizure events.

[0124] Table 2 Accuracy comparison with the state-of-the-art results on the CHB-MIT dataset

[0125]

[0126] Table 3. Sensitivity comparison with the state-of-the-art results on the CHB-MIT dataset

[0127]

[0128] Table 4 AUC comparison with the state-of-the-art results on the CHB-MIT dataset

[0129]

[0130] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0131] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

Claims

1. A method for predicting and diagnosing epilepsy based on multi-domain features of EEG signals, characterized in that: It includes the following steps: S1: Perform feature extraction; calculate the covariance matrix of the two types of EEG data, pre-ictal and interictal, to capture the linear relationship and variation characteristics between each channel; S2: Enhanced spatial features; Use the CSP algorithm to select a set of optimal spatial filters for projection and enhance the EEG spatial features; Firstly, the covariance matrix of the two types of EEG data, pre-ictal and interictal, was calculated to capture the linear relationship and variation characteristics between channels, thus reflecting the correlation between channels in the EEG data. Next, the covariance matrices of the pre-ictal and interictal periods are summed to obtain the total covariance matrix for subsequent feature extraction; After that, the total covariance matrix is ​​subjected to eigenvalue decomposition to obtain the eigenvalues ​​and eigenvectors; in order to standardize the data, the total covariance matrix is ​​subjected to whitening transformation; Next, the EEG data were linearly transformed to highlight the differences between the preictal and interictal periods; Then, the transformed covariance matrix is ​​subjected to generalized eigenvalue decomposition to find a set of optimal spatial filters that enhance the distinguishability of the two types of data, namely the CSP projection matrix. After obtaining the CSP projection matrix, it is applied to the original EEG data to perform a linear transformation on the data. After processing the spatial domain information, the EEG data is subjected to time-frequency analysis, and the STFT method is used to convert the EEG data into a time-frequency diagram that can provide both time and frequency information; After extracting the time-frequency features of the EEG data, the ResNet18 network with the SE module and spatial attention mechanism is used to pre-extract features from the time-frequency graph; S3: Perform capsule classification and build a capsule network prediction module.

2. The method for predicting and diagnosing epilepsy based on multi-domain features of EEG signals according to claim 1, characterized in that: Perform computational analysis on the data; The specific formula is as follows: (1) (2) (3) (4) (5) (6) (7) Among them, (1) is used to calculate the covariance matrix, is the covariance matrix, which is a matrix describing the linear relationship between EEG signals in various channels. It is EEG signal vector, Indicates the number of samples, that is, the total number of EEG signal samples; calculate The trace of , i.e. the sum of the diagonal elements, is used for normalization; Among them, (2) is used to calculate the total covariance matrix, represents the total covariance matrix, which is the sum of the covariance matrices of the preictal and interictal periods; is the covariance matrix of the preictal period; is the covariance matrix of the interictal intervals; Among them, (3) is the generalized eigenvalue decomposition, is the eigenvector matrix, is the total covariance matrix The characteristic vector of is the diagonal eigenvalue matrix, is the total covariance matrix The characteristic value of Among them, (4) is used to calculate the whitening matrix, is the whitening matrix, the total covariance matrix Perform whitening transformation to standardize the covariance matrix; represents a diagonal matrix containing The negative square root of each eigenvalue of is used for the whitening process; represents the transpose of the eigenvector matrix; Among them, (5) represents the whitening transformation of the covariance matrix, represents the whitening covariance matrix of the pre-ictal period; represents the whitened covariance matrix of the interictal period, is the transpose of the whitening matrix; Where (6) represents the generalized eigenvalue decomposition, Represents the CSP projection matrix, which is a matrix of eigenvectors used to map data to a new feature space; is a diagonal eigenvalue matrix, which represents the distribution of data in the new feature space; Among them, (7) is used to calculate the CSP projection matrix, is the final CSP projection matrix, which is used to project the original EEG data into a new feature space to enhance the discrimination between preictal and interictal periods.

3. The method for predicting and diagnosing epilepsy based on multi-domain features of EEG signals according to claim 2, characterized in that: The definition of STFT is as follows: (8) in, is the original signal, expressed at time The purpose of STFT is to analyze the characteristics of the EEG signal in time and frequency. represents a moving window function, where Indicates the position of the center point of the window. is a window function used to Up signal Weighted, is a time index used to identify the position of the window sliding on the signal, is the sliding step length, is a complex exponential term, representing the sinusoidal component in the frequency domain, where is an imaginary unit, is time, ω is angular frequency, and the time domain signal is converted to frequency domain representation by integrating the entire time domain signal.

4. The method for predicting and diagnosing epilepsy based on multi-domain features of EEG signals according to claim 3, characterized in that: The input image is preliminarily processed using a 7×7 convolution kernel and a convolution operation with a stride of 2, followed by a 3×3 maximum pooling layer to reduce computational complexity and extract local features.

5. The method for predicting and diagnosing epilepsy based on multi-domain features of EEG signals according to claim 4, characterized in that: The core of the feature pre-extraction module consists of four residual modules, each of which consists of two 3×3 convolutional layers; Each convolutional layer is followed by batch normalization and ReLU activation function is applied to ensure the stability and nonlinearity of the network; After the second convolution, the module is further batch normalized, and the SE module and spatial attention mechanism are introduced to enhance feature expression.

6. The method for predicting and diagnosing epilepsy based on multi-domain features of EEG signals according to claim 5, characterized in that: Perform global average pooling, and the pooled features , the feature map is recorded as , the calculation formula is: (9) Among them, by the feature map Global average pooling in the spatial dimension obtains the representation channel Compression feature , and are the height and width of the feature map, respectively. It is the first Channels in The value of .

7. The method for predicting and diagnosing epilepsy based on multi-domain features of EEG signals according to claim 6, characterized in that: Perform Excitation operation and generate a weight vector; The weight vector is normalized by the Sigmoid function to indicate the importance of each channel. The channel weighting process is as follows: (10) (11) In formula (10), and is the weight matrix of the fully connected layer, ReLU is the activation function used to introduce nonlinearity, Is the Sigmoid activation function, used to generate channel weights , Indicates The weight of each channel; In formula (11), Represents the weighted feature map value, through the channel weight Adjustment.

8. The method for predicting and diagnosing epilepsy based on multi-domain features of EEG signals according to claim 7, characterized in that: Generate spatial attention map; The calculation formula of the spatial attention map is: (12) in, is the output spatial attention map, is the Sigmoid activation function, It represents a 7×7 convolution operation, which is used to generate a spatial attention map. represents concatenation in the channel dimension, It is the input feature map The feature map obtained by average pooling is It is the input feature map The feature map obtained by performing maximum pooling.

9. The method for predicting and diagnosing epilepsy based on multi-domain features of EEG signals according to claim 8, characterized in that: Introduce capsule layer and dynamic routing algorithm to build capsule network prediction module; The capsule layer represents the features as the vector output of a group of neurons, namely the capsule form, and maps the feature map into a higher-dimensional capsule representation; The capsule network prediction module consists of two parts: the primary capsule layer and the advanced capsule layer.

10. The epilepsy prediction and diagnosis method based on multi-domain features of EEG signals according to claim 9, characterized in that: The primary capsule layer converts the output of the feature pre-extraction module into capsule form through 1×1 convolution. Each primary capsule contains multiple vectors representing different aspects of the input feature map. The output of the primary capsule layer is normalized by the capsule scaling (Squash) function to ensure that the norm of the output vector is between 0 and 1. The formula is as follows: (13) in, is the input vector, is the scaled output vector; The advanced capsule layer further processes the output of the primary capsule through a dynamic routing algorithm to achieve the prediction of epilepsy; Use the MarginLoss loss function to calculate the classification loss of the model; MarginLoss optimizes the model by calculating the Euclidean norm of each capsule output vector to evaluate the activation value, maximizing the activation value of the correctly predicted capsule and minimizing the activation value of the incorrectly predicted capsule, thereby enhancing the model's sensitivity to the prediction boundary. In order to improve the stability and efficiency of training, an exponential decay learning rate strategy is chosen; the Euclidean norm and MarginLoss loss function formulas are as follows: (14) (15) In formula (14), It is The output of a capsule, It is The first capsule output vector Quantity, It is The Euclidean norm of the capsule output vector, which is the square root of the sum of the squares of all components; In formula (15), It is The loss of capsules, is an indicator variable, when the category It is 1 if it is the target category, otherwise it is 0. is the target norm threshold of positive samples, is the target norm threshold of negative samples, Represents the weight of the negative sample loss and adjusts the loss impact of negative samples.