Epileptic seizure prediction method based on multi-scale mixed graph attention convolutional network

Abstract

The invention discloses an epileptic seizure prediction method based on a multi-scale mixed graph attention convolutional network. The epileptic seizure prediction method comprises the following steps: 1, performing data processing on electroencephalogram signals; 2, processing an electroencephalogram signal space sequence through graph construction and convolution; 3, the global correlation of the electroencephalogram signal time sequence is processed through an iTransform (Inverted Transform) model, and an element sequence is output by a decoder; 4, processing electroencephalogram signal time sequence local correlation through a multi-scale time adaptive network, and predicting epileptic seizure; and 5, classifying the epilepsy types of the discriminated seizures through a classifier. Compared with a traditional electroencephalogram diagnosis method, an iTransform model is introduced, input is converted into segment sequence level features from time node level features, and representation of time sequence features is optimized through an adaptive attention mechanism and a feedforward network. In addition, the method also adopts a multi-scale time adaptive network, so that the accuracy and timeliness of epileptic seizure prediction are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of general electroencephalogram signal data processing or generation, and particularly to a method for predicting epileptic seizures based on a multi-scale hybrid graph attention convolutional network. Background Art

[0002] Epilepsy is a common neurological disorder characterized by abnormal electrical discharges of neurons in the brain, resulting in recurrent transient brain dysfunction. Epileptic seizures can manifest as sudden loss of consciousness, convulsions, perceptual abnormalities, emotional abnormalities, etc., and the seizure types are diverse, with common types including generalized seizures and partial seizures.

[0003] Electroencephalogram (EEG) signals have specific frequency band characteristics during seizures. It contains rich spatio-temporal information of brain activities, which is used to discover the dynamic functional connections between different regions of the brain, identify the patterns of brain diseases, and localize lesions. Epileptic seizures can be predicted by observing EEG signals. Therefore, in clinical diagnosis, medical workers usually perform long-term EEG monitoring on patients and manually analyze relevant EEG characteristics to diagnose the condition. However, doctors usually have difficulty accurately predicting the occurrence of the disease. Moreover, this manual analysis is extremely time-consuming and laborious, with the key problems of high labor costs and low efficiency.

[0004] EEG data is sourced from electrodes at multiple locations in the brain, and there are complex spatial relationships between these electrodes. At the same time, due to the non-linearity and non-stationarity of EEG signals, it is difficult to effectively extract the temporal characteristic relationships of EEG.

[0005] When predicting epileptic seizures, it is not enough to merely predict whether an epileptic seizure occurs. If the type of epileptic seizure cannot be quickly determined, it will bring great difficulties to doctors in curing patients. Summary of the Invention

[0006] The purpose of the present invention is to propose a new multi-scale hybrid graph attention convolutional network model and apply it to epileptic seizure prediction. By deeply analyzing the EEG signals recorded from different regions of the patient's scalp at different time distances, the dynamic functional connections of the brain are discovered. This method realizes the dual attention to the global and local correlations of EEG signals, activates the multi-scale modeling ability through an adaptive approach, and endows it with adaptive modeling characteristics, further improving the accuracy of epileptic seizure prediction. At the same time, it quickly locates the epileptic seizure type, effectively promoting the precise prediction of epilepsy.

[0007] In terms of global correlation, a brand-new Transformer model, namely the iTransformer (Inverted Transformer) model, is introduced. Different from the general Transformer model, the iTransformer model changes the input from time-node-level features to segment-sequence-level features. Without changing the network architecture of the Transformer model, it transforms the roles of the attention mechanism and the feed-forward network. The iTransformer model considers different variables separately. Each variable is encoded into independent temporal features. The attention mechanism is used to model the correlation between different variables, and at the same time, the feed-forward network is used to model the temporal correlation of variables to obtain a better sequence temporal representation.

[0008] In terms of local correlation, a multi-scale time adaptive network is introduced. Compared with the Pathformer (PathwayTransformer) network, the multi-scale adaptive network divides the time series with different time steps, and adaptively combines these multi-scale features through weighted aggregation. Combined with the graph attention convolutional network, it can effectively capture the dynamic spatial connection relationship between brain electrodes, so as to extract the spatio-temporal relationship between nodes.

[0009] According to the present invention, a multi-scale hybrid graph attention convolutional network epilepsy seizure prediction method is provided, including the following steps:

[0010] Perform data processing on the electroencephalogram (EEG) signal. By obtaining the EEG signal data to be detected and preprocessing the obtained EEG data to be detected, a correlation coefficient matrix is obtained.

[0011] Construct and perform convolutional processing on the EEG signal spatial sequence. Extract the EEG signal electrode positions and construct them into a graph structure, and perform graph convolution operations to generate local and global spatial representations.

[0012] Process the global correlation of the EEG signal time series through the iTransformer model, and the decoder outputs an element sequence. Using the time features as input, the attention mechanism is used to model the correlation between different variables and the feed-forward network is used to model the temporal correlation of variables to obtain a better sequence temporal representation.

[0013] Process the local correlation of the EEG signal time series through the multi-scale time adaptive network and predict epileptic seizures. By analyzing different time scales, extract the features of the input data at multiple time levels, identify short-term and long-term dependence relationships, and predict the graph of future EEG signals according to the dependence relationships.

[0014] Classify the epileptic seizure types determined by the classifier, and further localize the category of epilepsy for the EEG signal graph predicting epileptic seizures.

[0015] Compared with the existing inventions, the beneficial effects of the present invention are as follows:

[0016] The present invention conducts multi-scale depth analysis on the electroencephalogram (EEG) signals recorded in different regions of the brain, and proposes an epilepsy seizure prediction method based on a multi-scale hybrid graph attention convolutional network. This method takes into account the global and local correlations between the frequencies of EEG signals. By combining multi-layer cross-attention mechanisms with graph convolution, it realizes the spatio-temporal collaborative representation of EEG signals, and captures the spatial and temporal features discovered by high-resolution dynamic functional connectivity, thereby reducing the loss of key information in EEG signals and further improving the accuracy of epilepsy prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flowchart of the epilepsy seizure prediction technology based on a multi-scale hybrid graph attention convolutional network.

[0018] Figure 2 is a schematic diagram of the epilepsy seizure prediction technology based on a multi-scale hybrid graph attention convolutional network.

[0019] Figure 3 is a structure diagram of the iTransformer model.

[0020] Figure 4 is a structure diagram of the multi-scale time adaptive network.

[0021] Figure 5 is a structure diagram of the classifier model. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] Next, the method solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the present invention. And some details in the invention solutions will be supplemented and explained in detail.

[0023] Based on multi-scale spatio-temporal attention and dynamic relationship modeling, the present invention proposes an epilepsy seizure prediction method based on a multi-scale hybrid graph attention convolutional network, and uses the Temple University Hospital EEG Epilepsy Corpus as the dataset. The Temple University Hospital EEG Epilepsy Corpus is the largest public EEG seizure database to date, containing 5,612 EEGs, 3,050 clinically recorded annotated seizures, and 8 seizure types. It is widely used for the research and development of EEG-based epilepsy detection and diagnosis algorithms. Its main process is as Figure 1 shown, and the method principle is as Figure 2 shown. The present invention mainly includes the following steps:

[0024] S100. Obtain the electroencephalogram (EEG) signal data to be detected through a data processing unit, preprocess the obtained EEG data to be detected, map the EEG signal feature data to the nodes in the graph, and map the correlation relationships existing between channels to the connecting edges in the graph. Relying on the processing advantage of the graph convolutional network for irregular structures, construct the electroencephalogram into a graph structure with the structure of H(x,t).

[0025] S101. Among them, x is the EEG signal data of each channel, and t is the connection relationship between channels.

[0026] S102. Before the feature data is input into the graph convolutional model, it is necessary to calculate the correlation coefficient between its channels.

[0027] S103. The present invention uses the Pearson correlation analysis method to calculate the correlation coefficient between channels. The Pearson Correlation Coefficient (PCC) is used to represent the correlation existing between element a and element b, reflecting the degree of their correlation. The specific calculation formula is as follows:

[0028] S104.

[0029] S105. Among them, E((a - μ a )(b - μ b )) = E(ab) - E(a)E(b)

[0030] S106. Among them,

[0031] S107. Among them, a and b are the signals of each node. The larger the absolute value of the Pearson correlation coefficient, the stronger the correlation between channels, and vice versa, the weaker the correlation.

[0032] S108. To ensure the good performance of the model and facilitate the calculation of the internal structure of the network, the obtained relationship matrix is thresholded. The threshold is set to 0.5. If the correlation relationship coefficient is higher than the threshold, it is recorded as 1, indicating that there is a relationship between channels, that is, there is a connecting edge in the graph structure; if it is lower than the threshold, it is recorded as 0, indicating that there is no connection relationship between channels.

[0033] S200. Process the spatial sequence of EEG signals through graph construction and convolution. After jointly inputting the feature data and relationship data into the graph convolutional layer, calculate the Laplacian matrix L (the difference between the degree matrix and the adjacency matrix) according to the constructed graph structure data. Among them, the degree matrix is a diagonal matrix, and the diagonal elements are the degrees of each vertex, representing the number of edges associated with that vertex; the adjacency matrix represents the relationship between vertices. If there is a connection relationship between the corresponding vertices, it is recorded as 1, and if there is no corresponding connection relationship, it is recorded as 0. Then perform spectral decomposition on the Laplacian matrix to obtain the corresponding eigenvalues and eigenvectors.

[0034] S201, L = D - A

[0035] S202, L = UΛU -1 = UΛU T

[0036] S203, After spectral decomposition, eigenvectors are obtained, forming an orthogonal basis in the space corresponding to the basis of the Fourier transform. Among them, U represents the unit eigenvector matrix, and U T is its orthogonal matrix, and Λ represents the diagonal matrix composed of eigenvalues. Then, through the Fourier transform, the conversion between the spatio-temporal domain and the frequency spectrum domain is carried out, thus effectively completing the graph convolution processing process. The Fourier transform of the graph data can be expressed as:

[0037] S204, GF(x) = U T x

[0038] S205, where x is the electroencephalogram feature data, and GF(x) is the data after Fourier transform, which is converted from the spatial domain dimension to the frequency spectrum domain dimension. U T represents the matrix of unit eigenvectors. The inverse Fourier transform on the graph data, that is, the conversion process from the frequency spectrum domain to the spatio-temporal domain, can be expressed as:

[0039] S206, IGF(x) = Ux

[0040] S207, On this basis, the convolution operation on the graph data can be expressed as:

[0041] S208, y = x * K

[0042] S209, y = IGF(GF(x) · GF(K))

[0043] S210, y = U(U T x ⊙ U T K)

[0044] S211, where x is the input feature data, K is the convolution kernel parameter, and ⊙ is the Hadamard product, that is, a dot multiplication operation is performed. The x and the convolution kernel K are transformed to the spectral domain through the graph Fourier transform, a parametric convolution is performed in the spectral domain, and then the data is transformed back to the node domain through the inverse graph Fourier transform. y is the feature data extracted by the graph convolution filter. If U T K is directly represented as the convolution parameter K α in the frequency spectrum domain, then the above convolution formula can be expressed as:

[0045] S212, Y = x * K = UK α U T x

[0046] S213, Kα = diag([α0, α1 ··· α n-1 )

[0047] S214,

[0048] S215, and finally the output result enters the ReLu activation function for sparsification processing. The above is the main processing process of the electroencephalogram entering the graph convolutional network, and the calculation formula is as follows:

[0049] S216, ReLu(Y) = max(0, Y)

[0050] S217, Z = σ(Y) = ReLu(Y)

[0051] S300, The iTransformer (Inverted Transformer) model is used to process the global correlation of the electroencephalogram signal time series, and the decoder outputs an element sequence. The iTransformer structure is as Figure 3 shown. First, through the iTransformer encoder, the attention mechanism is used to model the correlation between different variables and the feed-forward network is used to model the temporal correlation of variables to obtain a better sequence temporal representation.

[0052] S301, In multivariate time series prediction, given historical observation data X = {x1, …, x T} ∈ R T×N , X has T time steps and N variables, and predicts the future S time steps Y = {x T+1 , …, x T+S} ∈ S×N .

[0053] S302, First, obtain the sequence encoding of a single variable, and map X n ∈ R N×D to H n ∈ R N×T .

[0054] S303, represents the entire time series of each variable indexed by n, and H n represents the mapped representation indexed by n.

[0055] S304, Use multi-head attention (Multivariate-Attention) to analyze the features H n ∈ R N×TThe correlation between them. Query, Key, and Value represent three transformable matrices for training. Scale is calculated by multiplying Query and Key through matrix multiplication (Matmul), and the correlation coefficient is calculated through a multivariate-correlation map and then multiplied with Value through matrix multiplication (Matmul) to finally obtain the time series.

[0056] S305, Further model the temporal features within each variable through a feed-forward neural network to obtain an efficient temporal representation.

[0057] S306, The calculation formula of the feed-forward neural network. X is the input time series, W1 and W2 are weight parameters, and b1 and b2 are learnable hyperparameters:

[0058] S307, max(0, XW1 + b1)W2 + b2

[0059] S308, To ensure variable independence and avoid the impact of sampling misalignment or time lag on the cross-correlation of modeling variables, each variable is normalized separately. The normalization formula is:

[0060] S309,

[0061] S310, x is the time series, μ is the mean of x, and σ is the standard deviation of z.

[0062] S311, Reintroduce the normalized data into the attention mechanism to construct the relationship between graph structures. The attention weight α ij is calculated through the similarity between node features and is based on self-attention calculation:

[0063] S312,

[0064] S313, α ij represents the attention weight of node i to node j. It represents the degree of attention of node i to node j in the graph attention network. h i represents the feature vector of node i. The feature vector h i is usually obtained by linearly transforming or other processing of the node features in the previous layer. h j represents the feature vector of node j. represents the feature vector h of node i i and the feature vector h of node j j is the dot product. The dot product is used to measure the similarity between two feature vectors. represents the set of adjacent nodes of node i. This set contains all the nodes directly connected to node i. It is the exponential function of the dot product of feature vectors. This step performs an exponential transformation on the result of the dot product to ensure that all attention weights are positive and the similarity measure is non-linear. It is the exponential sum of the dot products of the feature vectors of node i with all its adjacent nodes. This part is used for normalization so that the sum of the attention weights of node i to all neighbor nodes is 1.

[0065] S314, the features of each node i are weighted and summed with the features of its adjacent nodes through the attention mechanism, and the updated feature representation is:

[0066] S315,

[0067] S316, where, represents the set of neighbor nodes of node i, α ij represents the attention weight between node i and node j, and Z j is the feature of node j.

[0068] S317, through the decoder, receives the context vector output by the encoder as the initial input.

[0069] S318, outputs the processed feature data as the original electroencephalogram signal map.

[0070] S400, processes the local correlation of the electroencephalogram signal time series through a multi-scale time adaptive method and predicts epileptic seizures. The multi-scale time adaptive network structure is as Figure 4 shown, defines a set S of M patch sizes S = {S1,..., S M}' and each patch size corresponds to a patch partitioning operation.

[0071] S401, for the input time series X ∈ R H×d , after being partitioned by the patch size S, it gets (X 1 , X 2 , …, X P ), and d is the dimension size.

[0072] S402, for the i-th patch, X i ∈ R S×d is linearly mapped to get:

[0073] S403,

[0074] S404, performs a linear mapping in the time dimension to get X trans ∈ R d ,

[0075] S405,

[0076] S406, W r represents a weight matrix for linear transformation; ∈ represents noise sampled from the standard normal distribution N(0, 1); W noise represents another weight matrix; Softplus(X trans W noise ) represents an activation function to ensure non - negative output; Softmax(X trans W r + ∈·Softplus(X trans W noise )) represents converting the output into a probability distribution.

[0077] S407, output the prediction result X out , where is the indicator function, which is 1 at and 0 otherwise,

[0078] S408,

[0079] S500, through the classifier, input the processed feature data into the self - attention layer. Perform time slicing on the encoded one - dimensional matrix, divide the time - series data of a single channel into four one - dimensional segments, and assign different weights through attention training. Then update the time segments through weighted summation, and finally recombine each slice component to obtain the original size for output, as Figure 5 shown. The specific process is as follows:

[0080] S501, T i =(Q i , K i , V i )

[0081] S502,

[0082] S503,

[0083] S504,

[0084] S505,

[0085] S506, where T represents each time slice, T0 - T3 are the four different time - series segments obtained by slicing each input segment, and each slice segment is divided into three parts, Q (Query), K (Key), V (Value) represent three transformable matrices for training. Z i is the input data of different segments is a set of trainable parameter matrices, whose function is to obtain the training matrices Q, K, and V of each time slice through linear transformation with the data part. d k represents the dimension size of K, that is, a scaling calculation is performed. The Q value in the same time period is calculated and compared with its own K value and the K values of the other three time periods, and then the weight coefficients obtained for each segment T are calculated after normalization by the Softmax function. Finally, weighted summation is performed for slice update. Through W O concatenates the outputs of each component and performs a linear transformation to obtain the final output vector of the original size.

[0086] S507, MHA(Z) = (T0, T1, ··· T n-1 )W O

[0087] S508, classify the data through pooling operations to predict different types of epileptic seizures.

Claims

1. A method for predicting epileptic seizures based on a multi-scale mixed graph attention convolutional network, characterized in that: include: Perform data processing on EEG signals; Processing EEG signal spatial sequences through graph construction and convolution; The iTransformer model is used to process the global correlation of the EEG signal time series, and the decoder outputs the element sequence; Processing local correlation of EEG signal time series through multi-scale time adaptive network and predicting epileptic seizures; The classifier is used to classify the type of epilepsy that occurs.

2. The epileptic seizure prediction method based on multi-scale mixed graph attention convolutional network according to claim 1 is characterized in that: The data processing of the EEG signal specifically includes the following steps: The EEG signal data to be detected is obtained through the data processing unit, and the EEG is constructed into a graph structure of structure H(x, t), wherein x is the EEG signal data of each channel, and t is the connection relationship between the channels; Before the feature data is input into the graph convolution model, the inter-channel correlation coefficient needs to be calculated using the Pearson correlation analysis method. The calculation formula is: Among them, a and b are the signals of each node. The larger the absolute value of the Pearson correlation coefficient, the stronger the correlation between channels, and vice versa. The obtained relationship matrix is ​​thresholded and the threshold is set to 0.

5. If the correlation coefficient is higher than the threshold, it is recorded as 1, indicating that there is a relationship between the channels; if it is lower than the threshold, it is recorded as 0, indicating that there is no connection relationship between the channels.

3. The epileptic seizure prediction method based on multi-scale mixed graph attention convolutional network according to claim 1 is characterized in that: The processing of the EEG signal spatial sequence by graph construction and convolution specifically includes the following steps: The Laplace matrix L is calculated based on the constructed graph structure data, and then the Laplace matrix is ​​spectrally decomposed to obtain the corresponding eigenvalues ​​and eigenvectors; The Fourier transform is used to convert the time-space domain to the spectrum domain, thereby effectively completing the Fourier transform of the graph convolution process. The formula is: GF(x) = U T x, where x is the EEG feature data, GF(x) is the data converted from the spatial domain dimension to the spectral domain dimension after Fourier transformation, and U T A matrix representing the unit eigenvector; The output result enters the ReLu activation function for sparse processing.

4. The epileptic seizure prediction method based on multi-scale mixed graph attention convolutional network according to claim 1 is characterized in that: The method of processing the global correlation of the EEG signal time series by the iTransformer model and outputting the element sequence by the decoder specifically includes the following steps: Through the iTransformer encoder, the original data X n Mapping to H n ; Use multi-head attention to analyze the features H after different variable mappings n ∈R N×T The correlation between The time series characteristics within each variable are further modeled through a feedforward neural network to obtain efficient time series representation; The timing representation is standardized and the standardization formula is: Where x is the time series, μ is the mean of x, and σ is the standard deviation; The normalized data is reintroduced into the attention mechanism to construct the relationship between the graph structures; The feature value is updated by performing weighted summation with the features of its neighboring nodes through the attention mechanism; Through the decoder, the context vector output by the encoder is received and the feature vector is converted into the target output.

5. The epileptic seizure prediction method based on multi-scale mixed graph attention convolutional network according to claim 1 is characterized in that: The method of processing the local correlation of the EEG signal time series by a multi-scale time adaptive network and predicting epileptic seizures specifically includes the following steps: The segment sequence features of the EEG signal graph are divided through a multi-scale time adaptive network; The output is converted into a probability distribution through a probability distribution function.

6. The epileptic seizure prediction method based on multi-scale mixed graph attention convolutional network according to claim 1 is characterized in that: The method of classifying the epilepsy type by a classifier specifically comprises the following steps: The processed feature data is input into the self-attention layer, and the encoded one-dimensional matrix is ​​time-sliced ​​to divide the time series data of individual channels into one-dimensional segments; Update the time slices by weighted summation, and finally reassemble each slice component to get the original size for output; The data is classified through pooling operations to predict different types of epileptic seizures.

Images