A spatio-temporal-frequency attention-based epilepsy electroencephalogram signal classification method

Abstract

The application belongs to the technical field of deep learning, and discloses a kind of epilepsy electroencephalogram signal classification method based on time-space-frequency attention, mainly divided into three stages of data set acquisition, data preprocessing, feature extraction.The application comprehensively considers the spatial domain, frequency domain, time domain features of electroencephalogram signal, and constructs a epilepsy classification model based on attention mechanism to complete the detection and multi-classification task of epilepsy electroencephalogram signal.The epilepsy detection task can be regarded as a epilepsy binary classification task, i.e., the electroencephalogram segment is divided into two categories of epilepsy seizure and non-epilepsy seizure;the epilepsy multi-classification task divides the epilepsy electroencephalogram segment into different categories.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of deep learning, and relates to a seizure electroencephalogram signal classification method based on time-space-frequency attention, which is used for improving the diagnosis and treatment efficiency of doctors on epilepsy. BACKGROUND

[0002] The technology related to the application includes three aspects: the first is the channel attention mechanism; the second is the spatial attention mechanism; and the third is the one-time aggregation module.

[0003] The channel attention mechanism aims to associate the features extracted by different convolution kernels, and enhances the learning ability of the network to the features by learning the correlation between the channels. Hu et al. introduced the channel attention mechanism in the document Squeeze-and-Excitation Networks, the core of which is to capture the global information of the features through a squeeze-and-excitation module, and simultaneously learn the relationship between the channels. The squeeze module collects the global spatial information of the features through a global average pooling layer; the excitation module learns the correlation between the channels by using a fully connected layer and a nonlinear activation function, and finally completes the weighting of the input features by performing element-wise multiplication of the attention scores and the input features. The specific calculation process of the attention scores can be expressed as follows:

[0004] s=F se (X,θ)=σ(W2δ(W1G(X))) (1)

[0005] where F se represents the squeeze-and-excitation module, sigma and delta are nonlinear activation functions, and G is a global average pooling function. For the three-dimensional representation of the electroencephalogram signal, the channel attention mechanism makes the model pay attention to the time steps that contribute more to the epilepsy classification task, but cannot process the space-frequency features of the electroencephalogram signal. In the multi-channel electroencephalogram signal, different channels correspond to electrodes at different positions, and according to the electrodes placed according to the international 10-20 system, different brain regions are corresponded. When patients suffer from different types of seizures, the activity rules and ranges of brain neurons may be different, so the space-frequency features of the electroencephalogram signal are crucial to the epilepsy classification task.

[0006] The core of spatial attention mechanism is to help the network determine the location of important features and improve the feature expression of key areas, which can be regarded as an adaptive spatial region selection mechanism. Jaderberg et al. proposed the spatial transformer network (STN) in the literature “Spatial Transformer Networks”. STN is the first attention mechanism that provides transformation invariance for neural networks. The core idea is to learn spatial transformation operations to enable the neural network to better perform geometric transformations such as rotation, scaling, and translation on input data when processing image or sequence data, thereby improving the robustness and generalization ability of the model. Dosovitskiy et al. proposed the Vision Transformer (ViT) in the literature “An image is worth 16x16 words: Transformers for image recognition at scale”. ViT is the first architecture that uses attention mechanism to process images completely and achieves comparable results to convolutional neural networks. ViT divides the image into several 16*16 image blocks and embeds them into vectors, which are then input into the Transformer model after position encoding. This allows ViT to process images of any size without relying on fixed-size convolution operations. Unlike traditional CNNs, ViT uses self-attention mechanism in Transformer, which allows the network to capture the relationship between different positions in the input image. The self-attention mechanism allows the network to assign different weights to each image block to better capture global and local information when processing images. The three-dimensional representation of EEG signals corresponds to the channel-time-frequency features of EEG signals. Spatial attention mechanism weights the space-frequency features, but ignores the time-domain information of EEG signals. Different types of seizures produce different features over time when they occur, so time-domain information cannot be ignored when performing seizure detection and classification tasks.

[0007] The one-shot aggregation module was proposed by Lee et al. in the literature “An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detection”. The one-shot aggregation (OSA) module proposed in the paper aggregates the features of all layers of the network only in the last layer, which not only inherits the advantage of DenseNet in representing multiple features with multiple branches, but also solves the problem of low efficiency caused by dense connection. The one-shot aggregation module can make the model better reuse features and control the overall parameter amount.

[0008] Based on the technical problems existing in the above-mentioned technologies, it is necessary to develop a method for continuously extruding a bottomed cylindrical shell by using a segmented core mold. SUMMARY

[0009] In order to extract the essential features of the electroencephalogram signal, the spatial domain, frequency domain and time domain features of the electroencephalogram signal are comprehensively considered, and a seizure classification model is constructed based on the attention mechanism to complete the detection and multi-classification tasks of the epilepsy electroencephalogram signal. The epilepsy detection task can be regarded as a two-class classification task of epilepsy, that is, the electroencephalogram segment is divided into two categories of epilepsy seizure and non-epilepsy seizure; the multi-classification task of epilepsy divides the epilepsy electroencephalogram segment into different categories.

[0010] The technical scheme of the present application is as follows:

[0011] A seizure electroencephalogram signal classification method based on time-space-frequency attention, mainly including three stages of data set acquisition, data preprocessing and feature extraction;

[0012] The specific steps are as follows:

[0013] (1) Data set acquisition stage

[0014] The data set at least contains epilepsy electroencephalogram signal, signal sampling frequency and signal category label;

[0015] (2) Electroencephalogram signal preprocessing stage

[0016] The preprocessing stage includes seven stages of data screening, TCP (Temporal Central Parasagittal) montage, resampling, filtering, data slicing, time series signal normalization, short-time Fourier transform and time-frequency matrix normalization;

[0017] (3) Feature extraction stage

[0018] An attention-based temporal-spatial-spectral (ATSS) model is proposed to distinguish different types of electroencephalogram (EEG) signals. The ATSS model mainly consists of a temporal attention module, a spatial-spectral attention module, a one-time aggregation module, a transition module, and a classification layer. A short-time Fourier transform is used to construct a three-dimensional representation of the multi-channel EEG signal as the input of the ATSS model. The temporal attention module and the spatial-spectral attention module are designed based on the squeeze-and-excitation model, so that the ATSS model focuses on the time steps, EEG channels, and frequency ranges that contribute more to the epilepsy classification task. The one-time aggregation module enhances feature reuse while limiting the overall parameter and computational load of the ATSS model. The transition module limits the growth rate of the feature channels and reduces the size of the feature maps. A three-dimensional convolution layer with a kernel size of 1*1*1 is used instead of a fully connected layer to further limit the parameter amount of the ATSS model.

[0019] 3.1) Temporal attention module

[0020] The time dimension is considered as the feature channel dimension, and the input feature of the temporal attention module is represented as Input feature After processing the spatial and frequency dimensions through the global average pooling layer and the global maximum pooling layer, the time dimension remains unchanged. After dimension reduction and transposition, the input is input into the one-dimensional convolution layer. The convolution kernel size of the one-dimensional convolution layer is K, and the padding operation is used to keep the feature map size unchanged. It is represented as the following two formulas:

[0021]

[0022]

[0023] wherein, Conv() represents one-dimensional convolution. After GMP and GAP, the spatial and frequency domain information is aggregated into a one-dimensional vector, and the one-dimensional convolution layer is used to complete the information exchange across time steps.

[0024] Then X avg and X max are spliced in the first dimension, and a convolution layer with a kernel size of 1 is used to achieve feature fusion and dimension reduction processing. The fused features are processed by an activation function to obtain attention weights, and then the broadcast mechanism is used to perform element-wise multiplication between the attention weights and the input features. It is represented as the following formula:

[0025]

[0026] where Concat() is concatenation operation, Conv() represents one-dimensional convolution, and Mul() represents element-wise multiplication;

[0027] 3.2) Spatial-temporal attention module

[0028] The spatial-temporal attention module mainly consists of a time-dimension three-dimensional max pooling layer, a time-dimension three-dimensional average pooling layer, and a two-dimensional convolution layer;

[0029] Input feature Meanwhile, the feature extraction and dimension reduction processing are performed through two parallel max pooling layers TMP and average pooling layers TAP; the features processed by the max pooling layer TMP and the average pooling layer TAP are represented as and Then, the two groups of features after the pooling are concatenated in the channel dimension, and the concatenated feature is represented as After the dimension reduction processing of the channel dimension by the two-dimensional convolution layer, the fused feature is obtained Then, the sigmoid activation function is used to obtain the attention weight Finally, the broadcast mechanism is used to perform element-wise multiplication between the attention weight and the input feature to complete the weighting of the feature; the overall operation of the spatial-temporal attention module on the input feature is represented by the following formula:

[0030]

[0031] where MaxPool() represents the max pooling layer, AvgPool() represents the average pooling layer, Concat() represents the channel dimension concatenation, and Mul() represents the element-wise multiplication;

[0032] 3.3) One aggregation module and transition module

[0033] The input feature of the one aggregation module is represented as where C is the feature channel dimension; each one aggregation module consists of M identical sub-modules, each sub-module consists of a batch normalization layer, a ReLU activation function, and a three-dimensional convolution layer; the input channel number of each sub-module is the output channel number of the previous module, the output channel numbers of all sub-modules are the same, and the feature reuse is achieved by concatenating the output features of the last sub-module in the feature channel dimension; the three-dimensional convolution layer in the one aggregation module has a uniform kernel size of 3*3*3, and the padding is set to same mode to ensure that the feature map size remains unchanged;

[0034] The transition module is composed of a batch normalization layer, a ReLU activation function, a three-dimensional convolution layer with a convolution kernel size of 1*1*1, and a three-dimensional average pooling layer with a pooling kernel size and a step of 2*2*2; wherein the three-dimensional convolution layer is used to reduce the number of feature channels without changing the size of the feature map, limit the growth rate of the feature channel, and the three-dimensional average pooling layer is used to change the size of the feature map to half of the size of the input feature map. The output of the transition module can be represented as X 3 ;

[0035] 3.4) Classification layer

[0036] The classification layer is composed of a three-dimensional adaptive average pooling layer, a three-dimensional convolution layer with a convolution kernel size and a step of 1*1*1, and a softmax layer; the input of the classification layer is represented as First, the three-dimensional adaptive average pooling layer is used to average pool the input features in the time-space-frequency three dimensions, obtaining the intermediate features Secondly, a three-dimensional convolution layer with an output channel number equal to the number of epilepsy categories is introduced instead of a fully connected layer to increase the nonlinear operation of the model while reducing the number of parameters; finally, the features are flattened and passed through the softmax function to obtain the epilepsy category predicted by the model.

[0037] The beneficial effects of the present application are: the present application uses short-time Fourier transform to construct a time-space-frequency three-dimensional representation of multi-channel electroencephalogram signal as the input of the model. The ATSS model uses an attention module with one-time aggregation to extract multi-level time-space-frequency electroencephalogram features, i.e. integrates time, space and frequency attention, so that the model pays attention to the most discriminative time step, electrode position and frequency range, and at the same time, through the one-time aggregation module, the feature reuse is strengthened and the parameter quantity of the model is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is a epilepsy classification model structure based on time-space-frequency attention mechanism.

[0039] Figure 2 is a time attention module structure.

[0040] Figure 3 is a space-frequency attention module structure.

[0041] Figure 4 is a one-time aggregation module and transition module structure.

[0042] Figure 5 is a classification layer structure.

[0043] In the figure: N* represents that the attention module is repeated N times, and represents that the elements are averaged, represents that the features are spliced along the channel dimension, Represents element-wise multiplication, ① represents one-dimensional convolution, Represents sigmoid activation function, ② represents two-dimensional convolution. DETAILED DESCRIPTION

[0044] The specific embodiments of the present application are further described below in conjunction with the accompanying drawings and technical solutions

[0045] Figure 1 is a seizure classification model structure based on a time-space-frequency attention mechanism. The electroencephalogram signal is preprocessed to obtain a time-space-frequency three-dimensional representation of the EEG segment, which represents the distribution of brain activity energy in the time, space (electroencephalogram channel) and frequency domain. The input data obtained after preprocessing each EEG segment can be represented as where N is the number of electroencephalogram channels, F and T represent the frequency and time domain dimensions, respectively. The three-dimensional time-frequency matrix is input into the network and then passes through N attention modules (Attention Module) to assign higher weights to important features through the attention mechanism. Each attention module includes a temporal attention (Temporal-Attention, TA) module, a spatial-spectral attention (Spatial-Spectral Attention, SSA) module, a one-shot aggregation (One-shot Aggregation, OSA) module and a transition module. The TA module enables the model to assign higher weights to the most representative temporal features, and the SSA module enables the model to focus on the frequency band and electroencephalogram channel with the most information. By combining the TA and SSA modules, the model simultaneously extracts time-space-frequency features from the EEG signal, thereby completing the seizure classification task.

[0046] Figure 2 is a temporal attention module structure. Different types of seizures produce different characteristics as they change over time, so this paper designs a temporal attention module to weight the electroencephalogram features at different time steps according to their contribution to the seizure classification task. In the TA module, this paper considers the time dimension as the feature channel dimension, i.e. the input data of the TA module can be represented as Figure 2 GAP is a global average pooling layer, and GMP is a global maximum pooling layer. The input feature After the global average pooling layer and the global maximum pooling layer process the spatial and frequency domain dimensions, the time dimension remains unchanged, and then after dimension reduction and transposition, it is input into the one-dimensional convolution layer; the convolution kernel size of the convolution layer is K, and the feature map size does not change through the padding operation, which can be specifically represented by the following two formulas.

[0047] X avg =Conv(GAP(X in ))(6)

[0048] X max = Conv(GMP(X in ))(7)

[0049] where Conv() represents one-dimensional convolution. After GMP and GAP, the spatial domain and frequency domain information are aggregated into one-dimensional vectors, and the one-dimensional convolution layer is used to complete the information exchange across time steps.

[0050] The above X avg is concatenated with X max in the first dimension, and a convolution layer with a kernel size of 1 is used to achieve feature fusion and dimension reduction. After the fused features are processed by the activation function, the attention weight is obtained, and then the broadcast mechanism is used to perform element-wise multiplication on the attention weight and the input feature, so that the model gives higher weight to important features. Specifically, it can be expressed as follows:

[0051] X att = Mul(Sigmoid(Conv(Concat(X max ,X avg ))),X in ) (8)

[0052] where Concat() is the concatenation operation, Conv() represents one-dimensional convolution, and Mul() represents element-wise multiplication.

[0053] Figure 3 is the spatial-frequency attention module structure. In multi-channel electroencephalogram signals, different channels correspond to electrodes at different positions, and electrodes placed according to the international 10-20 system correspond to different brain regions. When patients experience different types of seizures, the activity patterns and ranges of brain neurons may be different. Therefore, to extract the essential features of electroencephalogram signals, this study designed and implemented a spatial-frequency attention module to capture the correlation between the spatial position and frequency spectrum of electroencephalogram signals. The spatial-frequency attention module mainly consists of a time dimension three-dimensional maximum pooling layer, a time dimension three-dimensional average pooling layer, and a two-dimensional convolution layer, Figure 3 where TAP represents the time dimension average pooling, and TMP represents the time dimension maximum pooling.

[0054] The input feature is extracted and dimensionally reduced by two parallel pooling layers, which also saves computational cost. The features processed by the maximum pooling layer TMP and the average pooling layer TAP can be represented as and The two sets of features after pooling are concatenated in the channel dimension, and the concatenated features can be represented as After dimension reduction processing on the channel dimension through the two-dimensional convolution layer, the fusion feature is obtained Then the attention weight is obtained through the sigmoid activation function Finally, the attention weight and the input feature are multiplied element by element through the broadcast mechanism to complete the weighting of the feature. The overall operation of the space-frequency attention module on the input feature can be represented by the following formula.

[0055] X out = Mul(Sigmoid(Concat(MaxPool(X in ),AvgPool(X in ))),X in ) (9)

[0056] Where MaxPool() represents the maximum pooling layer, AvgPool() represents the average pooling layer, Concat() represents the channel dimension concatenation, and Mul() represents the element-wise multiplication

[0057] Figure 4 is a one-aggregation module and a transition module structure. In order to solve the gradient vanishing and enhance feature propagation and reuse, while limiting the model parameter quantity and calculation quantity, inspired by DenseNet and VoVNet, a one-aggregation (One-Shot Aggregation, OSA) module is used to process the features. The input feature of the OSA module can be represented as Where C is the feature channel dimension. Each OSA module is composed of M identical sub-modules, each of which is composed of a batch normalization layer, a ReLU activation function and a three-dimensional convolution layer. The input channel number of each sub-module is the output channel number of the previous module, and the output channel number of all sub-modules is the same. After the output feature of the last sub-module is output, the features are spliced in the feature channel dimension to achieve the purpose of feature reuse. The three-dimensional convolution layer in the OSA module has a uniform kernel size of 3*3*3, and the padding is set to same mode to ensure that the feature map size remains unchanged. The transition module is composed of a batch normalization layer, a ReLU activation function, a three-dimensional convolution layer with a kernel size of 1*1*1 and a three-dimensional average pooling layer with a kernel size and step of 2*2*2. The three-dimensional convolution layer is used to reduce the feature channel number without changing the feature map size, limiting the growth rate of the feature channel, and the three-dimensional average pooling layer is used to convert the feature map size to half of the input feature map size.

[0058] Figure 5is the classification layer structure. The classification layer is composed of a three-dimensional adaptive average pooling layer (AdaptiveAvgPool3d), a three-dimensional convolution layer (Conv3D) with a convolution kernel size and a step size of 1*1*1, and a softmax layer. The input of the classification layer can be represented as which is first subjected to a three-dimensional adaptive average pooling layer to perform average pooling operation on the input features in the time-space-frequency three dimensions to obtain intermediate features Secondly, this paper introduces a three-dimensional convolution layer with the number of output channels equal to the number of epilepsy categories to replace the fully connected layer, in order to increase the nonlinear operation of the model while reducing the number of parameters. Finally, the features are flattened and subjected to a softmax function to obtain the epilepsy category predicted by the model.

[0059] Embodiments

[0060] An epilepsy electroencephalogram signal classification method based on an attention mechanism, the steps are as follows:

[0061] The epilepsy electroencephalogram signal classification method mainly includes four stages of dataset acquisition, data preprocessing, feature extraction, classification and evaluation:

[0062] (1) Dataset acquisition

[0063] Acquire epilepsy electroencephalogram signal data, including epilepsy category labeling, electroencephalogram signal sampling frequency, etc.

[0064] (2) Data preprocessing

[0065] The preprocessing stage can be divided into seven stages of data screening, TCP montage, resampling, filtering, data slicing, time series signal normalization, short-time Fourier transform and time-frequency matrix normalization.

[0066] (2.1) Data screening

[0067] Exclude epilepsy category samples with very small data volume, and select electroencephalogram data containing A1-REF and A2-REF channels to facilitate TCP montage.

[0068] (2.2) TCP montage

[0069] This paper performs TCP montage processing on the electroencephalogram signal. The specific implementation method is to subtract the electroencephalogram signals of the two corresponding channels to obtain a new single-channel electroencephalogram signal.

[0070] (2.3) Resampling

[0071] If the epilepsy electroencephalogram signals in the dataset have multiple sampling frequencies, in order to unify the time resolution, the selected data need to be resampled first, and the unified sampling rate is 200Hz.

[0072] (2.4) Filtering

[0073] To eliminate the 60Hz power frequency interference, this paper uses a fourth-order Butterworth band-stop filter to filter the resampled data, filter out power supply noise, and the cutoff frequency is 59-61Hz.

[0074] (2.5) Data slicing

[0075] If the sampling time of the epileptic EEG signal data is different, in order to unify the data length, this study uses a 5-second sliding window to segment the data, and ensures that the start and end time of the sliding window are in the same epileptic seizure. The epileptic segment less than 5 seconds is discarded. At the same time, considering the data leakage problem, there is no overlap between adjacent sliding windows when the data is segmented.

[0076] (2.6) Time series normalization

[0077] Because of the great difference in the distribution of EEG signal data among different patients, even the EEG signal of the same patient in two seizures will be different, therefore, this study normalizes the segmented EEG data in the channel dimension by Min-Max normalization method, which is convenient for feature extraction. The normalization method is as follows:

[0078]

[0079] Where x is the sliced EEG signal segment, min represents the minimum value of x, and max represents the maximum value of x. After Min-Max normalization, the amplitude of the EEG signal is limited in the range of [0, 1].

[0080] (2.7) STFT and time-frequency matrix normalization

[0081] The normalized EEG signal segment will be converted from time domain to time-frequency domain by STFT. Traditional Fourier transform can only reflect the characteristics of stationary signal in frequency domain, and cannot show the time domain information. While EEG signal as a non-stationary signal, its time domain information cannot be ignored in the task of epilepsy detection and classification. The essence of STFT is the windowed Fourier transform, that is, before Fourier transform, the window function is multiplied with the signal, and it is assumed that the non-stationary signal is stationary within the added window, and through this way, the time domain information and frequency domain information of the signal are connected. The STFT of signal x(τ) is defined as the following formula:

[0082] S x (ω,t)=∫x(τ)m(τ-t)e -jωτ dτ (11)

[0083] Where m(τ-t) is the window function, and ω is the angular frequency.

[0084] To better extract the time-frequency features of EEG signals, the amplitude of the time-frequency matrix generated by STFT was taken and converted to decibels, and then Min-Max normalization was performed in the channel dimension. The processing method of the time-frequency matrix is shown in the following formula:

[0085] M abs =20*log 10 (H(M)) (12)

[0086]

[0087] In formula 3.3, M is the time-frequency matrix, H() is the modulo function, M abs is the result after taking the modulo and converting the unit to decibels. In formula 3.4, min is the minimum value of M abs , max is the maximum value of M abs , M * is the normalized result, i.e., the time-space-frequency three-dimensional representation of the EEG signal, which is used as the input of the model.

[0088] After preprocessing, EEG data of different lengths form EEG segments with equal length. Each segment has a length of 5s and a sampling frequency of 200Hz, so each epilepsy segment has 22 channels and 1000 data points. The kaiser window with a shape parameter of 1 is used for STFT of the EEG segment, where each time segment has a length of 63, an overlap length coefficient of 0.75, and a zero padding length of 256.

[0089] (3) Feature extraction

[0090] The dimension of the preprocessed EEG signal data is The data is input into the ATSS network for training, the model with the highest weighted F1 score is taken as the final model, and the test set is tested.

[0091] (4) Classification and evaluation

[0092] To test the performance of the ATSS model, the project analyzes the performance of the model according to the confusion matrix. In a binary classification task, True Label in the confusion matrix refers to the true label of the sample, PredictedLabel refers to the predicted label of the sample by the model, and True Negative (TN) and True Positive (TP) both refer to the correct prediction of the classification of the sample by the model, that is, the sample label is negative, and the model prediction is also negative, or the sample label is positive, and the model prediction is also positive. False Negative (FN) and False Positive (FP) represent the incorrect estimation of the classification of the sample by the model, that is, the sample label is positive, and the model judges it as negative, or the sample label is negative, and the model judges it as positive.

[0093] According to the confusion matrix, evaluation indicators can be further calculated. This paper uses the following three indicators to evaluate the performance of the model, and takes a binary classification task as an example to explain the calculation method of each indicator:

[0094] Accuracy: represents the percentage of the number of samples predicted correctly by the model to the total number of samples, reflecting the prediction performance of the model on the whole sample. The calculation is as follows:

[0095]

[0096] Precision: represents the percentage of samples with true label positive among samples predicted positive by the model. Precision is different from accuracy. Accuracy represents the prediction accuracy of the model for all samples, while precision is for the prediction result, reflecting the prediction accuracy of the model for positive samples in the result. The calculation method is as follows:

[0097]

[0098] F1-score: Recall is for the original sample, representing the percentage of samples with true label positive that are correctly predicted. However, in multi-classification tasks, the weighted recall and accuracy are equal in value, so this paper does not give the specific value of recall. Precision and recall are a pair of indicators that complement each other. To take both into account, this paper also selects F1-score as an evaluation indicator. The calculation method is as follows:

[0099]

[0100] Where P represents precision, and R represents recall.

[0101] When performing multi-classification tasks, due to the imbalance of the data set, the precision and F1 score are weighted and summed as the evaluation standard of the model performance. The specific calculation method is: after obtaining the index value of all categories, the weighted sum is obtained as the final index. Specifically, it can be expressed as the following formula:

[0102]

[0103] Where i represents the class, C represents the total number of classes, M i represents the precision or F1 score of class i, w i represents the proportion of the number of samples of class i in all samples in the test set. To alleviate the negative impact of extremely imbalanced data set on model training, this paper uses weighted cross-entropy as the loss function, and the weight array is defined as the inverse of the ratio of the number of samples of each class to the number of samples of CFSZ, so as to give higher weight to the class with less number of samples.

[0104] Due to the huge difference in the number of epilepsy class samples, the extremely imbalanced data set is used to fully utilize the data and objectively evaluate the model performance. This paper uses 5-fold cross-validation to evaluate the model. In the division process, all samples of the four types of epilepsy are randomly divided into 5 parts, and 1 part is taken out from each type of epilepsy sample to form a 1-fold data, so as to ensure that the proportion of each epilepsy class sample in each fold is the same. Finally, an accuracy of 95.12%, a weighted F1 score of 0.9510 and a weighted precision of 95.11% are obtained.

Claims

1. A method for classifying epileptic EEG signals based on spatiotemporal-frequency attention, characterized in that, This method for classifying epilepsy EEG signals mainly consists of three stages: dataset acquisition, data preprocessing, and feature extraction. The specific steps are as follows: (1) Data Acquisition Stage The dataset must contain at least epileptic EEG signals, signal sampling frequency, and signal category labeling; (2) EEG signal preprocessing stage The preprocessing stage is divided into seven stages: data filtering, TCP montage, resampling, filtering, data slicing, time-series signal normalization, short-time Fourier transform, and time-frequency matrix normalization. (3) Feature extraction stage A spatiotemporal-frequency (STF) model based on attention mechanisms is proposed to distinguish different types of epileptic EEG signals. This model mainly consists of a temporal attention module, a spatiotemporal attention module, a primary aggregation module, a transition module, and a classification layer. A three-dimensional spatiotemporal-frequency representation of multi-channel EEG signals is constructed using short-time Fourier transform and serves as the input to the STF model. Both the temporal and spatiotemporal attention modules are designed based on a squeeze-excitation pattern, allowing the model to focus attention on the time step, EEG channels, and frequency range that contribute significantly to the epilepsy classification task. The primary aggregation module enhances feature reuse while limiting the overall parameter count and computational cost of the STF model. The transition module limits the growth rate of feature channels and reduces the size of the feature maps. Replacing fully connected layers with 1*1*1 three-dimensional convolutional layers further limits the parameter count of the STF model. 3.1) Time Attention Module Treating the time dimension as a feature channel dimension, the input features of the time attention module are represented as follows: Input features After processing the spatial and frequency domains using global average pooling and global max pooling layers while maintaining the time dimension, the data is further reduced in dimension and transposed before being input into a one-dimensional convolutional layer. The kernel size of the one-dimensional convolutional layer is K, and padding operations are used to ensure that the feature map size remains unchanged. This is expressed by the following two formulas: in, Conv() represents one-dimensional convolution; after GMP and GAP, spatial and frequency domain information is aggregated into a one-dimensional vector. The one-dimensional convolutional layer is used to complete the information fusion across time steps. Then X avg With X max The first dimension is concatenated, and feature fusion and dimensionality reduction are achieved through convolutional layers with a kernel size of 1. The fused features are processed by an activation function to obtain attention weights, and then a broadcast mechanism is used to perform element-wise multiplication of the attention weights and input features; expressed as the following formula: Among them, Concat() is the concatenation operation, Conv() represents one-dimensional convolution, and Mul() represents element-wise multiplication; 3.2) Spatial-Frequency Attention Module The space-frequency attention module mainly consists of a three-dimensional max pooling layer in the time dimension, a three-dimensional average pooling layer in the time dimension, and a two-dimensional convolutional layer; Input features Simultaneously, feature extraction and dimensionality reduction are performed using two parallel max-pooling layers (TMP and TAP). The features after processing by the max-pooling layer (TMP) and the average-pooling layer (TAP) are represented as follows: and The two pooled feature sets are then concatenated along the channel dimension, and the concatenated feature is represented as follows: The fused features are obtained after dimensionality reduction of the channel dimensions through a two-dimensional convolutional layer. Then, pass the sigmoid activation function to obtain the attention weights. Finally, a broadcast mechanism is used to perform element-wise multiplication of the attention weights with the input features to complete the weighting of the features; the overall operation of the space-frequency attention module on the input features is represented by the following formula: Among them, MaxPool() represents the max pooling layer, AvgPool() represents the average pooling layer, Concat() represents channel concatenation, and Mul() represents element-wise multiplication. 3.3) Primary aggregation module and transition module The input features of the first aggregation module are represented as follows: Where C represents the feature channel dimension; each primary aggregation module consists of M identical sub-modules, each sub-module consisting of a batch normalization layer, a ReLU activation function, and a 3D convolutional layer; the number of input channels of each sub-module is the same as the number of output channels of the previous module, all sub-modules have the same number of output channels, and they are concatenated along the feature channel dimension after the last sub-module outputs features to achieve feature reuse; the kernel size of the 3D convolutional layer in the primary aggregation module is uniformly 3*3*3, and the padding is set to the same mode to ensure that the feature map size remains unchanged; The transition module consists of a batch normalization layer, a ReLU activation function, a 3D convolutional layer with a kernel size of 1*1*1, and a 3D average pooling layer with a kernel size and stride of 2*2*2. The 3D convolutional layer is used to reduce the number of feature channels and limit the growth rate of feature channels without changing the size of the feature map, and the 3D average pooling layer is used to transform the feature map size to half the size of the input feature map. 3.4) Classification layer The classification layer consists of a 3D adaptive average pooling layer, a 3D convolutional layer with a kernel size and stride of 1*1*1, and a softmax layer; the input to the classification layer is represented as... First, the input features are averaged using a three-dimensional adaptive average pooling layer in three dimensions: time, space, and frequency, to obtain intermediate features. Secondly, a three-dimensional convolutional layer with the number of output channels equal to the number of epilepsy categories is introduced to replace the fully connected layer, in order to increase the nonlinear operation of the model while reducing the number of parameters; finally, the features are flattened and subjected to a softmax function to obtain the epilepsy category predicted by the model.

Images