Seizure prediction system and method based on spatiotemporal multi-scale attention mechanism

Abstract

The application provides a seizure prediction system and method based on a space-time multi-scale attention mechanism, wherein the system comprises: an acquisition unit configured to acquire electroencephalogram data; a prediction unit configured to input the electroencephalogram data into a prediction model to obtain a seizure prediction result output by the prediction model; the prediction model comprises a backbone network module, a spatial feature extraction module, a time sequence feature fusion module and a classifier; the backbone network module is configured to perform multi-level attention extraction on the electroencephalogram data to obtain features at each level; the spatial feature extraction module is configured to perform multi-scale spatial attention extraction on the features at each level to obtain feature maps in each frequency domain; the time sequence feature fusion module is configured to perform feature fusion on the feature maps in each frequency domain to obtain fused features, and obtain hidden state features based on the fused features and the feature maps in each frequency domain; and the classifier is configured to classify the hidden state features to obtain the seizure prediction result, thereby improving the accuracy and robustness of seizure prediction.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an epileptic seizure prediction system and method based on a spatiotemporal multi-scale attention mechanism. Background Technology

[0002] Epilepsy is a common neurological disorder characterized by brief, recurrent episodes of abnormal brain electrical activity. Epileptic seizures severely impact patients' quality of life, mental health, and safety. Therefore, accurately predicting epileptic seizures has significant clinical importance.

[0003] Traditional epilepsy prediction methods are typically based on hand-designed feature extraction methods and conventional machine learning algorithms, such as support vector machines and random forests. The predictive performance of these methods is limited by the effectiveness of the feature extraction methods and the generalization ability of the machine learning algorithms. In recent years, deep learning technology has achieved significant results in many fields, especially the applications of CNNs (Convolutional Neural Networks) and RNNs (Recurrent Neural Networks) in image recognition and natural language processing.

[0004] However, existing deep learning methods still face many challenges in the task of predicting epileptic seizures, such as low sensitivity, high false alarm rate, and susceptibility to external noise. Summary of the Invention

[0005] This invention provides an epileptic seizure prediction system and method based on a spatiotemporal multi-scale attention mechanism, which addresses the many challenges faced by existing deep learning methods, such as low sensitivity, high false alarm rate, and susceptibility to external noise.

[0006] This invention provides an epileptic seizure prediction system based on a spatiotemporal multi-scale attention mechanism, comprising:

[0007] Acquisition unit, used to acquire electroencephalogram (EEG) data;

[0008] The prediction unit is used to input the electroencephalogram data into the prediction model to obtain the epilepsy prediction result output by the prediction model.

[0009] The prediction model includes a backbone network module, a spatial feature extraction module, a temporal feature fusion module, and a classifier;

[0010] The backbone network module is used to perform multi-level attention extraction on the EEG spectrogram data to obtain features at each level. The spatial feature extraction module is used to perform multi-scale spatial attention extraction on the features at each level to obtain feature maps in each frequency domain. The temporal feature fusion module is used to fuse the feature maps in each frequency domain to obtain fused features. Based on the fused features and the feature maps in each frequency domain, the final hidden state features are obtained.

[0011] The classifier is used to classify based on the features of the final hidden state to obtain the epilepsy prediction result.

[0012] The epileptic seizure prediction system based on a spatiotemporal multi-scale attention mechanism provided by the present invention further includes a data augmentation unit, the data augmentation unit comprising:

[0013] The raw EEG data acquisition unit is used to acquire raw EEG data;

[0014] The segmentation unit is used to resample the original EEG data to obtain resampled EEG data, and use a preset sliding window to segment the resampled EEG data for a preset duration to obtain multiple segmented EEG fragments.

[0015] The filtering unit is used to filter the plurality of extracted EEG segments using a band-stop filter and a high-pass filter to obtain filtered EEG data.

[0016] The data augmentation subunit is used to randomly swap the local time steps of the filtered EEG data and randomly add Gaussian noise to the filtered EEG data to obtain data-augmented EEG data, and to obtain the EEG spectrogram data based on the data-augmented EEG data.

[0017] According to the present invention, an epileptic seizure prediction system based on a spatiotemporal multi-scale attention mechanism is provided, wherein the data augmentation subunit is used for:

[0018] The data-enhanced EEG data were sequentially subjected to short-time Fourier transform and normalization to obtain the EEG spectrogram data.

[0019] According to the present invention, an epileptic seizure prediction system based on a spatiotemporal multi-scale attention mechanism further includes a model training unit, wherein the model training unit is used for:

[0020] Determine the initial prediction model, acquire sample EEG spectrogram data, and generate labels based on the episodic stage of the data-enhanced EEG data;

[0021] The sample EEG spectrogram data is input into the initial prediction model to obtain the epilepsy classification result output by the initial prediction model, as well as the previous embedding layer features of the epilepsy classification result;

[0022] Based on the epilepsy classification results and the labels, the classification loss is determined, and based on the features of the previous embedding layer, positive samples, and negative samples, the triplet loss is determined.

[0023] Based on the classification loss and the triplet loss, the total loss is determined, and the parameters of the initial prediction model are iterated based on the total loss to obtain the prediction model.

[0024] An epileptic seizure prediction system based on a spatiotemporal multi-scale attention mechanism provided by the present invention further includes a data balancing unit, wherein the data balancing unit is used for:

[0025] The preic and interictal phases of the electroencephalogram (EEG) data are balanced to obtain balanced EEG data.

[0026] Accordingly, the prediction unit is used for:

[0027] The balanced spectrum data is input into the prediction model to obtain the epilepsy prediction results output by the prediction model.

[0028] According to the present invention, an epileptic seizure prediction system based on a spatiotemporal multi-scale attention mechanism is provided. The spatial feature extraction module includes a hollow spatial pyramid pooling module and a convolutional channel compression module. The hollow spatial pyramid pooling module is used to extract multi-scale spatial frequency domain features from the features at each level to obtain each spatial frequency domain feature. The convolutional channel compression module is used to compress the channels of each spatial frequency domain feature to obtain each frequency domain feature map.

[0029] This invention also provides a method for predicting epileptic seizures based on a spatiotemporal multi-scale attention mechanism, comprising:

[0030] Acquire electroencephalogram (EEG) data;

[0031] The electroencephalogram (EEG) data is input into the prediction model to obtain the epilepsy prediction result output by the prediction model.

[0032] The prediction model includes a backbone network module, a spatial feature extraction module, a temporal feature fusion module, and a classifier;

[0033] The backbone network module is used to perform multi-level attention extraction on the EEG spectrogram data to obtain features at each level. The spatial feature extraction module is used to perform multi-scale spatial attention extraction on the features at each level to obtain feature maps in each frequency domain. The temporal feature fusion module is used to fuse the feature maps in each frequency domain to obtain fused features. Based on the fused features and the feature maps in each frequency domain, the final hidden state features are obtained.

[0034] The classifier is used to classify based on the features of the final hidden state to obtain the epilepsy prediction result.

[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the epileptic seizure prediction method based on spatiotemporal multi-scale attention mechanism as described above.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the epileptic seizure prediction method based on spatiotemporal multi-scale attention mechanism as described above.

[0037] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the epileptic seizure prediction method based on spatiotemporal multi-scale attention mechanism as described above.

[0038] The epilepsy seizure prediction system and method based on spatiotemporal multi-scale attention mechanism provided by this invention uses spatiotemporal multi-scale attention mechanism to extract features from EEG spectrogram data, which can effectively capture the features of EEG spectrogram data at different spatial and temporal scales. This allows the prediction model to better adapt to the diversity of EEG signals, improve the accuracy and robustness of epilepsy seizure prediction, and the backbone network module adopts a hierarchical feature learning strategy, which can learn features at different levels and levels of abstraction. This hierarchical feature learning method helps to extract more representative features to cope with complex patterns and noise in EEG signals. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1 This is one of the structural schematic diagrams of the epileptic seizure prediction system based on spatiotemporal multi-scale attention mechanism provided by the present invention;

[0041] Figure 2 This is the second schematic diagram of the epileptic seizure prediction system based on spatiotemporal multi-scale attention mechanism provided by the present invention;

[0042] Figure 3 This is a flowchart illustrating the epileptic seizure prediction method based on spatiotemporal multi-scale attention mechanism provided by the present invention.

[0043] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0045] This invention provides an epileptic seizure prediction system based on a spatiotemporal multi-scale attention mechanism. Figure 1 This is one of the structural schematic diagrams of the epileptic seizure prediction system based on spatiotemporal multi-scale attention mechanism provided by the present invention. Figure 2 This is the second schematic diagram of the structure of the epileptic seizure prediction system based on spatiotemporal multi-scale attention mechanism provided by the present invention, as shown below. Figure 1 , Figure 2 As shown, the system includes:

[0046] Acquisition unit 110 is used to acquire electroencephalogram (EEG) data;

[0047] The prediction unit 120 is used to input the electroencephalogram data into the prediction model to obtain the epilepsy prediction result output by the prediction model;

[0048] The prediction model includes a backbone network module, a spatial feature extraction module, a temporal feature fusion module, and a classifier;

[0049] The backbone network module is used to perform multi-level attention extraction on the EEG spectrogram data to obtain features at each level. The spatial feature extraction module is used to perform multi-scale spatial attention extraction on the features at each level to obtain feature maps in each frequency domain. The temporal feature fusion module is used to fuse the feature maps in each frequency domain to obtain fused features. Based on the fused features and the feature maps in each frequency domain, the final hidden state features are obtained.

[0050] The classifier is used to classify based on the features of the final hidden state to obtain the epilepsy prediction result.

[0051] Specifically, electroencephalography (EEG) data refers to data needed for predicting epileptic seizures. EEG signals are electrical signals generated by brain neural activity, reflecting cognitive activity and offering advantages such as high temporal resolution and ease of acquisition. EEG data can be obtained by filtering and data augmentation of raw EEG data.

[0052] After the acquisition unit obtains the EEG spectrogram data, the prediction unit inputs the EEG spectrogram data into the prediction model to obtain the epilepsy prediction result output by the prediction model. The prediction model here may include a backbone network module, a spatial feature extraction module, a temporal feature fusion module, and a classifier.

[0053] The backbone network module is used to perform multi-level attention extraction on the EEG spectrogram data, obtaining features at each level. These features reflect the different scales and levels of abstraction of the EEG spectrogram data. The backbone network module can include attention layer 1, attention layer 2, attention layer 3, and attention layer 4. It uses bilinear interpolation to upsample the EEG spectrogram data, performs channel compression using 1x1 convolutions, and then uses a pre-trained SwinTransformer model as the backbone (i.e., the attention layer) to learn features at different scales and levels of abstraction through a multi-level attention network. Features X1, X2, X3, and X4 are obtained from the four layers of the backbone.

[0054] The spatial feature extraction module here is used to perform multi-scale spatial attention extraction on features at each level to obtain frequency domain feature maps. These frequency domain feature maps reflect the frequency domain feature information of the EEG spectrogram data.

[0055] The spatial feature extraction module can include a dilated spatial pyramid pooling module and a convolutional channel compression module. For example, the dilated spatial pyramid pooling module can be used to extract multi-scale spatial frequency domain features from feature maps X1, X2, X3, and X4, and a 1*1 convolution can be used to compress the channels of the multi-scale frequency domain feature maps to obtain a multi-level, multi-scale frequency domain feature map X1. ′ X2 ′ X3 ′ X4 ′ .

[0056] The temporal feature fusion module here is used to fuse features from various frequency domain feature maps to obtain fused features. Based on the fused features and the various frequency domain feature maps, the final hidden state features are obtained. For example, four learnable parameters W1, W2, W3, and W4 can be set for each frequency domain feature map X1. ′ X2 ′ X3 ′ X4 ′ Feature fusion is performed to obtain fused features F. Multilayer perceptron is used to perform linear and nonlinear transformations on fused features F, and the feature representation is mapped to the LSTM (Long Short-Term Memory) state space as the initial state of LSTM. The initial state is used to initialize the cell state and hidden state.

[0057] Then, LSTM can be used for time series modeling, using four LSTM modules connected together, each receiving the extracted multi-level, multi-scale frequency domain features X1 as input. ′ X2 ′ X3 ′ X4 ′ This yields the final hidden state features.

[0058] The classifier here is used to classify based on the features of the final hidden state to obtain the epilepsy prediction result. The classifier here can be a multilayer perceptron (MLP) or the like, but the embodiments of the present invention do not specifically limit it.

[0059] The system provided in this invention employs a spatiotemporal multi-scale attention mechanism to extract features from EEG spectrogram data. This effectively captures the features of EEG spectrogram data at different spatial and temporal scales, enabling the prediction model to better adapt to the diversity of EEG signals and improve the accuracy and robustness of epileptic seizure prediction. Furthermore, the backbone network module adopts a hierarchical feature learning strategy, which can learn features at different levels and abstraction levels. This hierarchical feature learning method helps to extract more representative features to cope with complex patterns and noise in EEG signals.

[0060] Based on the above embodiments, a data enhancement unit is also included, the data enhancement unit comprising:

[0061] The raw EEG data acquisition unit is used to acquire raw EEG data;

[0062] The segmentation unit is used to resample the original EEG data to obtain resampled EEG data, and use a preset sliding window to segment the resampled EEG data for a preset duration to obtain multiple segmented EEG fragments.

[0063] The filtering unit is used to filter the plurality of extracted EEG segments using a band-stop filter and a high-pass filter to obtain filtered EEG data.

[0064] The data augmentation subunit is used to randomly swap the local time steps of the filtered EEG data and randomly add Gaussian noise to the filtered EEG data to obtain data-augmented EEG data, and to obtain the EEG spectrogram data based on the data-augmented EEG data.

[0065] Specifically, the raw EEG data acquisition unit is used to acquire raw EEG data, which refers to EEG data before preprocessing. Raw EEG data can be obtained by collecting and recording data from each patient at least twice during the interictal period, preictal period, ictal period, and postictal period.

[0066] After obtaining the raw EEG data, the raw EEG data can be resampled to obtain resampled EEG data. The sampling frequency can be determined as needed, such as 512Hz or 256Hz, etc. This embodiment of the invention does not make a specific limitation on this.

[0067] The resampled EEG data mentioned here refers to the EEG data obtained after resampling the original EEG data.

[0068] Then, a preset sliding window can be used to extract resampled EEG data for a preset duration to obtain multiple extracted EEG segments. The preset duration can be 5 seconds, 30 seconds, etc., and this embodiment of the invention does not specifically limit it.

[0069] After obtaining multiple truncated EEG segments, bandstop filters (BSFs) and high-pass filters can be used to filter these segments, resulting in filtered EEG data to remove noise within specific frequency ranges. These filters effectively remove interference and noise from the signal, allowing for better analysis of its spectral characteristics. Specifically, two bandstop filters were used to remove noise in the frequency ranges of 57-63Hz and 117-123Hz, respectively, and a high-pass filter was used to remove low-frequency noise.

[0070] This paper presents a band-stop filter implementation based on the Butterworth filter design method. The Butterworth filter is a commonly used IIR (Infinite Impulse Response) filter, characterized by a smooth passband and a smooth transition at the cutoff frequency. Its performance can be controlled by adjusting its order. The implementation steps of the band-stop filter are as follows:

[0071] The Nyquist frequency is calculated based on the sampling rate, which is half of the sampling rate.

[0072]

[0073] Among them, f Nyquist f represents the Nyquist frequency. s This indicates the sampling rate.

[0074] Then convert the low-frequency cutoff frequency and the high-frequency cutoff frequency into normalized frequencies, that is, divide by the Nyquist frequency.

[0075]

[0076]

[0077] Among them, f lowcut f represents the low-frequency cutoff frequency. higf represents the high-frequency cutoff frequency. low f represents the normalized frequency after the low-frequency cutoff frequency conversion. high This represents the normalized frequency after conversion from the high-frequency cutoff frequency.

[0078] Use the butter function to design a band-stop filter. This function takes the order and normalized frequency as input and returns the coefficients of the IIR filter.

[0079]

[0080] Among them, b i and a i Here, represents the filter coefficients, and N represents the order of the filter.

[0081] The lfilter function is used to apply the designed filter and filter the input data:

[0082] y(n)=b0x(n)+b1x(n-1)+…+b N x(nN)-a1y(n-1)-…-a N y(nN)

[0083] Where x(n) is the input data and y(n) is the filtered data.

[0084] The implementation of a high-pass filter is also based on the Butterworth filter design method. The implementation steps of a high-pass filter are as follows:

[0085] The Nyquist frequency is calculated based on the sampling rate, which is half of the sampling rate.

[0086]

[0087] Among them, f Nyquist f represents the Nyquist frequency. s This indicates the sampling rate.

[0088] Convert the lowest cutoff frequency to a normalized frequency, i.e., divide by the Nyquist frequency.

[0089]

[0090] Among them, f cutoff f represents the lowest cutoff frequency. cuto This represents the normalized frequency after conversion from the lowest cutoff frequency.

[0091] Use the butter function to design a high-pass filter. This function takes the order and normalized frequency as input and returns the coefficients of the IIR filter.

[0092]

[0093] Among them, b i and a i Here, represents the filter coefficients, and N represents the order of the filter.

[0094] The lfilter function is used to apply the designed filter to filter the input data.

[0095] y(n)=b0x(n)+b1x(n-1)+…+b N x(nN)-a1y(n-1)-…-a N y(nN)

[0096] Where x(n) is the input data and y(n) is the filtered data.

[0097] After obtaining the filtered EEG data, the local time steps of the filtered EEG data can be randomly swapped, and Gaussian noise can be randomly added to the filtered EEG data to obtain data-enhanced EEG data. The Gaussian noise here can be Gaussian white noise, which is calculated by the following formula:

[0098] Assuming a given signal-to-noise ratio (SNR) of 8, and the input epileptic EEG signal as x, with added Gaussian white noise as w, then the output signal is x + w. Here, w follows a signal with a mean of 0 and a variance of P. n Gaussian distributed random noise, P n The noise power is calculated using SNR.

[0099] The specific calculation process is as follows:

[0100] 1. Calculate the power P of the input signal x. s P s This represents the average power of the signal at each sample point:

[0101]

[0102] Where N is the number of sample points.

[0103] 2. Calculate the noise power P based on the SNR. n :

[0104]

[0105] SNR stands for Signal-to-Noise Ratio, expressed in decibels.

[0106] 3. Generate a sequence with a mean of 0 and a variance of P. n Gaussian distributed random noise w:

[0107]

[0108] 4. Add noise w to the input signal x to obtain the output signal x+w.

[0109] Therefore, the overall formula can be expressed as:

[0110] y[n] = x[n] + w[n]

[0111] Where x[n] is the input epileptic EEG signal, and w[n] is a signal that follows a normal frequency distribution with a mean of 0 and a variance of P. n Gaussian distributed random noise, P n The noise power is calculated using SNR, and y[n] is the output epileptic EEG signal with added Gaussian white noise, i.e., data-enhanced EEG data.

[0112] After obtaining augmented EEG data, EEG spectrogram data can be obtained based on the augmented EEG data.

[0113] The system provided in this embodiment of the invention performs preprocessing operations such as resampling and filtering on the original EEG data to improve the expressive power of features. At the same time, data augmentation operations, such as adding Gaussian noise, are performed before the data is input into the prediction model to improve the robustness of the prediction model.

[0114] Based on the above embodiments, the data enhancement subunit is used for:

[0115] The data-enhanced EEG data were sequentially subjected to short-time Fourier transform and normalization to obtain the EEG spectrogram data.

[0116] Specifically, augmented EEG data can be input into an STFT (Short-Time Fourier Transform) function to calculate the frequency components within each time window:

[0117] (1) The formula for STFT is as follows:

[0118]

[0119] Here, w(t) is a window function (e.g., the Hanning window). For the input epileptic EEG signal x(t), it is divided into several time windows. The signal within each time window is windowed (multiplied by the window function w(t-τ)). Then, a Fourier transform is performed within this time window to obtain the frequency domain representation X(t,f) of the signal within that time window. Repeating this process yields the time-frequency domain representation of the entire signal.

[0120] (2) The formula for the spectrum is as follows:

[0121]

[0122] For the input signal x(t), after windowing with a window function w(t) in the time domain, a Fourier transform is performed to obtain its frequency domain representation X(t,f). Then, the amplitude is squared to obtain the power spectral density SP. x (t,f) is used to describe the energy distribution of a signal at different frequencies.

[0123] 3. Process the spectrum data obtained from STFT calculation, including steps such as transposition, truncation, and normalization.

[0124] (1) The formula for normalization is as follows:

[0125]

[0126] Among them, P xx This is the spectral data obtained from STFT calculation. stft_data is the normalized spectral data, i.e., EEG spectral data. First, the logarithm of the power spectral density (PSD) is calculated by multiplying it by 10. Then, the minimum value is subtracted from it and divided by the range. This scales the data range to between 0 and 1.

[0127] 4. The final output data is the spectrum data normalized after STFT.

[0128] Based on the above embodiments, a model training unit is also included, the model training unit being used for:

[0129] Determine the initial prediction model, acquire sample EEG spectrogram data, and generate labels based on the episodic stage of the data-enhanced EEG data;

[0130] The sample EEG spectrogram data is input into the initial prediction model to obtain the epilepsy classification result output by the initial prediction model, as well as the previous embedding layer features of the epilepsy classification result;

[0131] Based on the epilepsy classification results and the labels, the classification loss is determined, and based on the features of the previous embedding layer, positive samples, and negative samples, the triplet loss is determined.

[0132] Based on the classification loss and the triplet loss, the total loss is determined, and the parameters of the initial prediction model are iterated based on the total loss to obtain the prediction model.

[0133] Specifically, in order to improve the accuracy and reliability of epilepsy prediction results, the prediction model needs to be trained:

[0134] It can collect sample EEG spectrogram data in advance, generate labels based on the stage of the attack in the data-enhanced EEG data, and build an initial prediction model.

[0135] First, the sample EEG spectrogram data can be input into the initial prediction model to obtain the epilepsy classification result output by the initial prediction model, as well as the features of the previous embedding layer of the epilepsy classification result.

[0136] After the epilepsy classification results are output based on the initial prediction model, the epilepsy classification results can be compared with the labels. The classification loss is calculated based on the degree of difference between the two, and the triplet loss is determined based on the degree of difference between the features of the previous embedding layer and the positive samples, and the degree of difference between the features of the previous embedding layer and the negative samples.

[0137] Understandably, the greater the difference between the epilepsy classification result and the label, the greater the classification loss; conversely, the smaller the difference between the epilepsy classification result and the label, the smaller the classification loss.

[0138] Given a sample x and its corresponding true label y, the binary cross-entropy loss function is formulated as follows:

[0139]

[0140] in, This represents the label probability predicted by the model for sample x. The calculation method depends on the model structure and the activation function of the output layer. For binary classification tasks, the sigmoid activation function is often used. The calculation method is as follows:

[0141]

[0142] Where z represents the output value of the model.

[0143] The formula for triplet loss is as follows:

[0144] Given a triple (a, p, n), where a represents the anchor sample (i.e., the feature from the previous embedding layer), p represents the positive sample, and n represents the negative sample, the triple contrastive loss function is used. The formula is as follows:

[0145]

[0146] Wherein, m is a predefined margin, and d(a,p) and d(a,n) respectively denote the distances between the anchor sample a and the positive sample p, and between the anchor sample a and the negative sample n. The distances can be calculated using metrics such as Euclidean distance, Manhattan distance or cosine distance. If the distance of the triplet satisfies d(a,p)+m<d(a,n), the loss is 0; otherwise, the loss is d(a,p)+m-d(a,n).

[0147] Here, a Cross Entropy Loss Function may be used, a Mean Squared Error (MSE) loss function may also be used, and a stochastic gradient descent method may further be used to update the parameters of the initial prediction model, which is not specifically limited in the embodiments of the present invention.

[0148] After the classification loss and the triplet loss are obtained, the total loss may be determined based on the sum or the weighted sum of the classification loss and the triplet loss, parameter iteration is performed on the initial prediction model based on the total loss, and the initial prediction model after completing the parameter iteration is determined as the prediction model.

[0149] Wherein, the formula of the total loss is as follows:

[0150] Assume that one batch contains N samples, and each sample has a triplet contrastive loss and a binary cross entropy loss (classification loss) then the total loss can be expressed as the following formula:

[0151]

[0152] Wherein, a i , p i , n i , x i and y i respectively denote the anchor (the feature of the previous embedding layer), the positive sample, the negative sample, the input data and the ground-truth label of the i-th sample. The constant 8 is a hyperparameter for balancing the weights of the triplet loss and the classification loss .

[0153] The system provided by the embodiments of the present invention adopts an end-to-end training mode, and the entire prediction model can simultaneously optimize feature extraction, feature fusion and classification tasks, which enables the model to automatically learn a more discriminative feature representation, and improves the accuracy and robustness of seizure prediction.

[0154] Multilayer perceptrons are used to map the final feature representations to the label space, and a loss function is used to measure the difference between the predicted label and the true label. In addition, triplet loss is used to map positive and negative samples to different feature spaces, which helps the model learn more discriminative feature representations, thereby improving the model's performance and the classification accuracy and robustness of binary classification tasks.

[0155] Based on the above embodiments, a data balancing unit is further included, the data balancing unit being used for:

[0156] The preic and interictal phases of the electroencephalogram (EEG) data are balanced to obtain balanced EEG data.

[0157] Accordingly, the prediction unit is used for:

[0158] The balanced spectrum data is input into the prediction model to obtain the epilepsy prediction results output by the prediction model.

[0159] Specifically, after obtaining the EEG spectrogram data, the preic and interictal phases of the EEG spectrogram data can be balanced to obtain balanced spectrogram data. Here, the data balancing of the preic and interictal phases of the EEG spectrogram data can be performed using techniques such as SMOTE (Synthetic Minority Oversampling Technique). This embodiment of the invention does not specifically limit this method.

[0160] The balanced spectrogram data here refers to the spectrogram data obtained by balancing the preic and interictal EEG spectrogram data.

[0161] Accordingly, the prediction unit is used for:

[0162] The balanced spectral data is input into the prediction model to obtain the epilepsy prediction results output by the prediction model.

[0163] Furthermore, during the prediction process, data is fed into the prediction model for forward propagation, and the model's output is obtained. Then, the output is processed using a softmax function to transform it into a probability distribution. The probability values ​​and labels output by the prediction model are saved and smoothed. Alarm determination is then based on the smoothed probability values ​​and a given threshold.

[0164] If the duration of a continuous exceedance of the threshold exceeds the set time, such as 30 seconds, an alarm is considered to have occurred. Finally, a confusion matrix is ​​calculated based on the prediction results, and the alarm results are analyzed, traversing the alarm results to distinguish between positive and false alarms. If the alarm occurs within the target prediction time window, i.e., in the early stage of an event, it is considered a positive alarm; otherwise, it is considered a false alarm. Sensitivity and the false alarm rate are calculated, i.e., the number of positive alarms divided by the total number of alarms, and the number of false alarms divided by the total observation time (in hours).

[0165] Specifically, smoothing uses convolution operations to smooth the original data and returns the smoothed result.

[0166] The primary goal of smoothing is to reduce noise and short-term fluctuations in data, thereby revealing underlying long-term trends or patterns. In this implementation, a moving average method is used to smooth the input data.

[0167] Given a window of length WSZ, the moving average method calculates the average of the data points within each window and uses this average as the smoothed data points. The window starts from the left of the data and slides one data point to the right until it has traversed the entire dataset. For each element of the input data 'a', the following moving average is calculated:

[0168]

[0169] Where S i It is the i-th element of the smoothed data, a i+j It is an element of the input data.

[0170] In the code, this embodiment of the invention uses NumPy's convolve function to calculate the convolution of input data a and a vector of length WSZ consisting of all 1s, and then divides the result by WSZ to obtain smooth data in the middle part.

[0171] In addition, for the training and testing steps, namely steps 3, 4, 5, and 6, the training data batch size is set to 256, the model learning rate is set to 0.0001, the optimizer is set to the Adam optimizer, and the epoch is set to 120. This process uses leave-one-out cross-validation.

[0172] This invention was validated on the publicly available Boston Children's Hospital epilepsy dataset CHB-MIT. The average sensitivity (Sn) for a single patient reached 97.1%, and the false positive rate (FPR) was less than 0.09 per hour. Experimental results demonstrate that this invention outperforms existing epilepsy seizure prediction methods in both prediction accuracy and model generalization ability, showing promising clinical application prospects.

[0173] In summary, the epilepsy seizure prediction method based on spatiotemporal multi-scale attention mechanisms of this invention exhibits unique advantages in EEG signal feature processing, effectively improving the accuracy and robustness of epilepsy seizure prediction. This makes this invention possess greater practical value and clinical application potential in epilepsy seizure prediction tasks.

[0174] The system provided in this embodiment of the invention performs data augmentation operations before the data is input into the prediction model, and uses SMOTE for data balancing to improve the robustness of the prediction model.

[0175] Based on the above embodiments, the spatial feature extraction module includes a hollow spatial pyramid pooling module and a convolutional channel compression module. The hollow spatial pyramid pooling module is used to extract multi-scale spatial frequency domain features from the features at each level to obtain each spatial frequency domain feature. The convolutional channel compression module is used to compress the channels of each spatial frequency domain feature to obtain each frequency domain feature map.

[0176] Specifically, the spatial feature extraction module may include a hollow spatial pyramid pooling module and a convolutional channel compression module. The hollow spatial pyramid pooling module is used to extract multi-scale spatial frequency domain features from features at each level to obtain spatial frequency domain features. These spatial frequency domain features reflect the feature information of EEG spectrogram data at the spatial frequency domain level.

[0177] The convolutional channel compression module here is used to compress the channels of each spatial frequency domain feature to obtain each frequency domain feature map. For example, a 1*1 convolution can be used to compress the channels of each spatial frequency domain feature to obtain each frequency domain feature map.

[0178] The epileptic seizure prediction method based on spatiotemporal multi-scale attention mechanism provided by the present invention is described below. The epileptic seizure prediction method based on spatiotemporal multi-scale attention mechanism described below can be referred to in correspondence with the epileptic seizure prediction device based on spatiotemporal multi-scale attention mechanism described above.

[0179] Based on any of the above embodiments, the present invention provides a method for predicting epileptic seizures based on a spatiotemporal multi-scale attention mechanism. Figure 3 This is a flowchart illustrating the epileptic seizure prediction method based on spatiotemporal multi-scale attention mechanisms provided by the present invention, as shown below. Figure 3 As shown, the method includes:

[0180] Step 310: Obtain EEG spectrogram data;

[0181] Step 320: Input the EEG data into the prediction model to obtain the epilepsy prediction result output by the prediction model;

[0182] The prediction model includes a backbone network module, a spatial feature extraction module, a temporal feature fusion module, and a classifier;

[0183] The backbone network module is used to perform multi-level attention extraction on the EEG spectrogram data to obtain features at each level. The spatial feature extraction module is used to perform multi-scale spatial attention extraction on the features at each level to obtain feature maps in each frequency domain. The temporal feature fusion module is used to fuse the feature maps in each frequency domain to obtain fused features. Based on the fused features and the feature maps in each frequency domain, the final hidden state features are obtained.

[0184] The classifier is used to classify based on the features of the final hidden state to obtain the epilepsy prediction result.

[0185] The method provided in this invention employs a spatiotemporal multi-scale attention mechanism to extract features from EEG spectrogram data. This effectively captures the features of EEG spectrogram data at different spatial and temporal scales, enabling the prediction model to better adapt to the diversity of EEG signals and improve the accuracy and robustness of epileptic seizure prediction. Furthermore, the backbone network module adopts a hierarchical feature learning strategy, which can learn features at different levels and abstraction levels. This hierarchical feature learning method helps to extract more representative features to cope with complex patterns and noise in EEG signals.

[0186] Based on any of the above embodiments, the acquisition of electroencephalogram (EEG) data further includes, prior to:

[0187] Acquire raw EEG data;

[0188] The original EEG data is resampled to obtain resampled EEG data, and a preset sliding window is used to extract the resampled EEG data for a preset duration to obtain multiple extracted EEG segments.

[0189] The multiple extracted EEG segments were filtered using a band-stop filter and a high-pass filter to obtain filtered EEG data.

[0190] The filtered EEG data is randomly subjected to local time step random swapping, and Gaussian noise is randomly added to the filtered EEG data to obtain data-enhanced EEG data, and the EEG spectrogram data is obtained based on the data-enhanced EEG data.

[0191] Based on any of the above embodiments, obtaining the EEG spectrogram data based on the data-enhanced EEG data includes:

[0192] The data-enhanced EEG data were sequentially subjected to short-time Fourier transform and normalization to obtain the EEG spectrogram data.

[0193] Based on any of the above embodiments, the training steps of the prediction model include:

[0194] Determine the initial prediction model, acquire sample EEG spectrogram data, and generate labels based on the stage of the seizure indicated by the enhanced EEG data;

[0195] The sample EEG spectrogram data is input into the initial prediction model to obtain the epilepsy classification result output by the initial prediction model, as well as the previous embedding layer features of the epilepsy classification result;

[0196] Based on the epilepsy classification results and the labels, the classification loss is determined, and based on the features of the previous embedding layer, positive samples, and negative samples, the triplet loss is determined.

[0197] Based on the classification loss and the triplet loss, the total loss is determined, and the parameters of the initial prediction model are iterated based on the total loss to obtain the prediction model.

[0198] Based on any of the above embodiments, the step of acquiring electroencephalogram (EEG) data further includes:

[0199] The preic and interictal phases of the electroencephalogram (EEG) data are balanced to obtain balanced EEG data.

[0200] Accordingly, the step of inputting the EEG data into the prediction model to obtain the epilepsy prediction result output by the prediction model includes:

[0201] The balanced spectrum data is input into the prediction model to obtain the epilepsy prediction results output by the prediction model.

[0202] Based on any of the above embodiments, the spatial feature extraction module includes a hollow spatial pyramid pooling module and a convolutional channel compression module. The hollow spatial pyramid pooling module is used to extract multi-scale spatial frequency domain features from the features at each level to obtain each spatial frequency domain feature. The convolutional channel compression module is used to compress the channels of each spatial frequency domain feature to obtain each frequency domain feature map.

[0203] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a method for predicting epileptic seizures based on a spatiotemporal multi-scale attention mechanism. This method includes: acquiring electroencephalogram (EEG) spectrogram data; inputting the EEG spectrogram data into a prediction model to obtain an epileptic seizure prediction result output by the prediction model; the prediction model includes a backbone network module, a spatial feature extraction module, a temporal feature fusion module, and a classifier; the backbone network module performs multi-level attention extraction on the EEG spectrogram data to obtain features at each level; the spatial feature extraction module performs multi-scale spatial attention extraction on the features at each level to obtain frequency domain feature maps; the temporal feature fusion module performs feature fusion on the frequency domain feature maps to obtain fused features, and based on the fused features and the frequency domain feature maps, obtains final hidden state features; the classifier performs classification based on the final hidden state features to obtain the epileptic seizure prediction result.

[0204] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0205] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the epilepsy seizure prediction method based on spatiotemporal multi-scale attention mechanism provided by the above methods. The method includes: acquiring electroencephalogram (EEG) spectrogram data; inputting the EEG spectrogram data into a prediction model to obtain an epilepsy prediction result output by the prediction model; the prediction model includes a backbone network module, a spatial feature extraction module, a temporal feature fusion module, and a classifier; the backbone network module is used to perform multi-level attention extraction on the EEG spectrogram data to obtain features at each level; the spatial feature extraction module is used to perform multi-scale spatial attention extraction on the features at each level to obtain frequency domain feature maps; the temporal feature fusion module is used to perform feature fusion on the frequency domain feature maps to obtain fused features, and based on the fused features and the frequency domain feature maps, obtain final hidden state features; the classifier is used to classify based on the final hidden state features to obtain the epilepsy prediction result.

[0206] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the epilepsy seizure prediction method based on spatiotemporal multi-scale attention mechanism provided by the above methods. The method includes: acquiring electroencephalogram (EEG) spectrogram data; inputting the EEG spectrogram data into a prediction model to obtain an epilepsy prediction result output by the prediction model; the prediction model includes a backbone network module, a spatial feature extraction module, a temporal feature fusion module, and a classifier; the backbone network module is used to perform multi-level attention extraction on the EEG spectrogram data to obtain features at each level; the spatial feature extraction module is used to perform multi-scale spatial attention extraction on the features at each level to obtain frequency domain feature maps; the temporal feature fusion module is used to perform feature fusion on the frequency domain feature maps to obtain fused features, and based on the fused features and the frequency domain feature maps, obtains a final hidden state feature; the classifier is used to classify based on the final hidden state feature to obtain the epilepsy prediction result.

[0207] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0208] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0209] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A seizure prediction system based on spatiotemporal multi-scale attention mechanisms, characterized in that, include: Acquisition unit, used to acquire electroencephalogram (EEG) data; The prediction unit is used to input the electroencephalogram data into the prediction model to obtain the epilepsy prediction result output by the prediction model. The prediction model includes a backbone network module, a spatial feature extraction module, a temporal feature fusion module, and a classifier; The backbone network module is used to perform multi-level attention extraction on the EEG spectrogram data to obtain features at each level. The spatial feature extraction module is used to perform multi-scale spatial attention extraction on the features at each level to obtain feature maps in each frequency domain. The temporal feature fusion module is used to fuse the feature maps in each frequency domain to obtain fused features. Based on the fused features and the feature maps in each frequency domain, the final hidden state features are obtained. The hierarchical features are used to reflect the feature information of the EEG spectrogram data at different scales and levels of abstraction, and the frequency domain feature maps are used to reflect the frequency domain feature information of the EEG spectrogram data. The classifier is used to classify based on the features of the final hidden state to obtain the epilepsy prediction result; The spatial feature extraction module includes a hollow spatial pyramid pooling module and a convolutional channel compression module. The hollow spatial pyramid pooling module is used to extract multi-scale spatial frequency domain features from the features at each level to obtain each spatial frequency domain feature. The convolutional channel compression module is used to compress the channels of each spatial frequency domain feature to obtain each frequency domain feature map. The backbone network module uses bilinear interpolation to upsample the EEG spectrogram data, uses 1*1 convolution for channel compression, and then uses a Swin Transformer pre-trained model as the backbone to learn features of different scales and levels of abstraction through a multi-layer attention network.

2. The epileptic seizure prediction system based on spatiotemporal multi-scale attention mechanism according to claim 1, characterized in that, It also includes a data enhancement unit, which comprises: The raw EEG data acquisition unit is used to acquire raw EEG data; The segmentation unit is used to resample the original EEG data to obtain resampled EEG data, and use a preset sliding window to segment the resampled EEG data for a preset duration to obtain multiple segmented EEG fragments. The filtering unit is used to filter the multiple extracted EEG segments using a band-stop filter and a high-pass filter to obtain filtered EEG data. The data augmentation subunit is used to randomly swap the local time steps of the filtered EEG data and randomly add Gaussian noise to the filtered EEG data to obtain data-augmented EEG data, and to obtain the EEG spectrogram data based on the data-augmented EEG data.

3. The epileptic seizure prediction system based on spatiotemporal multi-scale attention mechanism according to claim 2, characterized in that, The data augmentation subunit is used for: The data-enhanced EEG data were sequentially subjected to short-time Fourier transform and normalization to obtain the EEG spectrogram data.

4. The epileptic seizure prediction system based on spatiotemporal multi-scale attention mechanism according to claim 1, characterized in that, It also includes a model training unit, which is used for: Determine the initial prediction model, acquire sample EEG spectrogram data, and generate labels based on the episodic stage of the data-enhanced EEG data; The sample EEG spectrogram data is input into the initial prediction model to obtain the epilepsy classification result output by the initial prediction model, as well as the previous embedding layer features of the epilepsy classification result; Based on the epilepsy classification results and the labels, the classification loss is determined, and based on the features of the previous embedding layer, positive samples, and negative samples, the triplet loss is determined. Based on the classification loss and the triplet loss, the total loss is determined, and the parameters of the initial prediction model are iterated based on the total loss to obtain the prediction model.

5. The epileptic seizure prediction system based on spatiotemporal multi-scale attention mechanism according to any one of claims 1 to 4, characterized in that, It also includes a data balancing unit, which is used for: The preic and interictal phases of the electroencephalogram (EEG) data are balanced to obtain balanced EEG data. Accordingly, the prediction unit is used for: The balanced spectrum data is input into the prediction model to obtain the epilepsy prediction results output by the prediction model.

6. A method for predicting epileptic seizures based on spatiotemporal multi-scale attention mechanisms, characterized in that, include: Acquire electroencephalogram (EEG) data; The electroencephalogram (EEG) data is input into the prediction model to obtain the epilepsy prediction result output by the prediction model. The prediction model includes a backbone network module, a spatial feature extraction module, a temporal feature fusion module, and a classifier; The backbone network module is used to perform multi-level attention extraction on the EEG spectrogram data to obtain features at each level. The spatial feature extraction module is used to perform multi-scale spatial attention extraction on the features at each level to obtain feature maps in each frequency domain. The temporal feature fusion module is used to fuse the feature maps in each frequency domain to obtain fused features. Based on the fused features and the feature maps in each frequency domain, the final hidden state features are obtained. The hierarchical features are used to reflect the feature information of the EEG spectrogram data at different scales and levels of abstraction, and the frequency domain feature maps are used to reflect the frequency domain feature information of the EEG spectrogram data. The classifier is used to classify based on the features of the final hidden state to obtain the epilepsy prediction result; The spatial feature extraction module includes a hollow spatial pyramid pooling module and a convolutional channel compression module. The hollow spatial pyramid pooling module is used to extract multi-scale spatial frequency domain features from the features at each level to obtain each spatial frequency domain feature. The convolutional channel compression module is used to compress the channels of each spatial frequency domain feature to obtain each frequency domain feature map. The backbone network module uses bilinear interpolation to upsample the EEG spectrogram data, uses 1*1 convolution for channel compression, and then uses a Swin Transformer pre-trained model as the backbone to learn features of different scales and levels of abstraction through a multi-layer attention network.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the epileptic seizure prediction method based on spatiotemporal multi-scale attention mechanism as described in claim 6.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the epileptic seizure prediction method based on the spatiotemporal multi-scale attention mechanism as described in claim 6.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the epileptic seizure prediction method based on the spatiotemporal multi-scale attention mechanism as described in claim 6.

Images