An electroencephalogram epilepsy pattern recognition method based on time-frequency fusion features

Abstract

The method discloses a brain electrical seizure pattern recognition method based on time-frequency fusion features and belongs to the technical field of medical signal processing.The spatial information expression capability of the electroencephalogram data is effectively enhanced by introducing a differential sequence construction method; in combination with a short-time Fourier transform (STFT) and a mel filter bank, efficient modeling of the electroencephalogram signal in the time-frequency domain is realized, and local periodicity and rhythm characteristics in abnormal brain activity patterns are fully captured.EfficientNet-b0 is selected as a core classification model, and the core classification model has the characteristics of small parameter quantity, high calculation efficiency and strong generalization capability.Through image splicing, interpolation scaling and other processing, the model can be lightweight deployed without sacrificing performance.The present application realizes the full-process automatic processing from the original EEG signal input to the final classification result output, does not need manual participation in feature engineering or label marking, and greatly improves the system operation efficiency and practicability.

Description

Technical Field

[0001] This invention belongs to the field of medical signal processing technology, specifically involving the processing and conversion of electroencephalogram (EEG) signals and the training and inference of deep classification networks in the spectral domain. Background Technology

[0002] Electroencephalography (EEG) is widely used to detect seizures (SZ) and other abnormal brain activity. With advancements in EEG analysis technology, early detection and timely treatment of critically ill patients are becoming increasingly feasible. While the diagnosis of epilepsy typically requires a combination of EEG and clinical scales, accurate EEG interpretation is crucial for optimizing treatment outcomes. Although manual analysis of EEG data by specialized neurologists is of significant value, this remains a major bottleneck due to the labor-intensive nature of the process. Furthermore, different analysts (even experts) may interpret EEG data differently. Given these challenges, there is an urgent need for high-performance computer-aided diagnostic (CAD) tools that are highly accurate, fast, and easy to interpret; such tools can significantly improve detection rates, especially in cases of disagreement among experts, by improving efficiency and reducing discrepancies among assessors, thus providing patients with reliable and professional treatment recommendations. Traditional CAD systems heavily rely on predefined features. For example, rhythmic delta activity mixed with sharp discharges may indicate the end of a sharp transient response (SZ) or the beginning of an artifact, while a unilateral sharp transient in the frontal lobe with a frequency of approximately 1 Hz can be considered a potential brain activity (PD). The effectiveness of such methods largely depends on the successful preprocessing of noisy signals. In contrast, end-to-end deep learning methods show great potential for accurately identifying specific types of brain activity due to their powerful ability to extract information from complex digital signals without relying on feature engineering based on prior knowledge. Summary of the Invention

[0003] This invention proposes a fusion-based, lightweight, interpretable, and high-performance algorithm that can be deployed on any local computer for identifying and classifying abnormal brain activity patterns.

[0004] The specific implementation scheme of this technology is as follows: A method for EEG epilepsy pattern recognition based on time-frequency fusion features, the method comprising:

[0005] Step 1: Acquire EEG data. This data must be collected according to the IO20 standard and includes EEG pattern labels: LPD, GPD, LRDA, GRDA, Seizure, and Other. These labels represent left-sided periodic discharge, generalized periodic discharge, left-sided rhythmic delta activity, generalized rhythmic delta activity, epileptic seizure, and others, respectively.

[0006] Step 4: Obtain EEG sequences using differential computation;

[0007] LL: Fp1-F7-T3-T5-O1,

[0008] LP: Fp1-F3-C3-P3-O1,

[0009] LZ: Fp1-Fz-Cz-Pz-O1,

[0010] RZ: Fp2-Fz-Cz-Pz-O1,

[0011] RP: Fp2-F4-C4-P4-O2,

[0012] RR: Fp2-F8-T4-T6-O2;

[0013] Wherein, LL represents the lower left part, LP represents the posterior left part, LZ represents the left zygomatic part, RZ represents the right zygomatic part, RP represents the posterior right part, RR represents the lower right part, Fp1 represents the left frontal pole region, Fp2 represents the right frontal pole region, F3 represents the left frontal lobe region, F4 represents the right frontal lobe region, Fz represents the frontal midline region, F7 represents the left anterior temporal region, T3 represents the left temporal region, T4 represents the right temporal region, T5 represents the left posterior temporal region, T6 represents the right posterior temporal region, C3 represents the left central region, Cz represents the central midline region, C4 represents the right central region, P3 represents the left parietal lobe region, P4 represents the right parietal lobe region, Pz represents the parietal midline region, O1 represents the left occipital lobe region, and O2 represents the right occipital lobe region.

[0014] Step 5: Obtain Fourier spectrum data;

[0015] Step 6: Obtain Mel spectrum data;

[0016] Step 7: Time-Spectral Domain Model Training:

[0017] Step 8: Use the trained EfficientNet-b0 model to identify the newly obtained EEG data.

[0018] Furthermore, the specific method for step 5 is as follows:

[0019] The difference sequence obtained in step 4 is subjected to Fast Fourier Transform to obtain the spectrum and phase diagram, and the STFT of the spectrum is retained as a spectral domain feature:

[0020] STFT=∫x(t)w(t-τ)e iwt dt

[0021] Here, x(t) represents the original signal, t represents the time variable used to traverse the entire time signal axis, τ represents the center time, represents the position of the window function w(t-τ) on the time axis, w represents the angular frequency, which is the frequency component, and w(t-τ) is a window function that restricts the analysis to only the part of the signal x(t) close to time τ, thus giving the Fourier transform locality.

[0022] Furthermore, the specific method for step 6 is as follows:

[0023] Step 6.1: Convert the Fourier spectrum data into a power spectrum, i.e.:

[0024] power = |STFT| 2

[0025] Step 6.2: Convert the power spectrum frequency to Mel frequency and determine the upper bound m. max and lower bound m min :

[0026]

[0027] Among them, f min Minimum frequency of power spectrum, f max Indicates the maximum frequency of the power spectrum;

[0028] Step 6.3: Set up multiple Mel filters to capture different frequency distribution characteristics; apply each Mel filter to the result obtained in step 5, and weight and aggregate all frequency components to obtain S. Mel (m):

[0029]

[0030] Where S(k) represents the Mel filter output, H m (k) represents the amplitude of frequency k in the power spectrum, represents the total number of frequencies, and m represents the m-th Mel filter;

[0031] Step 6.4: Convert the Mel spectrum into a decibel signal dB(S):

[0032]

[0033] Where ref is the scaling factor, and dB(S) represents the decibel signal output result;

[0034] Step 6.5: Standardize the decibel signal obtained in Step 6.4:

[0035] dB(S) = (dB(S) + 40) / 40.

[0036] Furthermore, the specific method for step 7 is as follows:

[0037] Step 7.1: Select six sequences: LL, LP, RP, RR, LZ, and RZ. Step 7.2: Use all six sequences. Step 7.3: Use only four of them.

[0038] Step 7.2: Use a Mel filter to denoise the FFT spectra of the six sequences from Step 7.1;

[0039] Step 7.3: Combine the spectrogram obtained from Mel noise reduction with the FFT spectrograms of the four sequences LL, LP, RP, and RR in top-to-bottom order; then scale the combined spectrogram to 768x768 using bilinear interpolation.

[0040] Step 7.4: Use the spliced ​​data and corresponding labels obtained in Step 7.3 to train the EfficientNet-b0 model.

[0041] Furthermore, in step 6.4, the scaling factor ref is taken as the maximum value of the Mel filter output.

[0042] This invention effectively enhances the spatial information representation capability of EEG data by introducing a differential sequence construction method. Combining Short-Time Fourier Transform (STFT) and Mel filter banks, it achieves efficient modeling of EEG signals in the time-frequency domain, fully capturing the local periodicity and rhythmic features of abnormal brain activity patterns. EfficientNet-b0 is selected as the core classification model, characterized by few parameters, high computational efficiency, and strong generalization ability. By processing the input images through stitching, interpolation, and scaling, the model can be deployed in a lightweight manner without sacrificing performance. Furthermore, the design emphasizes the visualization and structured representation of features, such as using STFT spectrograms and Mel spectrograms as input features. This not only preserves the time-frequency information of the original signal but also makes the model output results easier for doctors to understand and verify. In addition, this invention achieves fully automated processing from the input of the original EEG signal to the output of the final classification result, eliminating the need for manual feature engineering or label annotation, greatly improving the system's operating efficiency and practicality. Attached Figure Description

[0043] Figure 1 This is an example of the model building process for this method;

[0044] Figure 2 Here is an example of the training data label distribution;

[0045] Figure 3 This is an example of the IO20 standard EEG electrode distribution;

[0046] Figure 4 This is an example comparing the spectrograms before and after using Mel noise reduction;

[0047] Figure 5 This is a schematic diagram of the splicing of Fourier spectrum and normalized Mel spectrum;

[0048] Figure 6This is a comparison of the accuracy of mainstream convolutional models as prediction backbone networks;

[0049] Figure 7 It is the AUC curve of the mainstream convolutional model used to predict the backbone network;

[0050] Figure 8 This is a comparison of the top 2 and top 3 accuracy rates of the Transformer model as the prediction backbone network;

[0051] Figure 9 This is a visual example of a deep network model;

[0052] Figure 10 These are examples of the most representative pathological fragments selected based on the model's prediction results; Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0055] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, including a series of steps, processes, methods, etc., is not limited to the steps listed, but may optionally include steps not listed, or may optionally include other steps inherent to these processes, methods, products, or devices.

[0056] The purpose of this invention is to provide a signal mapping and adaptive recognition method for detecting abnormal EEG patterns based on an end-to-end network. This method can provide a high-accuracy prediction probability of abnormal EEG patterns, reduce the workload of doctors, and can be easily deployed on any local computer.

[0057] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The relevant code for this method has been open-sourced on the website. https: / / github.com / hardhash / SMART4EEG The specific implementation method mentions that the code files can be downloaded from open-source websites. The code files involve one-stage training and two-stage training. The difference is that the two-stage training is based on the first-stage training and uses high-quality data (expert-reviewed labels greater than 3) to fine-tune the model. This method uses an RTX 3090 graphics card for local training, achieving 35.6 TFLOPS at FP16 data precision. The training time is based on this computing resource. See the detailed implementation steps below. Figure 1 .

[0058] Step A: Data Preparation and Hardware / Environment Preparation

[0059] The dataset used in this method comes from a publicly available dataset provided by Harvard Medical School and the CriticalCare EEG Monitoring Research Consortium (CCEMRC), containing 1950 patients and 106,800 EEG signal segments and 11,138 spectral data points. Specifically, it includes 20,933 cases of Seizure, 16,702 cases of GPD, 16,640 cases of LRDA, 18,861 cases of GRDA, 14,856 cases of LPD, and 18,808 other cases. Each EEG segment was assessed for epilepsy pattern by at least one expert; the specific expert label distribution can be found in [link to expert label distribution]. Figure 2 The data was generated using EEG signal identification and acquisition methods described in [link to relevant documentation]. Figure 3 ;

[0060] The dataset is divided into a training set and a validation set (80% training set and 20% validation set);

[0061] Hardware preparation. This method requires the use of a GPU for model training, so you need to prepare a hard drive that meets the requirements for data storage and an NVIDIA graphics card with sufficient video memory for the training data;

[0062] Configure the environment. This includes (1) a conda environment with Python 3.10 or higher; (2) CUDA and cuDNN for GPU-compatible computation acceleration; and (3) a GPU-compatible PyTorch tool with PyTorch 2.0 or higher.

[0063] Training / validation set partitioning. 80% of the original data is used for training, and 20% for validation. (Corresponding code file: divide_valid.ipynb)

[0064] Mel-frequency spectrum conversion. First, the six difference paths need to be converted into Fourier spectra, then converted into standardized Mel-frequency spectra, with the Mel-to-decibels conversion ref parameter set to the maximum value of the Mel-frequency spectrum. See the comparison before and after conversion. Figure 4 The corresponding code file is eeg2mel.ipynb;

[0065] Step B: Loss Function Design

[0066] This method aims to assist doctors in accurately identifying complex EEG patterns, helping patients receive precise treatment and reducing doctors' workload. Therefore, KL divergence is chosen as the loss function. KL divergence can characterize the similarity between the predicted probability distribution and the actual probability distribution; the closer the KL divergence is to 0, the closer the predicted probability is to the actual probability. The specific formula is as follows:

[0067]

[0068] Where P is the actual probability distribution and Q is the predicted probability distribution.

[0069] Step C: Model Training

[0070] Training data construction. Mel transform is used on the 6-channel data to obtain the final data, which is then concatenated with the original data to form the training data format. See the example for a specific concatenation step. Figure 5 .

[0071] Data standardization. Considering the alignment of Mel-frequency and Fourier spectrum data, the spliced ​​data needs to be standardized again. Two methods are used: simple standardization and trimmed standardization, with the specific formulas as follows:

[0072]

[0073] Model Training. EfficientNet-b0 was chosen as the backbone network, where EfficientNet represents the backbone network and b0 represents the model size. The channel parameter was adjusted to 1, and the final linear transformation matrix was set to (1000, 7), where 1000 represents the output dimension of the preceding network and 7 represents the output dimension. The performance of the EfficientNet-b0 model is compared with other mainstream convolutional models in terms of accuracy when predicting different subtypes. Figure 6 Models with parameter sizes b1 and b0 performed best. DenseNet-201 had a lower average prediction accuracy than the EfficientNet model, while ConvNeXt showed significant overfitting. Their average AUC curves are shown in [Figure showing...]. Figure 7 The closer the AUC curve is to the top left corner, the better the performance; EfficientNet has an advantage in this regard. See the comparison results of prediction accuracy with the Transformer model. Figure 8The model based on EfficientNet showed better classification performance.

[0074] Use softmax to transform the model's prediction results into a probability distribution:

[0075]

[0076] Five-fold cross-validation. Steps ③ and ④ are repeated for each fold to verify model performance through cross-validation. The complete training corresponds to the code files M3_1.ipynb and M3_2.ipynb. The former takes approximately 7 hours to run and generates several xxx.pt files, which are the model weight files for the first stage of training; the latter takes approximately 5 hours to run and generates several xxxv2.pt files, which are the model weight files for the second stage of training.

[0077] Save the model results. Save the final prediction results file of the model. The corresponding code files are m3_validation.ipynb and m3_save_with_label.ipynb.

[0078] Step D: Interpretive analysis and deployment feasibility analysis

[0079] To enhance model credibility, the GradCAM method was used to perform back-calculation on the model output to obtain an attention heatmap, see [link / reference]. Figure 9 The spectrograms can be further mapped onto the original differential time series of EEG. Representative EEGs with varying degrees of expert disagreement and agreement were selected for model prediction. The model accurately segmented pathological EEG fragments, and the segmentation results corresponded well with the professional knowledge base, demonstrating the practicality and feasibility of the method. (See [link to relevant documentation]). Figure 10 To ensure generalizability, the model has 6.52M parameters, which is 7.3% of the mainstream computer vision network ViT model. Experiments show that the model's accuracy is 3.6% higher than the ViT model. This means that for pathological inferences, only the computing power of a single GTX 1050 graphics card is needed to complete the calculation within seconds. A GTX 1050 currently costs approximately 300 RMB. Therefore, it is a lightweight and efficient model that can be deployed locally.

[0080] Step E: Reasoning

[0081] Once the model weight parameters are obtained, for subsequent EEG pattern inference using new data, simply adjust the data path and run the infer.ipynb file. Furthermore, the model can be packaged into a more lightweight ONXX format and deployed on a server, creating a WebUI to visually display the model's inference results and details.

Claims

1. A method for EEG epilepsy pattern recognition based on time-frequency fusion features, the method comprising: Step 1: Acquire EEG data. This data must be collected according to the IO20 standard and include EEG pattern labels: LPD, GPD, LRDA, GRDA, Seizure, and Other. These labels represent left-sided periodic discharges, generalized periodic discharges, left-sided rhythmic delta activity, generalized rhythmic delta activity, epileptic seizures, and others, respectively. Step 4: Obtain EEG sequences using differential computation; LL: Fp1-F7-T3-T5-O1, LP: Fp1-F3-C3-P3-O1, LZ: Fp1-Fz-Cz-Pz-O1, RZ: Fp2-Fz-Cz-Pz-O1, RP: Fp2-F4-C4-P4-O2, RR: Fp2-F8-T4-T6-O2; Wherein, LL represents the lower left part, LP represents the posterior left part, LZ represents the left zygomatic part, RZ represents the right zygomatic part, RP represents the posterior right part, RR represents the lower right part, Fp1 represents the left frontal pole region, Fp2 represents the right frontal pole region, F3 represents the left frontal lobe region, F4 represents the right frontal lobe region, Fz represents the frontal midline region, F7 represents the left anterior temporal region, T3 represents the left temporal region, T4 represents the right temporal region, T5 represents the left posterior temporal region, T6 represents the right posterior temporal region, C3 represents the left central region, Cz represents the central midline region, C4 represents the right central region, P3 represents the left parietal lobe region, P4 represents the right parietal lobe region, Pz represents the parietal midline region, O1 represents the left occipital lobe region, and O2 represents the right occipital lobe region. Step 5: Obtain Fourier spectrum data; Step 6: Obtain Mel spectrum data; Step 7: Time-Spectral Domain Model Training: Step 7.1: Select six sequences: LL, LP, RP, RR, LZ, and RZ. Step 7.2: Use all six sequences. Step 7.3: Use only four of them. Step 7.2: Use a Mel filter to denoise the FFT spectra of the six sequences from Step 7.1; Step 7.3: Combine the spectrogram obtained from Mel-denoising with the FFT spectrograms of the four sequences LL, LP, RP, and RR in top-to-bottom order; scale the combined spectrogram to 768x768 using bilinear interpolation. Step 7.4: Use the spliced ​​data and corresponding labels obtained in Step 7.3 to train the EfficientNet-b0 model; Step 8: Use the trained EfficientNet-b0 model to identify the newly obtained EEG data.

2. The EEG epilepsy pattern recognition method based on time-frequency fusion features as described in claim 1, characterized in that, The specific method for step 5 is as follows: Apply Fast Fourier Transform to the difference sequence obtained in step 4 to obtain the spectrum and phase diagram, and retain the spectrum. As a spectral domain feature: in, Represents the original signal. The time variable represents the signal axis used to traverse the entire time frame. Indicates the center time, representing the window function. Position on the timeline This represents angular frequency, which is a frequency component. It is a window function that restricts access only to signals. Close to time We analyze the parts and assign locality to the Fourier transform.

3. The method for EEG epilepsy pattern recognition based on time-frequency fusion features as described in claim 1, characterized in that, The specific method for step 6 is as follows: Step 6.1: Convert the Fourier spectrum data into a power spectrum, i.e.: ； Step 6.2: Convert the power spectrum frequency to Mel frequency and determine the upper bound. and the lower realm : ； in, Minimum frequency of power spectrum Indicates the maximum frequency of the power spectrum; Step 6.3: Set up multiple Mel filters to capture different frequency distribution characteristics; apply each Mel filter to the results obtained in step 5, and weight and aggregate all frequency components to obtain... : ； in, This represents the output of the Mel filter. Let represent the amplitude of frequency k in the power spectrum, and represent the total number of frequencies. This represents the m-th Mel filter; Step 6.4: Convert the Mel spectrum into a decibel signal. : ； in, It is the scaling factor. This indicates the output result of the decibel signal; Step 6.5: Standardize the decibel signal obtained in Step 6.4: 。 4. The EEG epilepsy pattern recognition method based on time-frequency fusion features as described in claim 3, characterized in that, The scaling factor in step 6.4 Take the maximum value of the Mel filter output.

Images