Electroencephalogram epilepsy mode recognition method based on time-frequency fusion features

Abstract

The invention discloses an electroencephalogram epilepsy mode recognition method based on time-frequency fusion features, and belongs to the technical field of medical signal processing. According to the method, a differential sequence construction method is introduced, so that the spatial information expression capability of electroencephalogram data is effectively enhanced; according to the method, short-time Fourier transform (STFT) and a Mel filter bank are combined, efficient modeling of electroencephalogram signals in a time-frequency domain is achieved, and local periodicity and rhythmicity characteristics in an abnormal brain activity mode are fully captured. The EfficientNet-b0 is selected as a core classification model, and the method has the advantages of being small in parameter quantity, high in calculation efficiency and high in generalization ability. By performing splicing, interpolation scaling and other processing on the input image, lightweight deployment of the model can be realized on the premise of not sacrificing performance. According to the method, full-process automatic processing from original EEG signal input to final classification result output is realized, manual participation in feature engineering or label labeling is not needed, and the system operation efficiency and practicability are greatly improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical signal processing, and particularly relates to processing and conversion of electroencephalogram signals and deep classification network training and reasoning in time and spectral domains. BACKGROUND

[0002] Electroencephalogram (EEG) is widely used to detect seizures (SZ) and other abnormal brain activities. With the advancement of electroencephalogram analysis technology, early detection and timely treatment of critically ill patients become increasingly feasible. Although the diagnosis of epilepsy usually requires the combination of electroencephalogram and clinical scales, accurate electroencephalogram interpretation is essential for optimizing treatment outcomes. Although manual analysis of electroencephalogram data by professional neurologists is of great value, due to the labor-intensive nature of the process, it remains a major bottleneck. In addition, different analysts (even experts) have different interpretations of electroencephalogram data. In view of these challenges, there is an urgent need for high-performance computer-aided diagnosis (CAD) tools, which should have high accuracy, high speed and easy-to-interpret features; at the same time, such tools can significantly improve the detection rate, especially in the case of expert disagreement, and can provide reliable and professional treatment recommendations for patients by improving efficiency and reducing inter-evaluator disagreement. Traditional CAD systems rely heavily on pre-defined features. For example, rhythmic delta activity mixed with sharp discharges may indicate the end of SZ or the beginning of artifacts, while frontal unilateral sharp transients with a frequency of about 1 Hz can be considered as PD, and the effectiveness of such methods largely depends on the successful preprocessing of noisy signals. In contrast, end-to-end deep learning methods, due to their strong ability to extract information from complex digital signals, do not rely on prior knowledge-based feature engineering, and show great potential in accurately identifying specific types of brain activity. SUMMARY

[0003] The application proposes a fused lightweight interpretable high-performance algorithm that can be deployed on any local computer to identify and classify abnormal brain activity patterns.

[0004] The specific implementation scheme of the technology is as follows: a brain electrical seizure pattern recognition method based on time-frequency fusion features, the method comprising:

[0005] Step 1: Obtain electroencephalogram (EEG) data, which should be collected according to the IO20 standard, containing electroencephalogram pattern labels: LPD, GPD, LRDA, GRDA, Seizure, Other, which represent left periodic discharge, generalized periodic discharge, left rhythmic delta activity, generalized rhythmic delta activity, seizure, and other;

[0006] Step 4: Use difference calculation to obtain EEG brain electrical sequence;

[0007] LL: Fp1-F7-T3-T5-O1,

[0008] LP: Fp1-F3-C3-P3-O1,

[0009] LZ: Fp1-Fz-Cz-Pz-O1,

[0010] RZ: Fp2-Fz-Cz-Pz-O1,

[0011] RP: Fp2-F4-C4-P4-O2,

[0012] RR: Fp2-F8-T4-T6-O2;

[0013] Wherein, LL represents left lower part, LP represents left rear part, LZ represents left zygomatic part, RZ represents right zygomatic part, RP represents right rear part, RR represents right lower part, Fp1 represents left frontal pole region, Fp2 represents right frontal pole region, F3 represents left frontal region, F4 represents right frontal region, Fz represents midline region, F7 represents left anterior temporal region, T3 represents left temporal region, T4 represents right temporal region, T5 represents left posterior temporal region, T6 represents right posterior temporal region, C3 represents left central region, Cz represents midline region, C4 represents right central region, P3 represents left parietal region, P4 represents right parietal region, Pz represents midline region, O1 represents left occipital region, O2 represents right occipital region;

[0014] Step 5: obtaining Fourier spectrum data;

[0015] Step 6: obtaining Mel spectrum data;

[0016] Step 7: time-domain-spectrum domain model training:

[0017] Step 8: using the trained EfficientNet-b0 model to identify the newly obtained electroencephalogram data.

[0018] Further, the specific method of step 5 is:

[0019] The difference sequence obtained in step 4 is used to obtain a spectrogram and a phase diagram using fast Fourier transform, and the spectrogram STFT is retained as a spectrum domain feature:

[0020] STFT = ∫x(t)w(t-τ)e iwt dt

[0021] Wherein, x(t) represents an original signal, t represents a time variable used to traverse the entire time signal axis, τ represents a center time, represents the position of the window function w(t-τ) on the time axis, w represents an angular frequency, which is a frequency component, w(t-τ) is a window function, which limits the analysis to only the part of the signal x(t) close to the time τ, giving the Fourier transform locality.

[0022] Further, the specific method of step 6 is:

[0023] Step 6.1: Transform the Fourier spectrum data into power spectrum, that is:

[0024] power = |STFT| 2

[0025] Step 6.2: Transform the power spectrum frequency into mel frequency, determine the upper bound m max and the lower bound m min :

[0026]

[0027] Where, f min is the minimum frequency of the power spectrum, f max represents the maximum frequency of the power spectrum;

[0028] Step 6.3: Set multiple mel filters to capture different frequency distribution characteristics; apply each mel filter to the result obtained in step 5, and aggregate all frequency components to obtain S Mel (m):

[0029]

[0030] Where, S(k) represents the output of the mel filter, H m (k) represents the amplitude of k frequency in the power spectrum, N represents the total number of frequencies, and m represents the mth mel filter;

[0031] Step 6.4: Convert the mel spectrum into decibel signal dB(S):

[0032]

[0033] Where, ref is the scaling coefficient, and dB(S) represents the output result of the decibel signal;

[0034] Step 6.5: Standardize the decibel signal obtained in step 6.4:

[0035] dB(S) = (dB(S) + 40) / 40.

[0036] Further, the specific method of step 7 is:

[0037] Step 7.1: Select LL, LP, RP, RR, LZ, and RZ six sequences, step 7.2 uses 6 sequences, and step 7.3 uses only 4 sequences among them;

[0038] Step 7.2: denoising the FFT spectrograms of the six sequences in step 7.1 using a Mel filter;

[0039] Step 7.3: concatenating the spectrograms obtained by Mel denoising and the FFT spectrograms of the LL, LP, RP, and RR sequences in an up-down order; and scaling the concatenated spectrograms to 768x768 using bilinear interpolation;

[0040] Step 7.4: training the EfficientNet-b0 model using the concatenated data obtained in step 7.3 and the corresponding labels.

[0041] Further, the scaling factor ref in step 6.4 takes the maximum value of the output of the Mel filter.

[0042] The present application effectively enhances the spatial information expression ability of electroencephalogram data by introducing a differential sequence construction method; combines short-time Fourier transform (STFT) and a Mel filter bank to realize efficient modeling of electroencephalogram signals in the time-frequency domain, and fully captures the local periodicity and rhythmicity characteristics in abnormal brain activity patterns. The EfficientNet-b0 is selected as the core classification model, which has the characteristics of small parameter quantity, high computational efficiency, and strong generalization ability. Through processing such as concatenation, interpolation scaling, etc. of the input image, the model can realize lightweight deployment without sacrificing performance. At the same time, attention is paid to the visualization and structured expression of features during the design process, for example, using STFT spectrograms and Mel spectrograms as input features, which not only preserves the time-frequency information of the original signal, but also makes the model output results easier for doctors to understand and verify. In addition, the present application realizes the full-process automation from the input of the original EEG signal to the output of the final classification result, without the need for human participation in feature engineering or label annotation, greatly improving the system running efficiency and practicality. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is an example of the model building process of the present method;

[0044] Figure 2 is an example of the label distribution of the training data;

[0045] Figure 3 is an example of the IO20 standard electroencephalogram electrode distribution;

[0046] Figure 4 is an example of the comparison of spectrograms before and after using Mel denoising;

[0047] Figure 5 is an example of the concatenation of Fourier spectrograms and standardized Mel spectrograms;

[0048] Figure 6is the accuracy comparison of mainstream convolutional models as the prediction backbone network;

[0049] Figure 7 is the AUC curve of mainstream convolutional models as the prediction backbone network;

[0050] Figure 8 is the TOP2 TOP3 accuracy comparison of the Transformer model as the prediction backbone network;

[0051] Figure 9 is a visual display example of a deep network model;

[0052] Figure 10 is an example of the most representative pathological fragments screened according to the model prediction results; DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0054] In this document, reference to“an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase“in an embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. Those skilled in the art will appreciate that embodiments described herein can be combined with other embodiments unless clearly indicated otherwise.

[0055] The terms“first”,“second”,“third”, and“fourth” and the like in the specification and claims of the application and the drawings are used to distinguish different objects, and are not used to describe a particular order. In addition, the terms“include” and“have” and any variations thereof are intended to cover non-exclusive inclusion. For example, a series of steps, processes, methods, etc. are not limited to the listed steps, but optionally include steps not listed, or optionally include other steps inherent to the processes, methods, products or devices.

[0056] The purpose of the present application is to provide an end-to-end network-based signal mapping and adaptive recognition method for detecting abnormal electroencephalogram patterns, which can give a high-accuracy prediction probability of abnormal electroencephalogram patterns, reduce the workload of doctors, and the method is easy to deploy on any local computer.

[0057] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below in combination with the drawings and specific embodiments. The method related code has been open sourced on the website https: / / github.com / hardhash / SMART4EEG The specific implementation method refers to the code file which can be downloaded from the open source website, and the code file involves 1-stage training and 2-stage training, the difference between which is that the 2-stage training is based on the 1-stage training and then uses high-quality data (expert evaluation label greater than 3) to fine-tune the model. The local training of the present method uses an RTX3090 graphics card, which has 35.6TFLOPS under FP16 data precision, and the training time is obtained according to the calculation resource. The specific implementation steps are shown in Figure 1 .

[0058] Step A: data preparation and hardware / environment preparation

[0059] The data set used in the present method is from the public data set provided by Harvard Medical School and Critical Care EEG Monitoring Research Consortium (CCEMRC), which has 1950 patients, a total of 106800 EEG signal segments, 11138 frequency spectrum data, specifically including 20933 cases of Seizure, 16702 cases of GPD, 16640 cases of LRDA, 18861 cases of GRDA, 14856 cases of LPD, and 18808 cases of Other. Each EEG segment is given a determination of epilepsy pattern by no less than one expert, and the specific expert label distribution is shown in Figure 2 . The EEG signal identification and acquisition method is shown in Figure 3 ;

[0060] The data set is divided into a training set and a validation set (80% of the training set and 20% of the validation set);

[0061] Hardware preparation. The present method needs to use GPU for model training, and needs to prepare hard disk for data storage and NVIDIA graphics card with sufficient display memory size to meet the training data;

[0062] Environment configuration. Including (1) conda environment of python >= 3.10; (2) cuda and cudnn matching the GPU for calculation acceleration; (3) GPU version of Pytorch tool of torch >= 2.0;

[0063] Training set / validation set division. The original data is 80% for training and 20% for validation. The corresponding code file is divide_valid.ipynb;

[0064] Mel-spectrogram transformation. First, we need to transform the six differential paths into Fourier spectrum, and then transform them into standardized Mel-spectrogram, and set the Mel-decibel transformation ref parameter value to the maximum value of the Mel-spectrogram. The comparison before and after the transformation is shown in Figure 4 , the corresponding code file is eeg2mel.ipynb;

[0065] Step B: Loss function design

[0066] The method aims to assist doctors in accurately identifying complex EEG patterns, help patients receive accurate treatment, and reduce the workload of doctors. Therefore, the KL divergence is selected as the loss function. The KL divergence can represent the similarity between the predicted probability distribution and the actual probability distribution. The closer the KL divergence is to 0, the closer the predicted probability is to the actual probability. The specific formula is as follows:

[0067]

[0068] where P is the actual probability distribution and Q is the predicted probability distribution.

[0069] Step C: Model training

[0070] Training data construction. The final data is obtained by using Mel transformation on the 6-channel data, and the original data is spliced to form the training data format. The specific splicing example is shown in Figure 5 .

[0071] Data standardization. Considering the alignment of Mel-spectrum and Fourier-spectrum data, the spliced data needs to be standardized again. Two standardization methods, simple standardization and clipping standardization, are used. The specific formula is as follows:

[0072]

[0073] Model training. EfficientNet-b0 is selected as the backbone network. EfficientNet represents the backbone network, and b0 represents the model size. The channel parameter is adjusted to 1, and the last linear transformation matrix dimension is (1000, 7). 1000 represents the output dimension of the previous network, and 7 represents the output dimension. The performance of EfficientNet-b0 model and other mainstream convolutional networks in predicting different subtypes is shown in Figure 6 , the models with b1 and b0 parameters have the best performance, and DenseNet-201 has lower average accuracy than EfficientNet models, and ConvNeXt has obvious overfitting. Their average AUC curves are shown in Figure 7 , the closer the AUC curve is to the upper left corner, the better the effect is, and EfficientNet has an advantage. The comparison results of the prediction accuracy of the Transformer model are shown in Figure 8, the model based on EfficientNet shows better classification results.

[0074] The model prediction results are converted into probability distribution using softmax:

[0075]

[0076] 5-fold cross-validation. Repeat steps ③ and ④ for each fold to verify the model performance through cross-validation. The complete training code files are M3_1.ipynb and M3_2.ipynb. The former takes about 7 hours to run and generates several xxx.pt files, which are the model weight files for the first stage training; the latter takes about 5 hours to run and generates several xxxv2.pt files, which are the model weight files for the second stage training.

[0077] Model result saving. Save the final prediction result file of the model. The corresponding code files are m3_validation.ipynb and m3_save_with_label.ipynb.

[0078] Step D: Explainable analysis and feasibility analysis of deployment

[0079] To enhance the credibility of the model, GradCAM method is used to calculate the attention heat map for the model output results. See Figure 9 The spectrum can be further mapped to the original differential time series of EEG. Representative EEGs with expert disagreement and consistency are selected for model prediction. The model can accurately segment pathological EEG segments, and the segmentation results are consistent with the professional knowledge base, proving the practicality and feasibility of the method. See Figure 10 To meet the generalizability, the model parameters are 6.52M, which is 7.3% of the mainstream computer vision network ViT model. Through experiments, the accuracy of the model is improved by 3.6% compared with the ViT model. This means that for pathological inference, only the computing power of a GTX 1050 graphics card is needed to obtain the result in a few seconds. The current market price of GTX1050 is about 300 RMB. Therefore, it is a completely local deployable lightweight and efficient model.

[0080] Step E: Inference

[0081] After obtaining the model weight parameters, for subsequent new data EEG pattern inference, only the data path needs to be adjusted to run the infer.ipynb file. Further, the model can be packaged into a more lightweight onxx format and deployed on a server to build a WebUI visual display of the model inference results and details.

Claims

1. A method for EEG epilepsy pattern recognition based on time-frequency fusion features, the method comprising: Step 1: Acquire EEG data. This data must be collected according to the IO20 standard and include EEG pattern labels: LPD, GPD, LRDA, GRDA, Seizure, and Other. These labels represent left-sided periodic discharge, generalized periodic discharge, left-sided rhythmic delta activity, generalized rhythmic delta activity, epileptic seizure, and others, respectively. Step 4: Obtain EEG sequences using differential computation; LL: Fp1-F7-T3-T5-O1, LP: Fp1-F3-C3-P3-O1, LZ: Fp1-Fz-Cz-Pz-O1, RZ: Fp2-Fz-Cz-Pz-O1, RP: Fp2-F4-C4-P4-O2, RR: Fp2-F8-T4-T6-O2; Wherein, LL represents the lower left part, LP represents the posterior left part, LZ represents the left zygomatic part, RZ represents the right zygomatic part, RP represents the posterior right part, RR represents the lower right part, Fp1 represents the left frontal pole region, Fp2 represents the right frontal pole region, F3 represents the left frontal lobe region, F4 represents the right frontal lobe region, Fz represents the frontal midline region, F7 represents the left anterior temporal region, T3 represents the left temporal region, T4 represents the right temporal region, T5 represents the left posterior temporal region, T6 represents the right posterior temporal region, C3 represents the left central region, Cz represents the central midline region, C4 represents the right central region, P3 represents the left parietal lobe region, P4 represents the right parietal lobe region, Pz represents the parietal midline region, O1 represents the left occipital lobe region, and O2 represents the right occipital lobe region. Step 5: Obtain Fourier spectrum data; Step 6: Obtain Mel spectrum data; Step 7: Time-Spectral Domain Model Training: Step 8: Use the trained EfficientNet-b0 model to identify the newly obtained EEG data.

2. The EEG epilepsy pattern recognition method based on time-frequency fusion features as described in claim 1, characterized in that, The specific method for step 5 is as follows: The difference sequence obtained in step 4 is subjected to Fast Fourier Transform to obtain the spectrum and phase diagram, and the STFT of the spectrum is retained as a spectral domain feature: STFT=∫x(t)w(t-τ)e iwt dt Where x(t) represents the original signal, t represents the time variable used to traverse the entire time signal axis, τ represents the center time, represents the position of the window function w(t-τ) on the time axis, w represents the angular frequency, which is the frequency component, and w(t-τ) is a window function that restricts the analysis to only the part of the signal x(t) close to time τ, thus giving the Fourier transform locality.

3. The EEG epilepsy pattern recognition method based on time-frequency fusion features as described in claim 1, characterized in that, The specific method for step 6 is as follows: Step 6.1: Convert the Fourier spectrum data into a power spectrum, i.e.: power=|STFT| 2 Step 6.2: Convert the power spectrum frequency to Mel frequency and determine the upper bound m. max and lower bound m min : Among them, f min Minimum frequency of power spectrum, f max Indicates the maximum frequency of the power spectrum; Step 6.3: Set up multiple Mel filters to capture different frequency distribution characteristics; apply each Mel filter to the result obtained in step 5, and weight and aggregate all frequency components to obtain S. Mel (m): Where S(k) represents the Mel filter output, H m (k) represents the amplitude of frequency k in the power spectrum, represents the total number of frequencies, and m represents the m-th Mel filter; Step 6.4: Convert the Mel spectrum into a decibel signal dB(S): Where ref is the scaling factor, and dB(S) represents the decibel signal output result; Step 6.5: Standardize the decibel signal obtained in Step 6.4: dB(S) = (dB(S) + 40) / 40.

4. The EEG epilepsy pattern recognition method based on time-frequency fusion features as described in claim 1, characterized in that, The specific method for step 7 is as follows: Step 7.1: Select six sequences: LL, LP, RP, RR, LZ, and RZ. Step 7.2: Use all six sequences. Step 7.3: Use only four of them. Step 7.2: Use a Mel filter to denoise the FFT spectra of the six sequences from Step 7.1; Step 7.3: Combine the spectrogram obtained from Mel noise reduction with the FFT spectrograms of the four sequences LL, LP, RP, and RR in top-to-bottom order; then scale the combined spectrogram to 768x768 using bilinear interpolation. Step 7.4: Use the spliced ​​data and corresponding labels obtained in Step 7.3 to train the EfficientNet-b0 model.

5. The EEG epilepsy pattern recognition method based on time-frequency fusion features as described in claim 3, characterized in that, In step 6.4, the scaling factor ref is taken as the maximum value of the Mel filter output.

Images