A new-born baby epilepsy electroencephalogram detection method based on multi-modal spatio-temporal feature fusion

Abstract

This invention discloses a method for detecting neonatal epilepsy using electroencephalography (EEG) based on multimodal spatiotemporal feature fusion, comprising: acquiring EEG signals through electrode pairs; preprocessing the acquired EEG signals to obtain preprocessed EEG signals; and inputting the preprocessed EEG signals into a trained epilepsy EEG detection model to obtain detection results. The preprocessed EEG signals include three-dimensional time-domain signals, three-dimensional frequency-domain signals, one-dimensional time-domain signals, and one-dimensional frequency-domain signals. The epilepsy EEG detection model includes: a first convolution branch for convolutional operations on the three-dimensional time-domain signals, a second convolution branch for convolutional operations on the three-dimensional frequency-domain signals, a third convolution branch for convolutional operations on the one-dimensional time-domain signals, a fourth convolution branch for convolutional operations on the one-dimensional frequency-domain signals, and a fusion decision layer. This invention extracts spatial-temporal-frequency-domain features, achieving accurate identification of neonatal epilepsy seizures under low signal-to-noise ratio conditions.

Description

Technical Field

[0001] This invention relates to the field of medical artificial intelligence technology, and in particular to a method for detecting electroencephalogram (EEG) in newborns with epilepsy based on multimodal spatiotemporal feature fusion. Background Technology

[0002] Neonatal epilepsy is a common and critical clinical condition. During a seizure, the brain's electrical signals are weak and transient, posing a significant challenge to traditional detection methods. Abnormal neuronal discharges during a seizure can lead to complications such as respiratory arrest and metabolic disorders. Without timely diagnosis and treatment, the mortality rate of neonatal epilepsy can reach as high as 30%, and it may also result in sequelae such as intellectual disability and paralysis. Currently, clinical practice mainly relies on physicians visually analyzing electroencephalograms (EEGs). However, neonatal epileptiform discharges are characterized by low amplitude (30%–50% attenuation compared to adults), short duration (0.5–2 seconds), and spatial diffusion. In the low signal-to-noise ratio NICU monitoring environment, manual interpretation carries a risk of a missed diagnosis rate as high as 40%.

[0003] In recent years, deep learning-based automatic detection methods have been gradually applied to epilepsy identification. For example, convolutional neural networks (CNNs) can effectively extract spatiotemporal features of EEG through local receptive fields. However, existing two-dimensional CNN models still have significant limitations in neonatal epilepsy detection: on the one hand, the fixed geometric structure of traditional convolutional kernels is difficult to adapt to the non-uniform distribution characteristics of EEG signals in the time-frequency-spatial domain, resulting in low efficiency in capturing transient epileptiform discharges; on the other hand, conventional feature fusion methods cannot solve the asynchronicity problem of multimodal bioelectrical signals on the time-frequency scale, and the simple concatenation of frequency domain information and time domain features results in the loss of effective information. According to tests on publicly available datasets, although three-dimensional convolutional neural networks (3D-CNNs) perform well in decoding electroencephalograms (EEGs), their accuracy in neonatal epilepsy detection tasks is only 83.2%, and their sensitivity to focal seizures is less than 75%. Summary of the Invention

[0004] Purpose of the invention: To address the limitations of existing two-dimensional CNN models in neonatal epilepsy detection, this invention proposes a method for epilepsy detection based on multimodal spatiotemporal feature fusion and attention mechanisms in EEG signals. This method can better integrate the spatial information between electrode pairs and extract spatial-temporal-frequency domain features, making it particularly suitable for accurate identification of neonatal epileptic seizures under low signal-to-noise ratio conditions, especially for the accurate identification of weak epileptiform discharges in neonatal EEG.

[0005] Technical solution: A method for detecting neonatal epilepsy based on multimodal spatiotemporal feature fusion, comprising the following steps:

[0006] Step 1: Acquire electroencephalogram (EEG) signals using electrode pairs;

[0007] Step 2: Preprocess the acquired EEG signals to obtain preprocessed EEG signals;

[0008] Step 3: Input the preprocessed EEG signals into the trained epilepsy EEG detection model to obtain the detection results;

[0009] The preprocessing of the acquired EEG signals to obtain preprocessed EEG signals specifically includes the following operations:

[0010] The acquired EEG signals were sequentially filtered, sliced, and converted into frequency domain signals to obtain three-dimensional time domain signals and three-dimensional frequency domain signals.

[0011] By performing a two-dimensional plane mapping between the three-dimensional time domain signal and the three-dimensional frequency domain signal, we obtain the mapped one-dimensional time domain signal and one-dimensional frequency domain signal.

[0012] After normalizing the three-dimensional time domain signal, the three-dimensional frequency domain signal, the one-dimensional time domain signal, and the one-dimensional frequency domain signal respectively, the preprocessed EEG signal is obtained.

[0013] The epilepsy EEG detection model includes: a first convolution branch for convolutional operation on three-dimensional time domain signals, a second convolution branch for convolutional operation on three-dimensional frequency domain signals, a third convolution branch for convolutional operation on one-dimensional time domain signals, a fourth convolution branch for convolutional operation on one-dimensional frequency domain signals, and a fusion decision layer.

[0014] The fusion decision layer is used to fuse the features output from the four convolutional branches, and perform epilepsy detection based on the fused features, outputting the epilepsy detection result.

[0015] Furthermore, the two-dimensional plane mapping of the three-dimensional time domain signal and the three-dimensional frequency domain signal specifically involves mapping the three-dimensional time domain signal and the three-dimensional frequency domain signal in a two-dimensional plane according to the relative positions of the electrode pairs on the plane.

[0016] Furthermore, the first convolution branch for performing convolution operations on three-dimensional time-domain signals includes at least two cascaded three-dimensional convolution units, which are sequentially a first three-dimensional convolution unit and a second three-dimensional convolution unit.

[0017] In both the first and second 3D convolutional units, 3D convolutional kernels are used to perform convolution operations in the spatial dimension, temporal dimension, and channel dimension.

[0018] In the second three-dimensional convolutional unit, the structure after convolution operation using three-dimensional convolutional kernels is input into the first three-dimensional convolutional attention module. The first three-dimensional convolutional attention module recalibrates the features in the spatial dimension, temporal dimension, and channel dimension to obtain the features output by the first convolutional branch.

[0019] Furthermore, the first 3D convolutional attention module includes a channel attention submodule and a spatiotemporal attention submodule;

[0020] The following calculations are performed in the channel attention submodule:

[0021] Let the elements in the feature map of the input 3D convolutional attention module be represented as: ;in, Indicates batch size, Indicates the number of channels. Indicates spatial dimension, Represents the time / frequency domain dimension;

[0022] Global average pooling is applied to the input feature map, as follows:

[0023]

[0024] In the formula, Represents the input feature map, This represents the channel dimension of the input feature map. This represents the spatial dimension of the input feature map. This represents the time dimension of the input feature map;

[0025] Global maximization of the input feature map is represented as:

[0026]

[0027] The weight coefficients are calculated using a multilayer perceptron with shared weights. , is represented as:

[0028]

[0029] In the formula, This represents the Sigmoid activation function;

[0030] The calculated weight coefficients Element-wise multiplication with each channel of the input feature map is performed to achieve channel recalibration, represented as:

[0031]

[0032] In the formula, This represents the product of each channel.

[0033] The following calculations are performed in the spatiotemporal attention submodule:

[0034] The feature map of the channel recalibration is average pooled along the channel dimension, as shown below:

[0035]

[0036] Max pooling is performed along the channel dimension on the recalibrated feature map, as follows:

[0037]

[0038] Will and Spatiotemporal feature splicing is represented as follows:

[0039]

[0040] A 3×3×3 three-dimensional convolution kernel is used to convolve the spatiotemporal feature concatenation. After convolution, a sigmoid function is applied to generate 0-1 attention weights, represented as follows:

[0041]

[0042] In the formula, This represents a 3×3×3 three-dimensional convolution kernel;

[0043] The calculated attention weights are then multiplied element-wise with each channel of the recalibrated feature map, as shown below:

[0044] .

[0045] Furthermore, the second convolution branch for performing convolution operations on three-dimensional frequency domain signals comprises at least two cascaded three-dimensional convolution units, including a third three-dimensional convolution unit and a fourth three-dimensional convolution unit in sequence.

[0046] In both the third and fourth three-dimensional convolutional units, three-dimensional convolutional kernels are used to perform convolution operations in the spatial dimension, frequency domain dimension, and channel dimension;

[0047] In the fourth three-dimensional convolutional unit, the structure after convolution operation using three-dimensional convolutional kernels is input into the second three-dimensional convolutional attention module. The second three-dimensional convolutional attention module recalibrates the features in the spatial dimension, frequency domain dimension, and channel dimension to obtain the features output by the second convolutional branch.

[0048] Furthermore, the third convolution branch for performing convolution operations on one-dimensional time-domain signals and the fourth convolution branch for performing convolution operations on one-dimensional frequency-domain signals both include: WTConv1d layer, batch normalization layer, Conv1d layer, batch normalization layer, activation layer, max pooling layer, one-dimensional Dropout layer, and one-dimensional mean layer.

[0049] Furthermore, the following steps are performed in the WTConv1d layer:

[0050] Assume the input one-dimensional time-domain / frequency-domain signal is represented as ;

[0051] Perform multi-level wavelet decomposition according to the following formula:

[0052]

[0053] In the formula, This is the Kth order low-frequency component. For the Kth level high-frequency component, Discrete wavelet transform;

[0054] An adaptive convolution kernel is generated for each high-frequency component according to the following formula:

[0055]

[0056] In the formula, For global attention modules, For kernel parameter generation functions, For adaptive convolution kernels;

[0057] Using adaptive convolution kernels for dynamic convolution processing of high-frequency components is represented as follows:

[0058]

[0059] In the formula, This represents a one-dimensional convolution operation. For learnable scaling bias parameters;

[0060] The signal is reconstructed step by step using inverse wavelet transform, and expressed as:

[0061]

[0062] In the formula, For inverse discrete wavelet transform, It is a static depthwise separable convolution kernel.

[0063] Furthermore, the specific operations of the fusion decision-making layer include:

[0064] Global average pooling is performed on the features output by all four convolutional branches.

[0065] The importance of each feature can be automatically adjusted using a learnable parameter matrix.

[0066] The epilepsy detection results are output from the fully connected layer.

[0067] Furthermore, when training the epilepsy EEG detection model, a bilinear cross-entropy loss function is used to optimize the decision boundary of the epilepsy EEG detection model.

[0068] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0069] (1) During neonatal epilepsy, the EEG signal is characterized by low amplitude (20-50μV) and short duration (0.5-2 seconds). Traditional convolutional neural networks are difficult to capture transient discharge features due to their fixed receptive fields, and conventional time-frequency analysis methods are easily affected by motion artifacts. Existing technologies have problems such as lack of three-dimensional EEG signal spatial topology modeling and asynchronous fusion of time-frequency domain features, resulting in a clinical false detection rate as high as 40%. The method of this invention innovatively constructs a two-dimensional dynamic topology mapping mechanism, projects 18 bipolar leads to anatomical positions onto a 5×5 electrode grid, and compensates for the potential of vacant positions by means of a sliding window, thus preserving the spatial diffusion characteristics of the brain region;

[0070] (2) The present invention addresses the problem of decoupling time-frequency domain features by innovatively constructing a multimodal spatiotemporal feature fusion architecture and designing a four-way parallel processing architecture: the three-dimensional time-domain path uses 3×3×3 deformable convolution to capture the spatial-temporal correlation mode; the three-dimensional frequency-domain path analyzes the spectral energy distribution through 5×5×5 convolution; the one-dimensional time-domain path combines three-level db1 wavelet decomposition and dynamic convolution to enhance transient high-frequency components; the one-dimensional frequency-domain path uses an improved Welch algorithm to extract multi-scale power spectrum features; a CBAM3D attention mechanism is added at the end of the three-dimensional time-frequency convolution path to perform weighted output processing on the output feature values; and the classification boundary is optimized through bilinear cross-entropy loss.

[0071] (3) The method of the present invention achieves dynamic alignment in the time and frequency domains through three-dimensional deformable convolution, and enhances feature discriminability by combining the improved CBAM3D attention mechanism. This can greatly improve the ability of the epilepsy detection model to extract and fuse features in the spatial-temporal-frequency domains, and make the model have higher accuracy and better robustness.

[0072] (4) While maintaining the advantage of lightweight model (only 10110 parameters), the method of the present invention has achieved a high detection accuracy of 95.8% after being verified by more than 10,000 clinical samples. It improves the epilepsy detection rate by 24.6% compared with the traditional CNN model, which is significantly better than the performance standard of the algorithm recommended by the International Epilepsy Federation.

[0073] (5) Clinical verification shows that the method of the present invention has a sensitivity of 94.7%, a false detection rate of 0.5 times / hour, a 24.6% improvement in the epilepsy detection rate compared with the traditional CNN model, a specificity of 95.4%, and an AUC of 98.4. Attached Figure Description

[0074] Figure 1 This is an overall flowchart of a method for detecting epilepsy based on multimodal spatiotemporal feature fusion and attention mechanism of EEG signals proposed in this invention;

[0075] Figure 2 This is a schematic diagram of the mapping rules for a two-dimensional plane.

[0076] Figure 3 This is a schematic diagram of the channel attention component in the CBAM3D attention mechanism.

[0077] Figure 4 This is a schematic diagram of the spatial attention component in the CBAM3D attention mechanism.

[0078] Figure 5 This is a schematic diagram of a wavelet convolution module;

[0079] Figure 6 This is a schematic diagram of the overall model. Detailed Implementation

[0080] The technical solution of this embodiment will now be further described in conjunction with the accompanying drawings and examples.

[0081] Example 1:

[0082] This embodiment discloses a method for detecting epilepsy based on multimodal spatiotemporal feature fusion and attention mechanisms of EEG signals, such as... Figure 1 As shown, it mainly includes the following steps;

[0083] Step 1: Convert the 19 unipolar lead signals into 18 bipolar differential signals to effectively eliminate common-mode interference. The specific operation includes: acquiring raw EEG signals from the 19 electrode channels; including one reference electrode channel among the 19 channels; subtracting the reference electrode channel's EEG signal from the EEG signals acquired from the other 18 electrode channels (i.e., performing a differential operation) to obtain the processed 18 bipolar differential signals, represented as follows:

[0084]

[0085] In the formula, This represents the original electrode channel signal data. This indicates that the original electrode channel signal is subtracted. Signal data of the electrode pair formed by the reference electrode potential signal. Indicates the reference electrode potential;

[0086] This operation can effectively suppress ECG artifacts and respiratory motion interference, and improve the signal-to-noise ratio of epileptiform discharge signals.

[0087] In this embodiment, a composite filtering strategy is used to eliminate noise in stages, including: using a Butterworth bandpass filter (0.5-64Hz) to filter the processed EEG signal to eliminate baseline drift and high-frequency noise, and using a dual notch filter to suppress power frequency interference. In this embodiment, a 50Hz / 60Hz comb filter is used.

[0088] Based on the relevant parameters of the EEG signal, the filtered EEG signal is sliced ​​to obtain EEG signal slices. For example, slicing can be performed using a sliding window with a sampling frequency of 128Hz and a sampling time of 3 seconds. In this embodiment, the relevant parameters of the EEG signal include, but are not limited to, sample length, slice length within the sample, and slice window shift. The length of the sliding window is the sampling time multiplied by the sampling frequency.

[0089] The EEG signal is a time-domain signal. An improved Welch method was used, in which the Hanning window length is 129 points to window the signal, the FFT number is 256 points to obtain the fine structure in the frequency domain, and the overlap rate is 50% to ensure the balance of time and frequency resolution. The power spectral density feature of the EEG signal slice was calculated to obtain the frequency domain signal of the EEG signal slice.

[0090] A triple verification mechanism is established, in which multiple experts label each slice window. For the same slice window, the slice window and its corresponding label will only be included in the training set if the labels labeled by at least three experts are completely consistent.

[0091] Step 2: Establish a 5×5 two-dimensional grid (row numbers 0-4, column numbers 0-4). Using channel mapping, map the EEG signals to the corresponding position coordinates within the 5×5 range according to the relative positions of the decoupling plane, resulting in a 5×5 electrode mapping matrix, represented as follows:

[0092]

[0093] See Figure 2 The mapping rule for a two-dimensional plane is:

[0094] Forehead area (row 0): -REF electrode: Mapped to grid coordinates (0,1); -REF electrode: Mapped to grid coordinates (0,3);

[0095] Frontal region (line 1): Central frontal region: -REF is located on the midline (1,2); left frontal region: -REF(1,1) -REF(1,0); Right frontal region: -REF(1,3) -REF(1,4);

[0096] Central area (row 2): Motor cortex area: -REF(2,1) -REF(2,3); Anterior temporal region: -REF(2,0) -REF(2,4);

[0097] Top occipital region (rows 3-4); Top region of row 3: -REF(3,1) -REF(3,2) -REF(3,3);

[0098] Posterior temporal region: -REF(3,0) -REF(3,4);

[0099] Row 4, pillow area: -REF(4,1) -REF(4,3);

[0100] Where REF is the electrode channel. .

[0101] In the 5×5 electrode mapping matrix, M represents unmapped areas, automatically identified by the difference between the traversed grid coordinates and the assigned electrode coordinates. A dynamic filling strategy is employed to dynamically fill the unmapped grid coordinates (a total of 7 vacant positions), as shown below:

[0102]

[0103] In the formula, C represents the number of channel electrode pairs, which is 18 in this case. This represents the signal data of the electrode pair channel.

[0104] This step effectively maintains the continuity of the spatial potential distribution and prevents abrupt changes in characteristics caused by zero filling.

[0105] After the above operations, we obtain the time-domain and frequency-domain data of the EEG signal after two-dimensional planar mapping, as well as the time-domain and frequency-domain data of the EEG signal before mapping. This data is used to construct a multimodal training dataset. The time-domain and frequency-domain data of the mapped EEG signal are simply referred to as: one-dimensional time-domain signal and one-dimensional frequency-domain signal; the time-domain and frequency-domain data of the EEG signal before mapping are simply referred to as: three-dimensional time-domain signal and three-dimensional frequency-domain signal. The shape of the three-dimensional time-domain signal is (batch, channel=1, length=5, width=5, time=328), where time=328 corresponds to 3 seconds of time-domain sampling points (128Hz×3s≈384 points, dimensionality reduced by PSD calculation). The shape of the three-dimensional frequency-domain signal is (batch, channel=1, length=5, width=5, freq=129), obtained through STFT transformation.

[0106] The one-dimensional time domain signal, one-dimensional frequency domain signal, three-dimensional time domain signal, and three-dimensional frequency domain signal are each normalized to obtain the data input to the model.

[0107] Step 3: As Figure 6 As shown, different convolution branches are designed for one-dimensional time-domain signals, one-dimensional frequency-domain signals, three-dimensional time-domain signals, and three-dimensional frequency-domain signals.

[0108] The convolutional branch for 3D temporal signals consists of an input layer that receives a 5D temporal tensor of shape (batch, 1, 5, 5, 328). This convolutional branch contains at least two cascaded 3D convolutional units (Convd3D and Convd3D with CBAM) to achieve hierarchical abstraction of features.

[0109] Each 3D convolutional unit performs the following operations: It uses a 3D convolutional kernel to perform convolution operations in the spatial dimension (N×N), temporal dimension (T), and channel dimension (C). Specifically, it performs the following operations: using a 1×1×3 convolutional kernel to effectively capture local time-frequency domain patterns; using a 3×3×1 convolutional kernel to effectively capture local spatial patterns; and using a 3×3×3 convolutional kernel to effectively capture time-frequency-spatial features. These three convolutional operations with different kernels are essentially cascaded, with the output of the previous unit serving as the input of the next. Figure 6 Both Convd3D and CNN with CBAM 3D Block are cascaded operations like this, but only CNN with CBAM 3D Block adds the CBAM attention module at the very end.

[0110] The 3D Convolutional Attention Module (CBAM3D) extends the traditional channel-space attention mechanism in multiple dimensions, forming an attention mechanism with spatiotemporal joint perception capabilities. Therefore, this 3D Convolutional Attention Module (CBAM3D) is used to achieve channel-space-temporal 3D feature recalibration. Its workflow includes two core stages: the channel attention stage and the spatiotemporal attention stage. The channel attention stage is used to achieve effective global feature compression across spatiotemporal dimensions. The spatiotemporal attention stage is used to implement 5×5 electrode spatial weight mapping.

[0111] Now combined Figure 3 Further explanation of the channel attention phase.

[0112] The main objectives of the channel attention phase are: to learn the importance of channels, suppress noise-related channels, and enhance epileptiform channels. Specific steps include:

[0113] Suppose the elements of the input feature map are: ;in, Indicates batch size, Indicates the number of channels. Indicates spatial dimension (electrode layout). Represents the time / frequency domain dimension.

[0114] The first step is to perform dual-path compression on the input feature map:

[0115] The first compression path performs global average pooling: compression along the spatial (H,W) and temporal (T) dimensions, represented as:

[0116]

[0117] The second compression path performs a global maximization to capture significant activations, as shown below:

[0118]

[0119]

[0120] The second step involves learning the channel relationships using a multilayer perceptron (MLP) with shared weights. This means the dual-path compression result is input into an MLP containing two fully connected layers (dimensionality reduction ratio r=4). The MLP structure is: FC(C→C / r) → ReLU → FC(C / r→C), where FC(C→C / r) represents a fully connected layer with input data of shape (B, C), where C is the original number of input channels, and the fully connected layer reduces the dimensionality to C / r channels. FC(C / r→C) represents another fully connected layer with input data of shape (Batch, C / r), where C / r is the number of input channels after dimensionality reduction, and here the dimensionality is increased back to C channels.

[0121] The output of the MLP is represented as:

[0122]

[0123] In the formula, This represents the Sigmoid activation function. This represents the weight coefficient of each channel obtained after calculation. .

[0124] The third step involves element-wise multiplication of the calculated weight coefficients with each channel of the input feature map to achieve channel recalibration, as shown below:

[0125]

[0126] In the formula, This represents the product of each channel.

[0127] Calculation steps:

[0128]

[0129]

[0130] in, , For MLP weights, This represents the Sigmoid activation function.

[0131] like Figure 4 As shown, the spatiotemporal attention stage will now be further explained.

[0132] The goal of this spatiotemporal attention phase is to focus on key spatial regions (such as the epileptic focus) and key temporal segments (such as the seizure onset). This includes the following steps:

[0133] The first step is to aggregate features across channel dimensions, including:

[0134] Average pooling along the channel dimension is represented as:

[0135]

[0136] Max pooling along the channel dimension is represented as:

[0137]

[0138] .

[0139] Spatiotemporal feature splicing is represented as:

[0140]

[0141] .

[0142] The second step involves using 3D convolution to capture local spatiotemporal patterns, including:

[0143] Using a 3×3×3 convolution kernel (3x3 spatially, 3 temporally), concatenating dual-path features, and then applying a sigmoid function to generate 0-1 attention weights, represented as follows:

[0144]

[0145] In the formula, This represents a 3D convolution kernel (3×3×3). .

[0146] The third step involves element-wise multiplication of the calculated attention weights with each channel of the input feature map to achieve spatiotemporal feature enhancement, represented as:

[0147]

[0148] In this embodiment, a 3×3×3 convolution kernel (3x3 in space, 3x3 in time) is used. The 3x3 space can cover most of the electrode-dense area of ​​the newborn's head (5×5 mapping), and can better capture the spatial features between electrodes. The 3x3 time can capture the short-term diffusion pattern of epileptic waves (0.5-2 seconds).

[0149] The computational representation of the global 3D convolutional attention module (CBAM3D) is as follows:

[0150]

[0151] Among them, the convolution branch for three-dimensional frequency domain signals is as follows: a series of three-dimensional convolution modules are used to process the 5×5×F frequency domain feature matrix and extract the spatial-spectral joint features; this convolution branch is the same type as the convolution branch for three-dimensional time domain signals, and each three-dimensional convolution module includes: a three-dimensional convolution kernel and a three-dimensional convolution attention module (CBAM3D).

[0152] It receives three-dimensional frequency domain signals and performs convolution operations in the spatial dimension (N×N), frequency domain dimension (F), and channel dimension (C) using a three-dimensional convolution kernel.

[0153] In the 3D Convolutional Attention Module (CBAM3D), channel attention computes global statistics across the spatial and frequency domains; spatial-frequency attention captures the correlation between the spatial distribution of electrodes and the frequency band energy through 3D convolution.

[0154] The convolutional branches for one-dimensional time-domain signals and one-dimensional frequency-domain signals both include: WTConv1d layer, batch normalization layer, Conv1d layer, batch normalization layer, activation layer, max pooling layer, one-dimensional Dropout layer, and one-dimensional mean layer.

[0155] Figure 5 The structure of the WTConv1d layer is shown, including the basic path ( Figure 5 In this embodiment, only the basic path (the topmost path), WTleval-1 path, and WTleval-2 path are convolved and residuals are connected. The WTleval-1 path represents a wavelet decomposition of the input signal with a layer of 1. First, the input signal is decomposed into low-frequency and high-frequency components. Then, convolution operations are performed on these two parts respectively, followed by inverse wavelet decomposition, and finally, the components are merged into a single signal of the same length as the original signal. The residual values ​​of this component are then connected to the basic path. The WTleval-2 path represents a further wavelet decomposition of the low-frequency and high-frequency components obtained from WTleval-1, decomposing them into low-frequency, second-low-frequency, high-frequency, and second-high-frequency components. Convolution operations are then performed, followed by inverse decomposition, resulting in processed low-frequency and high-frequency components. These processed components are then residually connected to the low-frequency and high-frequency components of WTleval-1.

[0156] Specifically, the WTConv1d layer in the convolution branch for one-dimensional time-domain signals includes: a multi-level wavelet decomposition layer, a high-frequency component nonlinear processing layer, and a multi-level deconvolution reconstruction layer. The multi-level wavelet decomposition layer separates low-frequency and high-frequency components step-by-step to reveal the multi-scale characteristics of the signal. The high-frequency component nonlinear processing layer uses 1D convolution to extract multi-resolution time-domain features, enhancing the signal's feature recognition capability. The multi-level deconvolution reconstruction layer restores the signal resolution through transposed convolution, achieving accurate signal reconstruction. This processing flow embodies the multi-resolution analysis concept: Wavelet decomposition: A three-level decomposition using the db1 wavelet basis effectively separates the low-frequency baseline signal and high-frequency transient components. High-frequency processing: Deformable 1D convolution kernels are used to extract sample discharge features. Cross-scale reconstruction: Signal resolution is restored through transposed convolution, preserving multi-level feature mapping.

[0157] The WTConv1d layer in the convolutional branch for one-dimensional frequency domain signals includes: a multi-scale frequency band decomposition layer, a frequency domain feature enhancement layer, and a cross-band fusion reconstruction layer. The multi-scale frequency band decomposition layer separates different frequency band components step-by-step to analyze the frequency domain characteristics of the signal. The frequency domain feature enhancement layer uses 1D convolution to analyze multi-scale frequency band features, improving feature recognizability. The cross-band fusion reconstruction layer reconstructs the frequency domain signal through transposed convolution, ensuring signal integrity.

[0158] The operations involved in the WTConv1d layer are now explained. These include:

[0159] Suppose the input one-dimensional time-domain / frequency-domain signal is represented as: .

[0160] Perform multi-level wavelet decomposition according to the following formula:

[0161]

[0162] In the formula, This is the Kth order low-frequency component. For the Kth level high-frequency component, This is a discrete wavelet transform.

[0163] An adaptive convolution kernel is generated for each high-frequency component according to the following formula:

[0164]

[0165] In the formula, For global attention modules, This is a kernel parameter generation function (MLP implementation). For dynamically generated convolution kernels ( (Nucleus size).

[0166] High-frequency feature enhancement is achieved by using adaptive convolution kernels for dynamic convolution processing of high-frequency components, as shown below:

[0167]

[0168] In the formula, This represents a one-dimensional convolution operation. This is a learnable scaling bias parameter.

[0169] The signal is reconstructed step by step using inverse wavelet transform, and expressed as:

[0170]

[0171] In the formula, For inverse discrete wavelet transform, It is a static depthwise separable convolution kernel.

[0172] Step 4: Through a collaborative fusion mechanism, the features of the four sets of signals are fused to achieve information complementarity. Specifically, the features of the four sets of signals are globally averaged and flattened before being input into the fully connected layer. Global average pooling is performed on each output to generate a 32-dimensional feature vector. A learnable parameter matrix is ​​used to automatically adjust the importance of each feature, achieving self-adaptation. In the fully connected stage, a bilinear cross-entropy loss function is used to optimize the model's decision boundary. Finally, the fused features are used for epilepsy detection, outputting the classification result.

[0173] This embodiment establishes a joint spatial-temporal-frequency domain representation system and utilizes a four-way heterogeneous feature extraction network to achieve multi-scale epilepsy feature fusion. The entire processing flow strictly follows the technical route of "spatial projection - feature decoupling - collaborative decision-making," and the modules interact with each other through feature tensor pipelines to form a closed-loop processing link.

[0174] After rigorous validation with 10,000 clinical cases, the method of this invention has demonstrated the following outstanding performance indicators: accuracy of 95.8%, sensitivity of 94.7%, specificity of 95.4%, and AUC of 98.4. The method of this invention has been validated through clinical trials and has significant progressiveness and industrial application value.

Claims

1. A method for detecting neonatal epilepsy based on multimodal spatiotemporal feature fusion, characterized in that: Includes the following steps: Step 1: Acquire electroencephalogram (EEG) signals using electrode pairs; Step 2: Preprocess the acquired EEG signals to obtain preprocessed EEG signals; Step 3: Input the preprocessed EEG signals into the trained epilepsy EEG detection model to obtain the detection results; The preprocessing of the acquired EEG signals to obtain preprocessed EEG signals specifically includes the following operations: The acquired EEG signals were sequentially filtered, sliced, and converted into frequency domain signals to obtain one-dimensional time domain signals and one-dimensional frequency domain signals. By performing a two-dimensional plane mapping between a one-dimensional time-domain signal and a one-dimensional frequency-domain signal, we obtain a mapped three-dimensional time-domain signal and a three-dimensional frequency-domain signal. After normalizing the three-dimensional time domain signal, the three-dimensional frequency domain signal, the one-dimensional time domain signal, and the one-dimensional frequency domain signal respectively, the preprocessed EEG signal is obtained. The epilepsy EEG detection model includes: a first convolution branch for convolutional operation on three-dimensional time domain signals, a second convolution branch for convolutional operation on three-dimensional frequency domain signals, a third convolution branch for convolutional operation on one-dimensional time domain signals, a fourth convolution branch for convolutional operation on one-dimensional frequency domain signals, and a fusion decision layer. The fusion decision layer is used to fuse the features output from the four convolutional branches, and perform epilepsy detection based on the fused features, and output the epilepsy detection results. The first convolution branch for performing convolution operations on three-dimensional time-domain signals includes at least two cascaded three-dimensional convolution units, namely a first three-dimensional convolution unit and a second three-dimensional convolution unit. In both the first and second 3D convolutional units, 3D convolutional kernels are used to perform convolution operations in the spatial dimension, temporal dimension, and channel dimension. In the second three-dimensional convolutional unit, the result of the convolution operation using the three-dimensional convolutional kernel is input into the first three-dimensional convolutional attention module. The first three-dimensional convolutional attention module recalibrates the features of the spatial dimension, temporal dimension and channel dimension to obtain the features output by the first convolutional branch. The first 3D convolutional attention module includes a channel attention submodule and a spatiotemporal attention submodule; The following calculations are performed in the channel attention submodule: Let the elements in the feature map of the input 3D convolutional attention module be represented as: ;in, Indicates batch size, Indicates the number of channels. Indicates spatial dimension, Represents the time / frequency domain dimension; Global average pooling is applied to the input feature map, as follows: ； In the formula, Represents the input feature map, This represents the channel dimension of the input feature map. This represents the spatial dimension of the input feature map. This represents the time dimension of the input feature map; Global maximization of the input feature map is represented as: ； The weight coefficients are calculated using a multilayer perceptron with shared weights. , is represented as: ； In the formula, This represents the Sigmoid activation function; The calculated weight coefficients Element-wise multiplication with each channel of the input feature map is performed to achieve channel recalibration, represented as: ； In the formula, This represents the product of each channel. The following calculations are performed in the spatiotemporal attention submodule: The feature map of the channel recalibration is average pooled along the channel dimension, as shown below: ； Max pooling is performed along the channel dimension on the recalibrated feature map, as follows: ； Will and Spatiotemporal feature splicing is represented as follows: ； A 3×3×3 three-dimensional convolution kernel is used to convolve the spatiotemporal feature concatenation. After convolution, a sigmoid function is applied to generate 0-1 attention weights, represented as follows: ； In the formula, This represents a 3×3×3 three-dimensional convolution kernel; The calculated attention weights are then multiplied element-wise with each channel of the recalibrated feature map, as shown below: ； The third convolution branch for performing convolution operations on one-dimensional time-domain signals and the fourth convolution branch for performing convolution operations on one-dimensional frequency-domain signals both include: WTConv1d layer, batch normalization layer, Conv1d layer, batch normalization layer, activation layer, max pooling layer, one-dimensional Dropout layer, and one-dimensional mean layer.

2. The method for detecting neonatal epilepsy based on multimodal spatiotemporal feature fusion according to claim 1, characterized in that: The aforementioned two-dimensional plane mapping of one-dimensional time-domain signals and one-dimensional frequency-domain signals specifically involves mapping the one-dimensional time-domain signals and one-dimensional frequency-domain signals in a two-dimensional plane according to the relative positions of the electrode pairs on the plane.

3. The method for detecting neonatal epilepsy based on multimodal spatiotemporal feature fusion according to claim 1, characterized in that: The second convolution branch for performing convolution operations on three-dimensional frequency domain signals includes at least two cascaded three-dimensional convolution units, namely a third three-dimensional convolution unit and a fourth three-dimensional convolution unit. In both the third and fourth three-dimensional convolutional units, three-dimensional convolutional kernels are used to perform convolution operations in the spatial dimension, frequency domain dimension, and channel dimension; In the fourth three-dimensional convolutional unit, the structure after convolution operation using three-dimensional convolutional kernels is input into the second three-dimensional convolutional attention module. The second three-dimensional convolutional attention module recalibrates the features in the spatial dimension, frequency domain dimension, and channel dimension to obtain the features output by the second convolutional branch.

4. The method for detecting neonatal epilepsy based on multimodal spatiotemporal feature fusion according to claim 1, characterized in that: Perform the following steps in the WTConv1d layer: Assume the input one-dimensional time-domain / frequency-domain signal is represented as ; Perform multi-level wavelet decomposition according to the following formula: ； In the formula, For the k-th order low-frequency component, For the k-th high-frequency component, Discrete wavelet transform; An adaptive convolution kernel is generated for each high-frequency component according to the following formula: ； In the formula, For global attention modules, For kernel parameter generation functions, For adaptive convolution kernels; Using adaptive convolution kernels for dynamic convolution processing of high-frequency components is represented as follows: ； In the formula, This represents a one-dimensional convolution operation. For learnable scaling bias parameters; The signal is reconstructed step by step using inverse wavelet transform, and expressed as: ； In the formula, For inverse discrete wavelet transform, It is a static depthwise separable convolution kernel.

5. The method for detecting neonatal epilepsy based on multimodal spatiotemporal feature fusion according to claim 1, characterized in that: The fusion decision layer, in its specific operations, includes: Global average pooling is performed on the features output by all four convolutional branches. The importance of each feature can be automatically adjusted using a learnable parameter matrix. The epilepsy detection results are output from the fully connected layer.

6. The method for detecting neonatal epilepsy based on multimodal spatiotemporal feature fusion according to claim 1, characterized in that: When training the epilepsy EEG detection model, the bilinear cross-entropy loss function is used to optimize the decision boundary of the epilepsy EEG detection model.

Images