A method and system for recognizing spatio-temporal features of electroencephalogram signals

Abstract

This invention discloses a method and system for spatiotemporal feature recognition of electroencephalogram (EEG) signals, relating to the field of EEG signal processing technology. The method includes: inputting epileptic EEG signals into a feature embedding module to obtain low-dimensional features; inputting the low-dimensional features into an improved waveform network to extract multi-scale local spatial features; filtering the multi-scale local spatial features through a linear channel embedding attention module to obtain spatial features; inputting epileptic EEG signals into a temporal feature embedding module to obtain dimensional temporal features; inputting the low-dimensional temporal features into an improved convolutional long short-term memory network to capture global features and contextual semantic details; inputting the captured results into a linear position embedding attention module to obtain temporal features; and inputting the spatial and temporal features into a spatiotemporal feature selection and aggregation module to obtain the recognition result of the epileptic EEG signal. This invention can explore the correlation between spatiotemporal features and reduce their differences.

Description

Technical Field

[0001] This invention relates to the field of electroencephalogram (EEG) signal processing technology, and in particular to a method and system for recognizing the spatiotemporal features of EEG signals. Background Technology

[0002] Electroencephalogram (EEG) signals are complex signals that require advanced signal processing and feature extraction methods for accurate identification and interpretation.

[0003] Considering that single-structure deep learning methods cannot fully represent epileptic EEG signals, existing methods combining spatiotemporal features to identify epileptic EEG signals, while compensating for the shortcomings of single-structure networks, often employ simple splicing to aggregate spatiotemporal features during feature fusion, failing to address the differences between various features. Furthermore, redundant information is repeatedly used during inter-layer transmission, reducing the representational power of salient features. Therefore, there is still significant room for improvement in the performance of epileptic EEG signal recognition. In conclusion, how to fully identify the spatiotemporal features of epileptic EEG signals is a crucial problem that urgently needs to be solved. Summary of the Invention

[0004] This invention provides a method and system for recognizing the spatiotemporal features of electroencephalogram (EEG) signals, which can fully identify the spatiotemporal features of epileptic EEG signals.

[0005] This invention provides a method for spatiotemporal feature recognition of electroencephalogram (EEG) signals, comprising the following steps:

[0006] Acquire electroencephalogram (EEG) signals that reflect brain activity in epilepsy;

[0007] The EEG signal is input into a feature embedding module to obtain low-dimensional features of the EEG signal; the low-dimensional features are input into an improved waveform network to extract multi-scale local spatial features of the EEG signal; and the multi-scale local spatial features are filtered through a linear channel embedding attention module to obtain spatial features of the EEG signal.

[0008] The EEG signal is input into a temporal feature embedding module to obtain low-dimensional temporal features of the EEG signal; the low-dimensional temporal features are input into an improved convolutional long short-term memory network to capture global features and contextual semantic details of the EEG signal; the captured results are input into a linear position embedding attention module to obtain temporal features of the EEG signal.

[0009] Spatial and temporal features are used as the spatiotemporal features of EEG signals.

[0010] Furthermore, the step of extracting multi-scale local spatial features of EEG signals by inputting low-dimensional features into an improved waveform network includes:

[0011] Low-dimensional features are input into causal convolutional layers to extract low-level spatial features. These low-level spatial features are then input into improved dilated causal convolutional units (DCUs). Each DCU consists of three dilated causal convolutional layers, two standard convolutional layers, and a gated activation unit. The results from multiple improved DCUs are summed using residual and skip connections, and long-term dependencies are established between the results using a gated activation unit to obtain fused high-level causal convolutional features. These fused high-level causal convolutional features are then input into continuous standard convolutional layers of different scales to obtain multi-scale local spatial features of the EEG signal.

[0012] Furthermore, the step of filtering multi-scale local spatial features through linear channel embedding attention modules to obtain spatial features of EEG signals includes the following specific steps:

[0013] Multi-scale local spatial features Input linear channels embed attention modules to obtain spatial features of EEG signals. The formula is:

[0014] ;

[0015] in, This represents the feature mapping operation. This represents the Hadamard product;

[0016] Linear channel attention coefficients in low dimension The formula is:

[0017] ;

[0018] in, This represents the activation function of sigmoid. Indicates a linear layer. This represents the activation function for ReLU.

[0019] Furthermore, the specific steps of inputting low-dimensional temporal features into an improved convolutional long short-term memory network to capture global features and contextual semantic details of EEG signals include:

[0020] Low-dimensional temporal features Input the improved convolutional long short-term memory network to obtain the first... t Output features of a forward-biased dilated convolutional long short-term memory network;

[0021] ;

[0022] Obtain the t Output features of dilated convolutional long short-term memory networks that are reversed at every moment;

[0023] ;

[0024] Where * denotes an expanded convolution operation; This indicates the Hadamard product operation. This represents the activation function of ReLU. This represents the activation function for tanh; They represent t Input gates in both forward and reverse directions at all times. They represent t Memory units that are always in both forward and reverse directions. They represent t The hidden state in both positive and negative directions at all times. They represent t Two opposing output gates at all times. They represent t The forgetting gate, which always faces both forward and backward. and These represent the weight matrices related to the input features, the previous hidden state, and the input gate of the previous memory unit, acting in opposite directions. These represent the weight matrices related to the forget gates of the input features, the previous hidden state, and the previous memory unit, respectively, acting in the positive and negative directions. These represent the weight matrices related to the input features, the previous hidden state, and the memory of the previous memory unit, acting in opposite directions. These represent the weight matrices that act in opposite directions on the input features, the previous hidden state, and the output gate of the previous memory unit, respectively. and They represent t The hidden states in both directions at time -1. and They represent t Memory units in both directions at time -1. This indicates the input gate bias. Indicates the forget gate bias. Indicates memory cell bias. Indicates output gate bias;

[0025] Based on the output features of the forward-dilated convolutional long short-term memory network and the backward-dilated convolutional long short-term memory network, the improved output features of the convolutional long short-term memory unit are obtained. This is used to capture global features and contextual semantic details of EEG signals.

[0026] .

[0027] Furthermore, the step of embedding the captured results into the linear position of the attention module to obtain the temporal features of the EEG signal includes the following specific steps:

[0028] Capture results By embedding an attention module into the linear input position, the temporal features of the EEG signal are obtained. The formula is:

[0029] ;

[0030] in, This represents the multiplication of characteristic matrices; Indicates along the time sequence The SoftMax operation of the normalized exponential function; This represents the average pooling operation, which in temporal EEG signals occurs along the channel. The mean-averaging operation; This indicates a linear operation.

[0031] Furthermore, the signal feature recognition method further includes:

[0032] Temporal characteristics of EEG signals Spatial characteristics of EEG signals Align and stitch them together to obtain fusion features with complementary information. The formula is:

[0033] ;

[0034] in, This indicates feature splicing along the channel direction;

[0035] One-dimensional decomposition of dilated convolutional layers with two different selection kernels is used to fuse features. To perform a decomposition, the formula is:

[0036] ;

[0037] in, and These represent one-dimensional decomposition dilation convolution operations with different kernel choices; the size of the one-dimensional decomposition dilation convolution kernel is... and ;

[0038] The decomposed and fused features are concatenated, and attention feature maps are obtained by one-dimensional max pooling and average pooling, respectively.

[0039] The decomposed fusion features are masked and weighted, and then mapped to the attention feature map through a standard convolutional layer to obtain the mapped features.

[0040] The Hadamard product is used to reconstruct the mapped features and the decomposed fused features to obtain the filtered aggregated features;

[0041] Combine the selected aggregated features with the fused features Then, the parts are assembled.

[0042] This invention provides a spatiotemporal feature recognition system for electroencephalogram (EEG) signals, comprising:

[0043] The signal acquisition module is used to acquire electroencephalogram (EEG) signals that reflect brain activity in epilepsy.

[0044] The spatial feature acquisition module is used to input EEG signals into the feature embedding module to obtain low-dimensional features of the EEG signals; input the low-dimensional features into an improved waveform network to extract multi-scale local spatial features of the EEG signals; and filter the multi-scale local spatial features through a linear channel embedding attention module to obtain the spatial features of the EEG signals.

[0045] The temporal feature acquisition module is used to input EEG signals into the temporal feature embedding module to obtain low-dimensional temporal features of the EEG signals; input the low-dimensional temporal features into an improved convolutional long short-term memory network to capture global features and contextual semantic details of the EEG signals; input the captured results into the linear position embedding attention module to obtain temporal features of the EEG signals.

[0046] The recognition module is used to identify spatial and temporal features as spatiotemporal features of EEG signals.

[0047] This invention provides a method and system for spatiotemporal feature recognition of electroencephalogram (EEG) signals, which has the following advantages compared with the prior art:

[0048] The EEG signal is input into a feature embedding module to obtain low-dimensional features of the EEG signal. These low-dimensional features are then input into an improved waveform network to extract multi-scale local spatial features of the EEG signal. A linear channel embedding attention module is used to filter these multi-scale local spatial features to obtain spatial features of the EEG signal. The EEG signal is then input into a temporal feature embedding module to obtain low-dimensional temporal features of the EEG signal. These low-dimensional temporal features are then input into an improved convolutional long short-term memory network to capture global features and contextual semantic details of the EEG signal. The captured results are then input into a linear position embedding attention module to obtain temporal features of the EEG signal. These spatial and temporal features are used as the spatiotemporal features of the EEG signal, ultimately fully identifying the spatiotemporal features of the EEG signal. Attached Figure Description

[0049] Figure 1The diagrams show a standard dilated causal convolutional unit and an improved dilated causal convolutional unit provided for embodiments of the present invention, wherein (a) is a standard dilated causal convolutional unit and (b) is an improved dilated causal convolutional unit.

[0050] Figure 2 This is a structural diagram of a linear channel embedded attention module provided in an embodiment of the present invention;

[0051] Figure 3 The overall structure diagram of the improved convolutional long short-term memory network provided in the embodiments of the present invention;

[0052] Figure 4 The following is a diagram of the overall network structure provided in the embodiments of the present invention, wherein (a) is the overall network structure of the epilepsy EEG signal recognition framework with spatiotemporal feature selection and aggregation, (b) is the improved convolutional long short-term memory network in the temporal feature extraction module, and (c) is the multi-scale dilated causal convolutional unit of the improved WaveNet in the spatial feature extraction branch SFEB.

[0053] Figure 5 The operation process and training procedure of the epilepsy EEG signal recognition framework provided in the embodiments of the present invention;

[0054] Figure 6 The identification flowchart is provided for an embodiment of the present invention. Detailed Implementation

[0055] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0056] This invention develops a spatiotemporal feature selection aggregation network (STFSANets) to explore the correlation between spatiotemporal features and reduce the differences between them.

[0057] First, an improved WaveNet method is used to extract multi-scale local spatial features of epileptic EEG signals to compensate for the inadequacy of single-structure spatial feature representation. Then, a newly designed linear channel embedding attention module is used to refine the multi-scale features along the channels and establish dependencies between channels to reduce the use of redundant information and highlight the representation of salient features.

[0058] Secondly, an improved convolutional long short-term memory network is used to model epileptic EEG signals from both positive and negative directions, obtaining effective global features and contextual semantics. The use of an expanded convolutional network in this module alleviates the limitation of standard convolutional networks in effectively modeling local details due to insufficient receptive fields. Furthermore, a newly designed linear position embedding attention module is used to perform positional encoding on temporal epileptic EEG signals and refine the temporal features to reduce the transmission and use of redundant information between layers.

[0059] Finally, to mitigate the differences between various features and establish complementary relationships among them, a newly designed spatiotemporal feature selection and aggregation module is employed to fuse temporal and spatial features hierarchically. At each level of feature fusion, filtering and re-aggregation are used to reduce differences while improving the representation of salient features, thereby enhancing the recognition performance of epileptic EEG signals. Furthermore, the proposed framework is evaluated and validated on baseline datasets such as the Temple University Hospital Epilepsy EEG Dataset TUSZ (TUH EEG SeiZure) and the Children's Hospital Boston-MIT Joint Dataset CHBMIT (Children's Hospital Boston-MIT), demonstrating its effectiveness and robustness.

[0060] See Figure 1 This invention provides a method for spatiotemporal feature recognition of electroencephalogram (EEG) signals, comprising the following steps:

[0061] Step 1: Obtain electroencephalogram (EEG) signals that reflect brain activity in epileptic patients.

[0062] Step 2: Input the EEG signal into the feature embedding module to obtain low-dimensional features of the EEG signal; input the low-dimensional features into the improved waveform network to extract multi-scale local spatial features of the EEG signal; filter the multi-scale local spatial features through linear channel embedding into the attention module to obtain the spatial features of the EEG signal.

[0063] Step 3: Input the EEG signal into the temporal feature embedding module to obtain low-dimensional temporal features of the EEG signal; input the low-dimensional temporal features into an improved convolutional long short-term memory network to capture global features and contextual semantic details of the EEG signal; input the captured results into the linear position embedding attention module to obtain temporal features of the EEG signal.

[0064] Step 4: Use spatial and temporal features as the spatiotemporal features of the EEG signal, input the spatiotemporal features into the spatiotemporal feature selection and aggregation module, and obtain the recognition results of the EEG signal.

[0065] The specific details of this invention are as follows:

[0066] 1. Spatial Feature Extraction Branch (SFEB):

[0067] The Spatial Feature Extraction (SFEB) branch mainly consists of three parts: a feature embedding module, an improved channel attention module, and a WaveNet waveform network. The channel attention module aims to filter local spatial features, reducing redundant noise while improving the representation of salient features in epileptic EEG signals. WaveNet primarily utilizes expanded causal convolutional components to model the multi-scale spatial information of epileptic EEG signals and acquires the global semantics of epileptic EEG signals through a larger receptive field. Simultaneously, simple causal convolutional layers and gated activation functions effectively address the long-range dependencies required in the spatial feature extraction stage. Since the input epileptic EEG signals have high-dimensional features and cannot be directly used for spatial feature extraction, this invention employs a feature embedding module to map the input epileptic EEG signals to a unified low-dimensional space to learn more effective local spatial features.

[0068] 1.1 Improved WaveNet:

[0069] The traditional WaveNet is a deep neural network model developed by DeepMind, primarily composed of causal convolutional layers and dilated causal convolutional units. It aims to directly process raw audio waveform signals to generate corresponding text data, performing particularly well in speech synthesis, music generation, and sound effects synthesis. However, compared to raw audio waveform signals, epileptic EEG signals exhibit channel correlations and contain significant amounts of high-dimensional raw noise, making it impossible to directly model spatial details using a simple WaveNet. Standard dilated causal convolutional units and improved dilated causal convolutional units, such as... Figure 1 (a) and Figure 1 As shown in (b). In the figure, This represents a standard 3×3 convolution operation. This represents a standard 1×1 convolution operation. This represents a 1×1 dilated causal convolution operation. This represents a 3×3 dilated causal convolution operation. σ1 represents the STReLU activation function. σ2 represents the Swish activation function with non-linear properties. σ represents the Sigmoid activation function. tanh represents the Tanh activation function.

[0070] The specific improvement steps are as follows:

[0071] Step 1. Assume the input epileptic EEG signal is ,in These represent the channel dimension and sequence dimension, respectively. The low-dimensional features output by the feature embedding module. As shown in the equation.

[0072] (1).

[0073] in, This indicates the feature embedding module.

[0074] Step 2. Considering that prior knowledge contains rich low-level semantic information, which can effectively represent the basic properties of epileptic EEG signals, the acquired low-dimensional embedding features are then used. Input causal convolutional layers to ensure the model does not disrupt the order of epileptic EEG signal modeling, while simultaneously realizing low-level spatial features. The extraction is shown in the equation.

[0075] (2).

[0076] in, This indicates a causal convolution operation.

[0077] Step 3. To further obtain effective multi-scale spatial local features, low-level spatial features are input into dilated causal convolutional units to learn more effective high-level spatial details. To avoid overfitting and increased model complexity due to excessively deep network layers, multiple dilated causal convolutional units are combined using residuals and skip connections, and gated activation units are used to establish long-term dependencies between these features. High-level spatial features The calculation is shown in the equation.

[0078] (3).

[0079] in, This represents an expanded causal convolution operation. The causal convolutional units representing this expansion have layer.

[0080] Each improved dilated causal convolutional unit primarily consists of three dilated causal convolutional layers, two standard convolutional layers, and one gated activation unit. The output features of the first unit... and As shown in the equation.

[0081] (4).

[0082] in, This represents a standard 1×1 convolution operation. This represents a standard 3×3 convolution operation. This represents feature concatenation. In the first unit, a gated activation strategy is used to concatenate the aggregated dilated causal convolutional features. Filtering and learning can effectively strengthen the long-range dependencies between spatial features of epileptic EEG signals. Gated activation strategy. The calculation is shown in the equation.

[0083] (5).

[0084] in This represents the activation function of STReLU. represents the Swish activation function with nonlinear properties. ∗ represents a simple convolution operator. This represents the Hadamard product. Representation layer index. These represent filters and gate devices, respectively. This represents a learnable convolutional filter. The first aggregated dilated causal convolutional feature... As shown in the equation.

[0085] (6).

[0086] in, This represents a 1×1 dilated causal convolution operation. This represents a 3×3 dilated causal convolution operation.

[0087] Step 4. To obtain effective spatially salient features and reduce computational complexity, the obtained high-level causal convolutional features are fused. By inputting a set of consecutive standard convolutional layers at different scales, feature compression and activation are achieved, and spatial details are obtained. The specific calculations are shown in the equation.

[0088] (7).

[0089] in, This represents the ReLU activation function. This represents a set of consecutive 3×3 standard convolutions.

[0090] In summary, using residuals and skip connections in expanded causal convolutional units enables feature reuse while accelerating the convergence speed of the model during the training phase.

[0091] 1.2 Linear Channel Embedded Attention Module (LCEAM):

[0092] In recent years, channel attention mechanisms have played a positive role in improving the performance of deep convolutional neural networks. However, most existing methods focus on embedding more complex attention modules into neural networks to obtain better feature representations and performance. This approach not only increases the computational complexity of the model but also fails to effectively filter redundant information distributed in low-dimensional space. Therefore, to address these issues, this invention develops a linear channel embedding attention module. This module has few parameters and, without adding extra features, effectively alleviates the reuse of redundant information in low-dimensional space, thus avoiding problems such as insufficient representation of salient features. Furthermore, considering the correlation between spatial local feature channels in epileptic EEG signals, the linear channel embedding attention module is integrated with an improved WaveNet to further improve the representation of salient information in spatial features. The structure of the linear channel embedding attention module is as follows: Figure 2 As shown in the figure, Indicates a linear operation. This represents the activation function of the Sigmoid function. This represents the ReLU activation function.

[0093] Output features of linear channel embedded attention module The calculation is shown in the equation.

[0094] (8).

[0095] in, This indicates a feature mapping operation. Represents the Hadamard product. Linear channel attention coefficients in low dimension. The calculation is shown in the equation.

[0096] (9).

[0097] in, This represents the activation function for ReLU. This indicates a linear layer.

[0098] In summary, combining linear channel-embedded attention modules with the improved WaveNet not only effectively reduces redundant information reuse and improves the representation of salient features without adding extra parameters and information, but also further obtains more effective multi-scale spatial local details from epileptic EEG signals.

[0099] 2. Temporal Feature Extraction Branch (TFEB):

[0100] The Temporal Feature Extraction (TFEB) branch mainly consists of three parts: a temporal feature embedding module, an improved Convolutional Long Short-Term Memory (ConvLSTM) network, and an improved positional attention module. The improved positional attention module aims to capture the positional deviations of epileptic EEG signals within the temporal features and refine these features. The temporal feature embedding module is structurally similar to the feature embedding module in the spatial feature extraction branch, primarily performing a low-dimensional unified mapping of the temporal epileptic EEG signals to enable the network to learn a better temporal representation. The improved Convolutional Long Short-Term Memory (ConvLSTM) network aims to model the global and contextual semantics of epileptic EEG signals from both positive and negative perspectives, and extracts effective local details by increasing the receptive field, thereby establishing effective correlations between local features, global and contextual semantics and improving the representation of temporal features.

[0101] 2.1 Improved Convolutional Long Short-Term Memory Network (ConvLSTM):

[0102] Traditional Convolutional Long Short-Term Memory (ConvLSTM) networks are architectures combining Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks, primarily designed for processing temporal data. Unlike traditional LSTMs, ConvLSTMs apply convolutional operations at each time step, which is beneficial for capturing spatial local information in temporal data. However, in the temporal feature extraction stage, this network only considers forward temporal information, neglecting the importance of backward temporal information. Furthermore, applying standard convolutional networks for spatial local feature extraction at each time step is limited by the receptive field size and cannot fully capture the local spatial details contained in the global temporal semantics, as well as the correlation between local and global features. Therefore, to address these limitations, this invention designs a bidirectional dilated convolutional long short-term memory network (Bi-DConvLSTM) to acquire temporal features of epileptic EEG signals. Figure 3 The overall structure of the improved convolutional long short-term memory network is shown in the figure. This indicates the input gates for both forward and reverse directions. Memory units representing the two directions. This represents the output features in both positive and negative directions. This represents the initial mapping characteristics after passing through the timing mapping module. This represents the activation function. This indicates the addition of characteristic elements. The weight matrix represents the output features in both positive and negative directions.

[0103] The specific improvement steps are as follows:

[0104] Step 1. A given set of epileptic EEG signals is ,in These represent the channel dimension and the temporal dimension, respectively. Temporal epilepsy EEG signals are input into the temporal feature embedding module to generate low-dimensional temporal information with a uniform spatial distribution. As shown in the equation.

[0105] (10).

[0106] in, This represents the temporal feature embedding module.

[0107] Step 2. To obtain effective temporal features, as well as global features and contextual semantic details of epileptic EEG signals, the low-dimensional temporal features are... An improved convolutional long short-term memory network is input to model temporal information and establish interactions between local features, global features, and contextual semantics. Simultaneously, long-distance dependencies are established between different temporal steps. Output features of forward-biased dilated convolutional long short-term memory networks As shown in the equation.

[0108] (11).

[0109] in, This indicates the Hadamard product operation. This represents the activation function of ReLU. Let tanh be the activation function, the first... Output features of dilated convolutional long short-term memory networks in reverse order As shown in the equation.

[0110] (12).

[0111] Here, * denotes dilated convolution (DConv) operation. Compared to standard convolutional networks, dilated convolutions can achieve better local and global semantics without changing the output feature size. The convolution kernel in a dilated convolutional layer... and feel the wild The calculation is shown in the equation.

[0112] (13).

[0113] Where S represents the product of the step sizes of all previous layers. D represents the expansion coefficient.

[0114] Based on the output features of the forward-dilated convolutional long short-term memory network and the backward-dilated convolutional long short-term memory network, the improved output features of the convolutional long short-term memory unit are obtained. This is used to capture global features and contextual semantic details of epileptic EEG signals. .

[0115] In summary, embedding dilated convolutions into bidirectional long short-term memory networks can not only effectively capture global features and contextual semantic details in epileptic EEG signals, but also explore local details in temporal epileptic EEG signals and establish interactions between them to improve the representation of temporal features.

[0116] 2.2 Linear Position Embedded Attention Module (LPEAM):

[0117] Considering the significant amount of redundant information in the temporal features of epileptic EEG signals obtained through improved convolutional memory networks, the insufficient representation of spatial features extracted by extended convolution, and the importance of positional information in temporal epileptic EEG signals, this invention develops a linear positional embedding attention module. This module can generate an effective positional encoding matrix without adding additional parameters and refines these temporal features to reduce the reuse of redundant information. Furthermore, this module can capture the dependencies between different time points and time periods in the temporal epileptic EEG signals and assign specific attention weights to each time point or time period, enabling the temporal feature extraction module to more effectively perceive subtle changes between different types of epileptic EEG signals and further improve the representation of global features and contextual semantics. Notably, the linear positional embedding attention module can further acquire contextual semantic details from the spatial dimension during the feature learning stage.

[0118] Output features of linear position embedding attention module The calculation is shown in the equation.

[0119] (14).

[0120] in, This indicates the multiplication of characteristic matrices. This represents the output features of an improved convolutional long short-term memory network. Indicates along the time sequence SoftMax operation. This represents the average pooling operation, which in temporal epileptic EEG signals is along the channel. The mean-averaging operation. This indicates a linear operation.

[0121] In summary, combining linear position-embedded attention modules with improved convolutional long short-term memory networks not only refines temporal features but also reduces redundant information and highlights salient semantic representations, thereby improving the performance of the epilepsy EEG signal recognition framework.

[0122] 3. Spatiotemporal Feature Selection and Aggregation Module:

[0123] Spatial features focus on local information in epileptic EEG signals, while temporal features are more beneficial for representing global and contextual semantics. Therefore, to obtain better spatiotemporal feature representations and to promote information complementarity between temporal and spatial features, and to establish effective interaction between them, this invention develops a novel spatiotemporal feature selection and aggregation module. This module aims to select and aggregate temporal information and spatial features of epileptic EEG signals to improve the network's recognition performance. Furthermore, unlike simple temporal and spatial feature fusion schemes, this module employs two strategies in the feature aggregation stage: max pooling and average pooling. This further highlights the representation of salient features in spatiotemporal semantics without losing detailed information of the epileptic EEG signals, and emphasizes the importance of local and global contextual semantics and long-distance dependencies for recognition performance. Notably, spatiotemporal features can share information weights during the selection stage. The specific operation steps of this module are as follows.

[0124] Step 1. Given global context temporal features and multi-scale spatial local features The aligned spatiotemporal features are then simply concatenated to generate fused features with complementary information. The splicing process is shown in the equation.

[0125] (15).

[0126] in, This represents a simple feature assembly along the channel direction.

[0127] Step 2. To promote interaction among the spatiotemporal features of epileptic EEG signals, enhance the complementarity of these information, and explore their potential correlations, a one-dimensional decomposition dilated convolutional layer with two large selection kernels is used to process the simply fused features. An explicit decomposition is performed, and the salient information contained in these features is adaptively selected and modeled. The specific calculation is shown in the equation.

[0128] (16).

[0129] in, and This represents the decomposition dilation convolution operation with different kernel selections. In this invention, the size of the one-dimensional decomposition dilation convolution kernel is... and The decomposition and dilation convolution with a large selection kernel mainly involves... The standard convolution is decomposed along the channel into a wave with an expansion coefficient of D. The standard dilated convolution of size, while the decomposed channels satisfy... Furthermore, these decomposed channels are linearly weighted to enhance the interaction between spatiotemporal features from a channel perspective. In short, the decomposed dilated convolution with a large selection kernel can provide a dilated convolution operation along the depth direction and reconstruct and filter the spatial local details of the fused features by increasing the receptive field. At the same time, the channel correlation between different features is explored by decomposing and weighting the channels, which is beneficial for remodeling long-distance dependencies and global contextual semantics.

[0130] This deep dilation convolution with a large selection kernel can generate effective spatiotemporal fusion features with a large receptive field. This strategy can more effectively promote the selection of kernels in the later stages. At the same time, effective decomposition along the channel is more conducive to promoting the interaction between spatiotemporal features in epileptic EEG signals than simply applying a single large kernel.

[0131] Step 3. To reduce the transmission of redundant information between layers and further improve the representation of salient features, the spatiotemporal features decomposed by different large selection kernels will be concatenated. Two different spatiotemporal selection mechanisms, one-dimensional max pooling and average pooling, will be used to effectively select the spliced ​​spatiotemporal semantics. The splicing process of spatiotemporal semantics is shown in the equation.

[0132] (17).

[0133] in, This indicates the number of decomposition nuclei along the channel. Input average pooling and max pooling operations to obtain effective spatiotemporal correlations and generate novel attention feature maps A with local saliency and global contextual semantics. The specific operations are shown in the equations.

[0134] (18).

[0135] in, This indicates the average pooling operation. This indicates a max pooling operation. This represents a local salient detail attention feature map. Attention feature maps represent global features and contextual semantic details. To achieve information interaction between different spatiotemporal features, these spatiotemporal attention features are concatenated, and a depthwise decomposition dilated convolutional layer (DW-DConv) is used to transform these features into N spatiotemporal spaces for reconstruction, making the spatiotemporal feature fusion more compact. The specific operation is shown below.

[0136] (19).

[0137] in, This represents the activation function for Softmax. This represents a standard 1×1 convolution operation. The decomposed and fused features are masked and weighted, and then mapped to the spatiotemporal semantic attention feature map through a standard convolutional layer, generating the mapped features. As shown in the equation.

[0138] (20).

[0139] in, Indicates feature splicing, i The channel is represented. To achieve effective complementarity and weight sharing among the spatiotemporal features of epileptic EEG signals, the filtered and aggregated spatiotemporal feature maps are... The features are embedded into the initial spatiotemporal features for reconstruction, and finally, filtered aggregated features are generated. The specific calculations are shown in the equation.

[0140] (twenty one).

[0141] in, This indicates the Hadamard product operation.

[0142] Step 4. Considering that prior knowledge contains rich low-level semantic details, the original spatiotemporal fusion features are... With filtered aggregated features The features are then concatenated to obtain the final features after processing by the spatiotemporal feature selection and aggregation module. The calculation process is shown in the equation.

[0143] .

[0144] 4. Loss function for reconstruction:

[0145] To ensure the optimal feature representation of the epilepsy EEG signal recognition network designed in this invention, a new weighted loss function L is proposed. Different loss functions are used for individually supervised learning of each branch to avoid the recognition network getting trapped in local optima, while simultaneously balancing the differences between different categories. The specific calculation of the weighted loss function is shown in the equation.

[0146] (twenty two).

[0147] in, This represents a learnable balance factor. The cross-entropy main loss function for multi-class classification is shown in the equation.

[0148] (twenty three).

[0149] in, Labels representing the true category, Labels indicating predictions Represents the total number of samples. Loss function of the spatial feature extraction branch. The calculation is shown in the equation.

[0150] (twenty four).

[0151] in, This represents the predicted probability of a sample. δ represents the learnable hyperparameter. The loss function for the temporal feature extraction branch. Both the spatial feature extraction branch and the spatial feature extraction branch use the multi-class Focal loss as their loss function.

[0152] In summary, the designed weighted loss function plays a balancing role among different categories and features, and the use of this individually supervised learning strategy enhances the representation of spatiotemporal features. At the same time, the individually supervised spatiotemporal feature selection and aggregation module strengthens the interaction between spatiotemporal features in epileptic EEG signals and more effectively captures the differences between different epileptic EEG signals, thereby improving the overall performance of the recognition framework.

[0153] 5. Process and Overview:

[0154] Figure 4 The proposed spatiotemporal feature selection and aggregation (STFSAM) framework for epileptic EEG signal recognition is presented in (a). This framework mainly consists of three parts: a spatial feature extraction module (SFEM), a temporal feature extraction module (TFEM), and a spatiotemporal feature selection and aggregation module (STFSAM). The SFEM uses an improved WaveNet to learn multi-scale local spatial details of epileptic EEG signals and establishes effective correlations between features at different scales. Simultaneously, it utilizes a linear channel embedding (LCEAM) module to establish dependencies between spatial feature channels and highlights the representation of salient features through a weighting strategy. The TFEM uses an improved convolutional long short-term memory network to explore the contextual semantic details of epileptic EEG signals from both positive and negative directions. It embeds local features into global information through dilated convolutions and utilizes a linear positional attention module to reduce the transmission of redundant information between layers and effectively alleviate problems such as gradient explosion and vanishing gradients. The STFSAM module can mitigate the distributional differences between spatial and temporal features and form a complementary relationship between them to improve the recognition performance of epileptic EEG signals. The process for recognizing epileptic EEG signals is as follows:

[0155] Step 1. Local spatial features can be obtained using the Spatial Feature Extraction (SFEM) module. and refined features Global contextual semantic features can be obtained through the Temporal Feature Extraction (TFEM) module. and refined features .

[0156] Step 2. In the feature selection and aggregation stage, this invention repeatedly uses the Spatiotemporal Feature Selection and Aggregation Module (STFSAM) to reduce the differences in the distribution of different features and to achieve hierarchical selection and aggregation (see...). Figure 4 (a) STFSAM) is used to further improve the performance of the proposed recognition framework. The hierarchical feature selection and aggregation operations are shown in the equations.

[0157] (25).

[0158] in, This indicates feature splicing.

[0159] Figure 4 (a) The overall network structure of the epilepsy EEG signal recognition framework that selects aggregation based on spatiotemporal features. Figure 4 (b) represents the improved convolutional long short-term memory network in the temporal feature extraction module. Figure 4 (c) represents the multi-scale dilated causal convolutional unit of the improved WaveNet in the Spatial Feature Extraction (SFEB) branch. Here, LCEAM represents the linear channel embedding attention module, LPEAM represents the linear location embedding attention module, and STFSAM represents the spatiotemporal feature selection aggregation module. "Outputs" represents the generated recognition results. and The input epileptic EEG signal is represented by H and T, which represent the spatial and temporal dimensions of the input, respectively. This indicates a space-embedded module. This represents the timing embedding module. The red, blue, and green lines represent the information transmission process of different features.

[0160] Step 3. To achieve accurate identification of epileptic EEG signals, this invention uses the feature map after spatiotemporal feature selection and aggregation. With feature map The process involves further layer-by-layer fusion, followed by processing using standard 1×1 convolutions and a set of consecutive linear layers. This ultimately achieves accurate identification of epileptic EEG signals. The specific operations are illustrated in the equations.

[0161] (26).

[0162] Where + indicates that corresponding elements in the feature matrix are added together. Represents continuous linear operations. This represents the classifier for Softmax. This represents the final feature representation. This represents the generated recognition result. The operation process and training flow of the proposed epilepsy EEG signal recognition framework are as follows: Figure 5 As shown.

[0163] 6. Experiments and Discussion:

[0164] The following sections first introduce the sources and preprocessing of experimental data, the environment and experimental parameters, and evaluation metrics. Secondly, they provide experimental results and analyses comparing the proposed method with other approaches. Finally, to demonstrate whether each module in the proposed epilepsy EEG signal recognition framework plays a positive role in the overall performance, a series of ablation experiments were conducted.

[0165] 6.1 Data Preparation:

[0166] TUSZ: This dataset comes from historical data from Temple University Hospital (TUH) and includes 3050 seizure events from over 300 different patients. It encompasses seven different event types: ABSZ (Absence Seizure), SPSZ (Simple Partial Seizure), TCSZ (Tonic-Clonic Seizure), TNSZ (Tonic Seizure), CPSZ (Complex Partial Seizure), GNSZ (Generalized Non-Specific Seizure), and FNSZ (Focal Non-Specific Seizure). To ensure the smooth conduct of the experiment, 4626 events of 20×512 size from four relatively balanced categories (ABSZ, SPSZ, TCSZ, and TNSZ) were selected.

[0167] CHBMIT: This dataset contains EEG signals from Boston Children's Hospital related to epilepsy, collected from the MIT EEG database. It consists of scalp EEG recordings from children with refractory epilepsy. It includes 23 records from 22 subjects.

[0168] 6.2 Related Settings:

[0169] 6.2.1 Evaluation Indicators:

[0170] To ensure fairness and consistency in the experiments, this invention uses four commonly used evaluation metrics—overall accuracy (OA), precision (P), recall (R), specificity (SPE), and the F-score—to measure the performance of all experimental methods. The calculations for these metrics are shown in the equations.

[0171] (27).

[0172] (28).

[0173] (29).

[0174] (30).

[0175] Where FN represents a sample that was initially classified as negative but is actually positive; FP represents a sample that was initially classified as positive but is actually negative; TN represents a sample that was initially classified as negative and is actually negative; and TP represents a sample that was initially classified as positive and is actually positive.

[0176] 6.2.2 Parameter Settings:

[0177] During the experimental training phase, this invention sets the number of training iterations to 100, the batch size to 160, and the learning rate to 1e-4. AdamW is used as the optimizer to adjust the recognition framework, and the decay rate is set to 1e-5. Furthermore, the learning rate is dynamically adjusted using a cosine annealing algorithm.

[0178] All experiments in this invention were conducted in the same environment, using deep learning libraries such as Python 3.8.19, cu117, torch 1.13.1+cu117, and NumPy, and were trained and tested on an NVIDIA GeForce RTX 4090 equipped with 8 graphics cards.

[0179] This invention provides a spatiotemporal feature recognition system for electroencephalogram (EEG) signals, comprising:

[0180] The signal acquisition module is used to acquire electroencephalogram (EEG) signals that reflect brain activity in epilepsy.

[0181] The spatial feature acquisition module is used to input EEG signals into the feature embedding module to obtain low-dimensional features of the EEG signals; input the low-dimensional features into the improved waveform network to extract multi-scale local spatial features of the EEG signals; and filter the multi-scale local spatial features through linear channel embedding into the attention module to obtain the spatial features of the EEG signals.

[0182] The temporal feature acquisition module is used to input EEG signals into the temporal feature embedding module to obtain low-dimensional temporal features of the EEG signals; input the low-dimensional temporal features into an improved convolutional long short-term memory network to capture global features and contextual semantic details of the EEG signals; and input the captured results into the linear position embedding attention module to obtain temporal features of the EEG signals.

[0183] The recognition module is used to identify spatial and temporal features as spatiotemporal features of EEG signals.

[0184] A specific example is as follows:

[0185] This embodiment discloses a method for spatiotemporal feature recognition of electroencephalogram (EEG) signals, the specific steps of which are as follows:

[0186] S1. An improved WaveNet method is used to extract multi-scale local spatial features of epileptic EEG signals, which makes up for the shortcomings of single-structure spatial feature representation. A newly designed linear channel embedding attention module is used to refine the multi-scale features along the channel and establish dependencies between channels to reduce the use of redundant information and highlight the representation of salient features.

[0187] S2. An improved convolutional long short-term memory network is used to model epileptic EEG signals from both positive and negative directions, obtaining effective global features and contextual semantics. The use of an expanded convolutional network in this module alleviates the limitation of standard convolutional networks in effectively modeling local details due to insufficient receptive fields. Furthermore, a newly designed linear position embedding attention module is used to perform positional encoding on temporal epileptic EEG signals and refine the temporal features to reduce the transmission and use of redundant information between layers.

[0188] S3. To alleviate the differences between different features and establish complementary relationships among them, a newly designed spatiotemporal feature selection and aggregation module is used to fuse temporal and spatial features layer by layer. At each level of feature fusion, filtering and re-aggregation are employed to reduce differences while improving the representation of salient features, thereby improving the recognition performance of epileptic EEG signals. The specific process is as follows: Figure 6 As shown.

[0189] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for spatiotemporal feature recognition of electroencephalogram (EEG) signals, characterized in that, Includes the following steps: Acquire electroencephalogram (EEG) signals that reflect brain activity in epilepsy; The EEG signal is input into a feature embedding module to obtain low-dimensional features of the EEG signal; the low-dimensional features are then input into an improved waveform network to extract multi-scale local spatial features of the EEG signal. The spatial features of EEG signals are obtained by filtering multi-scale local spatial features through linear channel embedding attention modules; wherein, the improved waveform network includes: causal convolutional layers and improved extended causal convolutional units; the improved extended causal convolutional units consist of three extended causal convolutional layers, two standard convolutional layers and one gated activation unit; The EEG signal is input into a temporal feature embedding module to obtain low-dimensional temporal features of the EEG signal; the low-dimensional temporal features are input into an improved convolutional long short-term memory network to capture global features and contextual semantic details of the EEG signal; the captured results are input into a linear positional embedding attention module to obtain temporal features of the EEG signal; wherein, the improved convolutional long short-term memory network is determined by embedding dilated convolutions into a bidirectional long short-term memory network; Spatial and temporal features are used as the spatiotemporal features of EEG signals.

2. The spatiotemporal feature recognition method for electroencephalogram (EEG) signals as described in claim 1, characterized in that, The process of inputting low-dimensional features into an improved waveform network to extract multi-scale local spatial features of EEG signals includes the following steps: Low-dimensional features are input into causal convolutional layers to extract low-level spatial features; Low-level spatial features are input into an improved dilated causal convolutional unit; wherein the improved dilated causal convolutional unit consists of three dilated causal convolutional layers, two standard convolutional layers, and one gated activation unit; The results obtained from multiple improved extended causal convolutional units are summed using residuals and skip connections, and long-term dependencies are established between the results obtained from multiple improved extended causal convolutional units through gated activation units to obtain fused high-level causal convolutional features. By inputting high-level causal convolutional features into continuous standard convolutional layers of different scales, multi-scale local spatial features of EEG signals can be obtained.

3. The spatiotemporal feature recognition method for electroencephalogram (EEG) signals as described in claim 1, characterized in that, The step of filtering multi-scale local spatial features through linear channel embedding attention modules to obtain spatial features of EEG signals includes: Multi-scale local spatial features Input linear channels are embedded with attention modules to obtain spatial features of EEG signals. The formula is: ； in, This represents the feature mapping operation. This represents the Hadamard product; Linear channel attention coefficients in low dimensions The formula is: ； in, This represents the activation function of sigmoid. Indicates a linear layer. This represents the activation function for ReLU.

4. The spatiotemporal feature recognition method for electroencephalogram (EEG) signals as described in claim 1, characterized in that, The process of inputting low-dimensional temporal features into an improved convolutional long short-term memory network to capture global features and contextual semantic details of EEG signals includes the following steps: Low-dimensional temporal features Input the improved convolutional long short-term memory network to obtain the first... t Output features of a forward-biased dilated convolutional long short-term memory network; ； Obtain the t Output features of dilated convolutional long short-term memory networks that are reversed at every moment; ； Where * denotes an expanded convolution operation; This indicates the Hadamard product operation. This represents the activation function of ReLU. This represents the activation function for tanh; They represent t Input gates in both forward and reverse directions at all times. They represent t Memory units that are always in both forward and reverse directions. They represent t The hidden state in both positive and negative directions at all times. They represent t Two opposing output gates at all times. They represent t The forgetting gate, which is always in both forward and reverse directions. and These represent the weight matrices related to the input features, the previous hidden state, and the input gate of the previous memory unit, acting in opposite directions. These represent the weight matrices related to the forget gates of the input features, the previous hidden state, and the previous memory unit, respectively, acting in the positive and negative directions. These represent the weight matrices related to the input features, the previous hidden state, and the memory of the previous memory unit, acting in opposite directions. These represent the weight matrices that act on the input features, the previous hidden state, and the output gates of the previous memory unit in both positive and negative directions, respectively. and They represent t The hidden states in both directions at time -1. and They represent t Memory units in both directions at time -1. This indicates the input gate bias. Indicates the forget gate bias. Indicates memory cell bias. Indicates output gate bias; Based on the output features of the forward-dilated convolutional long short-term memory network and the backward-dilated convolutional long short-term memory network, the improved output features of the convolutional long short-term memory unit are obtained. This is used to capture global features and contextual semantic details of EEG signals. 。 5. The spatiotemporal feature recognition method for electroencephalogram (EEG) signals as described in claim 1, characterized in that, The step of embedding the captured results into a linear positional attention module to obtain the temporal characteristics of the EEG signal includes: Capture results By embedding an attention module into the linear input position, the temporal features of the EEG signal are obtained. The formula is: ； in, This represents the multiplication of characteristic matrices; Indicates along the time sequence The SoftMax operation of the normalized exponential function; This represents the average pooling operation, which in temporal EEG signals occurs along the channel. Meaning operation; This indicates a linear operation.

6. A method for spatiotemporal feature recognition of electroencephalogram (EEG) signals as described in claim 3 or 5, characterized in that, Also includes: Temporal characteristics of EEG signals Spatial characteristics of EEG signals Align and stitch them together to obtain fusion features with complementary information. The formula is: ； in, This indicates feature splicing along the channel direction; One-dimensional decomposition of dilated convolutional layers with two different selection kernels is used to fuse features. To perform a decomposition, the formula is: ； in, and These represent one-dimensional decomposition dilation convolution operations with different kernel choices; the size of the one-dimensional decomposition dilation convolution kernel is... and ; The decomposed and fused features are concatenated, and attention feature maps are obtained by one-dimensional max pooling and average pooling, respectively. The decomposed fusion features are masked and weighted, and then mapped to the attention feature map through a standard convolutional layer to obtain the mapped features. The Hadamard product is used to reconstruct the mapped features and the decomposed fused features to obtain the filtered aggregated features; Combine the selected aggregated features with the fused features Then, the parts are assembled.

7. A spatiotemporal feature recognition system for electroencephalogram (EEG) signals, characterized in that, include: The signal acquisition module is used to acquire electroencephalogram (EEG) signals that reflect brain activity in epilepsy. The spatial feature acquisition module is used to input the EEG signal into the feature embedding module to obtain low-dimensional features of the EEG signal; the low-dimensional features are then input into an improved waveform network to extract multi-scale local spatial features of the EEG signal. The spatial features of EEG signals are obtained by filtering multi-scale local spatial features through linear channel embedding attention modules; wherein, the improved waveform network includes: causal convolutional layers and improved extended causal convolutional units; the improved extended causal convolutional units consist of three extended causal convolutional layers, two standard convolutional layers and one gated activation unit; A temporal feature acquisition module is used to input EEG signals into a temporal feature embedding module to obtain low-dimensional temporal features of the EEG signals; input the low-dimensional temporal features into an improved convolutional long short-term memory network to capture global features and contextual semantic details of the EEG signals; input the captured results into a linear position embedding attention module to obtain temporal features of the EEG signals; wherein, the improved convolutional long short-term memory network is determined by embedding dilated convolutions into a bidirectional long short-term memory network; The recognition module is used to identify spatial and temporal features as spatiotemporal features of EEG signals.

Images