Method and system for identifying spatial-temporal characteristics of electroencephalogram signals

Abstract

The invention discloses a spatial-temporal feature recognition method and system for electroencephalogram signals, and relates to the technical field of electroencephalogram signal process.The method comprises the steps that epilepsy electroencephalogram signals are input into a feature embedding module to obtain low-dimensional features; inputting the low-dimensional features into an improved waveform network to extract multi-scale local spatial features; embedding an attention module through a linear channel to screen multi-scale local spatial features so as to obtain spatial features; inputting the epilepsy electroencephalogram signal into a time sequence feature embedding module to obtain a dimension time sequence feature; inputting the low-dimensional time sequence features into an improved convolutional long short-term memory network to capture global features and context semantic details; inputting the capture result into a linear position embedding attention module to obtain time sequence features; and inputting the spatial features and the time sequence features into a spatial-temporal feature selection aggregation module to obtain an epilepsy electroencephalogram signal recognition result. The method can explore the association between the spatial-temporal characteristics and reduce the difference between the spatial-temporal characteristics.

Description

Technical Field

[0001] The present invention relates to the technical field of electroencephalogram (EEG) signal processing, and in particular to a method and system for identifying spatiotemporal features of EEG signals. Background Art

[0002] EEG signals are complex signals that require advanced signal processing and feature extraction methods to be correctly identified and interpreted.

[0003] Considering that single-structure deep learning methods cannot fully represent epileptic EEG signals when identifying them, existing epileptic EEG signal recognition methods that combine spatiotemporal features have made up for the problem of insufficient feature representation of single-structure networks. However, during the feature fusion stage, simple splicing processing is often used to aggregate spatiotemporal features, which does not address the differences between different features. At the same time, the information flow uses redundant information multiple times during inter-layer transmission, which reduces the ability to represent significant features. Therefore, the recognition performance of epileptic EEG signals still has significant room for improvement. In summary, how to fully identify the spatiotemporal characteristics of epileptic EEG signals is an important issue that needs to be addressed urgently. Summary of the Invention

[0004] The embodiments of the present invention provide a method and system for identifying the spatiotemporal features of EEG signals, which can fully identify the spatiotemporal features of epileptic EEG signals.

[0005] An embodiment of the present invention provides a method for identifying spatiotemporal features of an EEG signal, comprising the following steps: Obtaining EEG signals that reflect epileptic brain activity; The EEG signal is input into the feature embedding module to obtain the low-dimensional features of the EEG signal; the low-dimensional features are input into the improved waveform network to extract the multi-scale local spatial features of the EEG signal; the multi-scale local spatial features are filtered through the linear channel embedding attention module to obtain the spatial features of the EEG signal; The EEG signal is input into the temporal feature embedding module to obtain the low-dimensional temporal features of the EEG signal; the low-dimensional temporal features are input into the improved convolutional long short-term memory network to capture the global features and contextual semantic details of the EEG signal; the captured results are input into the linear position embedding attention module to obtain the temporal features of the EEG signal; Spatial features and temporal features are used as spatiotemporal features of EEG signals.

[0006] Furthermore, the step of inputting the low-dimensional features into the improved waveform network to extract the multi-scale local spatial features of the EEG signal specifically includes: Low-dimensional features are input into the causal convolution layer to extract low-level spatial features; the low-level spatial features are input into the improved dilated causal convolution unit; wherein the improved dilated causal convolution unit is composed of three dilated causal convolution layers, two standard convolution layers and a gated activation unit; the results obtained by multiple improved dilated causal convolution units are added together using residual and jump connections, and long-term dependencies are established between the results obtained by multiple improved dilated causal convolution units through the gated activation unit to obtain fused high-level causal convolution features; the fused high-level causal convolution features are input into continuous standard convolution layers of different scales to obtain multi-scale local spatial features of the EEG signal.

[0007] Furthermore, the method of embedding the attention module through the linear channel to screen the multi-scale local spatial features to obtain the spatial features of the EEG signal includes the following specific steps: Multi-scale local spatial features Input linear channels are embedded into the attention module to obtain the spatial features of EEG signals , the formula is: ; in, represents the feature mapping operation, represents the Hadamard product; Linear channel attention coefficient in low dimension , the formula is: ; in, represents the activation function of sigmoid, represents a linear layer, Represents the activation function of ReLu.

[0008] Furthermore, the low-dimensional temporal features are input into the improved convolutional long short-term memory network to capture the global features and contextual semantic details of the EEG signal. The specific steps include: Low-dimensional time series features Input the improved convolutional long short-term memory network and obtain the t Output features of the dilated convolutional long short-term memory network with positive time-of-flight; ; Get the first t Output features of the time-reversed dilated convolutional long short-term memory network; ; Among them, * represents the dilated convolution operation; represents the Hadamard product operation, represents the activation function of ReLu, represents the activation function of tanh; Respectively t Input gates in both positive and negative directions at all times, Respectively t Memory units that are always in both positive and negative directions, Respectively t Hidden state in both positive and negative directions at all times, Respectively t There are two reverse output gates at all times. Respectively t The forgetting gate in both positive and negative directions at all times, and Represents the weight matrices related to the input gate of the input features, the previous layer hidden state, and the previous layer memory unit in the positive and negative directions, respectively. Represents the weight matrices related to the input features, the previous layer hidden state, and the forget gate of the previous layer memory unit in the positive and negative directions respectively; Represents the weight matrices related to the input features, the previous layer hidden state, and the previous layer memory units in the positive and negative directions, respectively. Represents the weight matrices related to the input features, the hidden state of the previous layer, and the output gate of the memory unit of the previous layer in the positive and negative directions respectively. and Respectively t -1 moment hidden state in both positive and negative directions, and Respectively t -1 moment memory units in both positive and negative directions, represents the input gate bias, represents the forget gate bias, represents the memory cell bias, represents the output gate bias;

[0009] According to the output characteristics of the forward dilated convolutional long short-term memory network and the output characteristics of the reverse dilated convolutional long short-term memory network, the output characteristics of the improved convolutional long short-term memory unit are obtained. , used to capture the global features and contextual semantic details of EEG signals: .

[0010] Furthermore, the capture result is input into the linear position embedding attention module to obtain the temporal characteristics of the EEG signal, and the specific steps include: Will capture the results Input linear position embedding attention module to obtain the temporal characteristics of EEG signal , the formula is: ; in, Represents feature matrix multiplication; Indicates that along the time series Normalized exponential function SoftMax operation; Represents the average pooling operation, which is along the channel in the time series EEG signal The averaging operation; Represents a linear operation.

[0011] Furthermore, the signal feature recognition method further includes: The temporal characteristics of EEG signals and the spatial characteristics of EEG signals Align and splice to obtain fusion features with complementary information , the formula is: ; in, represents the feature splicing along the channel direction; Two one-dimensional decomposition dilated convolution layers with different selected kernels are used to fusion features Perform display decomposition, the formula is: ; in, and They represent the one-dimensional decomposition expansion convolution operation with different selection kernels; the one-dimensional decomposition expansion convolution kernel size is and ; The decomposed fusion features are spliced ​​together, and one-dimensional maximum pooling and average pooling are used to obtain the attention feature map; The decomposed fusion features are mask-weighted, and the mask-weighted fusion features are mapped to the attention feature map through a standard convolutional layer to obtain the mapping features; Use Hadamard product to reconstruct the mapped features and the decomposed fusion features to obtain the filtered aggregated features; Combine the filtered aggregate features with the fusion features Perform splicing.

[0012] An embodiment of the present invention provides a system for identifying spatiotemporal features of EEG signals, comprising: Signal acquisition module, used to obtain EEG signals reflecting epileptic brain activity; The spatial feature acquisition module is used to input the EEG signal into the feature embedding module to obtain the low-dimensional features of the EEG signal; the low-dimensional features are input into the improved waveform network to extract the multi-scale local spatial features of the EEG signal; the multi-scale local spatial features are filtered through the linear channel embedding attention module to obtain the spatial features of the EEG signal; The temporal feature acquisition module is used to input the EEG signal into the temporal feature embedding module to obtain the low-dimensional temporal features of the EEG signal; the low-dimensional temporal features are input into the improved convolutional long short-term memory network to capture the global features and contextual semantic details of the EEG signal; the captured results are input into the linear position embedding attention module to obtain the temporal features of the EEG signal; The recognition module is used to use spatial features and temporal features as spatiotemporal features of EEG signals.

[0013] The present invention provides a method and system for identifying spatiotemporal features of EEG signals. Compared with the prior art, the present invention has the following advantages: The EEG signal is input into the feature embedding module to obtain the low-dimensional features of the EEG signal; the low-dimensional features are input into the improved waveform network to extract the multi-scale local spatial features of the EEG signal; the multi-scale local spatial features are screened through the linear channel embedding attention module to obtain the spatial features of the EEG signal; the EEG signal is input into the timing feature embedding module to obtain the low-dimensional timing features of the EEG signal; the low-dimensional timing features are input into the improved convolutional long short-term memory network to capture the global features and contextual semantic details of the EEG signal; the captured results are input into the linear position embedding attention module to obtain the timing features of the EEG signal; the spatial features and timing features are used as the spatiotemporal features of the EEG signal; and finally the spatiotemporal features of the EEG signal are fully identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Schematic diagrams of a standard dilated causal convolution unit and an improved dilated causal convolution unit provided in an embodiment of the present invention, wherein (a) is a standard dilated causal convolution unit, and (b) is an improved dilated causal convolution unit; Figure 2 A structural diagram of a linear channel embedded attention module provided by an embodiment of the present invention; Figure 3 The overall structure diagram of the improved convolutional long short-term memory network provided by the embodiment of the present invention; Figure 4 Figure 1 shows the overall network structure of an embodiment of the present invention, including (a) the overall network structure of the epileptic EEG signal recognition framework with spatiotemporal feature selection and aggregation, (b) the improved convolutional long short-term memory network in the temporal feature extraction module, and (c) the improved WaveNet multi-scale dilated causal convolution unit in the spatial feature extraction branch SFEB. Figure 5 The operation process and training process of the epilepsy EEG signal recognition framework provided by the embodiment of the present invention; Figure 6 This is an identification flow chart provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0015] To make the above-mentioned objects, features, and advantages of the present invention more readily apparent, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. The following description sets forth numerous specific details to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art may make similar modifications without departing from the scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0016] In this paper, we develop spatiotemporal feature selection aggregation networks (STFSANets) to explore the correlation between spatiotemporal features and reduce their differences.

[0017] First, an improved WaveNet method is used to extract multi-scale local spatial features of epileptic EEG signals to make up for the problem of insufficient representation of single structural spatial features. A newly designed linear channel embedding attention module is used to refine the multi-scale features along the channel and establish dependencies between channels to reduce the use of redundant information and highlight the representation of significant features.

[0018] Secondly, an improved convolutional long short-term memory network is used to model epileptic EEG signals from both positive and negative directions, acquiring effective global features and contextual semantics. The use of a dilated convolutional network in this module alleviates the shortcomings of standard convolutions, which suffer from insufficient receptive field and are unable to effectively model local details. Furthermore, a newly designed linear position embedding attention module is used to positionally encode temporal epileptic EEG signals and refine temporal features to reduce the transfer and use of redundant information between layers.

[0019] Finally, to mitigate the disparity between different features and establish a complementary relationship between them, a newly designed spatiotemporal feature selection and aggregation module is used to fuse temporal and spatial features layer by layer. At each level of feature fusion, a screening and re-aggregation approach is used to reduce disparity while improving the representation of salient features, thereby improving the recognition performance of epileptic EEG signals. Furthermore, the proposed recognition framework is evaluated and demonstrated on baseline datasets, including the Temple University Hospital EEG SeiZure (TUH EEG SeiZure) dataset and the Children's Hospital Boston-MIT (Children's Hospital Boston-MIT) dataset, demonstrating its effectiveness and robustness.

[0020] See also Figure 1 The present invention provides a method for identifying spatiotemporal features of EEG signals, comprising the following steps: Step 1: Obtain EEG signals that reflect epileptic brain activity.

[0021] Step 2: Input the EEG signal into the feature embedding module to obtain the low-dimensional features of the EEG signal; input the low-dimensional features into the improved waveform network to extract the multi-scale local spatial features of the EEG signal; and filter the multi-scale local spatial features through the linear channel embedding attention module to obtain the spatial features of the EEG signal.

[0022] Step 3: Input the EEG signal into the timing feature embedding module to obtain the low-dimensional timing features of the EEG signal; input the low-dimensional timing features into the improved convolutional long short-term memory network to capture the global features and contextual semantic details of the EEG signal; input the captured results into the linear position embedding attention module to obtain the timing features of the EEG signal.

[0023] Step 4: Use the spatial features and temporal features as the spatiotemporal features of the EEG signal, input the spatiotemporal features into the spatiotemporal feature selection and aggregation module to obtain the recognition results of the EEG signal.

[0024] The specific introduction of the present invention is as follows: 1. Spatial Feature Extraction Branch (SFEB): The spatial feature extraction branch (SFEB) is mainly composed of three parts: a feature embedding module, an improved channel attention module, and a waveform network WaveNet. The channel attention module is designed to screen local features in space, reduce redundant noise, and improve the representation of epileptic EEG signals by significant features. WaveNet mainly uses an expanded causal convolution component to model the multi-scale spatial information of epileptic EEG signals, and obtains the global semantics of epileptic EEG signals through a larger receptive field. At the same time, a simple causal convolution layer and a gated activation function effectively solve the long-distance dependency required in the spatial feature extraction stage. Since the input epileptic EEG signal has high-dimensional features and cannot be directly used for spatial feature extraction, the present invention uses a feature embedding module to map the input brain epileptic electrical signal to a unified low-dimensional space to learn more effective spatial local features.

[0025] 1.1 Improved WaveNet: The traditional WaveNet is a deep neural network model developed by DeepMind, which mainly consists of causal convolution layers and dilated causal convolution units. It is designed to directly process the original audio waveform signal to generate effective corresponding text data, especially in the fields of speech synthesis, music generation and sound effect synthesis. However, compared with the original audio waveform signal, there is correlation between the channels of epileptic EEG signals. At the same time, these epileptic EEG signals contain a large amount of high-dimensional raw noise, and simple WaveNet cannot be used directly to model the spatial details. The standard dilated causal convolution unit and the improved dilated causal convolution unit are as follows: Figure 1 (a) and Figure 1 (b) As shown in the figure, Represents a standard 3×3 convolution operation. Represents a standard 1×1 convolution operation. Represents a 1×1 dilated causal convolution operation. represents a 3×3 dilated causal convolution operation. σ1 represents the STReLU activation function. σ2 represents the Swish activation function with nonlinear characteristics. σ represents the Sigmoid activation function. tanh represents the Tanh activation function.

[0026] The specific improvement steps are as follows: Step 1. Assume that the input epileptic EEG signal is ,in Represent the channel dimension and sequence dimension respectively. The low-dimensional features output by the feature embedding module As shown in Eq.

[0027] (1).

[0028] in, Represents the feature embedding module.

[0029] Step 2. Considering that the prior knowledge contains rich low-level semantic information, it can effectively represent the basic properties of epileptic EEG signals. The acquired low-dimensional embedding features Input causal convolution layer to ensure that the model does not disrupt the order of epileptic EEG signal modeling, and at the same time, realize low-level spatial features The specific calculation is shown in Eq.

[0030] (2).

[0031] in, Represents a causal convolution operation.

[0032] Step 3. In order to further obtain effective multi-scale spatial local features, the low-level spatial features are input into the dilated causal convolution unit to learn more effective high-level spatial details. To avoid overfitting caused by too deep network layers and increase the complexity of the model, multiple dilated causal convolution units are combined using residual and skip connections, and gated activation units are used to establish long-term dependencies between these features. High-level spatial features The calculation of is shown in Eq.

[0033] (3).

[0034] in, Represents a dilated causal convolution operation. The expanded causal convolution unit has layer.

[0035] Each improved dilated causal convolution unit is mainly composed of three dilated causal convolution layers, two standard convolution layers and a gated activation unit. The output feature of the first unit is and As shown in Eq.

[0036] (4).

[0037] in, Represents a standard 1×1 convolution operation. Represents a standard 3×3 convolution operation. Indicates feature concatenation. In the first unit, a gated activation strategy is used to aggregate the dilated causal convolution features. Filtering and learning can effectively strengthen the long-range dependencies between the spatial features of epileptic EEG signals. The calculation of is shown in Eq.

[0038] (5).

[0039] in Represents the activation function of STReLU. represents the Swish activation function with nonlinear characteristics. ∗ represents a simple convolution operator. represents the Hadamard product. Represents the layer index. Represent filter and gate devices respectively. Represents a learnable convolution filter. The first aggregated dilated causal convolution feature As shown in Eq.

[0040] (6).

[0041] in, Represents a 1×1 dilated causal convolution operation. Represents a 3×3 dilated causal convolution operation.

[0042] Step 4. In order to obtain effective spatial salient features and reduce computational complexity, the obtained fused high-level causal convolution features are Input a set of continuous standard convolutional layers of different scales to achieve feature compression and excitation, and obtain spatial details The specific calculation is shown in Eq.

[0043] (7).

[0044] in, Represents the ReLu activation function. Represents a set of consecutive 3×3 standard convolutions.

[0045] In summary, the use of residual and skip connections in the dilated causal convolutional unit enables feature reuse while accelerating the convergence of the model during the training phase.

[0046] 1.2 Linear Channel Embedding Attention Module (LCEAM): In recent years, the channel attention mechanism has played a positive role in improving the performance of deep convolutional neural networks. However, most existing methods are committed to embedding more complex attention modules into neural networks to obtain better feature representation and performance. This approach not only increases the computational complexity of the model, but also fails to effectively filter redundant information distributed in low-dimensional space. Therefore, in order to solve these problems, the present invention develops a linear channel embedding attention module, which has a small number of parameters and can effectively alleviate the reuse of redundant information in low-dimensional space without adding additional features, so as to avoid problems such as insufficient representation of significant features. In addition, considering the correlation between the spatial local feature channels of epileptic EEG signals, the linear channel embedding attention module is integrated with the improved WaveNet to further improve the representation of significant information in spatial features. The structure of the linear channel embedding attention module is as follows: Figure 2 As shown in the figure, represents a linear operation, represents the Sigmoid activation function, Represents the ReLu activation function.

[0047] Linear channel embedding of the output features of the attention module The calculation of is shown in Eq.

[0048] (8).

[0049] in, Represents a feature mapping operation. Represents Hadamard product. Linear channel attention coefficient in low dimension The calculation of is shown in Eq.

[0050] (9).

[0051] in, Represents the activation function of ReLu. represents a linear layer.

[0052] In summary, combining the linear channel embedding attention module with the improved WaveNet can not only effectively reduce the reuse of redundant information and improve the representation of salient features without adding additional parameters and information, but also further obtain more effective multi-scale spatial local details from epileptic EEG signals.

[0053] 2. Temporal Feature Extraction Branch (TFEB): The temporal feature extraction branch (TFEB) consists primarily of three components: a temporal feature embedding module, an improved convolutional long short-term memory (ConvLSTM) network, and an improved positional attention module. The improved positional attention module is designed to capture positional deviations in epileptic EEG signals within temporal features and refine these temporal features. The temporal feature embedding module is structurally similar to the feature embedding module in the spatial feature extraction branch. It primarily performs a low-dimensional unified mapping of temporal epileptic EEG signals to enable the network to learn a more optimal temporal representation. The improved ConvLSTM network is designed to model the global and contextual semantics of epileptic EEG signals from both positive and negative perspectives. It extracts effective local details by increasing the receptive field, forming effective associations between local features, global, and contextual semantics, and improving the representation of temporal features.

[0054] 2.1 Improved Convolutional Long Short-Term Memory Network (ConvLSTM): The traditional convolutional long short-term memory network (ConvLSTM) is an architecture that combines convolutional neural networks (CNN) and long short-term memory networks (LSTM), mainly for processing time series data. Unlike traditional LSTM, ConvLSTM applies convolution operations at each time step, which is beneficial for capturing spatial local information in time series data. However, in the time series feature extraction stage, the network only considers the forward time series information and ignores the importance of reverse time series information. At the same time, the standard convolutional network is applied in the time step for the spatial local feature extraction stage. Due to the size of the receptive field, it cannot fully capture the local spatial details contained in the global semantics of the time series, as well as the correlation between local and global features. Therefore, in order to address these limitations, the present invention designs a bidirectional dilated convolutional long short-term memory network (Bi-DConvLSTM) to obtain the time series features of epileptic EEG signals. Figure 3 The overall structure of the improved convolutional long short-term memory network is shown. Represents the input gate for both positive and negative directions. A memory cell that represents both positive and negative directions. Represents the output features in both positive and negative directions. Represents the initial mapping features after the timing mapping module. Represents the activation function. Indicates the addition of characteristic elements. The weight matrix representing the output features in both positive and negative directions.

[0055] The specific improvement steps are as follows: Step 1. Given a set of epileptic EEG signals: ,in Represent the channel dimension and time series dimension respectively. The time series epileptic EEG signal is input into the time series feature embedding module to generate low-dimensional time series information with uniform spatial distribution. As shown in Eq.

[0056] (10).

[0057] in, Represents the temporal feature embedding module.

[0058] Step 2. In order to obtain effective time series features, as well as the global features and contextual semantic details of epileptic EEG signals, the low-dimensional time series features are transformed into The improved convolutional long short-term memory network is input to model the temporal information and form interactivity between local features, global features and contextual semantics, while establishing long-distance dependencies between different temporal steps. Output features of the dilated convolutional long short-term memory network with positive time-of-flight As shown in Eq.

[0059] (11).

[0060] in, represents the Hadamard product operation, represents the activation function of ReLu, Represents the activation function of tanh, Output features of time-reversed dilated convolutional long short-term memory network As shown in Eq.

[0061] (12).

[0062] Where * represents the dilated convolution (DConv) operation. Compared with the standard convolutional network, the dilated convolution can obtain better local and global semantics without changing the output feature size. and receptive field The calculation of is shown in Eq.

[0063] (13).

[0064] Where S represents the product of the strides of all previous layers, and D represents the dilation coefficient.

[0065] According to the output characteristics of the forward dilated convolutional long short-term memory network and the output characteristics of the reverse dilated convolutional long short-term memory network, the output characteristics of the improved convolutional long short-term memory unit are obtained. , used to capture the global features and contextual semantic details of epileptic EEG signals: .

[0066] In summary, embedding dilated convolution into a bidirectional long short-term memory network can not only effectively capture the global features and contextual semantic details in epileptic EEG signals, but also explore the local details in the temporal epileptic EEG signals and establish interactions between them to improve the representation of temporal features.

[0067] 2.2 Linear Position Embedding Attention Module (LPEAM): Taking into account that the temporal features of epileptic EEG signals obtained by the improved convolutional memory network contain a large amount of redundant information, the spatial features extracted by the extended convolution are not fully represented, and the importance of position information in temporal epileptic EEG signals, the present invention has developed a linear position embedding attention module, which can generate an effective position encoding matrix without adding additional parameters, and refine these temporal features to reduce the reuse of redundant information. In addition, the module can capture the dependencies between different time points and different time periods in the temporal epileptic EEG signals, and assign specific attention weights to each time point or time period in the temporal epileptic EEG signals, so as to enable the temporal feature extraction module to more effectively perceive the subtle changes between different types of epileptic EEG signals, and further improve the representation of global features and contextual semantics. It is worth noting that the linear position embedding attention module can further obtain contextual semantic details from the spatial dimension during the feature learning stage.

[0068] Linear position embedding of the output features of the attention module The calculation of is shown in Eq.

[0069] (14).

[0070] in, Represents feature matrix multiplication. Represents the output features of the improved convolutional long short-term memory network. Indicates that along the time series SoftMax operation. Represents the average pooling operation, which is along the channel in the time series epileptic EEG signal The averaging operation. Represents a linear operation.

[0071] In summary, combining the linear position embedding attention module with the improved convolutional long short-term memory network can not only refine the temporal features, but also reduce the redundant information and highlight the representation of salient semantics, thereby further improving the performance of the epileptic EEG signal recognition framework.

[0072] 3. Spatiotemporal feature selection and aggregation module: Spatial features focus on the local information of epileptic EEG signals, while temporal features are more beneficial to the representation of global and contextual semantics. Therefore, in order to obtain better spatiotemporal feature representation, and to promote information complementarity between temporal features and spatial features, and to establish an effective interactive relationship between them. The present invention has developed a novel spatiotemporal feature selection and aggregation module, which aims to select and aggregate the temporal information and spatial features of epileptic EEG signals to improve the recognition performance of the network. In addition, unlike the simple temporal information and spatial feature fusion scheme, this module adopts two strategies, such as maximum and average pooling, in the feature aggregation stage, which further highlights the representation of significant features in spatiotemporal semantics without losing the detailed information of epileptic EEG signals, and focuses on the importance of local, global contextual semantics and long-distance dependencies to recognition performance. It is worth noting that spatiotemporal features can share information weights in the selection stage. The specific operating steps of this module are as follows.

[0073] Step 1. Given global context temporal features and multi-scale spatial local features , simply splice the aligned spatiotemporal features to generate fusion features with complementary information The splicing process is shown in Eq.

[0074] (15).

[0075] in, Represents a simple feature concatenation along the channel direction.

[0076] Step 2. To promote the interaction between the spatiotemporal features of epileptic EEG signals, enhance the complementarity of these information, and explore the potential correlation between them, two one-dimensional decomposition dilated convolutional layers with larger selection kernels are used to perform the simple fusion of the features. Perform explicit decomposition and adaptively select and model the significant information contained in these features. The specific calculation is shown in the equation.

[0077] (16).

[0078] in, and Represents the decomposition and expansion convolution operation of different selection kernels. In the present invention, the decomposition and expansion convolution kernel size of one dimension is and The decomposition of the large selection kernel dilated convolution is mainly to convert a The standard convolution is decomposed into a dilation factor D along the channel. The standard dilated convolution of size, and the decomposed channels meet , and linearly weights these decomposed channels to strengthen the interaction between spatiotemporal features from a channel perspective. In short, the decomposed dilated convolution with a large selective kernel can provide a dilated convolution operation along the depth direction, and reconstruct and filter the spatial local details of the fused features by increasing the receptive field. At the same time, the channel correlation between different features is explored by weighting the channel decomposition, which is beneficial for remodeling long-range dependencies and global contextual semantics.

[0079] Through this deep dilated convolution with a large selection kernel, effective large receptive field spatiotemporal fusion features can be generated. This strategy can more effectively promote the subsequent kernel selection. At the same time, the effective decomposition along the channel is easier to promote the interaction between spatiotemporal features in epileptic EEG signals than simply applying a single large kernel.

[0080] Step 3. To reduce the transmission of redundant information between layers and further improve the representation of salient features, the spatiotemporal features decomposed using different large selection kernels are concatenated. Two different spatiotemporal selection mechanisms, one-dimensional max pooling and average pooling, are used to effectively select the reconstructed spatiotemporal semantics. The splicing process of spatiotemporal semantics is shown in Eq.

[0081] (17).

[0082] in, represents the number of decomposition kernels along the channel. The input average pooling and maximum pooling operations are used to obtain effective spatiotemporal associations and generate a new type of attention feature map A with local saliency and global contextual semantics. The specific operation is shown in the equation.

[0083] (18).

[0084] in, Represents an average pooling operation. Represents the maximum pooling operation. Represents the local salient detail attention feature map. Attention feature maps represent global features and contextual semantic details. To enable information exchange between different spatiotemporal features, these spatiotemporal attention features are concatenated and reconstructed using a deep-seated dilated convolutional layer (DW-DConv), transforming them into N spatiotemporal layers. This makes spatiotemporal feature fusion more compact. The specific operation is shown below.

[0085] (19).

[0086] in, Represents the Softmax activation function. Represents a standard 1×1 convolution operation. The decomposed fusion features are masked and weighted, and the fusion features are mapped to the spatiotemporal semantic attention feature map through the standard convolution layer to generate the mapping feature. As shown in Eq.

[0087] (20).

[0088] in, represents feature splicing, i In order to form effective complementarity and weight sharing between the spatiotemporal features of epileptic EEG signals, the spatiotemporal feature maps after filtering and aggregation are Embedded into the initial spatiotemporal features for reconstruction and generate the final filtered aggregate features The specific calculation is shown in Eq.

[0089] (twenty one).

[0090] in, Represents the Hadamard product operation.

[0091] Step 4. Considering that the prior knowledge contains rich low-level semantic details, the original spatiotemporal features are fused With the filtered aggregate features Splicing is performed to obtain the final features after processing by the spatiotemporal feature selection aggregation module , the calculation process is shown in Eq.

[0092] .

[0093] 4. Reconstructed loss function: To ensure optimal feature representation for the epilepsy EEG signal recognition network designed in this paper, a new weighted loss function, L, was designed. Using different loss functions, each branch undergoes independent supervised learning to prevent the recognition network from falling into local optima while also balancing the differences between different classes. The specific calculation of the weighted loss function is shown in the equation.

[0094] (twenty two).

[0095] in, represents a learnable balancing factor. It represents the main cross entropy loss function of multi-classification, and its specific calculation is shown in the equation.

[0096] (twenty three).

[0097] in, The label of the true category, represents the predicted label, Represents the total number of samples. The loss function of the spatial feature extraction branch is The calculation of is shown in Eq.

[0098] (twenty four).

[0099] in, Represents the predicted probability of the sample. δ represents a learnable hyperparameter. The loss function of the temporal feature extraction branch is The loss function of the spatial feature extraction branch is multi-classification Focal loss.

[0100] In summary, the designed weighted loss function plays a balancing role between different categories and different features, and adopts this individually supervised learning strategy to enhance the representation of spatiotemporal features. At the same time, the interaction between spatiotemporal features in epileptic EEG signals is strengthened through the individually supervised spatiotemporal feature selection and aggregation module, and the differences between different epileptic EEG signals are more effectively captured, thereby improving the overall performance of the recognition framework.

[0101] 5. Process and Overview: Figure 4 The spatiotemporal feature selection and aggregation framework for epileptic EEG signal recognition proposed in (a) above is presented. This framework consists of three main components: a spatial feature extraction module (SFEM), a temporal feature extraction module (TFEM), and a spatiotemporal feature selection and aggregation module (STFSAM). The spatial feature extraction module (SFEM) uses an improved WaveNet to learn the multi-scale local spatial details of epileptic EEG signals and establish effective correlations between features at different scales. Furthermore, a linear channel embedding module (LCEAM) establishes dependencies between spatial feature channels and emphasizes the representation of significant features through a weighted distribution strategy. The temporal feature extraction module (TFEM) utilizes an improved convolutional long short-term memory network to explore the contextual semantic details of epileptic EEG signals from both positive and negative directions. It embeds local features within global information through dilated convolutions. Furthermore, a linear positional attention module is used to reduce the transfer of redundant information between layers and effectively alleviate problems such as gradient explosion and vanishing gradients. The spatiotemporal feature selection and aggregation module (STFSAM) can alleviate the distributional differences between spatial and temporal features and complement them to improve the recognition performance of epileptic EEG signals. The recognition process of epileptic EEG signals is as follows:

[0102] Step 1. Use the spatial feature extraction module (SFEM) to obtain local spatial features and refined features The global context semantic features can be obtained through the Temporal Feature Extraction Module (TFEM) and refined features .

[0103] Step 2. In the feature screening and aggregation stage, this study invented the use of spatiotemporal feature selection and aggregation module (STFSAM) multiple times to reduce the differences in the distribution of different features and achieve hierarchical screening and aggregation (see Figure 4 (a) of STFSAM) to further promote the proposed recognition framework to achieve the best performance. The level-by-level feature selection and aggregation operations are shown in Eq.

[0104] (25).

[0105] in, Represents feature splicing.

[0106] Figure 4 (a) The overall network structure of the epileptic EEG signal recognition framework for spatiotemporal feature selection aggregation. Figure 4 (b) Represents the improved convolutional long short-term memory network in the temporal feature extraction module. Figure 4 (c) shows the improved WaveNet multi-scale dilated causal convolutional unit in the spatial feature extraction branch (SFEB). LCEAM represents the linear channel embedding attention module, LPEAM represents the linear position embedding attention module, and STFSAM represents the spatiotemporal feature selection aggregation module. “Outputs” represents the generated recognition results. and represents the input epileptic EEG signal, H and T represent the input spatial high dimension and temporal dimension. Representational spatial embedding module. Represents the temporal embedding module. The red, blue, and green lines represent the information transfer process of different features.

[0107] Step 3. In order to achieve accurate recognition of epileptic EEG signals, the present invention selects and aggregates the feature maps after spatiotemporal feature selection. With feature map The layers are then fused layer by layer, processed using standard 1×1 convolutions and a series of consecutive linear layers. This ultimately enables accurate recognition of epileptic EEG signals. The specific operation is shown in the equation.

[0108] (26).

[0109] Among them, + means adding the corresponding elements in the feature matrix. represents a continuous linear operation, represents the Softmax classifier, represents the final feature expression, The operation process and training process of the proposed epilepsy EEG signal recognition framework are as follows: Figure 5 shown.

[0110] 6. Experiments and Discussion The following content first describes the experimental data sources and related preprocessing, the environment and experimental parameters, and the evaluation metrics. Secondly, experimental results and analysis are provided in comparison with other methods. Finally, a series of ablation experiments were conducted to demonstrate the positive contribution of each module in the proposed epileptic EEG signal recognition framework to the overall performance.

[0111] 6.1 Data Preparation: The TUSZ dataset, derived from historical data from Temple University Hospital (TUH), includes 3,050 seizure events from over 300 patients. The dataset includes seven different event types: ABSZ (Absence Seizure), SPSZ (Simple Partial Seizure), TCSZ (Tonic-Clonic Seizure), TNSZ (Tonic Seizure), CPSZ (Complex Partial Seizure), GNSZ (Generalized Non-Specific Seizure), and FNSZ (Focal Non-Specific Seizure). To ensure smooth experimentation, we selected 4,626 events of 20 × 512 size from four well-balanced categories: ABSZ, SPSZ, TCSZ, and TNSZ.

[0112] CHBMIT: This dataset is an MIT EEG database containing epilepsy EEG signals from Boston Children's Hospital. It consists of scalp EEG recordings from children with intractable epilepsy. It contains 23 recordings from 22 subjects.

[0113] 6.2 Related settings: 6.2.1 Evaluation indicators: To ensure fairness and consistency across experiments, this paper uses four commonly used evaluation metrics: overall accuracy (OA), precision (P), recall (R), specific effectiveness (SPE), and the comprehensive evaluation index F-score to measure the performance of all experimental methods. The calculation of these metrics is shown in the equations.

[0114] (27).

[0115] (28).

[0116] (29).

[0117] (30).

[0118] Among them, FN means that the sample is judged as a negative sample, but is actually a positive sample. FP means that the sample is judged as a positive sample, but is actually a negative sample. TN means that the sample is judged as a negative sample, but is actually a negative sample. TP means that the sample is judged as a positive sample, but is actually a positive sample.

[0119] 6.2.2 Parameter settings: During the experimental training phase, the present invention sets the number of training times to 100, the batch size to 160, the learning rate to 1e-4, uses AdamW as the optimizer to adjust the recognition framework, and sets the decay rate to 1e-5. Secondly, the cosine annealing algorithm is used to dynamically adjust the learning rate.

[0120] All experiments in this paper were conducted in the same environment, using deep learning libraries such as python 3.8.19, cu117, torch 1.13.1 + cu117, and numpy, and were trained and tested on an NVIDIA GeForce RTX 4090 equipped with 8 graphics cards.

[0121] An embodiment of the present invention provides a system for identifying spatiotemporal features of EEG signals, comprising: The signal acquisition module is used to obtain EEG signals that reflect epileptic brain activity.

[0122] The spatial feature acquisition module is used to input the EEG signal into the feature embedding module to obtain the low-dimensional features of the EEG signal; the low-dimensional features are input into the improved waveform network to extract the multi-scale local spatial features of the EEG signal; the multi-scale local spatial features are screened through the linear channel embedding attention module to obtain the spatial features of the EEG signal.

[0123] The temporal feature acquisition module is used to input the EEG signal into the temporal feature embedding module to obtain the low-dimensional temporal features of the EEG signal; the low-dimensional temporal features are input into the improved convolutional long short-term memory network to capture the global features and contextual semantic details of the EEG signal; the captured results are input into the linear position embedding attention module to obtain the temporal features of the EEG signal.

[0124] The recognition module is used to use spatial features and temporal features as spatiotemporal features of EEG signals.

[0125] A specific embodiment is as follows: This embodiment discloses a method for identifying spatiotemporal features of EEG signals, and the specific steps are as follows: S1. An improved WaveNet method is used to extract multi-scale local spatial features of epileptic EEG signals to make up for the problem of insufficient representation of single structural spatial features. A newly designed linear channel embedding attention module is used to refine the multi-scale features along the channel and establish dependencies between channels to reduce the use of redundant information and highlight the representation of significant features.

[0126] S2. An improved convolutional long short-term memory network models epileptic EEG signals from both positive and negative directions, acquiring effective global features and contextual semantics. The use of a dilated convolutional network in this module mitigates the shortcomings of standard convolutions, which suffer from insufficient receptive field and are unable to effectively model local details. Furthermore, a newly designed linear position embedding attention module positionally encodes temporal epileptic EEG signals and refines temporal features to reduce the transfer and use of redundant information between layers.

[0127] S3. In order to alleviate the differences between different features and establish a complementary relationship between them, a newly designed spatiotemporal feature selection and aggregation module is used to fuse temporal features and spatial features layer by layer. At each level of feature fusion, screening and re-aggregation are used to reduce differences while improving the representation of significant features, thereby improving the recognition performance of epileptic EEG signals. The specific process is as follows: Figure 6 shown.

[0128] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A method for identifying spatiotemporal features of EEG signals, characterized in that: The following steps are involved: Obtaining EEG signals that reflect epileptic brain activity; The EEG signal input features are embedded into the module to obtain low-dimensional features of the EEG signal; the low-dimensional features are input into the improved waveform network to extract the multi-scale local spatial features of the EEG signal; The linear channel embedding attention module is used to filter multi-scale local spatial features to obtain the spatial characteristics of EEG signals; The EEG signal is input into the time series feature embedding module to obtain the low-dimensional time series features of the EEG signal; The low-dimensional temporal features are input into the improved convolutional long short-term memory network to capture the global features and contextual semantic details of the EEG signal; the captured results are input into the linear position embedding attention module to obtain the temporal features of the EEG signal; Spatial features and temporal features are used as spatiotemporal features of EEG signals.

2. The method for identifying spatiotemporal features of an EEG signal according to claim 1, wherein: The step of inputting the low-dimensional features into the improved waveform network to extract the multi-scale local spatial features of the EEG signal specifically includes: Input low-dimensional features into the causal convolution layer to extract low-level spatial features; Inputting the low-level spatial features into an improved dilated causal convolutional unit; wherein the improved dilated causal convolutional unit is composed of three dilated causal convolutional layers, two standard convolutional layers and a gated activation unit; The results obtained by multiple improved dilated causal convolution units are summed using residual and skip connections, and long-term dependencies are established between the results obtained by multiple improved dilated causal convolution units through gated activation units to obtain fused high-level causal convolution features; The fused high-level causal convolutional features are input into consecutive standard convolutional layers of different scales to obtain multi-scale local spatial features of EEG signals.

3. The method for identifying spatiotemporal features of an EEG signal according to claim 1, wherein: The method uses the linear channel embedding attention module to filter multi-scale local spatial features to obtain the spatial features of the EEG signal. The specific steps include: Multi-scale local spatial features Input linear channels are embedded into the attention module to obtain the spatial features of EEG signals , the formula is: ； in, represents the feature mapping operation, represents the Hadamard product; Linear channel attention coefficient in low dimension , the formula is: ； in, represents the activation function of sigmoid, represents a linear layer, Represents the activation function of ReLu.

4. The method for identifying spatiotemporal features of an EEG signal according to claim 1, wherein: The low-dimensional temporal features are input into the improved convolutional long short-term memory network to capture the global features and contextual semantic details of the EEG signal. The specific steps include: Low-dimensional time series features Input the improved convolutional long short-term memory network and obtain the t Output features of the dilated convolutional long short-term memory network with positive time-of-flight; ； Get the first t Output features of the time-reversed dilated convolutional long short-term memory network; ； Among them, * represents the dilated convolution operation; represents the Hadamard product operation, represents the activation function of ReLu, represents the activation function of tanh; Respectively t Input gates in both positive and negative directions at all times, Respectively t Memory units that are always in both positive and negative directions, Respectively t Hidden state in both positive and negative directions at all times, Respectively t There are two reverse output gates at all times. Respectively t The forgetting gate in both positive and negative directions at all times, and Represents the weight matrices related to the input gate of the input features, the previous layer hidden state, and the previous layer memory unit in the positive and negative directions, respectively. Represents the weight matrices related to the input features, the previous layer hidden state, and the forget gate of the previous layer memory unit in the positive and negative directions respectively; Represents the weight matrices related to the input features, the previous layer hidden state, and the previous layer memory units in the positive and negative directions, respectively. Represents the weight matrices related to the input features, the hidden state of the previous layer, and the output gate of the memory unit of the previous layer in the positive and negative directions respectively. and Respectively t -1 moment hidden state in both positive and negative directions, and Respectively t -1 moment memory units in both positive and negative directions, represents the input gate bias, represents the forget gate bias, represents the memory cell bias, Represents the output gate bias.

5. According to the output characteristics of the forward dilated convolutional long short-term memory network and the output characteristics of the reverse dilated convolutional long short-term memory network, the output characteristics of the improved convolutional long short-term memory unit are obtained. , used to capture the global features and contextual semantic details of EEG signals: 。 6. The method for identifying spatiotemporal features of an EEG signal according to claim 1, wherein: The capture result is input into the linear position embedding attention module to obtain the temporal characteristics of the EEG signal. The specific steps include: Will capture the results Input linear position embedding attention module to obtain the temporal characteristics of EEG signal , the formula is: ； in, Represents feature matrix multiplication; Indicates that along the time series Normalized exponential function SoftMax operation; Represents the average pooling operation, which is along the channel in the time series EEG signal The averaging operation; Represents a linear operation.

7. A method for identifying spatiotemporal features of EEG signals according to claim 3 or 5, characterized in that: The signal feature recognition method further includes: The temporal characteristics of EEG signals and the spatial characteristics of EEG signals Align and splice to obtain fusion features with complementary information , the formula is: ； in, represents the feature splicing along the channel direction; Two one-dimensional decomposition dilated convolution layers with different selected kernels are used to fusion features Perform display decomposition, the formula is: ； in, and They represent the one-dimensional decomposition expansion convolution operation with different selection kernels; the one-dimensional decomposition expansion convolution kernel size is and ; The decomposed fusion features are spliced ​​together, and one-dimensional maximum pooling and average pooling are used to obtain the attention feature map; The decomposed fusion features are mask-weighted, and the mask-weighted fusion features are mapped to the attention feature map through a standard convolutional layer to obtain the mapping features; Use Hadamard product to reconstruct the mapped features and the decomposed fusion features to obtain the filtered aggregated features; Combine the filtered aggregate features with the fusion features Perform splicing.

8. A spatiotemporal feature recognition system for EEG signals, characterized in that: include: Signal acquisition module, used to obtain EEG signals reflecting epileptic brain activity; The spatial feature acquisition module is used to embed the EEG signal input features into the module to obtain the low-dimensional features of the EEG signal; the low-dimensional features are input into the improved waveform network to extract the multi-scale local spatial features of the EEG signal; The linear channel embedding attention module is used to filter multi-scale local spatial features to obtain the spatial characteristics of EEG signals; A time series feature acquisition module is used to input the EEG signal into the time series feature embedding module to obtain the low-dimensional time series features of the EEG signal; The low-dimensional temporal features are input into the improved convolutional long short-term memory network to capture the global features and contextual semantic details of the EEG signal; the captured results are input into the linear position embedding attention module to obtain the temporal features of the EEG signal; The recognition module is used to use spatial features and temporal features as spatiotemporal features of EEG signals.

Images