A neural network method for brain electrical source imaging based on local-global temporal information integration

Abstract

This invention proposes a brain source imaging method based on the integration of local and global temporal information. This method establishes a direct mapping from cerebrospinal EEG to intracranial activity through supervised learning. It uses a temporal convolutional network to mine local temporal information from EEG signals and a bidirectional long short-term memory network to mine global temporal information. Simultaneously, a multi-head attention mechanism is introduced to extract key local and global information. Fully connected layers are used to integrate the local and global features of the model, achieving effective estimation of intracranial source activity. Based on deep learning, this invention integrates local and global information from input cerebrospinal signals to establish a mapping between EEG and intracranial signals, avoiding the sensitivity to prior conditions of classical source imaging methods. Compared to classical source imaging methods, it has stronger noise resistance and can capture deep source imaging that classical methods cannot achieve. This method is of great significance for high spatiotemporal resolution exploration based on EEG, especially for the exploration of deep activity.

Description

Technical Field

[0001] This invention belongs to the field of brain power imaging, and specifically relates to a brain power imaging neural network method based on the integration of local and global temporal information. Background Technology

[0002] Electroencephalography (EEG) is one of the most influential methods for recording brain activity. It generates postsynaptic potentials from a large number of neurons synchronously during brain activity and is recorded in real-time by electrodes placed on the scalp. It offers advantages such as being non-invasive and having high temporal resolution. However, reliable analysis of EEG at high spatial resolution is challenging due to the unavoidable volumetric effects of the skull and noise from various sources such as head movements, muscle movements, and blinking, especially for source activity located deep within the brain. Therefore, it is crucial to demystify the skull and probe the rapidly changing activity characteristics within the brain. Electroencephalography (EEG) is a technique aimed at achieving high spatiotemporal resolution understanding of the brain. Its core is the inverse solution of EEG, reversing the scalp signal to the cortical source space. This inverse solution process is called the inverse problem process; however, the inverse problem is often pathological because the number of sources in the cortical space is far greater than the number of scalp electrodes, and various source configurations in the cortex may produce the same topology. Traditional EEG analytical methods typically solve underdetermined problems by imposing various physical or mathematical prior constraints. However, these methods suffer from problems such as discrepancies between prior assumptions and reality, often resulting in unstable results due to noise, source configuration, and other issues. The located sources typically tend to be superficial activities, falling short in resolving source activities in deep structures. In recent years, deep neural networks, with their superior performance, have been applied to various problems. This paper proposes a neural network method for brain energy imaging based on the integration of local and global temporal information. This method primarily uses temporal convolutional networks and bidirectional long short-term memory networks to extract and integrate local and global temporal information from EEG signals recorded from the cephalogram (CSE) to achieve precise localization of cortical source activities. In this method, a temporal convolutional network is used to extract local temporal information from the CSE input, while a bidirectional long short-term memory network extracts global temporal information. Subsequently, based on the attention mechanism, key local and global information from the input EEG signal is extracted and integrated to achieve accurate estimation of cortical source activities. This method breaks free from the constraints of prior conditions in classical brain energy imaging methods, estimating real cortical activity in a data-driven manner, which is of great significance for high spatiotemporal resolution understanding of intrinsic brain activity (especially deep activity). Summary of the Invention

[0003] To address the problems of traditional brain power imaging methods, such as assumption constraints, inaccurate localization results, and difficulty in capturing deep activity, this invention proposes a brain power imaging neural network method based on the integration of local and global temporal information.

[0004] This invention proposes a brain power imaging neural network method based on the integration of local and global temporal information, comprising the following steps:

[0005] Step S1: The preprocessing steps for the EEG signals acquired from the head scale are as follows:

[0006] The EEG signal was rereferenced using reference electrode normalization technology, and noise components were filtered out using bandpass filtering. The signal was downsampled, segmented, and artifact-containing signal segments were removed.

[0007] Step S2: Using the mne-python package, generate a corresponding simulated head model based on the lead system and electrode names used to collect EEG signals. Simulate and generate surface EEG signal data and intracranial cortical signals, constructing sample pairs (surface EEG signals, intracranial cortical signals) as a dataset.

[0008] Step S3: Construct a neural network model for brain power imaging based on the integration of local and global temporal information, and divide the dataset obtained in step S2 into a training set and a validation set. Use the training set to perform supervised training on the model: use the adaptive moment estimation Adam optimizer to calculate the fitting loss function between the fitted intracranial cortical signal and the actual intracranial cortical signal, and update the parameters of the neural network. Use the validation set to evaluate the performance of the template after each training and update and save the optimal model parameters.

[0009] Step S4: Import the optimal neural network parameters determined after training, process the cerebrospinal EEG signals to be analyzed using the method in step S1, and input the processed cerebrospinal EEG signals into the trained neural network to obtain intracranial cortical signals.

[0010] Step S5: Using the triangular division information of the brain surface obtained by the positive model and the three-dimensional coordinate information of each cortical source in the cranium, the intracranial cortical signal output by the neural network is plotted on the three-dimensional brain map to complete brain power imaging.

[0011] Furthermore, the steps in step S2 for generating electroencephalogram (EEG) signals and intracranial cortical signals are as follows:

[0012] Step S21: Based on the lead system used to collect EEG signals and the electrode channels used, create the MNE information for the corresponding lead combination;

[0013] Step S22: Based on the MNE information, create a source space using the FreeSurfer fsaverage template. Based on the obtained source space and the MNE information, build a forward model.

[0014] Step S23: Generate simulated intracranial cortical signal data using a forward model. Where m represents the number of cortical sources and T represents the number of time points;

[0015] Step S24: Use the obtained intracranial cortical signals to positively model and obtain the electroencephalogram (EEG) signals of the head surface. The specific process is as follows: Calculate the EEG signals of the head surface using V = LX + E. in This is represented as the transmission matrix between intracranial cortical signals and surface EEG signals, where n represents the number of surface EEG channels, m represents the number of cortical sources, and T represents the number of time points. Indicates intracranial cortical signal, Indicates noise.

[0016] Furthermore, the brain power imaging neural network model based on local-global temporal information integration in step S3 is constructed as follows:

[0017] The neural network is mainly composed of a spatial-temporal-temporal-spatial template sequence; the first-layer spatial module learns and mines the spatial information of EEG signals, increasing the number of spatial features; the two-layer temporal modules mine the local-global temporal features of EEG signals; the second-layer spatial module learns the spatial characteristics of signal transmission and infers the temporal activity signals of the sending source from the received source signal to obtain intracranial signals.

[0018] Furthermore, the training process of the brain power imaging neural network model based on the integration of local and global temporal information in step S3 is as follows:

[0019] Step S31: Divide the dataset into a training set and a validation set, set the number of iterations and initialize the model hyperparameters;

[0020] Step S32: Standardize the EEG signals and intracranial signals in the training set separately. Perform channel-based Z-score standardization on the EEG signals: Where V represents the original EEG signal, V * The EEG signal is represented by the standardized value, μ represents the mean of the original dataset, and σ represents the standardization of the original dataset, which standardizes the original dataset to a mean of 0 and a variance of 1. Channel-based normalization is performed on the intracranial signals. Where X represents the original intracranial signal, X * This represents the normalized intracranial signal;

[0021] Step S33: Input the processed data into the neural network to obtain the fitted intracranial signal. The loss function is: Where Bsize represents the training batch size, T represents the time series length of the simulated signal, m represents the number of cortical sources, and X i,j,zThis represents the i-th cortical source data of the j-th time series of the z-th intracranial signal sample in the current batch simulation. This represents the i-th cortical source data of the j-th time series of the z-th fitted intracranial signal sample output by the neural network;

[0022] Step S34: After each iteration of training, the validation set is input into the current neural network to obtain the fitted intracranial signal. The loss function is calculated based on the corresponding simulated intracranial signal. If the loss value is less than the recorded minimum validation loss value, the current neural network model parameters are saved, and the minimum validation loss value is set as the current loss value.

[0023] Step S35: Repeat steps S33 and S34 for each iteration of training, and complete the initial set number of iterations for 3 iterations to complete the training.

[0024] Furthermore, the first spatial module in the brain power imaging neural network model based on local-global temporal information integration in step S3 is constructed as follows:

[0025] It consists of a fully connected layer, a Tanh layer, a random deactivation layer, and a multi-head attention module. The random deactivation layer sets the deactivation ratio of neurons (which only works during training). The multi-head attention module integrates multiple different single attention parts, then splices the multiple different attention distributions through a splicing layer, and then integrates them through a fully connected layer, and superimposes them with the features of the original input information.

[0026] Furthermore, the single attention distribution calculation involves three fully connected layers that respectively obtain the Q, K, and V vectors corresponding to the input. The attention distribution is then obtained using a scaled dot product operation. in Represents the query vector. Represents the key vector. Let m represent the value vector, T represent the total dimension of the model, and T represent the time series. These three vectors are the input vectors and are respectively coupled to three matrices W. Q W K W V Multiplying them together, we get d k Let K represent the dimension of the vector K, and T represent the length of the time series.

[0027] Furthermore, the operation of the Tanh layer: Where x represents the input data.

[0028] Furthermore, the first temporal module in the brain power imaging neural network model based on local-global temporal information integration in step S3 is constructed as follows:

[0029] It consists of a variant temporal convolutional network, a convolutional network module, and a multi-head attention module. The variant temporal convolutional network is composed of multiple residual modules. The input sequence is processed in the residual module as follows: the input sequence is processed by an expanded convolutional layer, a weight normalization layer, and an activation function layer to obtain two feature encoding representations. Then, the corresponding element-wise multiplication of the two feature encoding representations is performed. The result is processed by a random deactivation layer and a one-dimensional convolutional layer. The input and the obtained output are summed to obtain the output of the residual module. The convolutional network module consists of a ReLU layer, a one-dimensional convolutional layer, a ReLU layer, a one-dimensional convolutional layer, and a Tanh layer.

[0030] Furthermore, the implementation of the dilated convolutional layer is as follows: the equivalent kernel size formula is: k′=k+(k-1)×(d-1), where k represents the kernel size and d represents the dilation rate; the recursive relationship of the receptive field of the next layer is: RF i+1 =RF i +(k′ i -1)×S i , Among them, Stride i S represents the step size of the i-th layer. i-1 This represents the product of the strides of the first i-1 layers. To ensure that the input and output sequences have the same length, the sequence is padded with zeros before the convolution operation, with a padding length of d, and then the regular one-dimensional sequence convolution operation is performed.

[0031] Furthermore, the implementation of the weight normalization layer: Where V represents the original weights, which is a parameter vector, and g represents a parameter scalar;

[0032] Furthermore, the one-dimensional convolutional layer is implemented as follows: C represents the bias at the j-th position of the output of a one-dimensional convolutional layer. in The length of the input data vector. This represents the weight of the k-th position in the input data and the j-th position in the output data, where input(N) represents the weight of the k-th position in the input data and the j-th position in the output data. i (,k) represents the data at the k-th position of the input data. This indicates the data at the j-th position of the output data;

[0033] Furthermore, the operation of the ReLU layer is: ReLU(x) = max(0,x), where x represents the input data.

[0034] Furthermore, the second-layer temporal module in the brain power imaging neural network model based on local-global temporal information integration in step S3 is constructed as follows:

[0035] It consists of a bidirectional long short-term memory network and a multi-head attention module in sequence. The bidirectional long short-term memory network is composed of a forward long short-term memory network and a backward long short-term memory network. The outputs of the two networks are spliced ​​together to form the output of the bidirectional long short-term memory network.

[0036] Furthermore, the structure of the Long Short-Term Memory network is as follows:

[0037] Long Short-Term Memory (LSTM) networks have four core components: forget gate, input gate, cell renewal, and output gate;

[0038] Among them, the forget gate operation:

[0039] Selectively forget the data passed from the previous node: f t =σ(W f ·h t-1 +U f ·x t +b f ), where W f U represents the weight coefficient of the forget gate for the hidden state of the previous node. f b represents the weighting coefficient of the forget gate on the current input. f h represents the bias value of the forget gate. t-1 X represents the hidden state at the previous time step. t f represents the input at the current moment. t This represents the probability distribution of the forget gate at the current moment;

[0040] Input gate operation:

[0041] Store the important information input at the current moment into the cell state: i t =σ(W i ·h t-1 +U i ·x t +b i ), where W i U represents the weight coefficient of the input gate relative to the hidden state of the previous node. i b represents the weighting coefficient of the input gate with respect to the input at the current time step. i Indicates the bias value of the input gate, i t This represents the probability distribution of the input gates at the current moment; Among them W C U represents the weight coefficient of feature extraction with respect to the hidden state of the previous node. C b represents the weighting coefficient of the feature extraction on the input at the current time step. C This represents the bias value for feature extraction. This represents the candidate value for the cell state at the current moment;

[0042] Cell renewal operations include:

[0043] Calculate the cell state at the current moment: Where f t Let C represent the probability distribution of the forget gate at the current time. t-1 Indicates the cell state at the previous time step, i t This represents the probability distribution of the input gate at the current time. This represents the candidate cell state value at the current time step. The cell state at the previous time step is multiplied by the probability distribution of the forget gate at the current time step, discarding some information from the previous sequence; the candidate cell state value at the current time step is multiplied by the probability distribution of the input gate at the current time step, retaining the current input information.

[0044] Output gate operations include:

[0045] Calculate how many cell states are output to the next time step: o t =σ(W o ·h t-1 +U o ·x t +b o ), where W o U represents the weight coefficient of the output gate relative to the hidden state of the previous node. o b represents the weighting coefficient of the output gate with respect to the input at the current time step. o This represents the bias value of the output gate, o t This represents the output gate probability distribution at the current moment; The current input is filtered by the previous hidden state and multiplied by the mapped current cell state to obtain the current hidden state, which is then output.

[0046] Furthermore, the second spatial module in the neural network structure is as follows:

[0047] The second spatial module includes a fully connected layer, which mainly learns the spatial characteristics of signal transmission and infers the temporal activity signals of the sending source from the received source signal to obtain the intracranial cortical signal.

[0048] Secondly, this invention proposes a brain power imaging system based on the integration of local and global temporal information, comprising:

[0049] Preprocessing module: Preprocesses the electroencephalogram (EEG) signals from the head surface collected by the sensors to be analyzed;

[0050] Data generation module: simulates electroencephalogram (EEG) signals and intracranial signals to provide samples for training neural networks;

[0051] Training module: Inputs simulated EEG signal training samples into the neural network to obtain fitted intracranial signals. Calculate the intracranial signal X generated by simulation and the fitted intracranial signal. The cosine similarity and gradient are calculated, and the Adam optimizer is used to update the model parameters. After each round of training, the validation set is used to validate and save the optimal parameters of the neural network.

[0052] Imaging module: Using a trained neural network, the collected electroencephalogram (EEG) signals from the head surface are input to obtain the corresponding intracranial signals. Using the coordinate information of each point in the intracranial cavity and the obtained signals, the intracranial signals are mapped onto the brain image to complete the imaging.

[0053] The advantages of this invention are:

[0054] The temporal convolutional network enables the neural network to learn the temporal local information of EEG signals, the bidirectional long short-term memory network enables the network to learn the temporal forward global information and temporal backward global information of EEG signals, and the multi-head attention module can simultaneously focus on multiple key points, effectively integrating global and local information to obtain accurate source imaging results. Attached Figure Description

[0055] Figure 1 This is a schematic diagram illustrating the steps of a neural network method for brain power imaging based on the integration of local and global temporal information, as provided by the present invention.

[0056] Figure 2 This invention provides a schematic diagram of a multi-head attention module in a neural network structure.

[0057] Figure 3 This invention provides a schematic diagram of the first spatial module in a neural network structure.

[0058] Figure 4 This invention provides a schematic diagram of the first-layer time module in a neural network structure.

[0059] Figure 5 This invention provides a schematic diagram of the second-layer time module in a neural network structure.

[0060] Figure 6 This invention provides a schematic diagram of the second-layer spatial module in a neural network structure.

[0061] Figure 7 This is a comparison of the deep brain power source imaging results of this method on simulated data with the results of the classical source imaging method.

[0062] Figure 8 The bar chart compares the brain power imaging results of this method on simulated data with those of the classical source imaging method on three indicators: mean localization error, normalized mean square error, and area under the receiver operating characteristic curve (AUC). Detailed Implementation

[0063] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0064] In a first aspect, according to embodiments of the present invention, a brain power imaging neural network method based on the integration of local-global temporal information is proposed. (See also...) Figure 1 This includes the following steps:

[0065] Step S1: The EEG signals collected from the head table are preprocessed as follows: the signal is rereferenced using reference electrode normalization technology, the signal noise is filtered out using bandpass filtering, the signal is downsampled, the signal is segmented and sliced, and signal segments containing artifacts are removed.

[0066] Step S2: Using the mne-python package, generate a corresponding simulated head model based on the lead system and electrode names used to collect EEG signals. Simulate and generate EEG signals and intracranial cortical signals to construct a dataset consisting of (EEG signals and intracranial cortical signals).

[0067] Step S3: Construct a neural network model for brain power imaging based on the integration of local and global temporal information, and divide the dataset obtained in step S2 into a training set and a validation set. Use the training set to perform supervised training on the model; use the adaptive moment estimation (Adam) optimizer to calculate the fitting loss function between the fitted intracranial cortical signal and the actual intracranial cortical signal, and update the parameters of the neural network; use the validation set to evaluate the performance of the model after each training, and update and save the optimal model parameters.

[0068] Step S4: Import the optimal neural network parameters determined after training, process the cerebrospinal EEG signals to be analyzed using Step S1, and input the processed cerebrospinal EEG signals into the trained neural network to obtain intracranial cortical signals.

[0069] Step S5: Using the triangular division information of the brain surface obtained by the positive model and the three-dimensional coordinate information of each cortical source in the cranium, the intracranial cortical signal output by the neural network is plotted on the three-dimensional brain map to complete brain power imaging.

[0070] Furthermore, step S1 specifically includes:

[0071] Step S11: Rereference the EEG signals using the Reference Electrode Standardization Technique (REST);

[0072] Step S12: Use a bandpass filter to remove noise interference;

[0073] Step S13: Downsample the EEG signal;

[0074] Step S14: Divide the continuously acquired potential signal into segments at equal intervals L in a non-overlapping manner;

[0075] Step S15: Remove artifact segments from the signal segments obtained in step S14. That is, compare each time point within the signal segment with the set artifact threshold one by one. If any time point of the segment exceeds the threshold, the signal segment is removed.

[0076] Furthermore, the steps in step S2 for generating electroencephalogram (EEG) signals and intracranial cortical signals are as follows:

[0077] Step S21: Based on the lead system used to collect EEG signals and the electrode channels used, create the MNE information for the corresponding lead combination;

[0078] Step S22: Based on the MNE information, create a source space using the FreeSurfer fsaverage template. Based on the obtained source space and the MNE information, build a forward model.

[0079] Step S23: Generate simulated intracranial cortical signal data using a forward model. Where m represents the number of cortical sources and T represents the number of time points;

[0080] Step S24: Use the obtained intracranial cortical signals to positively model and obtain the electroencephalogram (EEG) signals of the head surface. The specific process is as follows: Calculate the EEG signals of the head surface using V = LX + E. in This is represented as the transmission matrix between intracranial cortical signals and the head surface, where n represents the number of EEG channels on the head surface, m represents the number of cortical sources, and T represents the number of time points. Indicates intracranial cortical signal, Indicates noise.

[0081] Furthermore, the brain power imaging neural network model based on local-global temporal information integration in step S3 is constructed as follows:

[0082] The neural network is mainly composed of spatial-temporal-temporal-spatial modules in sequence. The first-layer spatial module learns and mines the spatial information of EEG signals, increasing the number of spatial features. The two-layer temporal modules mine the local-global temporal features of EEG signals. The second-layer spatial module learns the spatial characteristics of signal transmission and infers the temporal activity signals of the sending source from the received source signal to obtain intracranial signals.

[0083] Furthermore, the training process of the brain power imaging neural network model based on the integration of local and global temporal information in step S3 is as follows:

[0084] Step S31: Divide the dataset into a training set and a validation set, set the number of iterations and initialize the model hyperparameters (including the learning rate);

[0085] Step S32: Standardize the EEG signals and intracranial signals in the training set separately. Perform channel-based Z-score standardization on the EEG signals: Where V represents the original EEG signal, V * The EEG signal is represented by the standardized value, μ represents the mean of the original dataset, and σ represents the standardization of the original dataset, which standardizes the original dataset to a mean of 0 and a variance of 1. Channel-based normalization is performed on the intracranial signals. Where X represents the original intracranial signal, X * This represents the normalized intracranial signal;

[0086] Step S33: Input the processed data into the neural network to obtain the fitted intracranial signal. The loss function is: Where Bsize represents the training batch size, T represents the time series length of the simulated signal, m represents the number of cortical sources, and X i,j,z This represents the i-th cortical source data of the j-th time series of the z-th intracranial signal sample in the current batch simulation. This represents the i-th cortical source data of the j-th time series of the z-th fitted intracranial signal sample output by the neural network;

[0087] Step S34: After each iteration of training, the validation set is input into the current neural network to obtain the fitted intracranial signal. The loss function is calculated based on the corresponding simulated intracranial signal. If the loss value is less than the recorded minimum validation loss value, the current neural network model parameters are saved, and the minimum validation loss value is set as the current loss value.

[0088] Step S35: Repeat steps S33 and S34 for each iteration of training, and complete the initial set number of iterations for 3 iterations to complete the training.

[0089] Further, please refer to Figure 3 The first spatial module in the brain power imaging neural network model based on local-global temporal information integration in step S3 is constructed as follows:

[0090] It consists of a fully connected layer, a Tanh layer, a random deactivation layer, and a multi-head attention module. The random deactivation layer sets the deactivation ratio of neurons (which only works during training). The multi-head attention module integrates multiple different single attention parts, then splices the multiple different attention distributions through a splicing layer, and then integrates them through a fully connected layer, and superimposes them with the features of the original input information.

[0091] Further, please refer to Figure 2 The single attention distribution calculation involves three fully connected layers that obtain the Q, K, and V vectors corresponding to the input, and a scaled dot product operation is used to obtain the attention distribution. in Represents the query vector. Represents the key vector. Let m represent the value vector, T represent the total dimension of the model, and T represent the time series. These three vectors are the input vectors and are respectively coupled to three matrices W. Q W K W V Multiplying them together, we get d k Let K represent the dimension of the vector K, and T represent the length of the time series.

[0092] Furthermore, the operation of the Tanh layer: Where x represents the input data.

[0093] Further, please refer to Figure 4 The first temporal module in the brain power imaging neural network model based on local-global temporal information integration in step S3 is constructed as follows:

[0094] It consists of a variant temporal convolutional network, a convolutional network module, and a multi-head attention module. The variant temporal convolutional network is composed of multiple residual modules. The input sequence is processed in the residual module as follows: the input sequence is processed by an expanded convolutional layer, a weight normalization layer, and an activation function layer to obtain two feature encoding representations. Then, the corresponding element-wise multiplication of the two feature encoding representations is performed. The result is processed by a random deactivation layer and a one-dimensional convolutional layer. The input and the obtained output are summed to obtain the output of the residual module. The convolutional network module consists of a ReLU layer, a one-dimensional convolutional layer, a ReLU layer, a one-dimensional convolutional layer, and a Tanh layer.

[0095] Furthermore, the implementation of the dilated convolutional layer is as follows: the equivalent kernel size formula is: k′=k+(k-1)×(d-1), where k represents the kernel size and d represents the dilation rate; the recursive relationship of the receptive field of the next layer is: RF i+1 =RF i +(k′ i -1)×S i , Among them, Stride i S represents the step size of the i-th layer. i-1 This represents the product of the strides of the first i-1 layers. To ensure that the input and output sequences have the same length, the sequence is padded with zeros before the convolution operation, with a padding length of d, and then the regular one-dimensional sequence convolution operation is performed.

[0096] Furthermore, the implementation of the weight normalization layer: Where V represents the original weights, which is a parameter vector, and g represents a parameter scalar;

[0097] Furthermore, the one-dimensional convolutional layer is implemented as follows: C represents the bias at the j-th position of the output of a one-dimensional convolutional layer. in The length of the input data vector. This represents the weight of the k-th position in the input data and the j-th position in the output data, where input(N) represents the weight of the k-th position in the input data and the j-th position in the output data. i (,k) represents the data at the k-th position of the input data. This indicates the data at the j-th position of the output data;

[0098] Furthermore, the operation of the ReLU layer is: ReLU(x) = max(0,x), where x represents the input data.

[0099] Further, please refer to Figure 5 The second-layer temporal module in the brain power imaging neural network model based on local-global temporal information integration in step S3 is constructed as follows:

[0100] It consists of a bidirectional long short-term memory network and a multi-head attention module in sequence. The bidirectional long short-term memory network is composed of a forward long short-term memory network and a backward long short-term memory network. The outputs of the two networks are spliced ​​together to form the output of the bidirectional long short-term memory network.

[0101] Furthermore, the structure of the Long Short-Term Memory network is as follows:

[0102] Long Short-Term Memory (LSTM) networks have four core components: forget gate, input gate, cell renewal, and output gate;

[0103] Among them, the forget gate operation:

[0104] Selectively forget the data passed from the previous node: f t =σ(W f ·h t-1 +U f ·x t +b f ), where W f U represents the weight coefficient of the forget gate for the hidden state of the previous node. f b represents the weighting coefficient of the forget gate on the current input. f h represents the bias value of the forget gate. t-1 X represents the hidden state at the previous time step. t f represents the input at the current moment.t This represents the probability distribution of the forget gate at the current moment;

[0105] Input gate operation:

[0106] Store the important information input at the current moment into the cell state: i t =σ(W i ·h t-1 +U i ·x t +b i ), where W i U represents the weight coefficient of the input gate relative to the hidden state of the previous node. i b represents the weighting coefficient of the input gate with respect to the input at the current time step. i Indicates the bias value of the input gate, i t This represents the probability distribution of the input gates at the current moment; Among them W C U represents the weight coefficient of feature extraction with respect to the hidden state of the previous node. C b represents the weighting coefficient of the feature extraction on the input at the current time step. C This represents the bias value for feature extraction. This represents the candidate value for the cell state at the current moment;

[0107] Cell renewal operations include:

[0108] Calculate the cell state at the current moment: Where f t Let C represent the probability distribution of the forget gate at the current time. t-1 Indicates the cell state at the previous time step, i t This represents the probability distribution of the input gate at the current time. This represents the candidate cell state value at the current time step. The cell state at the previous time step is multiplied by the probability distribution of the forget gate at the current time step, discarding some information from the previous sequence; the candidate cell state value at the current time step is multiplied by the probability distribution of the input gate at the current time step, retaining the current input information.

[0109] Output gate operations include:

[0110] Calculate how many cell states are output to the next time step: o t =σ(W o ·h t-1 +U o ·x t +b o ), where W o Y represents the weight coefficient of the output gate relative to the hidden state of the previous node. o b represents the weighting coefficient of the output gate with respect to the input at the current time step. o This represents the bias value of the output gate, o tThis represents the output gate probability distribution at the current moment; The current input is filtered by the previous hidden state and multiplied by the mapped current cell state to obtain the current hidden state, which is then output.

[0111] Further, please refer to Figure 6 The second spatial module in the neural network structure is as follows:

[0112] The second spatial module includes a fully connected layer, which mainly learns the spatial characteristics of signal transmission and infers the temporal activity signals of the sending source from the received source signal to obtain the intracranial cortical signal.

[0113] Please see Figure 7 Figure 8 To compare the effectiveness of this method with that of classical EEG imaging methods, three evaluation metrics were used: mean localization error (MLE), normalized mean error (nMSE), and area under the receiver operating characteristic curve (AUC).

[0114] Therefore, this invention proposes a neural network method for brain power imaging based on the integration of local and global temporal information. By using signals collected from the head surface and a constructed neural network model, the local and global temporal information of the head surface EEG signals are learned in a supervised learning manner, the corresponding intracranial signals are directly estimated, and the intracranial signals are mapped onto the brain image to obtain brain power imaging results.

Claims

1. A brain energy imaging neural network method based on the integration of local-global temporal information, comprising the following steps: Step S1: The preprocessing steps for the EEG signals acquired from the head scale are as follows: The EEG signal was rereferenced using reference electrode normalization technology, and noise components were filtered out using bandpass filtering. The signal was downsampled, segmented, and artifact-containing signal segments were removed. Step S2: Using the mne-python package, generate a corresponding simulated head model based on the lead system and electrode names used to collect EEG signals. Simulate and generate surface EEG signal data and intracranial cortical signals, constructing sample pairs (surface EEG signals, intracranial cortical signals) as a dataset. Step S3: Construct a neural network model for brain power imaging based on the integration of local and global temporal information, and divide the dataset obtained in step S2 into a training set and a validation set. Use the training set to perform supervised training on the model: use the adaptive moment estimation Adam optimizer to calculate the fitting loss function between the fitted intracranial cortical signal and the actual intracranial cortical signal, and update the parameters of the neural network. Use the validation set to evaluate the performance of the template after each training and update and save the optimal model parameters. The neural network model for brain power imaging based on the integration of local and global temporal information is constructed as follows: The neural network is mainly composed of a spatial-temporal-temporal-spatial template sequence; the first-layer spatial module learns and mines the spatial information of EEG signals, increasing the number of spatial features; the two-layer temporal modules mine the local-global temporal features of EEG signals; The second-layer spatial module learns the spatial characteristics of signal transmission and infers the temporal activity signals of the sending source from the received source signal to obtain intracranial signals. Step S4: Import the optimal neural network parameters determined after training, process the cerebrospinal EEG signals to be analyzed using the method in step S1, and input the processed cerebrospinal EEG signals into the trained neural network to obtain intracranial cortical signals. Step S5: Using the triangular division information of the brain surface obtained by the positive model and the three-dimensional coordinate information of each cortical source in the cranium, the intracranial cortical signal output by the neural network is plotted on the three-dimensional brain map to complete brain power imaging.

2. The brain power imaging neural network method based on local-global temporal information integration as described in claim 1, characterized in that, The steps in step S2 for generating electroencephalogram (EEG) signals and intracranial cortical signals are as follows: Step S21: Based on the lead system used to collect EEG signals and the electrode channels used, create the MNE information for the corresponding lead combination; Step S22: Based on the MNE information, create a source space using the FreeSurfer fsaverage template. Based on the obtained source space and the MNE information, build a forward model. Step S23: Generate simulated intracranial cortical signal data using a forward model. Where m represents the number of cortical sources and T represents the number of time points; Step S24: Use the obtained intracranial cortical signals to positively model and obtain the electroencephalogram (EEG) signals of the head surface. The specific process is as follows: Calculate the EEG signals of the head surface using V = LX + E. in This is represented as the transmission matrix between intracranial cortical signals and surface EEG signals, where n represents the number of surface EEG channels, m represents the number of cortical sources, and T represents the number of time points. Indicates intracranial cortical signal, Indicates noise.

3. The brain power imaging neural network method based on local-global temporal information integration as described in claim 1, characterized in that, The training process of the brain power imaging neural network model based on the integration of local and global temporal information in step S3 is as follows: Step S31: Divide the dataset into a training set and a validation set, set the number of iterations and initialize the model hyperparameters; Step S32: Standardize the EEG signals and intracranial signals in the training set separately. Perform channel-based Z-score standardization on the EEG signals: Where V represents the original EEG signal, V * The EEG signal is represented by the standardized value, μ represents the mean of the original dataset, and σ represents the standardization of the original dataset, which standardizes the original dataset to a mean of 0 and a variance of 1. Channel-based normalization is performed on the intracranial signals. Where X represents the original intracranial signal, X * This represents the normalized intracranial signal; Step S33: Input the processed data into the neural network to obtain the fitted intracranial signal. The loss function is: Where Bsize represents the training batch size, T represents the time series length of the simulated signal, m represents the number of cortical sources, and X i,j,z This represents the i-th cortical source data of the j-th time series of the z-th intracranial signal sample in the current batch simulation. This represents the i-th cortical source data of the j-th time series of the z-th fitted intracranial signal sample output by the neural network; Step S34: After each iteration of training, the validation set is input into the current neural network to obtain the fitted intracranial signal. The loss function is calculated based on the corresponding simulated intracranial signal. If the loss value is less than the recorded minimum validation loss value, the current neural network model parameters are saved, and the minimum validation loss value is set as the current loss value. Step S35: Repeat steps S33 and S34 for each iteration of training, and complete the initial set number of iterations for 3 iterations to complete the training.

4. The brain power imaging neural network method based on local-global temporal information integration as described in claim 3, characterized in that, The first-layer spatial module is constructed as follows: It consists of a fully connected layer, a Tanh layer, a random deactivation layer, and a multi-head attention module. The random deactivation layer sets the deactivation ratio of neurons. The multi-head attention module integrates multiple different single attention parts, and then splices the multiple different attention distributions through a splicing layer, and then integrates them through a fully connected layer, and superimposes them with the features of the original input information. The first-level time module is constructed as follows: It consists of a variant temporal convolutional network, a convolutional network module, and a multi-head attention module. The variant temporal convolutional network is composed of multiple residual modules. The input sequence is processed in the residual module as follows: the input sequence is processed by an expanded convolutional layer, a weight normalization layer, and an activation function layer to obtain two feature encoding representations. Then, the corresponding element-wise multiplication of the two feature encoding representations is performed. The result is processed by a random deactivation layer and a one-dimensional convolutional layer. The input and the obtained output are summed to obtain the output of the residual module. The convolutional network module consists of a ReLU layer, a one-dimensional convolutional layer, a ReLU layer, a one-dimensional convolutional layer, and a Tanh layer. The second-layer time module is constructed as follows: It consists of a bidirectional long short-term memory network and a multi-head attention module in sequence. The bidirectional long short-term memory network is composed of a forward long short-term memory network and a backward long short-term memory network. The outputs of the two networks are spliced ​​together to form the output of the bidirectional long short-term memory network. The second-layer spatial module is as follows: The second spatial module includes a fully connected layer, which mainly learns the spatial characteristics of signal transmission and infers the temporal activity signals of the sending source from the received source signal to obtain the intracranial cortical signal.

5. The brain power imaging neural network method based on local-global temporal information integration as described in claim 4, characterized in that, In the first spatial module of the brain power imaging neural network model based on local-global temporal information integration in step S3, the calculation method of the single attention part is as follows: three fully connected layers respectively obtain the Q, K, and V vectors corresponding to the input, and the attention distribution is obtained by scaling dot product operation: in Represents the query vector. Represents the key vector. Let m represent the value vector, T represent the total dimension of the model, and T represent the time series. These three vectors are the input vectors and are respectively coupled to three matrices W. Q , W K , W V Multiplying them together, we get d k The dimension of the K vector is represented by T, and the length of the time series is represented by T. The operations of the Tanh layer: Where x represents the input data.

6. The brain power imaging neural network method based on local-global temporal information integration as described in claim 4, characterized in that, In step S3, during the construction of the first temporal module in the brain power imaging neural network model based on the integration of local and global temporal information, The implementation of the dilated convolutional layer: The equivalent kernel size formula is: k′=k+(k-1)×(d-1), where k represents the kernel size and d represents the dilation rate; the recursive relationship of the receptive field of the next layer is: RF i+1 =RF i +(k i ′-1)×S i , Among them, Stride i S represents the step size of the i-th layer. i-1 This represents the product of the strides of the first i-1 layers. To ensure that the input and output sequences have the same length, the sequence is padded with zeros before the convolution operation, with a padding length of d, and then the regular one-dimensional sequence convolution operation is performed. Implementation of the weight normalization layer: Where V represents the original weights, which is a parameter vector, and g represents a parameter scalar; The one-dimensional convolutional layer is implemented as follows: C represents the bias at the j-th position of the output of a one-dimensional convolutional layer. in The length of the input data vector. This represents the weight of the k-th position in the input data and the j-th position in the output data, where input(N) represents the weight of the k-th position in the input data and the j-th position in the output data. i (,k) represents the data at the k-th position of the input data. This indicates the data at the j-th position of the output data; The operation of the ReLU layer is: ReLU(x) = max(0,x), where x represents the input data.

7. The brain power imaging neural network method based on local-global temporal information integration as described in claim 4, characterized in that, In the second-layer temporal module of the brain power imaging neural network model based on local-global temporal information integration in step S3; The structure of the Long Short-Term Memory network is as follows: Long Short-Term Memory (LSTM) networks have four core components: forget gate, input gate, cell renewal, and output gate; Among them, the forget gate operation: Selectively forget the data passed from the previous node: f t =σ(W f ·h t-1 +U f ·x t +b f ), where W f U represents the weight coefficient of the forget gate for the hidden state of the previous node. f b represents the weighting coefficient of the forget gate on the current input. f h represents the bias value of the forget gate. t-1 X represents the hidden state at the previous time step. t f represents the input at the current moment. t This represents the probability distribution of the forget gate at the current moment; Input gate operation: Store the important information input at the current moment into the cell state: i t =σ(W i ·h t-1 +U i ·x t +b i ), where W i U represents the weight coefficient of the input gate relative to the hidden state of the previous node. i b represents the weighting coefficient of the input gate with respect to the input at the current time step. i Indicates the bias value of the input gate, i t This represents the probability distribution of the input gates at the current moment; Where W C U represents the weight coefficient of feature extraction with respect to the hidden state of the previous node. C b represents the weighting coefficient of the feature extraction on the input at the current time step. C This represents the bias value for feature extraction. This represents the candidate value for the cell state at the current moment; Cell renewal operations include: Calculate the cell state at the current moment: Where f t Let C represent the probability distribution of the forget gate at the current time. t-1 Indicates the cell state at the previous time step, i t This represents the probability distribution of the input gate at the current time. This represents the candidate value of the cell state at the current time step; the cell state at the previous time step is multiplied by the probability distribution of the forget gate at the current time step, discarding some information from the previous sequence; the candidate value of the cell state at the current time step is multiplied by the probability distribution of the input gate at the current time step, retaining the current input information; Output gate operations include: Calculate how many cell states are output to the next time step: o t =σ(W o ·h t-1 +U o ·x t +b o ), where W o U represents the weight coefficient of the output gate relative to the hidden state of the previous node. o b represents the weighting coefficient of the output gate with respect to the input at the current time step. o This represents the bias value of the output gate, o t This represents the output gate probability distribution at the current moment; The current input is filtered by the previous hidden state and multiplied by the mapped current cell state to obtain the current hidden state, which is then output.

8. A brain power imaging system employing the brain power imaging neural network method based on local-global temporal information integration as described in claim 1, comprising: Preprocessing module: Preprocesses the electroencephalogram (EEG) signals from the head surface collected by the sensors to be analyzed; Data generation module: simulates electroencephalogram (EEG) signals and intracranial signals to provide samples for training neural networks; Training module: Inputs simulated EEG signal training samples into the neural network to obtain fitted intracranial signals. Calculate the intracranial signal X generated by simulation and the fitted intracranial signal. The cosine similarity and gradient are calculated, and the Adam optimizer is used to update the model parameters. After each round of training, the validation set is used to validate and save the optimal parameters of the neural network. Imaging module: Using a trained neural network, the collected electroencephalogram (EEG) signals from the head surface are input to obtain the corresponding intracranial signals. Using the coordinate information of each point in the intracranial cavity and the obtained signals, the intracranial signals are mapped onto the brain image to complete the imaging.

Images