A spatio-temporal adaptive end-to-end brain network construction method based on graph neural network

Abstract

The present invention proposes an end-to-end spatio-temporal adaptive brain network construction method based on graph neural networks, including preprocessing electroencephalogram (EEG) signals; constructing an end-to-end model of a spatio-temporal adaptive neural network based on graph neural networks; loading the trained model parameters; inputting signals into the constructed model to estimate the directed brain network matrix; and setting a connection threshold to obtain significant directed connections. Based on sequence decomposition and recombination, the model enables the graph neural network model integrating long short-term memory based on time segment inputs to have spatio-temporal adaptability. The trained model can be used as a generalized and efficient neural relationship extractor to directly obtain the unique directed network patterns of different EEG signal instances (different numbers of channels, different signal lengths) in a data-driven manner. This avoids problems such as model and hypothesis constraints faced by traditional analytical methods, and at the same time expands the applicability of the end-to-end solution to different application scenarios (constructing brain networks under different durations and different numbers of sensor sources), which is of great significance for exploring the potential interaction mechanisms of brain networks, improving the reliability and repeatability of related research.

Description

Technical Field

[0001] The present invention belongs to the field of brain network construction, and particularly relates to a spatio-temporal adaptive end-to-end brain network construction method based on graph neural network. Background Art

[0002] Electroencephalogram (EEG) is a non-stationary signal with high temporal resolution. Mining the inherent directional interaction, spatio-temporal information mining, and long-term dependence information loss from non-stationary signals are the key issues to be solved. The common construction method is based on the multivariate autoregressive model, such as the Granger causality method. It defines the causal relationship based on the predictability between the past information and future information of the signal, establishes an autoregressive model for each input signal data respectively, and solves the corresponding model parameters to construct the corresponding directed brain network. Therefore, it not only faces model assumptions and hyperparameter setting limitations such as linearity and signal stationarity, but also the computational cost brought by the explicit solution of the single-instance model to a certain extent limits its applicability in a wider spatio-temporal adaptive scenario. Although many neural network-based solutions for constructing brain networks have been proposed in recent years, most of them are based on the idea of the Granger causality method. For example, Talebi et al. used an artificial neural network (ANN) to simulate the multivariate autoregressive model, and obtained the multivariate autoregressive model through the optimization training of the network, thereby constructing a brain network; used a recurrent neural network to extend the information flow process defined by the Granger causality method based on the fitting error, etc. These solutions have made up for the defects of the traditional model in constructing brain networks based on signal fitting to a certain extent, but their applicability is still limited by the framework.

[0003] The end-to-end solution is to mine general and extensive features based on a data-driven approach to construct a brain network, avoiding the process of separately modeling each input signal. However, different input signals may have spatio-temporal mismatch problems such as inconsistent time series ranges and different numbers of sensor sources. Constructing corresponding models for the adaptation problems of the same time series range and sensor sources obviously has problems such as waste of computing resources. Therefore, the present invention proposes an end-to-end spatio-temporal adaptive brain network construction method and system to improve the applicability of graph neural networks in constructing brain networks in application scenarios with inconsistent time series ranges and different numbers of sensor sources. Summary of the Invention

[0004] To avoid the problem of re-modeling for constructing brain networks with different instances and to expand the applicability of the end-to-end solution to different application scenarios (constructing brain networks with different durations and different numbers of sensor sources), the present invention proposes a spatio-temporal adaptive end-to-end brain network construction method based on graph neural network.

[0005] The technical solution of the present invention proposes a spatiotemporal adaptive end-to-end brain network construction method based on Graph Neural Network (GNN), which constructs the brain network through the following steps:

[0006] Step 1: Preprocess the multi-channel EEG signals to be analyzed to improve the signal-to-noise ratio;

[0007] Step 2: Load the trained graph neural network-based deep learning model parameters;

[0008] Step 3: Input the signal processed in step 1 into the constructed model to obtain a directed brain network matrix;

[0009] Step 4: Set the connection threshold to obtain significant directed connections;

[0010] The specific process of step 3 is as follows:

[0011] The multi-channel EEG signal is first encoded with temporal features by an encoder, and the learned features are then decoded by a decoder, and then a brain network matrix is output; the encoder includes: a temporal feature aggregation module and a system interaction encoding module, and the multi-channel EEG signal input to the encoder first passes through the temporal feature aggregation module, and then passes through the system interaction encoding module before being output;

[0012] The signal processing method in the time series feature aggregation module is as follows: the input signal is first cut into multiple time segments of sliding window lengths by "sequence slicing", and then input into a multi-layer long short-term memory network (Long Short-Term Memory, LSTM) based on time segment input, the encoded time segments are grouped, and then the group sampling method is used to take out the latest time segments of each group and reorganize them into a new time series, and then the time features are aggregated by multi-layer LSTM, and then output;

[0013] The signal processing method in the system interaction coding module is as follows: the input signal of the system interaction coding module first passes through the first edge interaction coding layer to obtain the shallow coding features of the interaction between nodes, then enters the node coding layer to obtain the average coding features of the node aggregation, and then enters the second edge interaction coding layer to further mine the deep hidden features that can express the interaction between system nodes, and then passes through the splicing layer to splice the shallow and deep interaction coding features, and then uses the group sampling method to take out each group of the latest time segments and reorganize them into a new time series, and finally uses the multi-layer LSTM based on the time segment input to mine the interaction coding features of the time series, and takes the features of the segments corresponding to the closest time for reshaping to adapt to the input of the decoder;

[0014] The described decoder structure consists of multiple multilayer perceptrons (MLPs) and a mapping scoring function of a rectified linear unit (ReLU). Each multilayer perceptron consists of two fully connected layers activated by two exponential linear units (f B ()) and a batch normalization layer (f E ()).

[0015] Furthermore, the structure of the temporal feature aggregation module based on inputting multiple LSTMs for time segments consists of a forget gate, a memory gate, and an output gate unit; in each time slice, the neurons in the gate unit receive the current and past slice features in a fully connected mode simultaneously. The repeated chain of the multiple LSTMs based on inputting time segments allows the time information of the accumulated historical slices to be added to the time information of the current slice. Among them, the forget gate controls the degree of forgetting of the historical cell state, which is calculated by , the memory gate controls the incorporation of the current information into the current cell state, which is calculated by and , and updates the cell state within the unit: The output value is determined by the output gate: and σ(·) is the non-linear sigmoid function; δ(ω) is the hyperbolic tangent function, ⊙ is the Hadamard product, represents the weight matrix of the forget gate of the k-th layer, represents the bias term of the forget gate of the k-th layer, represents the output feature of the forget gate of the k-th layer for the q-th sequence slice, represents the weight matrix of the first branch of the memory gate of the k-th layer, represents the bias term of the first branch of the memory gate of the k-th layer, represents the output feature of the first branch of the memory gate of the k-th layer for the q-th sequence slice, represents the weight matrix of the second branch of the memory gate of the k-th layer, represents the bias term of the second branch of the memory gate of the k-th layer, represents the output feature of the second branch of the memory gate of the k-th layer for the q-th sequence slice, represents the cell state of the k-th layer for the q-th sequence slice, represents the weight matrix of the output gate of the k-th layer, represents the bias term of the output gate of the k-th layer, represents the temporary output feature of the output gate of the k-th layer for the q-th sequence slice, represents the output feature of the output gate of the k-th layer for the q-th sequence slice, represents the output feature of the output gate of the k-th layer for the (q - 1)-th sequence slice, and represent the iterative initial values of the unit state and layer hidden state of the k-th layer, where k ≥ 2 and q ≥ 1.

[0016] Furthermore, the edge interaction encoding layer in the system interaction encoding module is:

[0017] H υe = Multi-LSTMs([F(RF -1 (H in )), F(SF -1 (H in ))])

[0018] where R and S are the position encoding matrices of the sending node and the receiving node respectively, H in is the layer input, F -1 () is the sequence recombination process, F() is the sequence decomposition process, and Multi-LSTMs represents a multi-layer LSTM structure.

[0019] Furthermore, the node encoding layer in the system interaction encoding module is:

[0020] H eυ = Multi-LSTMs(F((R T F -1 (H in ) / M)))

[0021] where R T is the transpose of the position encoding matrix of the sending node, and M is the number of nodes in the system.

[0022] Furthermore, multiple multi-layer perceptron combinations are set in the decoder, and each multi-layer perceptron is defined as:

[0023] H in and H out are the input and output respectively, represents the weight matrix of the first fully connected layer in the i-th multi-layer perceptron, represents the bias term of the first fully connected layer in the i-th multi-layer perceptron, represents the weight matrix of the second fully connected layer in the i-th multi-layer perceptron, represents the bias term of the second fully connected layer in the i-th multi-layer perceptron, f E represents the batch normalization layer, and f B represents the exponential linear unit activation function.

[0024] Furthermore, the rectified linear unit activation function:

[0025] ReLu(x) = max(0, x);

[0026] The exponential linear unit activation function:

[0027]

[0028] The batch normalization layer:

[0029]

[0030] where μ and σ are the mean and standard deviation, and ε is e -6 , and the trainable parameters γ, β.

[0031] Furthermore, the parameter determination process in the encoder and decoder is as follows:

[0032] Step S1: Divide the dataset into a training set and a validation set, set the number of iterations and initialize the model hyperparameters;

[0033] Step S2: Add random noise to the training set and input it to the graph neural network after channel normalization processing to obtain a brain network, and calculate the predefined brain network matrix and the matrix of the brain network estimated by the network Mean mean square error: Adopt the stochastic optimization algorithm with adaptive momentum, update the model parameters by the learning rate weighted loss error gradient, and Bsize represents the number of samples in each batch of training of the training set;

[0034] Step S3: After an iteration is completed, fix the model parameters, input the validation set into the graph neural network mentioned above to obtain a brain network, calculate the mean mean square error between the predefined brain network and the brain network estimated by the network. If this error is less than the recorded minimum validation error, save the current model parameters and set the minimum validation error as the current mean mean square error value;

[0035] Step S4: Repeat Step S2 and S3 for each iteration until the number of iterations reaches the initially set number of iterations, and the training is completed to obtain the model parameters.

[0036] The advantages of the present invention are as follows:

[0037] Through sequence decomposition and recombination, the GNN model integrating LSTM based on time segment input has spatio-temporal adaptability characteristics, adapts to different signal inputs with inconsistent adaptive time series ranges and different numbers of sensors, constructs corresponding brain networks, and verifies the reliability of this method through simulation datasets and real acquisition datasets. Description of the Drawings

[0038] Figure 1Schematic diagram of the steps of an end-to-end spatio-temporal adaptive brain network construction method provided by the present invention.

[0039] Figure 2 Schematic diagram of a spatio-temporal adaptive model based on graph neural network provided by the present invention.

[0040] Figure 3 Brain network estimation accuracy for different time series range signal inputs (signal-to-noise ratio is 0 dB), * indicates p < 0.001.

[0041] Figure 4 Brain network estimation accuracy for different numbers of sensing sources (signal-to-noise ratio is 0 dB), * indicates p < 0.001.

[0042] Figure 5 Resting-state brain network topology for different time series range signal inputs (threshold is 0.15).

[0043] Figure 6 Similarity of the resting-state brain network topology for different time series range signal inputs. Detailed implementation mode

[0044] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0045] According to the embodiments of the present invention, an end-to-end spatio-temporal adaptive brain network construction method is proposed. Please refer to Figure 1 , including the following steps:

[0046] Step 1: Preprocess the multi-channel electroencephalogram signals to be analyzed to improve the signal-to-noise ratio. The preprocessing steps include Reference Electrode Standardization Technique (REST) reference, band-pass filtering, downsampling, sequence segmentation and slicing, and artifact removal;

[0047] Step 2: Load the trained model parameters;

[0048] Step 3: Input the signals processed in Step 1 into the constructed model to obtain a directed brain network matrix;

[0049] Step 4: Set the connection threshold to obtain significant directed connections;

[0050] Further, the specific steps of Step 1 include:

[0051] Step S11: Process the signals using REST reference, with an infinitely distant point in space as the reference, to obtain the converted potential signals;

[0052] Step S12: Use band-pass filtering to filter out the interference of power frequency noise and environmental noise, and select the EEG signals in a specific frequency band. In this embodiment, the upper filtering limit is 30 Hz and the lower limit is 1 Hz;

[0053] Step S13: Reduce the sampling rate of the filtered signal. In this embodiment, the sampling rate of the EEG signal is downsampled from 1000 Hz to 256 Hz;

[0054] Step S14: Segment and slice the downsampled signal sequence. Set a sliding window with a window length of L, and slide the sliding window to divide the signal time series into multiple sequence slices in a non-overlapping manner. In this embodiment, the window length of the training samples is 6 seconds (s), and the window lengths of the test samples are 2 s, 4 s, 6 s, and 8 s respectively;

[0055] Step S15: Compare the sampling points of each obtained signal segment with the set artifact threshold. If the potential amplitude at a certain moment in the signal slice is greater than the artifact threshold, then discard the signal slice. In this embodiment, the artifact threshold is set to 75 μV;

[0056] Step S16: Normalize the data based on the channel dimension to obtain the processed EEG signal;

[0057] Further, the step 2 is based on the spatio-temporal adaptive model structure of the graph neural network. Please refer to Figure 2 :

[0058] This model design adopts an encoder-decoder framework. The encoder is designed as a spatio-temporal aggregator, integrating a temporal feature aggregation module for aggregating temporal features and a system interaction encoding module. The decoder is designed as an artificial neural network for decoding the interaction coupling features of the encoder to obtain a directed brain network;

[0059] Further, the encoder structure includes two modules: a temporal feature aggregation module and a system interaction encoding module:

[0060] The temporal feature aggregation module consists of sequence slices and a multi-layer long short-term memory network (LSTM) based on time segment input. The specific process of the input signal being processed by this module is as follows: the input signal is cut into multiple time segments of the sliding window length by "sequence slices", and then input into the multi-layer LSTM based on time segment input. The encoded time segments are grouped and the latest time segment of each group is taken out and recombined into a new time series, and then the temporal features are aggregated through the LSTM;

[0061] The system interaction encoding module is a graph neural network integrated with LSTM, which consists of two layers of edge interaction encoding layers, one layer of node encoding layer, a splicing layer, multiple layers of LSTM and a reshaping layer. The specific process of the input signal being processed by this module is as follows: the output of the encoder is used as the input of the module, input into the first edge interaction encoding layer, and the shallow coding features of the interaction between nodes are obtained. Then, it is input into the node encoding layer to obtain the average coding features of the node aggregation, and then it is input into the second edge interaction encoding layer to further mine the deep hidden features that can represent the interaction between system nodes. Then, the shallow and deep interaction coding features are spliced through the splicing layer, and the multi-layer LSTM based on the time segment input is used to mine the interaction coding features of the time series. The features of the segment corresponding to the closest time are input into the reshaping layer to adapt to the input of the decoder.

[0062] Furthermore, the sequence slicing process in the temporal feature aggregation module is as follows:

[0063] A sliding window with a window length of L is set to divide the input multi-channel signal time series into multiple sequence slices in a non-overlapping sliding window manner, and multiple input samples are grouped and arranged in time sequence. In this embodiment, the sliding window length is 32;

[0064] Furthermore, the LSTM design based on time segment input in the time series feature aggregation module is:

[0065] The structure consists of forget gate, memory gate, and output gate units. In each time slice, the neurons in the gate unit receive the current and past slice features simultaneously in a fully connected mode. The repeated chain of LSTM allows the accumulated time information of historical slices to be added to the time information of the current slice. The forget gate controls the degree of forgetting of the historical unit state. It is calculated that the memory gate controls how much information is incorporated into the current unit state, which is determined by and Calculate and update the cell state in the unit: The output value is determined by the output gate: and σ(·) is the nonlinear sigmoid function; δ(·) is the hyperbolic tangent function, ⊙ is the Hadamard product, represents the weight matrix of the k-th layer forget gate, represents the bias term of the forget gate of the kth layer, represents the output feature of the k-th layer forget gate of the q-th sequence slice, represents the weight matrix of the first branch of the k-th layer memory gate, represents the bias term of the first branch of the k-th layer memory gate, represents the output feature of the first branch of the qth sequence slice memory gate in the kth layer, The weight matrix representing the second branch of the memory gate at the k-th layer The bias term representing the second branch of the memory gate at the k-th layer The output feature representing the second branch of the memory gate at the k-th layer for the q-th sequence slice The cell state representing the k-th layer and the q-th sequence slice The weight matrix representing the output gate at the k-th layer The bias term representing the forget gate at the k-th layer The temporary output feature representing the output gate at the k-th layer for the q-th sequence slice The output feature representing the output gate at the k-th layer for the q-th sequence slice The output feature representing the output gate at the k-th layer for the (q - 1)-th sequence slice and Represents the iterative initial values of the k-th layer cell state and layer hidden state, where k ≥ 2 and q ≥ 1;

[0066] Furthermore, the edge interaction encoding layer in the system interaction encoding module:

[0067] Can be expressed as H υe = Multi-LSTMs([F(RF -1 (H in ), F(SF -1 (H in ))]), where R and S are the position encoding matrices of the sending node and the receiving node respectively, H in Is the layer input, F -1 () is the sequence recombination process, and F() is the sequence decomposition process;

[0068] Furthermore, the node encoding layer in the system interaction encoding module:

[0069] Can be expressed as H eυ = Multi-LSTMs(F((R T F -1 (H in ) / M))), R T Where is the transpose of the sending node position encoding matrix, and M is the number of nodes in the system;

[0070] Furthermore, the sequence recombination F -1 () process:

[0071] Input Traverse n trans_dim , in the i-th traversal, X(:,:,i) ← X (i)T , after the traversal ends, obtain Then traverse t, in the i-th traversal, X u (:,(i - 1) × ntrans_dim +1:i×n trans_dim ) ← X(:, i, :), obtaining Then traverse Bsize. At the i-th traversal, there is X out (i, :, :) ← X u ((i - 1)×n dim +1:i×n dim , :), after the traversal is completed, obtaining Bsize is the number of samples trained in each batch of the training set, n dim is the number of features of the layer input X, t is the sequence length of the layer input X, n trans dim is the number after feature transformation;

[0072] Furthermore, the process of the sequence decomposition F(·):

[0073] Input Traverse Bsize. At the i-th traversal, there is X u ((i - 1)×n dim +1:i×n dim , :) ← X(i, :, :), then traverse t. At the i-th traversal, there is X out (t, :, :) ← X u (:, (i - 1)×n trans_dim +1:i×n trans_dim ), after the traversal is completed, obtaining n dim is the number of features of the layer input X, t is the sequence length of the layer input X, n trans_dim is the number after feature transformation;

[0074] Furthermore, the structure of the decoder is as follows:

[0075] It consists of multiple multilayer perceptrons (MLPs) and a mapping scoring function of a rectified linear unit (ReLU). Each MLP module consists of two fully connected layers activated by two exponential linear units (f B ()) and a batch normalization layer (f E ()). In this embodiment, 3 combinations of multilayer perceptrons are set. Each multilayer perceptron can be expressed as: H in and H out are the input and output respectively, represents the weight matrix of the first fully connected layer in the i-th multilayer perceptron, represents the bias term of the first fully connected layer in the i-th multilayer perceptron, It represents the weight matrix of the second fully connected layer in the i-th multi-layer perceptron, It represents the bias term of the second fully connected layer in the i-th multi-layer perceptron, f E It represents the batch normalization layer, f B It represents the exponential linear unit activation function;

[0076] Furthermore, the rectified linear unit activation function:

[0077] ReLu(x) = max(0, x);

[0078] Furthermore, the exponential linear unit activation function:

[0079]

[0080] Furthermore, for the batch normalization layer (μ and σ are converted into the mean and standard deviation, ε is e -6 , the trainable parameters γ, β):

[0081]

[0082] Furthermore, the process of determining the parameters of the method is as follows:

[0083] Step S1: Divide the data set into a training set and a validation set, set the number of iterations and initialize the model hyperparameters (including the learning rate). In this embodiment, the initial learning rate is set to 0.008, and the learning rate is adjusted by fixed-step decay. Every time the number of iterations reaches 200, the learning rate is reduced to 0.8 times the original;

[0084] Step S2: Add random noise to the training set and input it into the graph neural network after channel normalization processing to obtain a brain network, and calculate the predefined brain network matrix and the matrix of the brain network estimated by the network The average mean square error: Adopt the stochastic optimization algorithm with adaptive momentum, and update the model parameters by the learning rate weighted loss error gradient;

[0085] Step S3: After one iteration is completed, fix the model parameters, input the validation set into the graph neural network, obtain a brain network, and calculate the average mean square error of the predefined brain network matrix and the matrix of the brain network estimated by the network: If the error is less than the recorded minimum validation error, save the current model parameters and set the minimum validation error to the current average mean square error value;

[0086] Step S4: Repeat Step S2 and S3 for each iteration until the number of iterations reaches the initially set number of iterations, and the training is completed to obtain the model parameters.

[0087] In this embodiment, the generation of simulated EEG signals with different durations is used to illustrate the model performance:

[0088] Collect the EEG signals of a normal person in the resting state for 5 minutes. Preprocess the collected signals according to step 1, and iteratively generate simulated signals with different durations (2s, 4s, 6s, 8s) and predefined brain networks (a total of 50 test data) according to the autoregressive equation. Add random noise to make the signal-to-noise ratio of the signal 0 dB, and input it into the graph neural network to obtain the brain network matrix. Among them, each test data is subjected to 50 noise simulations. Each simulation will calculate the accuracy rate by calculating the brain network matrix and the predefined brain network matrix, and the average value of the 50 simulation results is used as the brain network estimation accuracy rate of this test data. The average brain network estimation accuracy rate of all test data is used as the evaluation performance;

[0089] Furthermore, the calculation of the accuracy rate of the brain network matrix is as follows:

[0090]

[0091] Among them, TP represents the number of edges that correctly estimate the information flow direction between nodes when the connection exists, TN represents the number of edges that correctly estimate the non-existence of the connection between nodes, FP represents the incorrect estimation of the information flow direction between nodes when the connection exists, and FN represents the number of edges of the incorrectly estimated nodes;

[0092] Please refer to Figure 3 , compare the performance of the brain network constructed by the present invention and the Granger causality method. The performance of the brain network constructed by the present invention with different durations of EEG signals is close to that of the Granger causality method, and the performance of constructing the brain network at 4s and 6s is better than that of the Granger causality method. However, when using the Granger causality method to construct the brain network corresponding to different durations of EEG signals, it is necessary to model and solve them separately to obtain the corresponding parameters to obtain the corresponding brain network, while the present invention does not need to re-model and solve the model parameters for different durations of EEG signals, but uses the trained model to process different durations of EEG signals to construct the brain network;

[0093] In this embodiment, the generation of simulated EEG signals with different numbers of channels is used to illustrate the model performance:

[0094] The 5-minute EEG signals of normal people in the resting state were collected, preprocessed according to Step 1, and simulated signals with different numbers of sensing sources (5, 10, 15, 20) and a predefined brain network (a total of 50 test data) were iteratively generated according to the autoregressive equation. Random noise was added to make the signal-to-noise ratio of the signal 0 dB, and the resulting signal was input into the graph neural network to obtain the brain network matrix. Among them, each test data was subjected to 50 noise simulations, and the accuracy was calculated by comparing the brain network matrix obtained each time with the predefined brain network matrix. The average value of the 50 simulation results was used as the brain network estimation accuracy of the test data, and the average brain network estimation accuracy of all test data was used as the evaluation performance;

[0095] Please refer to Figure 4 , to compare the performance of the brain network constructed by the present invention and the Granger causality method. The accuracy of the brain network constructed by the present invention with different numbers of sensing sources is comparable to that of the Granger causality method, indicating that the present invention has reliable generalization performance;

[0096] The construction of the brain network for the EEG signals of normal people in the resting state in this embodiment is described as follows:

[0097] The 5-minute EEG signals of normal people in the resting state were collected. The collected signals were preprocessed according to Step 1, and signals of different durations were input into the graph neural network for processing. The brain network weights of all signal segments corresponding to different seconds of each normal person were summed and averaged, and the brain network matrices of all subjects were averaged, and a threshold was set to obtain significant connections;

[0098] Please refer to Figure 5 , to compare the resting-state brain network topologies constructed by the present invention and the Granger causality method:

[0099] When the slice lengths are 2s, 4s, 6s, and 8s, the brain network topologies constructed by the present invention are relatively consistent, mainly involving connections symmetrically distributed in the frontal lobe, posterior parietal-occipital region, and posterior temporal region. As the slice length increases, the central role of Fz in the brain networks constructed by the present invention and the Granger causality method becomes gradually clear. However, for the brain networks constructed by the Granger causality method for 4s, 6s, and 8s EEG signals: the long-range connections between the frontal lobe and the occipital lobe are not very significant;

[0100] Please refer to Figure 6 , to compare the similarities of the resting-state brain network topologies constructed by the present invention and the Granger causality method for different durations:

[0101] The brain networks constructed by the present invention maintain a high consistency (greater than 0.57±0.05) at different durations, and in most cases are higher than the Granger causality method (paired t-test, one-tailed, p<0.000). For the Granger causality method, the consistency of the brain networks constructed at different durations is relatively low. In particular, the consistency between the brain networks constructed with short durations (2s) and short durations (6s or 8s) is even lower than 0.5 (2s-6s: 0.49±0.10; 2s-8s: 0.31±0.07).

[0102] Therefore, under low signal-to-noise ratio, the spatio-temporal adaptive end-to-end brain network construction method based on graph neural network proposed by the present invention expands the applicability of the end-to-end scheme to different application scenarios (different number of channels, different durations).

[0103] The above are only the preferred embodiments of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope of the present invention.

Claims

1. A spatiotemporal adaptive end-to-end brain network construction method based on graph neural network, which constructs the brain network through the following steps: Step 1: Preprocess the multi-channel EEG signals to be analyzed to improve the signal-to-noise ratio; Step 2: Load the trained graph neural network-based deep learning model parameters; Step 3: Input the signal processed in step 1 into the constructed model to obtain a directed brain network matrix; Step 4: Set the connection threshold to obtain significant directed connections; The specific process of step 3 is as follows: The multi-channel EEG signal is first encoded with temporal features by an encoder, and the learned features are decoded by a decoder, and then the brain network matrix is output; the encoder includes: Temporal feature aggregation module and system interaction encoding module. The multi-channel EEG signal input to the encoder first passes through the temporal feature aggregation module, and then passes through the system interaction encoding module before being output; The signal processing method in the time series feature aggregation module is as follows: the input signal is first cut into multiple time segments of sliding window lengths by "sequence slicing", and then input into a multi-layer long short-term memory network based on time segment input, the encoded time segments are grouped, and then the group sampling method is used to take out the latest time segments of each group and reorganize them into a new time series, and then the time features are aggregated by multi-layer LSTM, and then output; The signal processing method in the system interaction coding module is as follows: the input signal of the system interaction coding module first passes through the first edge interaction coding layer to obtain the shallow coding features of the interaction between nodes, then enters the node coding layer to obtain the average coding features of the node aggregation, and then enters the second edge interaction coding layer to further mine the deep hidden features that can express the interaction between system nodes, and then passes through the splicing layer to splice the shallow and deep interaction coding features, and then uses the group sampling method to take out each group of the latest time segments and reorganize them into a new time series, and finally uses the multi-layer LSTM based on the time segment input to mine the interaction coding features of the time series, and takes the features of the segments corresponding to the closest time for reshaping to adapt to the input of the decoder; The described decoder structure consists of multiple multi-layer perceptrons and a mapping scoring function of a rectified linear unit. Each multi-layer perceptron consists of two fully connected layers activated by an exponential linear unit (f B ()), and a batch normalization layer (f E ()).

2. The spatio-temporal adaptive end-to-end brain network construction method based on graph neural network according to claim 1, characterized in that In the described temporal feature aggregation module, the structure of the multi-layer LSTM based on time segment input consists of a forget gate, an input gate, and an output gate unit; in each time slice, the neurons in the gate unit receive the current and past slice features in a fully connected mode, and the repeating chain of the multi-layer LSTM based on time segment input allows the time information of the accumulated historical slices to be added to the time information of the current slice. Among them, the forget gate controls the degree of forgetting of the historical cell state, which is calculated by The input gate controls the incorporation of the current information into the current cell state, which is calculated by and Calculate to update the cell state within the unit: The output value is determined by the output gate: and σ(·) is the non-linear sigmoid function; δ(·) is the hyperbolic tangent function, ⊙ is the Hadamard product, represents the weight matrix of the k-th layer forget gate, represents the bias term of the k-th layer forget gate, represents the output feature of the k-th layer forget gate for the q-th sequence slice, represents the weight matrix of the first branch of the k-th layer input gate, represents the bias term of the first branch of the k-th layer input gate, represents the output feature of the first branch of the k-th layer input gate for the q-th sequence slice, represents the weight matrix of the second branch of the k-th layer input gate, represents the bias term of the second branch of the k-th layer input gate, represents the output feature of the second branch of the k-th layer input gate for the q-th sequence slice, represents the cell state of the k-th layer for the q-th sequence slice, represents the weight matrix of the k-th layer output gate, represents the bias term of the k-th layer output gate, represents the temporary output feature of the k-th layer output gate for the q-th sequence slice, represents the output feature of the k-th layer output gate for the q-th sequence slice, represents the output feature of the k-th layer output gate for the (q - 1)-th sequence slice, and represent the iterative initial values of the k-th layer cell state and layer hidden state, k ≥ 2, q ≥ 1.

3. The spatio-temporal adaptive end-to-end brain network construction method based on graph neural network according to claim 2, wherein, The edge interaction coding layer in the system interaction coding module is: H υe = Multi-LSTMs([F(RF -1 (H in )), F(SF -1 (H in ))]) where R and S are the position encoding matrices of the sending node and the receiving node respectively, and H in is the layer input, and F -1 (·) is the sequence recombination process, F(·) is the sequence decomposition process, and Multi-LSTMs represent the multi-layer LSTM structure.

4. The spatio-temporal adaptive end-to-end brain network construction method based on graph neural network according to claim 3, characterized in that, The node coding layer in the system interaction coding module is: H eυ = Multi-LSTMs(F((R T F -1 (H in ) / M))) where R T is the transpose of the sending node position encoding matrix, and M is the number of nodes in the system.

5. The method for constructing a spatio-temporal adaptive end-to-end brain network based on a graph neural network according to claim 4, wherein, The decoder is provided with a plurality of multi-layer perceptron combinations, each of which is defined as: H in and H out are the input and output respectively, represents the weight matrix of the first fully connected layer in the i-th multi-layer perceptron, represents the bias term of the first fully connected layer in the i-th multi-layer perceptron, represents the weight matrix of the second fully connected layer in the i-th multi-layer perceptron, represents the bias term of the second fully connected layer in the i-th multi-layer perceptron, f E represents the batch normalization layer, f B represents the exponential linear unit activation function.

6. The spatiotemporal adaptive end-to-end brain network construction method based on a graph neural network according to claim 5, characterized in that The rectified linear unit activation function: ReLu(x)=max(0,x); The exponential linear unit activation function: The batch normalization layer: Among them, μ and σ are transformed into the mean and standard deviation, and ε is e -6 , and the trainable parameters are γ and β.

7. The method for constructing a spatio-temporal adaptive end-to-end brain network based on a graph neural network according to claim 6, wherein The parameter determination process in the encoder and decoder is as follows: Step S1: Divide the dataset into training set and validation set, set the number of iterations and initialize the model hyperparameters; Step S2: Add random noise to the training set and input it into the graph neural network after channel normalization to obtain a brain network, and calculate the predefined brain network matrix and the matrix of the network-estimated brain network Average mean square error: Adopt the stochastic optimization algorithm with adaptive momentum, and update the model parameters by weighting the loss error gradient with the learning rate. Bsize represents the number of samples in each batch of training of the training set; Step S3: After one iteration is completed, the model parameters are fixed, the validation set is input into the graph neural network, the brain network is obtained, and the average mean square error of the predefined brain network and the network estimated brain network is calculated. If the error is less than the recorded minimum validation error, the current model parameters are saved, and the minimum validation error is set to the current average mean square error value; Step S4: Repeat steps S2 and S3 in each iteration until the number of iterations reaches the number of iterations set initially, the training is completed, and the model parameters are obtained.

Images