Construction method of effective connection brain network model based on LSTM

Abstract

The application discloses a kind of based on the construction method of effective connection brain network model of LSTM, first the brain function image data of subject personnel is preprocessed, to obtain better imaging brain function image data, then the time series signal of the brain region of interest in brain function image data is proposed, form brain region activity vector and as training data to the effective connection brain network model of LSTM is trained, the weight parameter of the effective connection brain network model of LSTM after training is as brain effective connection relationship data;The application uses the LSTM model in deep learning to model, process is simple, greatly reduce the time required for modeling, improve the modeling efficiency;In addition, the LSTM model used in the application, whether based on single subject or multiple subjects, can be modeled simultaneously, so it can be better adapted to group-level modeling.

Description

Technical Field

[0001] This invention relates to, specifically, an efficient connectivity brain network model based on LSTM. Background Technology

[0002] With the advent of functional magnetic resonance imaging (fMRI), its non-invasive, radiation-free, and high spatiotemporal resolution characteristics have led to its widespread application in the research of mental illnesses. Currently, brain network modeling and analysis based on fMRI mainly focuses on functional connectivity analysis. However, because functional connectivity analysis cannot characterize the direction of brain connections, it cannot provide information about the flow of signals within brain circuits. In recent years, effective connectivity analysis, which can characterize the directional information of brain connections, has gradually gained widespread attention. Among effective connectivity analysis methods, Granger causality analysis and dynamic causal modeling are the most widely used.

[0003] Granger causality analysis, proposed by Granger in 1969, was initially used in economics and began to be applied to fMRI image analysis in 2003. Its basic idea is to use a multivariate regression model to model the interactions between brain regions, and by fitting this model, determine whether a causal relationship exists between two variables, while also clarifying the directionality of the causal relationship. Granger causality analysis is simple to model and has low computational complexity, and it is currently widely used in large-scale network operations. However, because Granger causality analysis cannot accurately model hemodynamic response processes, it is not suitable for fMRI data analysis.

[0004] The dynamic causal modeling method was first proposed by Professor Friston in 2003. Its basic idea is to model the human brain as a multi-input, multi-output nonlinear dynamic system. The dynamic causal model mainly consists of two elements. First, the neuronal activity state equation is used to model the interaction between neural activity signals in different brain regions at the level of neural activity. This mainly includes the intrinsic connectivity (A) between brain regions in the absence of external input, the direct influence (C) of external input on brain region activity, and the changes in connectivity between brain regions (B) caused by external input. The other element of the dynamic causal model—the hemodynamic response equation—is used to model the conversion process from brain region activity signals to the observed BOLD signal. By using the neuronal activity state equation and the hemodynamic response state equation, the dynamic causal model can achieve a more accurate modeling of human brain connectivity. While dynamic causal models can effectively model brain activity, several problems remain: First, dynamic causal models are complex, with computational complexity increasing exponentially with the number of brain regions. Second, they simplistically treat the brain system as a Markov process, assuming current brain activity is only related to previous activity and external input, neglecting past brain activity and lacking effective memory mechanisms. Finally, dynamic causal models are individual-level brain network models and cannot be easily modeled at the group level. Therefore, there is an urgent need for a computationally efficient connectivity model with low complexity to significantly improve the modeling speed of traditional efficient connectivity models while providing a more scientific and rational explanation of brain function from a higher perspective. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides an efficient connectivity brain network model based on LSTM, which can effectively improve the modeling speed of traditional efficient connectivity models, and at the same time, can provide a more scientific and reasonable explanation of brain function from a higher dimension.

[0006] This invention is achieved through the following technical solution:

[0007] A method for constructing an efficient connectivity brain network model based on LSTM includes the following steps:

[0008] Step 1: Preprocess the acquired raw brain function image data to remove various noises and obtain brain function image data with better imaging quality.

[0009] Step 2: Extract the time-series signals of the brain regions of interest from the brain functional image data obtained in Step 1, and form a vector of brain region activity with timestamps based on the time-series signals;

[0010] Step 3: Train the constructed LSTM effective connectivity brain network model using the vectors of brain region activity to obtain the weight parameters of the trained LSTM effective connectivity brain network model, and use these weight parameters as brain effective connectivity relationship data.

[0011] Preferably, the preprocessing in step 1 includes multiple operations such as DICOM to Nifti conversion, removal of the first n images, SliceTiming, head motion correction, standardization, spatial smoothing, delinear drift, regression covariates, frequency domain filtering, and Scrubbing.

[0012] Preferably, the method for obtaining the time-stamped vectors of brain region activity in step 2 is as follows:

[0013] For the brain function image data obtained from step 1 preprocessing, select its region of interest, use the brain function template to extract the time series of the corresponding brain region from the region of interest, and use it as the brain region activity signal. Then calculate the mean of the brain region activity signal at the current time point, and use it as the brain region activity intensity at the current time point, thereby forming a brain region activity vector with timestamp.

[0014] Preferably, the weight parameters in step 3 include the weight parameters of the input gate, forget gate, and output gate, and the three weight parameters serve as three effective connection models between regions of interest.

[0015] Preferably, the weighting parameter is a square matrix, the number of rows and columns of the square matrix is ​​equal to the number of regions of interest, each value in each square matrix represents the strength of the effective connection, and the direction of the effective connection is from the brain region corresponding to the column of each value to the brain region corresponding to the row of each value.

[0016] Preferably, the training method for the LSTM-based efficient connectivity brain network model in step 3 is as follows:

[0017] Step 3.1: Construct an efficient connectivity brain network model based on LSTM, which introduces an input gate, a forget gate, and an output gate;

[0018] S3.2, Input X according to the current time step t Hidden state H from the previous time step t-1 Determine the input gate I for time step t. t Forgotten Gate F t and output gate O t ;

[0019] S3.3, Input X according to the current time step t Hidden state H from the previous time step t-1 Calculate candidate memory cells

[0020] S3.4, According to the forgetting gate Ft Input gate I t and candidate memory cells Calculate the memory cell C at the current time step. t ;

[0021] S3.5, based on output gate O t and the current time step memory cell C t Determine the hidden state H at the current time step. t ;

[0022] S3.6. The backpropagation and gradient descent algorithms are used to update the weights and biases of the candidate memory cells of the input gate, forget gate, and output gate.

[0023] S3.7 Repeat the above training process until the maximum number of training iterations is reached or the hidden state H of two adjacent time steps is reached. t-1 and H t If the error between the two is less than a certain specific value, an effective connectivity brain network model based on LSTM is obtained after training.

[0024] Preferably, the input gate I described in step S3.2 t Forgotten Gate F t and output gate O t The method for determining it is as follows:

[0025] I t =σ(X) t W xi +H t-1 W hi +b i )

[0026] F t =σ(X) t W xf +H t-1 W hf +b f )

[0027] O t =σ(X) t W xo +H t-1 W ho +b o )

[0028] Where σ is the sigmoid function, W xi W xf W xo W hi W hf W ho These are the weight parameters, b i b f bo These are the deviation parameters for each gate.

[0029] Preferably, the selection of memory cells described in step S3.3 The method for determining it is as follows:

[0030]

[0031] Among them, W hc and W xc b is the weight parameter. c This is the deviation parameter.

[0032] Preferably, the memory cell C at the current time step in step S3.4 t The method for determining it is as follows:

[0033] Based on the information from the memory cells of the previous time step and the candidate memory cells of the current time step, the flow of information is controlled through the forget gate and the input gate to determine the memory cells of the current time step.

[0034] Preferably, the hidden state H of the current time step in step S3.5 t The determination method is as follows:

[0035] The output gate is used to control the memory cell at the current time step to the hidden state H. t The flow of information determines the hidden state of the current time step.

[0036] Compared with the prior art, the present invention has the following beneficial technical effects:

[0037] This invention provides a method for constructing an efficient connectivity brain network model based on LSTM. First, the brain function image data of the subjects is preprocessed to obtain better-imaged brain function images. Then, the time-series signals of brain regions of interest in the brain function image data are extracted to form brain region activity vectors, which are used as training data to train the LSTM efficient connectivity brain network model. The weight parameters of the trained LSTM efficient connectivity brain network model are used as the brain's efficient connectivity relationship data. Traditional dynamic causal modeling requires two factors: neuronal activity state equations and hemodynamic response equations. The modeling process is complex, and the modeling time increases exponentially with the number of brain regions. This invention uses the LSTM model from deep learning for modeling, which simplifies the process, greatly reduces the modeling time, and improves modeling efficiency. Furthermore, traditional dynamic causal modeling is based on individual-level brain network models, where each subject can only be modeled individually, making it unsuitable for group-level modeling needs. The LSTM model used in this invention can model simultaneously, whether based on a single subject or multiple subjects, thus making it well-suited for group-level modeling. Attached Figure Description

[0038] Figure 1 This is a flowchart of the method for constructing an efficient connectivity brain network model based on LSTM according to the present invention;

[0039] Figure 2 This is a block diagram of the LSTM model of the present invention;

[0040] Figure 3 This is a schematic diagram of the input gate of the DMN network in Embodiment 1 of the present invention;

[0041] Figure 4 This is a schematic diagram of the forget gate in the DMN network of Embodiment 1 of the present invention;

[0042] Figure 5 This is a schematic diagram of the output gate of the DMN network in Embodiment 1 of the present invention;

[0043] Figure 6 This is a comparison chart of the runtime of the traditional dynamic causal modeling and the efficient connectivity brain network model based on LSTM in this invention.

[0044] Figure 7 This is a diagram illustrating the dynamic causal modeling pattern of the present invention. Detailed Implementation

[0045] The present invention will now be described in further detail with reference to the accompanying drawings. These descriptions are intended to explain the invention and not to limit it.

[0046] See Figure 1-7 An efficient connectivity brain network model based on LSTM includes the following steps:

[0047] Step 1: Preprocess the raw brain function image data of all subjects to remove various noises and other irrelevant information, enhance the detectability of useful information, and thus obtain brain function images with better image quality.

[0048] Specifically, each raw brain function image data acquired is preprocessed, and the preprocessing operations include, but are not limited to, the following:

[0049] DICOM to Nifty conversion, removal of the first n images, slice timing, head motion correction, standardization, spatial smoothing, delinear drift, regression covariate, frequency domain filtering, and scrubbing.

[0050] Let A be the set of operations used for preprocessing, and let the original brain functional image dataset be... The brain functional image dataset obtained after preprocessing is D = {subj i |i∈[1,N]}.

[0051] Where N represents the number of subjects, subj i The brain function signals of subject i can be further represented as This represents the brain function signals of subject i at time t.

[0052] During operation, first determine the preprocessing operations to be performed, and construct the preprocessing operation set A = {act i |i∈[1,M]}, and then each preprocessing operation item act will be taken out from set A in sequence. i The parameters of this preprocessing operation were identified, and it was applied to the brain functional signals of the subjects. The above, and then the processed brain functional signals subj, are obtained. i ,Right now To generate higher quality brain function images.

[0053] Step 2: Extract the time-series signals of the brain regions of interest from the brain functional image data obtained in Step 1, and form a vector of brain region activity with timestamps based on the time-series signals.

[0054] For the brain function image data obtained from step 1 preprocessing, select its region of interest (ROI), use the brain function template to extract the time series of the corresponding brain region from the ROI and use it as the brain region activity signal. Then calculate the mean of the brain region activity signal at the current time point and use it as the brain region activity intensity at the current time point. This forms a brain region activity vector with a timestamp, and the brain region activity vector is used as training data for training the long short-term memory network model.

[0055] According to the definition in step 1, This represents the brain function signals of subject i, namely, This represents the brain functional signal of subject i at time t, where M is the number of brain regions. Let represent the functional signal of the m-th brain region of subject i at time t, and K represent the number of functional signals in the m-th brain region. Corresponding to the selected brain template, the mean value of the activity signal of each brain region for each subject at each time point is calculated, and this mean value is used as the activity intensity of that brain region for that subject at the current time point, denoted as . This leads to the formation of brain region activity vectors with timestamps. Therefore, the average functional signal of subject i's brain region at time t can be expressed as: Then the average functional signal of the brain regions of N subjects at time t is Then the set of average functional signals of brain regions of N subjects during the time interval [1, T] can be represented as: This data was used as training data to train the following Long Short-Term Memory (LSTM) network model. It is a three-dimensional array with dimension R. M ×N×T , where M, N, and T represent the number of brain regions, the number of subjects, and the duration of the time, respectively.

[0056] Step 3: Construct an LSTM-based effective connectivity brain network model. Use the brain region activity vectors of a single or multiple subjects as training data. Use the training data to train the LSTM-based effective connectivity brain network model. Use the weight parameters of the trained LSTM-based effective connectivity brain network model as brain effective connectivity relationship data.

[0057] Specifically, an efficient connectivity brain network model based on LSTM is constructed. This model introduces three gates: an input gate, a forget gate, and an output gate, as well as memory cells with the same shape as the hidden state to record additional information, such as... Figure 2 As shown.

[0058] The inputs to the gates of this effective brain network model are all the current time step input X. t Hidden state H from the previous time step t-1 The output is calculated by a fully connected layer with the sigmoid function (σ) as the activation function. The threshold values ​​of these three gate elements are [0,1].

[0059] S3.1, Input X according to the current time step. t Hidden state H from the previous time step t-1 Determine the input gate I for time step t. t Forgotten Gate F t and output gate O t The determination method is as follows:

[0060] I t =σ(X) t W xi +H t-1 W hi +b i (1)

[0061] Ft=σ(X t W xf +H t-1 W hf +b f (2)

[0062] O t =σ(X) t W xo +H t-1 W ho +b o (3)

[0063] Assume that the number of hidden units in the LSTM model is h, the batch size is m (m < M), and the dimension of the input data is N, that is, the number of subjects. Given the input X at time step t t ∈R m×N , that is, the average activity signals of N subjects in m brain regions at time step t, the hidden state H at the previous time step t-1 ∈R m×h , the memory cell C at the previous step t-1 ∈R m×h , according to formulas (1), (2), and (3), calculate the input gate I t , the forget gate F t , and the output gate O t , all of which have the dimension of R m×h , and the value range is [0, 1]. h is the number of hidden units, and W xi , W xf , W xo , W hi , W hf , W ho are weight parameters respectively, and b i , b f , b o are the bias parameters corresponding to each gate respectively. The dimension of the weight parameters W xi , W xf , W xo is all R N×h , and the dimension of W hi , W hf , W ho is all R h×h , and the dimension of the bias parameters b i , b f , b o is all R 1×h .

[0064] S3.2. Calculate the candidate memory cell according to the input X at the current time step t and the hidden state H at the previous time step t-1

[0065]

[0066] LSTM needs to calculate the candidate memory cell Its calculation is similar to that of the three gates, and the tanh function with a value range of [-1, 1] is used as the activation function. The candidate memory cell at time step t is calculated by the following formula:

[0067]

[0068] Candidate memory cell Its dimension is R m×h ​The value range is [-1, 1], and the weight parameter W xc The dimension is R N×h W hc The dimension is R h×h Deviation parameter b c The dimension is R 1×h .

[0069] S3.3, According to the forgetting gate F t Input gate I t and candidate memory cells Calculate the memory cell C at the current time step. t .

[0070] Current time step memory cell C t The calculation requires combining information from memory cells in the previous time step and candidate memory cells in the current time step. The flow of information is controlled through forgetting and input gates. The calculation formula is as follows:

[0071]

[0072] Among them, memory cells C t The dimension is R m×h , e indicates element-wise multiplication between matrices.

[0073] S3.4, based on output gate O t and the current time step memory cell C t Determine the hidden state H at the current time step. t .

[0074] After obtaining the memory cell at the current time step, the output gate is used to control the transition from the memory cell to the hidden state H. t The flow of information is used to obtain the hidden state at the current time step, and the calculation formula is as follows:

[0075]

[0076] Hidden state H t The dimension is R m×h .

[0077] S3.5. Use backpropagation and gradient descent algorithms to update the weights and biases of formulas (1)(2)(3)(4).

[0078] S3.6 Repeat the above training process until the maximum number of training iterations is reached or the hidden state H of two adjacent time steps is reached. t-1 and H t If the error between the input and output gates is less than a certain specific value, the trained Long Short-Term Memory (LSTM) network-based model is obtained. t Forgotten Gate F tand output gate O t Weight parameter W hi W hf W ho Since these three parameters are all square matrices, and the number of rows and columns equals the number of regions of interest (ROIs), these three square matrices can serve as three effective connection models between ROIs. Each value in each square matrix represents the strength of the effective connection, and the direction of the effective connection is from the brain region corresponding to the column containing that value to the brain region corresponding to the row containing that value.

[0079] Example 1

[0080] Taking the default network (DMN) as an example, we perform calculations for an efficient connectivity brain network model based on LSTM. Figure 3 , Figure 4 , Figure 5 These are the effective connectivity models of the DMN obtained by input gate, forget gate, and output gate, respectively. The DMN contains four brain regions: the medial prefrontal cortex (mPFC), the left posterior parietal cortex (LPC), the right posterior parietal cortex (RPC), and the posterior cingulate cortex (PCC). The arrows between the brain regions indicate the direction of the effective connectivity, and the numbers represent the strength of the effective connectivity.

[0081] Three subjects were selected for the experiment, and the number of brain regions was increased from 2 to 6. The computer configuration for running the program was: Intel Core i7-4790 CPU, frequency 3.60GHz, and 16.0GB of memory. Traditional dynamic causal modeling was run using MATLAB R2017b. The efficient connectivity brain network model based on LSTM proposed in this invention was run using the PyTorch 1.13.0 framework on a CPU, with 1000 training iterations. The running time of the two methods is compared as follows: Figure 6 As shown, the runtime of traditional dynamic causal modeling increases dramatically with the number of brain regions, while the runtime of the model proposed in this invention is much shorter than that of traditional methods, and the runtime increases slowly with the number of brain regions.

[0082] Traditional dynamic causal modeling assumes that brain activity at any given moment is only related to brain activity in the previous moment and external input, without considering previous states. For example... Figure 7 In traditional dynamic causal modeling, the influence on brain region Z2 includes the previous activity state S, the influence of brain region Z1 A, and external input B, and is independent of other brain regions. The LSTM model used in this invention has certain advantages in sequence modeling problems, possesses long-term memory capabilities, is simple to implement, and is more in line with the ideas of causal modeling. Figure 2 The LSTM model uses memory cells C t and hidden state H t The process of information transmission enables the memorization of different brain regions, demonstrating the causal relationship between brain regions.

[0083] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A method for constructing an efficient connectivity brain network model based on LSTM, characterized in that, Includes the following steps: Step 1: Preprocess the acquired raw brain function image data to remove various noises and obtain brain function image data with better imaging quality. Step 2: For the brain function image data obtained from Step 1, select its region of interest, use the brain function template to extract the time series of the corresponding brain region from the region of interest, and use it as the brain region activity signal. Then calculate the mean of the brain region activity signal at the current time point, and use it as the brain region activity intensity at the current time point, thereby forming a brain region activity vector with timestamp. Step 3: Train the constructed LSTM effective connectivity brain network model using the vectors of brain region activity to obtain the weight parameters of the trained LSTM effective connectivity brain network model, and use these weight parameters as brain effective connectivity relationship data. The weight parameters include the weight parameters of the input gate, forget gate, and output gate. These three weight parameters serve as three effective connection models between regions of interest. The weight parameters are square matrices, with the number of rows and columns equal to the number of regions of interest. Each value in each square matrix represents the strength of an effective connection, and the direction of an effective connection is from the brain region corresponding to the column to the brain region corresponding to the row.

2. The method for constructing an efficient connectivity brain network model based on LSTM according to claim 1, characterized in that, The preprocessing described in step 1 includes converting DICOM to Nifty and removing pre-processing components. n Multiple operations including image processing, slice timing, head motion correction, standardization, spatial smoothing, delinear drift, regression covariates, frequency domain filtering, and Scrubbing.

3. The method for constructing an efficient connectivity brain network model based on LSTM according to claim 1, characterized in that, The training method for the efficient connectivity brain network model based on LSTM described in step 3 is as follows: Step 3.1: Construct an efficient connectivity brain network model based on LSTM, which introduces an input gate, a forget gate, and an output gate; S3.2, Input X according to the current time step t Hidden state H from the previous time step t-1 Determine the input gate I for time step t. t Forgotten Gate F t and output gate O t ; S3.3, Input X according to the current time step t Hidden state H from the previous time step t-1 Calculate candidate memory cells S3.4, According to the forgetting gate F t Input gate I t and candidate memory cells Calculate the memory cell C at the current time step t ; S3.5, based on output gate O t and the current time step memory cell C t Determine the hidden state H at the current time step. t ; S3.

6. The backpropagation and gradient descent algorithms are used to update the weights and biases of the candidate memory cells of the input gate, forget gate, and output gate. S3.7 Repeat the above training process until the maximum number of training iterations is reached or the hidden state H of two adjacent time steps is reached. t-1 and H t If the error between the two is less than a certain specific value, an effective connectivity brain network model based on LSTM is obtained after training.

4. The method for constructing an efficient connectivity brain network model based on LSTM according to claim 3, characterized in that, Step S 3.2, the input gate I t Forgotten Gate F t and output gate O t The method for determining it is as follows: I t =σ(X t W xi +H t-1 W hi +b i ) F t =σ(X t W xf +H t-1 W hf +b f ) O t =σ(X t W xo +H t-1 W ho +b o ) Where σ is the sigmoid function, W xi W xf W xo W hi W hf W ho These are the weight parameters, b i b f b o These are the deviation parameters for each gate.

5. The method for constructing an efficient connectivity brain network model based on LSTM according to claim 3, characterized in that, Step S 3.3 describes the selection of memory cells. The method for determining it is as follows: Among them, W hc and W xc b is the weight parameter. c This is the deviation parameter.

6. The method for constructing an efficient connectivity brain network model based on LSTM according to claim 3, characterized in that, Step S3.4 describes the current time step memory cell C t The method for determining it is as follows: Based on the information from the memory cells of the previous time step and the candidate memory cells of the current time step, the flow of information is controlled through the forget gate and the input gate to determine the memory cells of the current time step.

7. The method for constructing an efficient connectivity brain network model based on LSTM according to claim 3, characterized in that, The hidden state H of the current time step in step S3.5 t The determination method is as follows: The output gate is used to control the memory cell at the current time step to the hidden state H. t The flow of information determines the hidden state of the current time step.

Images