Deep neural network topological structure constraint learning method

Abstract

The invention belongs to the field of artificial intelligence and neurobiology, and particularly relates to a deep neural network topological structure constraint learning method. According to the method, firstly, tested rs-fMRI data are collected and preprocessed, correlation coefficients between different brain regions are calculated to obtain a biological brain topological matrix, and a deep neural network model is constructed; then, a learning process of a biological brain topological structure constraint neural network is used to train a model, neural network parameters are updated by adopting a back propagation algorithm, and a loss function of the back propagation algorithm simultaneously comprises negative logarithm likelihood loss and topological matrix similarity loss; therefore, after the model training, the topological matrix of the neural network is deviated to the biological topological matrix, so that the technology of directly integrating the neurophysiology record into the artificial neural network is realized. According to the method, the technical blank of directly converting the neurophysiology record into the artificial neural network improvement is filled, so that the engineering performance of the neural network is improved.

Description

Technical Field

[0001] The present invention belongs to the fields of artificial intelligence and neurobiology, and specifically relates to a deep neural network topology structure constraint learning method. Background Art

[0002] Some key concepts in deep neural network research can be traced back to the study of the brain, and its research often draws inspiration from the field of neuroscience. Currently, these inspirations mainly focus on the functional structure of the network, the similarity of visual representations, or the biological plausibility of the network training process. There are also some studies inspired by biological network topology, but they mainly focus on the abstract similarity between the neural network topology and the biological brain network topology. For example, using a topological structure with abstract similarity to the biological brain network to build a neural network, rather than manually designing and building an artificial neural network, such as the Watts-Strogatz model or the Barabási-Albert model.

[0003] Although these methods utilize the network topology of the biological brain, they do so only at an abstract level and do not incorporate real neural data of the biological brain network topology. The brain has a typical global network, including modularity and short path lengths, which reflects how neuronal populations in different regions of the brain are interconnected. Computational neuroscience has also confirmed that brain networks are the basis of cognition, and the highly structured network topology of the biological brain is important prior knowledge that may be used to improve the engineering performance of neural networks. However, there is currently a lack of technology that can directly convert neurophysiological recordings into improvements in artificial neural networks. Therefore, how to integrate real biological brain topology data into artificial neural networks and improve the performance of artificial neural networks remains an urgent problem to be solved. Summary of the invention

[0004] In response to the above-mentioned problems or shortcomings, the present invention provides a deep neural network topology structure constraint learning method, which aims to solve the current lack of technology to directly integrate neurophysiological recordings into artificial neural networks, so as to improve the performance of artificial neural networks.

[0005] A deep neural network topology structure constraint learning method, the specific steps are as follows:

[0006] Step 1: Obtain biological brain topology matrix data and build a deep neural network model.

[0007] The acquired resting-state functional MRI data were preprocessed.

[0008] The preprocessed data is divided into m brain regions based on the brain partition template, where the value of m ranges from [2,116]. The biological brain topology matrix B is obtained by calculating the Pearson correlation coefficient between different brain regions. m×m. The matrix B m×m The calculation formula for the element in the i-th row and j-th column (i.e., the Pearson correlation coefficient between the i-th brain region and the j-th brain region) is:

[0009]

[0010] Where t i and t j are the time series of the i-th brain region and the j-th brain region, and are the average values ​​of the time series of the ith and jth brain regions, respectively.

[0011] RNNs recurrent neural network is used as the model of deep neural network, which consists of input layer, hidden layer and output layer. The state equation of its hidden layer is:

[0012] c t =W ih x t +b ih +W hh r t-1 +b hh , r t =f h (c t )

[0013] The output equation of the output layer is:

[0014] y t =f y (W y r t +b y )

[0015] where c t is the state variable of the neuron at time t, W ih , W hh and W y They are the neuron connection weight moments of the input layer, hidden layer, and output layer, respectively, and x t is the input of RNNs at time t, r t is the neural activity of the hidden layer at time t, b ih , b hh and b y are the bias items of the input layer, hidden layer, and output layer, respectively, y t is the output of RNNs at time t, f h (·) and f y (·) are the activation functions of the hidden layer and the output layer respectively.

[0016] Furthermore, the RNNs recurrent neural network: the number of neurons in the input layer is 2, the first neuron N1 is used to receive the one-dimensional vector of the task, and the second neuron N2 is used to receive the state instruction; when N2 receives 0, it indicates that the model is in the stimulus display stage of the cognitive task, and when it receives 1, it indicates that the model is in the reaction stage of the cognitive task. The number of neurons in the hidden layer is the same as the number of brain regions, which is m. The number of neurons in the output layer is 3, and 3 labels are output, defined as 1, 0, and -1; 1 indicates that the output is a match in the reaction stage of the cognitive task, -1 indicates that the output is a mismatch in the reaction stage of the cognitive task, and 0 indicates the output label of the stimulus display stage (non-reaction stage) of the cognitive task.

[0017] Step 2: Use the biological brain topology to constrain the learning process of the neural network and then train the model.

[0018] The cognitive task experimental paradigm used in biological experiments is used to train the deep neural network built in step 1 to perform working memory tasks in the field of life sciences. During training, the inputs of the deep neural network are:

[0019] The randomly generated number a is input to the input neuron N1, and 0 is input to the neuron N2. This stage is the stimulus display stage;

[0020] Next, the randomly generated number b is input to the input neuron N1, and 0 is input to the neuron N2. This stage is the stimulus display stage;

[0021] Next, the randomly generated number c is input to the input neuron N1, and 1 is input to neuron N2 at the same time. This stage is the reaction stage;

[0022] In the above input process, the corresponding correct model output is: if the randomly generated number c is the same as the randomly generated number a, the model response stage should output label 1, indicating a match; if the randomly generated number c is different from the randomly generated number a, the model response stage should output label -1, indicating a mismatch; in the stimulus display stage, the model should output label 0.

[0023] The randomly generated numbers a, b, and c are randomly generated d times to form d cognitive tasks as training sets for training the neural network.

[0024] At the same time, the randomly generated numbers a, b, and c are randomly generated e times to form e cognitive tasks as a test set for testing the performance of the trained neural network.

[0025] When using d of the above cognitive tasks as training sets to train the neural network, the back propagation algorithm is used to update the neural network parameters. The loss function of the back propagation algorithm also includes the negative log-likelihood loss cost task Similarity loss with topological matrix cost similarity, specifically defined as:

[0026] cost=cost task +αcost similarity

[0027] cost task =logp(y i |x i )

[0028]

[0029] Among them, α is an adjustable parameter; p(y i |x i ) indicates that the neural network input is x i When , the output is the correct output y i probability; For B m×m The mean of the matrix, W hh The mean of the matrix.

[0030] During training, the batch size is batch ≥ 32, and the number of iterations is epoch ≥ 100. When the number of iterations reaches epoch, the training is completed. The input data of the e cognitive tasks mentioned above as the test set are input into the input layer of the neural network respectively. The number of times the actual output of the corresponding neural network is equal to the correct output is counted as corrt, and the accuracy is corrt / e. At the same time, the cost after the training is completed is calculated. similarity , as the similarity between the topological structure of the neural network and the biological brain. The above two indicators are used as the final performance parameters of the neural network.

[0031] Furthermore, the preprocessing methods and purposes of step 1 are: removing time points (avoiding the influence of the initialization time stage), time layer correction (improving the temporal consistency of the signal), head motion correction (reducing artifacts caused by head motion), spatial standardization (converting to standard space to facilitate comparison of data from different individuals), smoothing (reducing signal noise and improving signal-to-noise ratio), removing linear drift (reducing the influence of false signals) and filtering (removing noise and irrelevant frequency components).

[0032] Furthermore, in step 2, the back propagation algorithm uses a gradient descent algorithm to iteratively update the loop layer (neural network parameters), and the formula of the gradient descent algorithm is:

[0033]

[0034] where θ k represents the parameter set of the neural network at the kth iteration, cost(θ k ) represents the loss function cost for θk The specific expression of I r Represents the network learning rate.

[0035] Furthermore, the network learning rate controls the amplitude of each iteration parameter update. Too high or too low a learning rate will reduce the training effect of the model. In the iterative training process, the present invention attenuates the learning rate to improve the network training efficiency. r The attenuation formula is:

[0036]

[0037] In summary, the goal of the present invention through neural network training is to minimize the loss function. An indicator for measuring the similarity between the neural network topological matrix and the biological topological matrix (neurophysiological recording) is added to the loss function to minimize the negative log-likelihood loss and minimize the topological matrix similarity loss. Therefore, after the model is trained, the topological matrix of the neural network will be biased towards the biological topological matrix, thereby realizing the technology of directly integrating neurophysiological recordings into artificial neural networks, filling the technical gap of directly converting neurophysiological recordings into artificial neural network improvements, thereby improving the engineering performance of neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flow chart of the present invention.

[0039] Figure 2 This is a schematic diagram of the training of the deep neural network of the present invention.

[0040] Figure 3 4 is a comparison chart of the accuracy of the embodiment of the present invention and the control group.

[0041] Figure 4 It is a comparison diagram of the similarity of the topological matrices of the embodiment of the present invention and the control group. DETAILED DESCRIPTION

[0042] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0043] A deep neural network topology structure constraint learning method, the specific steps are as follows:

[0044] Step 1: Obtain biological brain topology matrix data and build a deep neural network model

[0045] Thirty subjects were selected as data collection objects, and rs-fMRI data were collected using a magnetic resonance imaging device and then preprocessed. The preprocessing steps were: time point removal, time layer correction, head motion correction, spatial standardization, smoothing, linear drift removal, and filtering.

[0046] The preprocessed data were divided into m = 90 brain regions based on the AAL brain partition template, and the biological brain topology matrix B was obtained by calculating the correlation coefficients between different brain regions. 90×90 . The matrix B 90×90 The calculation formula for the element in the i-th row and j-th column is:

[0047]

[0048] Use Python to build RNNs recurrent neural network as a model of deep neural network, which is generally composed of input layer, hidden layer and output layer. The state equation of its hidden layer is:

[0049] c t =W ih x t +b ih +W hh r t-1 +b hh , r t =f h (c t )

[0050] The output equation of the output layer is:

[0051] y t =f y (W y r t +b y )

[0052] The number of neurons in the RNNs input layer is 2. The first neuron N1 is used to receive the one-dimensional vector of the task, and the second neuron N2 is used to receive the state instruction. When N2 receives 0, it means that the model is in the stimulus display stage of the cognitive task, and when it receives 1, it means that the model is in the reaction stage of the cognitive task. The number of neurons in the hidden layer is the same as the number of brain regions, which is m = 90. The number of neurons in the output layer is 3, and the output of three labels is 1, 0, and -1. 1 indicates that the output is matched in the reaction stage of the cognitive task, -1 indicates that the output is mismatched in the reaction stage of the cognitive task, and 0 indicates the output label of the stimulus display stage of the cognitive task.

[0053] Step 2: Use the biological brain topology to constrain the learning process of the neural network and then train the model. Figure 2 The specific training diagram of the deep neural network of the present invention is as follows: RNN is trained to perform working memory tasks in the field of life sciences. During the back propagation process of the network, the loss function simultaneously includes negative log-likelihood loss and topological matrix similarity loss calculated by the Pearson correlation coefficient. The goal of network training is to minimize negative log-likelihood loss and topological matrix similarity loss.

[0054] The cognitive task experimental paradigm used in biological experiments is used to train the deep neural network built in step 1 to perform working memory tasks in the field of life sciences. During training, the inputs of the deep neural network are:

[0055] The randomly generated number a is input into the input neuron N1, and 0 is input into neuron N2 at the same time. This stage is the stimulus display stage.

[0056] Next, the randomly generated number b is input to the input neuron N1, and 0 is input to the neuron N2. This stage is the stimulus display stage.

[0057] Next, the randomly generated number c is input to the input neuron N1, and 1 is input to the neuron N2 at the same time. This stage is the reaction stage.

[0058] In the above input process, the corresponding correct model output is: if the randomly generated number c is the same as the randomly generated number a, the model response stage should output label 1, indicating a match; if the randomly generated number c is different from the randomly generated number a, the model response stage should output label -1, indicating a mismatch; in the stimulus display stage, the model should output label 0.

[0059] The randomly generated numbers a, b, and c are randomly generated d(800) times to form d(800) cognitive tasks as training sets for training the neural network.

[0060] At the same time, the randomly generated numbers a, b, and c are randomly generated e(200) times to form e(200) of the above cognitive tasks as a test set for testing the performance of the trained neural network.

[0061] When using the 800 cognitive tasks mentioned above as training sets to train the neural network, the gradient descent algorithm is used to update the neural network parameters. The loss function of the gradient descent algorithm also includes the negative log-likelihood loss cost. task Similarity loss with topological matrix cost similarity , specifically:

[0062] cost=cost task +αcost similarity

[0063] cost task =logp(y i |x i )

[0064]

[0065] Among them, α is 0.01, p(y i |x i ) indicates that the neural network input is x iWhen , the output is the correct output y i probability; For B 90×90 The mean of the matrix, W hh The mean of the matrix.

[0066] During training, the batch size is batch = 128 and the number of iterations is epoch = 350. The formula of the gradient descent algorithm is:

[0067]

[0068] where θ k represents the parameter set of the neural network at the kth iteration, cost(θ k ) represents the loss function cost for θ k The specific expression of I r In this embodiment, the network learning rate is I during the iterative training process. r The attenuation formula is:

[0069]

[0070] When the number of iterations reaches epoc, the training is completed. The input data of the 200 cognitive tasks mentioned above as the test set are input into the input layer of the neural network respectively. The number of times the actual output of the corresponding neural network is equal to the correct output corrt and the accuracy corrt / e are counted. At the same time, the cost after the training is completed is calculated. similarity , as the similarity between the topological structure of the neural network and the biological brain. The above two indicators are used as the final performance parameters of the neural network.

[0071] The experimental results of the traditional method and the present method were further compared. The experimental results show that compared with the traditional method, the accuracy rate and similarity with the biological brain of the present method are the highest. Table 1 shows the accuracy rate and similarity with the biological brain of the present embodiment method and the traditional method.

[0072] Table 1:

[0073] method Accuracy <![CDATA[Cost with the topological structure of the biological brain similarity > Embodiments of the present invention 67.85％ 53.91％ Traditional training methods 66.96％ 4.63％

[0074] In order to demonstrate the superiority of the present invention, several different situations are set as comparison models. As a baseline comparison model, the present invention deletes the topological matrix similarity loss, that is, the parameter α is set to 0 (defined as "None"), and the method is used as a baseline. Since the method of the present invention belongs to a regularization method in a broad sense, a commonly used regularization method, i.e., L2 regularization (defined as "L2"), is selected as a comparison.

[0075] Finally, the present invention also uses a randomly generated matrix (defined as "Random") and a method of shuffling the biological brain topology matrix (defined as "Shuffle") as a control. The comparison of the accuracy and the similarity of the topology matrix is ​​as follows Figure 3 , Figure 4 The experimental results show that the method of the present invention can significantly improve the accuracy of the model and the similarity of the topological matrix.

[0076] It can be seen from the above embodiments that the present invention fills the technical gap of directly converting neurophysiological records into improved artificial neural networks. The recurrent neural network has been used as a standard model for modeling biological advanced cognition in the field of life sciences, improving its performance in cognitive tasks (such as working memory tasks commonly used in studying human cognitive levels), and improving the similarity between the topological matrix of the neural network and the biological topological matrix, which can improve its brain-likeness, allowing researchers to obtain a more brain-like model, which is more conducive to further research. As in real biological experiments, researchers are not allowed to perform brain resection on subjects, but simulation can be performed in a brain-like model, deleting certain neurons of the brain-like neural network, and conducting research from the perspective of simulation, and the credibility of the conclusion depends on the brain-likeness of the neural network. It can be seen that the present invention improves the engineering performance of the neural network.

Claims

1. A deep neural network topology constraint learning method, characterized in that: The specific steps are as follows: Step 1: Obtain biological brain topology matrix data and build a deep neural network model; The acquired resting-state functional magnetic resonance imaging data were preprocessed; the preprocessed data were divided into m brain regions based on the brain partition template, 2≤m≤116, and the biological brain topology matrix B was obtained by calculating the Pearson correlation coefficient between different brain regions. m ×m ; where matrix B m×m The calculation formula for the Pearson correlation coefficient of the i-th row and column element in , that is, the i-th brain region and the i-th brain region, is: Where t i and t j are the time series of the i-th brain region and the j-th brain region, and are the average values ​​of the time series of the i-th brain region and the j-th brain region, respectively; RNNs recurrent neural network is used as the model of deep neural network, which consists of input layer, hidden layer and output layer; the state equation of its hidden layer is: c t =W ih x t +b ih +W hh r t-1 +b hh ,r t =f h (c t ) The output equation of the output layer is: y t =f y (W y r t +b y ) where c t is the state variable of the neuron at time t, W ih , W hh and W y They are the neuron connection weight moments of the input layer, hidden layer, and output layer, respectively, and x t is the input of RNNs at time t, r t is the neural activity of the hidden layer at time t, b ih 、b hh and b y are the bias items of the input layer, hidden layer, and output layer, respectively, y t is the output of RNNs at time t, f h (·) and f y (·) are the activation functions of the hidden layer and the output layer respectively; Step 2: Use the biological brain topology to constrain the learning process of the neural network and then train the model; The cognitive task experimental paradigm used in biological experiments is used to train the deep neural network built in step 1 to perform working memory tasks in the field of life sciences; during training, the inputs of the deep neural network are: The randomly generated number a is input to the input neuron N1, and 0 is input to the neuron N2. This stage is the stimulus display stage; Next, the randomly generated number b is input to the input neuron N1, and 0 is input to the neuron N2. This stage is the stimulus display stage; Next, the randomly generated number c is input to the input neuron N1, and 1 is input to neuron N2 at the same time. This stage is the reaction stage; In the above input process, the corresponding correct model output is: if the randomly generated number c is the same as the randomly generated number a, the model response stage should output label 1, indicating a match; if the randomly generated number c is different from the randomly generated number a, the model response stage should output label -1, indicating a mismatch; in the stimulus display stage, the model should output label 0; Randomly generate the randomly generated numbers a, b, and c d times to form d cognitive tasks as training sets for training the neural network; At the same time, the randomly generated numbers a, b, and c are randomly generated e times to form e cognitive tasks as a test set for testing the performance of the trained neural network; When using d of the above cognitive tasks as training sets to train the neural network, the back propagation algorithm is used to update the neural network parameters. The loss function of the back propagation algorithm also includes the negative log-likelihood loss cost task Similarity loss with topological matrix cost similarity , specifically defined as: cost=cost task +αcost similarity cost task =log p(y i |x i ) Among them, α is an adjustable parameter; p(y i |x i ) indicates that the neural network input is x i When , the output is the correct output y i probability; For B m×m The mean of the matrix, W hh The mean of the matrix; During training, the batch size is batch ≥ 32, and the number of iterations is epoch ≥ 100. When the number of iterations reaches epoch, the training is completed. The input data of the E cognitive tasks mentioned above as the test set are input into the input layer of the neural network respectively. The number of times the actual output of the corresponding neural network is equal to the correct output is counted as corrt, and the accuracy is corrt / e. At the same time, the cost after the training is completed is calculated. similarity , as the similarity in topological structure between neural networks and biological brains.

2. The deep neural network topology structure constraint learning method according to claim 1, characterized in that: The preprocessing method of step 1 includes: removing time points, time layer correction, head motion correction, spatial standardization, smoothing, removing linear drift and filtering.

3. The deep neural network topology structure constraint learning method according to claim 1, characterized in that: The RNNs recurrent neural network in step 1 specifically has: the number of input layer neurons is 2, the first neuron N1 is used to receive the one-dimensional vector of the task, the second neuron N2 is used to receive the state instruction, when N2 receives 0, it means that the model is in the stimulus display stage of the cognitive task, and when it receives 1, it means that the model is in the reaction stage of the cognitive task; the number of hidden layer neurons is the same as the number of brain regions, which is m; the number of output layer neurons is 3, and 3 labels are output, which are defined as 1, 0, and -1; 1 indicates that the output is matched in the reaction stage of the cognitive task, -1 indicates that the output is mismatched in the reaction stage of the cognitive task, and 0 indicates the output label of the stimulation display stage of the cognitive task.

4. The deep neural network topology structure constraint learning method according to claim 1, characterized in that: In step 1, the brain partition template is AAL.

5. The deep neural network topology structure constraint learning method according to claim 1, characterized in that: The back propagation algorithm in step 2 adopts the gradient descent algorithm, and the formula of the gradient descent algorithm is: where θ k represents the parameter set of the neural network at the kth iteration, cost(θ k ) represents the loss function cost for θ k The specific expression of I r Represents the network learning rate.

6. The deep neural network topology structure constraint learning method according to claim 5, characterized in that: The network learning rate I r The attenuation formula is:

Images