A method for learning the constraints of a deep neural network topology

Abstract

The present invention belongs to the fields of artificial intelligence and neurobiology, and particularly relates to a method for learning the constraints of a deep neural network topology. First, the rs-fMRI data of the subject is collected and preprocessed, the correlation coefficients between different brain regions are calculated to obtain the biological brain topology matrix, and a deep neural network model is constructed; then, the learning process of the neural network is constrained by the biological brain topology to train the model, and the backpropagation algorithm is used to update the neural network parameters. The loss function of the backpropagation algorithm simultaneously includes the negative log-likelihood loss and the topology matrix similarity loss. Therefore, after the model is trained, the topology matrix of the neural network will tend to be the biological topology matrix, thereby realizing the technology of directly integrating neurophysiological recordings into artificial neural networks. The present invention fills the technical gap in directly converting neurophysiological recordings into improvements in artificial neural networks, thereby improving the engineering performance of neural networks.

Description

Technical Field

[0001] The present invention belongs to the fields of artificial intelligence and neurobiology, and particularly relates to a method for learning constraints on the topological structure of a deep neural network. Background Art

[0002] Some key concepts in the research of deep neural networks can be traced back to the research of the brain, and its research often obtains inspiration from the field of neuroscience. At present, these inspirations mainly focus on the functional structure of the network, the similarity of visual representations, or the biological rationality of the network training process, etc. There are also some studies inspired by biological network topologies, but mainly focus on the abstract similarity between neural network topologies and biological brain network topologies. For example, constructing a neural network using a topological structure that has an abstract similarity to a biological brain network, rather than manually designing and constructing an artificial neural network, such as the Watts-Strogatz model or the Barabási-Albert model.

[0003] Although these methods utilize the network topologies of biological brains, they are only at an abstract level and do not incorporate the real neural data of biological brain network topologies. The brain has typical global networks, including modularity and short path lengths, which reflect how neuron populations in different regions of the brain are interconnected. Computational neuroscience has also confirmed that brain networks are the basis of cognition, and the highly structured network topology of the biological brain is important prior knowledge, and this prior knowledge may be able to be used to improve the engineering performance of neural networks. However, there is currently a lack of technology to directly translate neurophysiological recordings into improvements in artificial neural networks. Therefore, how to integrate real biological brain topology data into artificial neural networks and improve the performance of artificial neural networks remains an urgent problem to be solved. Summary of the Invention

[0004] In view of the above existing problems or deficiencies, the present invention provides a method for learning constraints on the topological structure of a deep neural network, aiming to solve the current lack of technology to directly integrate neurophysiological recordings into artificial neural networks in order to improve the performance of artificial neural networks.

[0005] A method for learning constraints on the topological structure of a deep neural network, the specific steps are as follows:

[0006] Step 1, obtain biological brain topology matrix data and build a deep neural network model.

[0007] Preprocess the obtained resting-state functional magnetic resonance data.

[0008] Based on a brain partition template, divide the preprocessed data into m brain regions, where the value range of m is [2, 116], and obtain a biological brain topology matrix B by calculating the Pearson correlation coefficient between different brain regions m×mAmong them, matrix B m×m The calculation formula for the element in the i-th row and j-th column of (that is, the Pearson correlation coefficient between the i-th brain region and the j-th brain region) is as follows:

[0009]

[0010] In the formula, t i and t j are the time series of the i-th brain region and the j-th brain region respectively, and are the average values of the time series of the i-th brain region and the j-th brain region respectively.

[0011] The RNNs recurrent neural network is used as the model of the deep neural network, which consists of an input layer, a hidden layer, and an output layer. The state equation of its hidden layer is:

[0012] c t = W ih x t + b ih + W hh r t-1 + b hh , r t = f h (c t )

[0013] The output equation of the output layer is:

[0014] y t = f y (W y r t + b y )

[0015] Among them, c t is the state variable of the neuron at time t, W ih , W hh and W y are the neuron connection weight matrices of the input layer, the hidden layer, and the output layer respectively, x t is the input of the RNNs at time t, r t is the neural activity of the hidden layer at time t, b ih , b hh and b y are the bias terms of the input layer, the hidden layer, and the output layer respectively, y t is the output of the RNNs at time t, f h (·) and f y (·) are the activation functions of the hidden layer and the output layer respectively.

[0016] Furthermore, the RNNs (Recurrent Neural Networks): The number of neurons in the input layer is 2. The first neuron N1 is used to receive a one-dimensional vector of the task, and the second neuron N2 is used to receive a state instruction. When N2 receives 0, it indicates that the model is in the stimulus display stage of the cognitive task, and when it receives 1, it indicates that the model is in the response stage of the cognitive task. The number of neurons in the hidden layer is the same as the number of brain regions, which is m. The number of neurons in the output layer is 3, and 3 labels are output, defined as 1, 0, -1; 1 indicates a match in the output during the response stage of the cognitive task, -1 indicates a mismatch in the output during the response stage of the cognitive task, and 0 indicates the output label during the stimulus display stage (non-response stage) of the cognitive task.

[0017] Step 2: Use the biological brain topological structure to constrain the learning process of the neural network and then train the model.

[0018] Apply the cognitive task experimental paradigm adopted in biological experiments to train the deep neural network built in Step 1 to perform the working memory task in the field of life sciences. During training, the inputs to the deep neural network are as follows:

[0019] Randomly generate a number a and input it into the input neuron N1, and at the same time input 0 into the neuron N2. This stage is the stimulus display stage;

[0020] Next, randomly generate a number b and input it into the input neuron N1, and at the same time input 0 into the neuron N2. This stage is the stimulus display stage;

[0021] Next, randomly generate a number c and input it into the input neuron N1, and at the same time input 1 into the neuron N2. This stage is the response stage;

[0022] In the above input process, the correct output of the corresponding model is: If the randomly generated number c is the same as the randomly generated number a, the model should output the label 1 during the response stage, indicating a match; if the randomly generated number c is different from the randomly generated number a, the model should output the label -1 during the response stage, indicating a mismatch; during the stimulus display stage, the model should output the label 0.

[0023] Randomly generate the above randomly generated numbers a, b, c d times, and form d of the above cognitive tasks as a training set to train the neural network.

[0024] At the same time, randomly generate the above randomly generated numbers a, b, c e times, and form e of the above cognitive tasks as a test set to test the performance of the trained neural network.

[0025] When using d of the above cognitive tasks as a training set to train the neural network, use the backpropagation algorithm to update the neural network parameters. The loss function of the backpropagation algorithm includes both the negative log-likelihood loss cost task and the topological matrix similarity loss cost similarity, specifically defined as:

[0026] cost = cost task + αcost similarity

[0027] cost task = logp(y i |x i )

[0028]

[0029] where α is an adjustable parameter; p(y i |x i ) represents the probability that the output of the neural network is the correct output y i when the input is x i ; is the mean of matrix B m×m , is the mean of matrix W hh .

[0030] During training, the batch size is batch ≥ 32, and the number of iterations is epoch ≥ 100. After the number of iterations reaches epoch, the training is completed. The input data of e cognitive tasks as the test set are respectively input into the input layer of the neural network, and the number of times that the actual output of the corresponding neural network is equal to the correct output is counted as corrt. Then the accuracy rate is corrt / e; at the same time, calculate the cost similarity after training is completed, as the similarity between the neural network and the biological brain in topological structure. The above two indicators are used as the final performance parameters of the neural network.

[0031] Furthermore, the preprocessing methods and purposes of step 1 are as follows: removing time points (to avoid the influence of the initialization time stage), time layer correction (to improve the time consistency of the signal), head motion correction (to reduce artifacts caused by head motion), spatial normalization (to convert to the standard space for comparing data of different individuals), smoothing (to reduce signal noise and improve the signal-to-noise ratio), removing linear drift (to reduce the influence of pseudo signals), and filtering (to remove noise and irrelevant frequency components).

[0032] Furthermore, in step 2, the backpropagation algorithm uses the gradient descent algorithm to iteratively update the recurrent layer (neural network parameters). The formula of the gradient descent algorithm is:

[0033]

[0034] where θ k represents the set of neural network parameters at the k-th iteration, and cost(θ k ) represents the loss function cost with respect to θk The specific representation, I r represents the network learning rate.

[0035] Furthermore, the network learning rate controls the amplitude of parameter update in each iteration. An overly high or low learning rate will reduce the training effect of the model. In the iterative training process of the present invention, the learning rate is decayed to improve the network training efficiency. The network learning rate I r The decay formula is:

[0036]

[0037] In summary, the objective of the present invention through neural network training is to minimize the loss function. An index for measuring the similarity between the neural network topology matrix and the biological topology matrix (neurophysiological recording) is added to the loss function to minimize the negative log-likelihood loss and the topology matrix similarity loss. Therefore, after the model is trained, the topology matrix of the neural network will tend to the biological topology matrix, thereby realizing the technology of directly integrating neurophysiological recordings into artificial neural networks, filling the technical gap in directly converting neurophysiological recordings into improvements of artificial neural networks, and thus enhancing the engineering performance of neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flowchart of the present invention.

[0039] Figure 2 is a schematic block diagram of the training of the deep neural network of the present invention.

[0040] Figure 3 is a comparison chart of the correct rates of the embodiments and the control group of the present invention.

[0041] Figure 4 is a comparison chart of the topology matrix similarities of the embodiments and the control group of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] The present invention will be further described in detail below with reference to the drawings and embodiments.

[0043] A method for learning the topology structure constraint of a deep neural network, the specific steps are as follows:

[0044] Step 1: Obtain biological brain topology matrix data and build a deep neural network model

[0045] Thirty subjects are selected as data collection objects, and rs-fMRI data is collected using a nuclear magnetic resonance imaging device and then preprocessed. The preprocessing steps are, in sequence: removing time points, time slice correction, head motion correction, spatial normalization, smoothing, removing linear drift, and filtering.

[0046] The preprocessed data is divided into m = 90 brain regions based on the AAL brain partition template, and the biological brain topology matrix B is obtained by calculating the correlation coefficients between different brain regions. 90×90 Among them, for matrix B 90×90 the calculation formula for the element in the i-th row and j-th column is:

[0047]

[0048] Use Python to build an RNNs recurrent neural network as the model of the deep neural network, which generally consists of an input layer, a hidden layer, and an output layer. The state equation of its hidden layer is:

[0049] c t = W ih x t + b ih + W hh r t-1 + b hh , r t = f h (c t )

[0050] The output equation of the output layer is:

[0051] y t = f y (W y r t + b y )

[0052] The number of neurons in the input layer of RNNs is 2. The first neuron N1 is used to receive the one-dimensional vector of the task, and the second neuron N2 is used to receive the status instruction. When N2 receives 0, it means the model is in the stimulus display stage of the cognitive task, and when it receives 1, it means the model is in the response stage of the cognitive task. The number of neurons in the hidden layer is the same as the number of brain regions, which is m = 90. The number of neurons in the output layer is 3, and 3 labels are output as 1, 0, -1. 1 means the output is a match in the response stage of the cognitive task, -1 means the output is a mismatch in the response stage of the cognitive task, and 0 means the output label in the stimulus display stage of the cognitive task.

[0053] Step 2: Use the biological brain topology structure to constrain the learning process of the neural network and then train the model. Figure 2 This is the specific training schematic diagram of the deep neural network of the present invention: Train the RNN to perform the working memory task in the field of life sciences. During the backpropagation process of the network, the loss function simultaneously includes the negative log-likelihood loss and the topology matrix similarity loss calculated by the Pearson correlation coefficient. The goal of network training is to minimize the negative log-likelihood loss and the topology matrix similarity loss.

[0054] The cognitive task experimental paradigm used in biological experiments is applied to the deep neural network built in training step 1 to perform the working memory task in the field of life sciences. During training, the inputs to the deep neural network are, in sequence:

[0055] A randomly generated number a is input into the input neuron N1, and at the same time, 0 is input into the neuron N2. This stage is the stimulus presentation stage.

[0056] Next, a randomly generated number b is input into the input neuron N1, and at the same time, 0 is input into the neuron N2. This stage is the stimulus presentation stage.

[0057] Next, a randomly generated number c is input into the input neuron N1, and at the same time, 1 is input into the neuron N2. This stage is the response stage.

[0058] In the above input process, the corresponding correct output of the model is: if the randomly generated number c is the same as the randomly generated number a, the model should output the label 1 in the response stage, indicating a match; if the randomly generated number c is different from the randomly generated number a, the model should output the label -1 in the response stage, indicating a mismatch; in the stimulus presentation stage, the model should output the label 0.

[0059] The above randomly generated numbers a, b, and c are randomly generated d(800) times, and d(800) of the above cognitive tasks are formed as a training set to train the neural network.

[0060] At the same time, the above randomly generated numbers a, b, and c are randomly generated e(200) times, and e(200) of the above cognitive tasks are formed as a test set to test the performance of the trained neural network.

[0061] When using 800 of the above cognitive tasks as a training set to train the neural network, the gradient descent algorithm is used to update the neural network parameters. The loss function of the gradient descent algorithm simultaneously includes the negative log-likelihood loss cost task and the topological matrix similarity loss cost similarity , specifically:

[0062] cost = cost task + αcost similarity

[0063] cost task = logp(y i |x i )

[0064]

[0065] where α is taken as 0.01, and p(y i |x i ) represents that the input of the neural network is x iWhen it is correct, the output is the correct output y i probability; is B 90×90 the mean of the matrix, is W hh the mean of the matrix.

[0066] During training, the batch size batch = 128 and the number of iterations epoch = 350. The formula for the gradient descent algorithm is:

[0067]

[0068] where θ k represents the set of parameters of the neural network at the k-th iteration, and cost(θ k ) represents the specific representation of the loss function cost with respect to θ k , and I r represents the network learning rate. In this embodiment, the network learning rate I r decays according to the formula:

[0069]

[0070] When the number of iterations reaches epoc, the training is completed. Then, the input data of 200 cognitive tasks used as the test set are respectively input into the input layer of the neural network, and the number of times corrt when the actual output of the corresponding neural network is equal to the correct output is counted. The accuracy rate is corrt / e. At the same time, cost similarity after the training is completed is calculated as the similarity between the neural network and the biological brain in terms of topological structure. The above two indicators are used as the final performance parameters of the neural network.

[0071] The experimental results of the traditional method and the present method are further compared. The experimental results show that compared with the traditional method, the present invention method has the highest accuracy rate and similarity with the biological brain. Table 1 shows the results of the accuracy rate and similarity with the biological brain of the method of the present invention embodiment and the traditional method.

[0072] Table 1:

[0073] Method Accuracy rate <![CDATA[Cost with the biological brain topology similarity > Method of the embodiment of the present invention 67.85％ 53.91％ Traditional training method 66.96％ 4.63％

[0074] To prove the superiority of the present invention, several different situations are set as comparison models. As the baseline comparison model, the present invention deletes the topological matrix similarity loss, that is, the parameter α is set to 0 (defined as "None"), and this method is used as the baseline. Since the method of the present invention belongs to the regularization method in a broad sense, a commonly used regularization method, that is, L2 regularization (defined as "L2"), is selected for comparison.

[0075] Finally, the present invention also uses the methods of randomly generating a matrix (defined as "Random") and shuffling the biological brain topology matrix (defined as "Shuffle") as controls respectively. The comparison of the accuracy rate and the topology matrix similarity is as Figure 3 , Figure 4 shown. The experimental results show that the method of the present invention can significantly improve the accuracy rate of the model and the topology matrix similarity.

[0076] As can be seen from the above embodiments, the present invention fills the technical gap in directly converting neurophysiological recordings into improvements in artificial neural networks. The recurrent neural network has been used as a standard model for modeling biological higher-level cognition in the field of life sciences. Improving its performance in cognitive tasks (such as the working memory task commonly used in studying human cognitive levels), and improving the similarity between the topology matrix of the neural network and the biological topology matrix can enhance its brain-likeness, enabling researchers to obtain a more brain-like model, which is more conducive to further research. In real biological experiments, researchers are not allowed to perform brain region resection on the subjects, but it can be simulated in a brain-like model by deleting certain neurons in the brain-like neural network. From the perspective of simulation, research can be carried out, and the credibility of the conclusion depends on the brain-likeness of the neural network. It can be seen that the present invention improves the engineering performance of the neural network.

Claims

1. A method for learning constraints on the topology of a deep neural network, characterized in that, The specific steps are as follows: Step 1: Obtain the biological brain topological matrix data and build a deep neural network model; Preprocess the acquired resting-state functional magnetic resonance data; divide the preprocessed data into m brain regions based on a brain parcellation template, where 2 ≤ m ≤ 116, and obtain a biological brain topological matrix B by calculating the Pearson correlation coefficients between different brain regions m ×m ; where for matrix B m×m the element in the i-th row and j-th column is the calculation formula for the Pearson correlation coefficient between the i-th brain region and the j-th brain region: where t i and t j are the time series of the i-th brain region and the j-th brain region respectively, and are the average values of the time series of the i-th brain region and the j-th brain region respectively; Use the RNNs recurrent neural network as the model of the deep neural network, which consists of an input layer, a hidden layer, and an output layer; the state equation of its hidden layer is: c t = W ih x t + b ih + W hh r t-1 + b hh ,R t = F h (C t ) The output equation of the output layer is: y t = f y (W y r t + b y ) where c t is the state variable of the neuron at time t, W ih , W hh and W y are the neuron connection weight matrices of the input layer, hidden layer, and output layer respectively, x t is the input of the RNNs at time t, r t is the neural activity of the hidden layer at time t, b ih , b hh and b y are the bias terms of the input layer, hidden layer, and output layer respectively, y t is the output of the RNNs at time t, f h (·) and f y (·) are the activation functions of the hidden layer and output layer respectively; The resting-state functional magnetic resonance data is rs-fMRI data; the preprocessing steps are, in sequence: removing time points, time slice correction, head motion correction, spatial normalization, smoothing, removing linear drift, and filtering; Step 2: Use the biological brain topological structure to constrain the learning process of the neural network and then train the model; Use the cognitive task experimental paradigm adopted in biological experiments to train the deep neural network built in Step 1 to perform the working memory task in the field of life sciences; during training, the inputs to the deep neural network are, in sequence: Randomly generate a number a and input it into the input neuron N1, and at the same time input 0 into the neuron N2. This stage is the stimulus presentation stage; Next, randomly generate a number b and input it into the input neuron N1, and at the same time input 0 into the neuron N2. This stage is the stimulus presentation stage; Next, randomly generate a number c and input it into the input neuron N1, and at the same time input 1 into the neuron N2. This stage is the response stage; In the above input process, the correct outputs of the corresponding model are: if the randomly generated number c is the same as the randomly generated number a, the model should output the label 1 in the response stage, indicating a match; if the randomly generated number c is different from the randomly generated number a, the model should output the label -1 in the response stage, indicating a mismatch; in the stimulus presentation stage, the model should output the label 0; Randomly generate d times of the above randomly generated numbers a, b, c, and form d of the above cognitive tasks as a training set for training the neural network; At the same time, randomly generate e times of the above randomly generated numbers a, b, c, and form e of the above cognitive tasks as a test set for testing the performance of the trained neural network; When using the above d cognitive tasks as a training set to train a neural network, the parameters of the neural network are updated using the backpropagation algorithm, and the loss function of the backpropagation algorithm includes both the negative log-likelihood loss cost task and the topological matrix similarity loss cost similarity , which is specifically defined as: cost = cost task + αcost similarity cost task = log p(y i | x i ) where α is an adjustable parameter; p(y i |x i ) represents the probability that when the input of the neural network is x i , the output is the correct output y i ; is the mean of matrix B m×m , and is the mean of matrix W hh ; During training, the batch size is batch≥32, and the number of epochs is epoch≥100; after the number of epochs reaches epoch, the training is completed. Then, the input data of e cognitive tasks used as the test set are respectively input into the input layer of the neural network. The number of times when the actual output of the corresponding neural network is equal to the correct output is counted as corrt, and the accuracy rate is corrt / e; at the same time, the cost after training is calculated. similarity , as the similarity between the neural network and the biological brain in topological structure; The specific training objective of the deep neural network model: Train the RNNs to perform the working memory task in the field of life sciences. During the backpropagation process of the network, minimize the negative log-likelihood loss and the topological matrix similarity loss; and use the finally trained deep neural network model to build a brain-like model for biological simulation experiments.

2. The method for learning with constraints on the topological structure of a deep neural network as described in claim 1, wherein: The RNNs recurrent neural network in Step 1 specifically has: The number of neurons in the input layer is 2. The first neuron N1 is used to receive the one-dimensional vector of the task, and the second neuron N2 is used to receive the state instruction. When N2 receives 0, it means the model is in the stimulus presentation stage of the cognitive task, and when it receives 1, it means the model is in the response stage of the cognitive task; the number of neurons in the hidden layer is the same as the number of brain regions, which is m; the number of neurons in the output layer is 3, and 3 labels are output, defined as 1, 0, -1; 1 indicates a match in the response stage of the cognitive task, -1 indicates a mismatch in the response stage of the cognitive task, and 0 indicates the output label in the stimulus presentation stage of the cognitive task.

3. The method for learning the constraints of the deep neural network topology structure according to claim 1, wherein: The brain partition template in Step 1 is AAL.

4. The method for learning the constraints of the deep neural network topology structure according to claim 1, wherein: In step 2, the backpropagation algorithm uses the gradient descent algorithm, and the formula of the gradient descent algorithm is: where θ k represents the set of parameters of the neural network at the k-th iteration, and cost(θ k ) represents the specific representation of the loss function cost with respect to θ k , and I r represents the network learning rate.

5. The method for learning the constraints of the deep neural network topology structure according to claim 4, wherein: The network learning rate I r The decay formula is:

Images