Digital twin brain visual task state data assimilation system and method
By generating individualized computing neural network models and assimilating data, the problem of high-precision emotional representation of digital twin brains under visual tasks is solved, and accurate simulation and prediction of emotional responses to visual stimuli are achieved, supporting brain scientific research and disease diagnosis and treatment.
Patent Information
- Application Number
- CN202510467134.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-15
- Publication Date
- 2025-07-25
AI Technical Summary
The prior art is difficult to reconstruct a digital twin brain with significant correlation with the spatial and temporal dynamic mechanism of the biological brain on a computer, and lacks the ability to simulate and predict emotional representations under visual tasks.
The multimodal neural image data of the human brain obtained by high-precision magnetic resonance devices is used, combined with T1 weighted imaging and diffusion tensor imaging, an individualized computational neural network model is generated, and data assimilation is performed through integrated Kalman filtering and multi-layer Bayesian estimation, which simulates the brain activation mode under visual tasks, and accelerates neural simulation in parallel with GPU to predict the evaluation score of visual stimuli.
It realizes high-precision emotional representation simulation and prediction under visual tasks, reveals the internal mechanism of visual stimulation triggers emotional responses in the brain, provides individualized digital brain visual evaluation data, and supports brain scientific research and disease diagnosis and treatment evaluation.
Smart Images
Figure CN120373372A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data assimilation, and specifically, to a digital twin brain visual task state data assimilation system and method. Background Art
[0002] The human brain consists of 86 billion neurons, and each neuron is connected to up to one thousand to ten thousand other neurons, forming the most complex neural network known so far, which can generate emotions, self-learn, think, and has the ability to adapt to the external complex environment. The goal of visual task state assimilation is to attempt to reconstruct the simulated brain on a computer so that there is a significant correlation between the digital brain and the biological brain in terms of the spatio-temporal dynamics mechanism at the mesoscopic scale.
[0003] Currently, according to the mesoscopic scale data assimilation theory, using the multi-modal neuroimaging data of the human brain obtained by high-precision nuclear magnetic resonance equipment, on a supercomputer cluster connected by a high-speed network, based on a computational neural network model with voxels or functional columns as units, the brain stimulation signals under visual tasks have been successfully decoded, and the emotional representation of the real human brain under visual tasks is simulated through a digital twin brain. Summary of the Invention
[0004] In order to explore the potential application of the digital twin brain in cognitive decision-making, reverse solve to form the dynamic and information transmission mechanism of intelligent behavior and function at the mesoscopic scale, and study the working principle of brain intelligence, the present invention provides a digital twin brain visual task state data assimilation system and method; the present invention studies brain function by performing computational simulations on the digital twin brain, and designs digital experiments to analyze the spatio-temporal dynamic behavior at the mesoscopic scale, mainly for cognitively simulating the emotional evaluation of human vision.
[0005] Based on the existing brain imaging technology, the present invention uses the structural data of the real brain (such as T1 weighted imaging data (T1WI), diffusion tensor imaging (DTI), etc.), task state functional data (such as functional magnetic resonance imaging data (fMRI), etc.), and the external "input" current injected into each task "input" brain region decoded by the data assimilation method to simulate the dynamic brain in the task state, inputs the functional data of the V1 cortex of the reference brain in the visual evaluation task, and obtains the individualized digital brain visual evaluation data of new participants. Using these simulation data, it is possible to analyze and predict the evaluation scores of different individuals for visual stimuli. The technical solution of the present invention is specifically introduced as follows.
[0006] The present invention provides a digital twin brain visual task state data assimilation system, which includes a human brain emotion representation data acquisition module, a network structure generation module, a data assimilation module, a neural network simulation module, and a digital brain emotion representation prediction module; wherein: The human brain emotion representation data acquisition module is used to obtain the emotion evaluation of the subject in the real task experiment scenario; it includes: first, using the general linear model to evaluate the brain activation pattern under visual stimulation in each trial for the blood oxygenation level-dependent (BOLD) signal collected from the experiment, and then using the activation pattern of the biological brain during the stimulation evaluation as the response variable, and using the true score of the emotion picture as the predictor variable to train a linear regression model. The network structure generation module sets network scale parameters according to the measured biological structure data of the subject to generate a computational brain model of the subject (i.e., the digital twin brain); among them, the collected T1-weighted imaging data (T1WI) is used to obtain grey matter volume data (GMV) through voxel-based morphometry (VBM), which is used to model the relative proportion of the number of neurons in different voxels; the structural connectivity (SC) map between voxels is obtained through diffusion tensor imaging (DTI) and is used to model the connection probability between different voxels. By setting the total number of neurons in the computational brain model and the average in-degree of each voxel, the number of neurons in different voxels, the average in-degree of different voxels, and the distribution probability of connection edges between different voxels are obtained, and the connection structure of different neurons in the computational brain model is sampled according to the connection edge distribution probability, thus completing the generation of the model. The data assimilation module assimilates the external input current of the neurons of the minimum neural network based on the ensemble Kalman filter method in data assimilation and combines the multi-layer Bayesian estimation theory, so that the output of the brain model is similar to the blood oxygenation level-dependent (BOLD) signal of the subject. The neural network simulation module performs network simulation based on the assimilated parameters to obtain the BOLD signal data output by the computational brain model, and this data is used to obtain the brain activation pattern of the computational brain model through the general linear model in the digital brain emotion representation prediction module. The digital brain emotion representation prediction module uses the digital twin brain to simulate the real task experiment scenario, applies the general linear model to the BOLD signal output by the computational brain model obtained by the neural network simulation module to obtain the brain activation pattern of the model under visual stimulation, and applies the trained linear regression model in the human brain emotion representation data acquisition module to the brain activation pattern of the computational brain model to obtain the prediction of the model for the picture score of the subject in the real task experiment scenario.
[0007] In the present invention, in the human brain emotion representation data acquisition module, when training a linear regression model, LASSO regularization is adopted to obtain a sparse coefficient vector.
[0008] In the present invention, in the network structure generation module, the network scale parameters include the total number of neurons, the number of neurons in the smallest neural network, the network connection degree, and the synaptic neurotransmitter conductivity coefficient.
[0009] In the present invention, in the neural network simulation module, the Euler-Maruyama method is used for numerical iteration of the differential equation satisfied by the computational brain model, and GPU multi-card parallel computing is utilized; meanwhile, in view of the structural characteristics of brain heterogeneity and sparsity, a two-level routing scheme between the optimized mapping from neurons to computing units and the computational graphics processor is constructed.
[0010] In the present invention, a drawing module is encapsulated in the neural network simulation module, and the drawing module draws and displays the assimilation parameter sequence, the simulated BOLD signal and the original BOLD signal throughout the process; and draws and displays the average firing rate of the smallest neural network, the average neuron membrane potential, and the firing time of the sampled neurons.
[0011] In the present invention, the digital brain emotion representation prediction module also uses the brain activation pattern obtained from the digital twin brain and the sparse coefficient vector obtained by LASSO to predict the score of the subject for the pictures in the real task experimental scenario, and calculates the Pearson correlation coefficient between the score of the digital twin brain for the visual task pictures and the human brain score to measure the similarity between the digital twin brain and the human brain in terms of the dynamic mechanism.
[0012] The present invention provides a method for assimilating visual task state data of a digital twin brain, including the following steps: (1) Acquisition of human brain emotion representation data Obtain the emotional evaluation of the subject in the real task experimental scenario; including: first, use a generalized linear model to evaluate the brain activation pattern under visual stimulation in each trial for the blood oxygenation level-dependent (BOLD) signal collected in the experiment, and then use the activation pattern of the biological brain during the stimulation evaluation as the response variable, and use the true score of the emotional pictures as the predictor variable to train a linear regression model; (2) Network structure generation Set the network scale parameters according to the measured biological structure data of the subject to generate the computational brain model of the subject. Among them, the T1-weighted imaging data collected is used to obtain the gray matter volume data through voxel-based morphometry analysis, which is used to model the relative proportion of the number of neurons in different voxels; the structural connection map between voxels is obtained through diffusion tensor imaging, which is used to model the connection probability between different voxels. By setting the total number of neurons in the computational brain model and the average in-degree of each voxel, the number of neurons in different voxels, the average in-degree of different voxels, and the distribution probability of the connection edges between different voxels are obtained, and the connection structure of different neurons in the computational brain model is sampled according to the connection edge distribution probability, so as to complete the generation of the model; (3) Data assimilation Based on the ensemble Kalman filter method in data assimilation and combined with the multi-layer Bayesian estimation theory, assimilate the external input current of neurons in the smallest neural network, so that the output of the brain model is similar to the blood oxygenation level-dependent (BOLD) signal of the subject; (4) Neural network simulation Based on the assimilated parameters, perform network simulation to obtain the BOLD signal data output by the computational brain model. This data is used to obtain the brain activation pattern of the computational brain model through a generalized linear model in the digital brain emotion representation prediction module; (5) Digital brain emotion representation prediction Use the digital twin brain to simulate the real task experimental scenario. Use the digital twin brain to simulate the real task experimental scenario. Apply the generalized linear model to the BOLD signal output by the computational brain model obtained by the neural network simulation module to obtain the brain activation pattern of the model under visual stimulation, and apply the trained linear regression model in the human brain emotion representation data acquisition module to the brain activation pattern of the computational brain model to obtain the prediction of the model's picture score for the subject in the real task experimental scenario.
[0013] In summary, the present invention uses real brain visual processing signals to simulate a digital brain with dynamic changes in the task state over time in a visual stimulation task, obtains individualized digital brain information processing data, and predicts the emotional representation of the human brain in a real task experimental scenario; the present invention can analyze and predict the evaluation scores of different individuals for visual stimuli. Compared with the prior art, the beneficial effects of the present invention are as follows: 1. The present invention integrates real biological structure data such as T1-weighted imaging and diffusion tensor imaging to generate a neural network model that better conforms to the individual physiological structure characteristics; constructs a parameter learning framework based on the ensemble Kalman filter and multi-layer Bayesian estimation, and realizes high-precision simulation of the real biological brain.
[0014] 2. The computational brain model of the present invention can be used as a "dry" experimental platform to help brain scientists conduct digital experiments, explore and verify neuroscience theories and the mechanism of brain intelligence; the computational brain model can be used to explore and verify the pathology of brain diseases, and evaluate and develop diagnosis and treatment methods.
[0015] 3. The present invention improves the large-scale neural simulation architecture with GPU parallel acceleration, which can greatly improve the running efficiency of the computational brain model.
[0016] 4. For the BOLD signals obtained from fMRI scans of the human brain in visual tasks, the present invention assimilates the external input current of neurons in the primary visual cortex (V1) in the neural network model to decode visual stimulus signals, so as to realize the computational model brain reproduction and simulation of visual tasks; the present invention can accurately reproduce the visual cognitive process of the real biological brain and reveal the internal mechanism of how visual stimuli trigger emotional responses in the brain. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 Schematic diagram of the experimental paradigm in the visual task state.
[0018] Figure 2 Schematic diagram of using a digital twin brain to simulate a real task experimental scenario and predict the emotional representation of the subject.
[0019] Figure 3 Schematic diagram of the visual task state assimilation method and the emotional evaluation effect diagram. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0020] The technical solutions of the present invention will be introduced in detail below with reference to the drawings and embodiments.
[0021] The components of this system are: a human brain emotional representation data acquisition system, a network structure generation system, a data assimilation module, a neural network simulation system, and a digital brain emotional representation prediction system.
[0022] The human brain emotional representation data acquisition system first uses a generalized linear model to evaluate the brain activation pattern under visual stimuli in each trial for the BOLD signals collected from the experiment, and then uses the activation pattern of the biological brain during the stimulus evaluation as the response variable, and the true score of the emotional pictures as the predictive variable to train a linear regression model, and adopts LASSO regularization to obtain a sparse coefficient vector. fMRI BOLD (blood oxygenation level-dependent brain signal) signal processing: Using the spm software package, operations such as detrending, band-pass filtering, and standardization are performed on the signals to achieve the standardized processing of fRMI signals.
[0023] The described network structure generation system generates a computational brain model of the subject according to the measured fMRI signals of the experimental subjects, including diffusion tensor imaging (DTI) of the brain and the gray matter density of the cerebral cortex. The adjustable parameters include the total number of neurons, the number of neurons in the smallest neural network (voxel or functional column), the network connection degree, the synaptic neurotransmitter conductivity coefficient, etc.
[0024] The data assimilation module is the core of the algorithm. It mainly uses the ensemble Kalman filter method in data assimilation and combines the multi-layer Bayesian estimation theory to assimilate the external input current of neurons in the smallest neural network (voxel or functional column). Here, we assume that the external input currents of neurons in the same smallest neural network (voxel or functional column) follow the same probability distribution, so only the hyperparameters of the distribution need to be assimilated.
[0025] The neural network simulation system mainly performs network simulation on the given assimilated parameters. A plotting program is encapsulated in this system, which can plot the average firing rate of the smallest neural network, the average neuron membrane potential, etc. It is also possible to observe a specific neuron by inputting a specific number, and the input includes specific information such as the membrane potential, synaptic current, and firing time of the neuron.
[0026] The digital brain emotion representation prediction system mainly uses the digital twin brain to simulate the real task experimental scenario and predict the score of the human brain for the pictures in the real task experimental scenario. By comparing the emotion representation predicted by the digital brain with the collected human brain visual task score data, the similarity between the digital twin brain and the human brain in the dynamic mechanism is illustrated.
[0027] As Figure 2 shown, the present invention provides a data assimilation method for the visual task state of the digital twin brain, which uses the digital twin brain to simulate the real task experimental scenario and predict the emotion representation of the subject. The specific steps are as follows: Step 1: Obtain human brain emotion representation data. The subject needs to perform 30 visual assessment tasks. As Figure 1 shown, for each trial, after a 3-second graphic cue, the subject uses a Likert scale to rate his feeling towards the real-world stimulus from 0 to 10 within 4 seconds.
[0028] Subsequently, the BOLD signals collected in the experiment are used to evaluate the brain activation pattern in the trial using a generalized linear model; the brain activation pattern refers to the activity of different regions of the brain under different tasks or states. According to functional magnetic resonance imaging (fMRI) studies, the brain activation pattern can be manifested as an increase or decrease in the activity of different brain regions under specific tasks or stimuli. These patterns help to understand the functions and behaviors of the brain in different situations.
[0029] Finally, the activation pattern of the biological brain during the stimulation evaluation period and the emotional picture scores are trained into a linear regression model, and LASSO regularization is used to obtain a sparse coefficient vector.
[0030] Step 2: Network structure generation: The biological structure data obtained by scanning the subject (including T1-weighted imaging T1WI and diffusion tensor imaging DTI) is input, and the relative proportion of the number of neurons in different voxels and the connection probability between different voxels are obtained. By presetting the calculation of the total number of neurons in the brain model and the average in-degree of each voxel, the number of neurons in different voxels, the average in-degree of different voxels, and the allocation probability of connection edges between different voxels are obtained. Finally, by sampling the connection edge allocation probability, the connection structure of different neurons in the calculation brain model is obtained, and the storage address of the brain model is output to the data assimilation module.
[0031] Step 3: When the data assimilation module receives the brain model storage address and the subject's BOLD signal, it runs the data assimilation algorithm to update the parameters in the assimilated brain model so that the output of the brain model is similar to the subject's BOLD signal. And the assimilated parameters are stored and input into the neural network simulation system.
[0032] Step 4: After receiving the assimilated parameters, the neural network simulation system starts network simulation, that is, the Euler-Maruyama method is used for numerical iteration of the differential equation satisfied by the calculation brain model, and GPU multi-card parallel computing is utilized; at the same time, aiming at the structural characteristics of brain heterogeneity and sparsity, a two-level routing scheme between the optimized mapping from neurons to computing units and the computing graphics processor is constructed, so as to effectively reduce the communication volume between GPU / nodes and balance the communication traffic between different computing units. Note that at this time, the simulation system does not take the subject's standardized BOLD signal as input, but the system can still simulate an output signal similar to the subject's BOLD signal.
[0033] Step 5: Plotting and storage: After the simulation system runs to completion, the plotting module will plot and display the assimilated parameter sequence, simulated BOLD signal, and original BOLD signal throughout the process. In addition, it is also possible to choose to display information such as the average membrane potential, average firing rate of the smallest neural network (brain region or voxel), and firing time of sampled neurons, and store all the results in the memory.
[0034] Step 6: Apply the generalized linear model to the BOLD signal output from the computational brain model obtained in Step 4 to obtain the brain activation pattern of the digital twin brain. Then, use the linear regression model trained in Step 1 to predict the subject's score for the pictures in the real task experimental scenario, and calculate the Pearson correlation coefficient between the score of the digital twin brain for the visual task pictures and the human brain score to measure the similarity between the digital twin brain and the human brain in terms of dynamic mechanisms. Estimating the similarity is to verify the simulation effect of the digital twin brain. By providing the digital twin brain with "input" signals comparable to those of the human brain, we make the task-state functional data output by the model as similar as possible to the real biological data.
[0035] As Figure 3 shown, Figure 3 the right figure in... gives a comparison between the real scores of the subjects for the pictures under a certain set of visual tasks and the predictions of the digital twin brain model for the picture scores. The correlation between the two reaches 0.575, indicating that the digital twin brain model can achieve an efficient simulation of the interaction between the real human brain and visual perception input, and reveal the internal mechanism of how visual stimuli trigger emotional responses in the brain.
[0036] In summary, the present invention discloses a digital twin brain visual task-state data assimilation system and method. The present invention utilizes real brain visual processing signals to simulate a digital brain with dynamic changes in task state over time in a visual stimulation task, obtains individualized digital brain information processing data, and predicts the emotional representation of the human brain in the real task experimental scenario. The present invention can achieve an accurate reproduction of the visual cognitive process of the real biological brain and reveal the internal mechanism of how visual stimuli trigger emotional responses in the brain.
Claims
1. A digital twin brain visual task state data assimilation system, characterized in that, It includes a human brain emotion representation data acquisition module, a network structure generation module, a data assimilation module, a neural network simulation module, and a digital brain emotion representation prediction module; among which: The human brain emotion representation data acquisition module is used to obtain the emotion assessment of the subject in the real task experimental scenario; including: first, using a generalized linear model to evaluate the brain activation pattern under visual stimulation in each trial for the blood oxygenation level-dependent (BOLD) signal collected in the experiment, and then using the activation pattern of the biological brain during the stimulus assessment as the response variable and the true score of the emotion picture as the predictor variable to train a linear regression model; The network structure generation module sets network scale parameters according to the measured biological structure data of the subject to generate a computational brain model of the subject; among which, the T1-weighted imaging (T1WI) data collected is used to obtain the gray matter volume (GMV) data through voxel-based morphometry (VBM), which is used to model the relative proportion of the number of neurons in different voxels; the structural connectivity map (SC) between voxels is obtained through diffusion tensor imaging (DTI) and is used to model the connection probability between different voxels; by setting the total number of neurons in the computational brain model and the average in-degree of each voxel, the number of neurons in different voxels, the average in-degree of different voxels, and the distribution probability of the connection edges between different voxels are obtained, and the connection structure of different neurons in the computational brain model is sampled according to the connection edge distribution probability, thus completing the generation of the model; The data assimilation module assimilates the external input current of the neurons of the minimum neural network based on the ensemble Kalman filter method in data assimilation and combines the multi-layer Bayesian estimation theory to make the output of the brain model similar to the blood oxygenation level-dependent (BOLD) signal of the subject; The neural network simulation module performs network simulation based on the assimilated parameters to obtain the BOLD signal data output by the computational brain model, and this data is used to obtain the brain activation pattern of the computational brain model through a generalized linear model in the digital brain emotion representation prediction module; The digital brain emotion representation prediction module uses a digital twin brain to simulate the real task experimental scenario, applies the generalized linear model to the BOLD signal output by the computational brain model obtained by the neural network simulation module to obtain the brain activation pattern of the model under visual stimulation, and applies the trained linear regression model in the human brain emotion representation data acquisition module to the brain activation pattern of the computational brain model to obtain the prediction of the model for the picture scoring of the subject in the real task experimental scenario.
2. The digital twin brain visual task state data assimilation system according to claim 1, wherein In the human brain emotion representation data acquisition module, when training the linear regression model, LASSO regularization is adopted to obtain a sparse coefficient vector.
3. The digital twin brain visual task state data assimilation system according to claim 1, wherein In the human brain emotion representation data acquisition module, after preprocessing the blood oxygenation level-dependent (BOLD) signal by detrending, band-pass filtering, and standardization, a generalized linear model is used for evaluation.
4. The digital twin brain visual task state data assimilation system according to claim 1, wherein In the network structure generation module, the network scale parameters include the total number of neurons, the number of neurons included in the minimum neural network, the network link degree, and the synaptic neurotransmitter conductivity coefficient.
5. The digital twin brain visual task state data assimilation system according to claim 1, wherein In the neural network simulation module, the Euler-Maruyama method is used for numerical iteration of the differential equations satisfied by the computational brain model, and GPU multi-card parallel computing is utilized. Meanwhile, in view of the structural characteristics of brain heterogeneity and sparsity, a two-level routing scheme between the optimized mapping from neurons to computing units and the computational graphics processor is constructed.
6. The digital twin brain visual task state data assimilation system according to claim 1, characterized in that The digital brain emotion representation prediction module also uses the brain activation pattern obtained from the digital twin brain and the sparse coefficient vector obtained by LASSO to predict the subject's score for the pictures in the real task experimental scenario, and calculates the Pearson correlation coefficient between the score of the digital twin brain for the visual task pictures and the human brain score to measure the similarity between the digital twin brain and the human brain in terms of dynamic mechanisms.
7. A digital twin brain visual task state data assimilation method, characterized in that, It includes the following steps: (1) Acquisition of human brain emotion representation data Obtain the emotional evaluation of the subject in the real task experimental scenario, including: First, use the generalized linear model to evaluate the brain activation pattern under visual stimulation in each trial for the blood oxygenation level-dependent (BOLD) signal collected from the experiment. Then, use the activation pattern of the biological brain during the stimulation evaluation as the response variable, and use the true score of the emotional pictures as the predictor variable to train a linear regression model. (2) Generation of network structure Set the network scale parameters according to the biological structure data measured from the subject to generate the computational brain model of the subject. Among them, the T1-weighted imaging data (T1WI) collected is used to obtain the gray matter volume data (GMV) through voxel-based morphometry (VBM), which is used to model the relative proportion of the number of neurons in different voxels. The structural connection map (SC) between voxels is obtained through diffusion tensor imaging (DTI) and is used to model the connection probability between different voxels. By setting the total number of neurons in the computational brain model and the average in-degree of each voxel, the number of neurons in different voxels, the average in-degree of different voxels, and the distribution probability of the connection edges between different voxels are obtained, and the connection structure of different neurons in the computational brain model is sampled according to the connection edge distribution probability, thus completing the generation of the model. (3) Data assimilation Based on the ensemble Kalman filter method in data assimilation and combined with the multi-layer Bayesian estimation theory, the external input current of the neurons in the smallest neural network is assimilated to make the output of the brain model similar to the subject's blood oxygenation level-dependent (BOLD) signal. (4) Neural network simulation Based on the assimilated parameters, network simulation is carried out to obtain the BOLD signal data output by the computational brain model, which is used to obtain the brain activation pattern of the computational brain model through the generalized linear model in the digital brain emotion representation prediction module. (5) Digital brain emotion representation prediction Use the digital twin brain to simulate the real task experimental scenario. Apply the generalized linear model to the BOLD signal output by the computational brain model obtained from the neural network simulation module to obtain the brain activation pattern of the model under visual stimulation, and apply the trained linear regression model in the human brain emotion representation data acquisition module to the brain activation pattern of the computational brain model to obtain the prediction of the model for the subject's score of the pictures in the real task experimental scenario.