Neuron function analysis method and device, electronic equipment and storage medium
By acquiring activation maps and functional magnetic resonance imaging signals, a voxel-encoded model was constructed, which solved the problem of the difficulty in interpreting the black-box characteristics of deep neural networks, and enabled accurate analysis of neuronal function and interpretation of brain function.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-25
- Publication Date
- 2026-04-07
AI Technical Summary
Current technology lacks a solution for accurately analyzing the functional attributes of different brain regions in the human cerebral cortex, and the black-box nature of deep neural networks makes it difficult to explain their internal structure and function.
By acquiring activation maps and inputting them into a deep neural network model, the activation information of neurons is determined. Combined with functional magnetic resonance imaging signals, a voxel encoding model is constructed to map the relationship between neurons and voxels, thereby inferring functional information of brain regions.
This study achieves functional analysis of neurons in deep neural network models, reveals the perception mechanism and information processing method of brain neurons, and provides in-depth insights into the working principle of the human brain.
Smart Images
Figure CN121809561A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and storage medium for analyzing neuronal function. Background Technology
[0002] Deep Neural Networks (DNNs), the most typical artificial intelligence models, have demonstrated powerful performance in various machine learning tasks and applications such as computer vision and language processing. However, their extremely complex internal structure makes DNNs highly "black box" in nature. Explainable artificial intelligence (XAI) is research that attempts to overcome the black box nature of AI and provide explanations for models. In neuroscience research, the functional identification of different brain regions in the human cerebral cortex is an important direction, but current technologies lack a solution for accurately analyzing the functional attributes of voxels or brain regions. Summary of the Invention
[0003] This invention provides a method, apparatus, electronic device, and storage medium for neuronal functional profiling, in order to overcome the deficiencies in the prior art.
[0004] This invention provides a method for analyzing neuronal function, comprising: Obtain a first activation image set; wherein, the first activation image set includes multiple activation images; Multiple activation images are input into a deep neural network model to obtain the activation information of each neuron in the deep neural network for each activation image. Based on the activation information, a second activation map corresponding to each neuron is determined, and based on the second activation map, a functional description of each neuron is determined; Acquire functional magnetic resonance imaging signals of the subject while performing the target visual task, and acquire activation response data of each neuron of the deep neural network while performing the target visual task; Based on the functional magnetic resonance imaging signal and the activation response data, a voxel encoding model is constructed; wherein, the voxel encoding model reflects the mapping relationship between the neuron and the voxel; Based on the voxel encoding model, a first target neuron corresponding to the voxel is determined, and based on the functional description of the first target neuron, the functional information of the voxel is determined.
[0005] According to a method for neuronal functional profiling provided by the present invention, determining the second activation map corresponding to each neuron based on the activation information includes: The activation information of each neuron is sorted in descending order, and a candidate activation map set for each neuron is determined based on the result of the descending order. In each of the candidate activation image sets, a first similarity between the activation images is calculated; The second activation map set is determined based on the first similarity.
[0006] According to a method for profiling neuronal function provided by the present invention, the step of determining the functional description of each neuron based on a second activation map includes: Establish a visual question-answering task; wherein the visual question-answering task aims to generate a functional description; The activation images in the second activation set are input into the multimodal visual language large model to perform the visual question answering task and obtain the functional description of each neuron.
[0007] According to a method for neuronal functional profiling provided by the present invention, the step of acquiring activation response data of each neuron in the deep neural network when performing the target visual task includes: Extract the text concepts of the activated images in the second activation set, extract the text features of the text concepts using a text encoder, and extract the image features of the activated images in the second activation set using an image encoder; A second similarity is calculated based on the text features and image features corresponding to the same neuron; The required neurons are determined based on the second similarity, and the target visual task is performed based on the required neurons to obtain the activation response data.
[0008] According to a method for neuronal functional profiling provided by the present invention, the step of constructing a voxel-encoded model based on the functional magnetic resonance imaging signal and the activation response data includes: The functional magnetic resonance imaging signal and the activation response data are fitted using the ridge regression algorithm to construct the voxel encoding model for each voxel.
[0009] According to a method for neuronal functional profiling provided by the present invention, determining the first target neuron corresponding to the voxel based on the voxel encoding model includes: Extract regression weight parameters from the voxel encoding model; Based on the regression weight parameters, the first target neuron corresponding to the voxel is determined.
[0010] According to a neuronal function profiling method provided by the present invention, after determining the functional information of the voxel based on the functional description of the first target neuron, the method further includes: Obtain the target semantic concept and determine the second target neuron corresponding to the target semantic concept; Based on the voxel encoding model, the target voxel corresponding to the second target neuron is determined; The target voxel is mapped onto a planar image of the subject's brain.
[0011] The present invention also provides a neuronal function profiling device, comprising: The first acquisition module is configured to acquire a first activation image set; wherein, the first activation image set includes multiple activation images; The input module is configured to input multiple activation images into a deep neural network model to obtain activation information of each neuron in the deep neural network for each activation image. The first determining module is configured to determine the second activation map corresponding to each neuron based on the activation information, and to determine the functional description of each neuron based on the second activation map. The second acquisition module is configured to acquire the functional magnetic resonance imaging signal of the subject when performing the target visual task, and to acquire the activation response data of each neuron of the deep neural network when performing the target visual task; The module is configured to construct a voxel encoding model based on the functional magnetic resonance imaging signal and the activation response data; wherein the voxel encoding model reflects the mapping relationship between the neuron and the voxel; The second determining module is configured to determine the first target neuron corresponding to the voxel based on the voxel encoding model, and to determine the functional information of the voxel based on the functional description of the first target neuron.
[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the neuronal functional profiling method as described above.
[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the neuron functional profiling method as described above.
[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the neuron function profiling method as described above.
[0015] The present invention provides a method, apparatus, electronic device, and storage medium for neuronal functional profiling. Multiple activation images are input into a deep neural network model to obtain activation information of each neuron in the deep neural network in response to each activation image. Based on this activation information, highly activated images are selected to form a second activation set. The second activation set is used to determine the functional description of each neuron, thus enabling the analysis of individual neurons in the deep neural network model. Subsequently, based on the neuronal activation response data and functional magnetic resonance imaging signals perceived by the real brain, a voxel encoding model is constructed between artificial neurons (i.e., neurons in the deep neural network model) and brain neurons (i.e., voxels). The functional information of brain neurons is further inferred from the results of the functional profiling of the artificial neurons. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0017] Figure 1 This is a flowchart illustrating the neuronal function profiling method provided by the present invention.
[0018] Figure 2 This is a schematic diagram of the neuronal function profiling method provided by the present invention.
[0019] Figure 3 This is a schematic diagram of the neuronal function analysis device provided by the present invention.
[0020] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0022] The data involved in this application (including but not limited to data used for analysis, data stored, data displayed, etc.) are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.
[0023] Figure 1 This is a flowchart illustrating a method for neuronal functional profiling according to an exemplary embodiment. Figure 1 As shown in an exemplary embodiment, the neuronal function profiling method includes steps 110 to 160, which are described in detail below.
[0024] Step 110: Obtain the first activation image set; wherein the first activation image set includes multiple activation images.
[0025] In this embodiment of the invention, the first activation map set is obtained. The first activation set includes multiple activation images. For example... Figure 2 As shown, Figure 2 The image dataset in the dataset is the first activation set.
[0026] Step 120: Input the multiple activation images into the deep neural network model respectively to obtain the activation information of each neuron in the deep neural network for each activation image; In this embodiment of the invention, the deep neural network model consists of multiple neurons. A neuron is the basic computational unit in a neural network, mimicking the working principle of neurons in the biological brain, specifically as follows: Figure 2 The neurons shown are illustrated. To obtain functional annotations for individual neurons in a deep neural network model, it is first necessary to obtain the activation information of neurons after they are input into the deep neural network model with different activation images as input. For a given deep neural network model to be analyzed, taking the execution of a visual task as an example, after inputting the first activation map set, the activation information of each neuron in the model when processing the input image is obtained. A single neuron in a given deep neural network model. Calculate the first activation map of the neuron for the input. Each activated image in Activation information .
[0027] Step 130: Determine the second activation map corresponding to each neuron based on the activation information, and determine the functional description of each neuron based on the second activation map.
[0028] In this embodiment of the invention, the obtained activation information can demonstrate the response weights and selectivity of a single neuron to different input activation images. To reflect the functional tendencies of neurons in performing visual tasks, a second activation set with high activation is determined based on the activation information. The second activation set indicates the image concepts or features that the neuron focuses on during the processing of activation images. A functional description of each neuron is then determined based on the second activation set.
[0029] Step 140: Acquire functional magnetic resonance imaging signals of the subject when performing the target visual task, and acquire activation response data of each neuron of the deep neural network when performing the target visual task.
[0030] In this embodiment of the invention, functional magnetic resonance imaging (fMRI) signals are based on the blood oxygenation level-dependent (BOLD) effect. When the brain receives external stimuli, it triggers brain activity, resulting in changes in blood oxygenation levels in different brain regions. Blood oxygenation-dependent magnetic resonance imaging (fMRI) can reflect the phenomenon of neural activity in active brain regions when stimuli occur, thus enabling further research on the function of different brain regions of the cerebral cortex. Therefore, voxel-scale fMRI, as a non-invasive method, has high spatial resolution and good temporal resolution, making it a primary means of acquiring information for observing and studying neural activity in the human cerebral cortex.
[0031] Brain activity is recorded while subjects perceive stimuli or perform a target visual task, thus acquiring the necessary functional magnetic resonance imaging (fMRI) signals. These fMRI signals represent the activity information of multiple voxels in the three-dimensional space of the subject's brain. The same target visual task is then input into a deep neural network, and the activation response data of each neuron is recorded.
[0032] Step 150: Based on the functional magnetic resonance imaging signal and the activation response data, construct a voxel encoding model; wherein the voxel encoding model reflects the mapping relationship between the neuron and the voxel.
[0033] In this embodiment of the invention, for each voxel in the functional magnetic resonance imaging signal, a corresponding voxel encoding model is constructed based on the functional magnetic resonance imaging signal and activation response data. The voxel encoding model aims to find a mapping function that enables the activation response data of the neuron to best predict the activity of the voxel.
[0034] Step 160: Based on the voxel encoding model, determine the first target neuron corresponding to the voxel, and based on the functional description of the first target neuron, determine the functional information of the voxel.
[0035] In this embodiment of the invention, the voxel encoding model achieves the fitting of the cerebral cortex and the neural network. Using the voxel encoding model, the functional information of voxels is further inferred from the functional description of neurons.
[0036] In this embodiment of the invention, multiple activation images are input into a deep neural network model to obtain the activation information of each neuron in the deep neural network in response to each activation image. Based on the activation information, highly activated images are selected to form a second activation set. The functional description of each neuron is determined by the second activation set, thereby realizing the analysis of a single neuron in the deep neural network model. Subsequently, the activation response data of the neuron is fitted with the functional magnetic resonance imaging signal of real brain perception. By constructing a voxel encoding model between artificial neurons (i.e., neurons in the deep neural network model) and brain neurons (i.e., voxels), the functional information of brain neurons is further inferred from the results of the functional analysis of artificial neurons.
[0037] Precise analysis of artificial neurons allows for a deeper understanding of the characteristics and functions of brain neurons. This functional analysis is characterized by comparing deep neural network models with real brain data, thereby providing an interpretation of brain neuron function. This analysis process helps reveal the brain's perceptual mechanisms, information processing methods, and cognitive processes, providing profound insights into how the human brain works. Compared to other methods of analyzing deep neural networks and the brain, the technical solution provided in this invention focuses on analyzing large-scale neural network models and aligning them with the brain at the neuron level. The results of neuron analysis on the neural network model show a structural similarity to the functions of brain neurons, thus obtaining functional annotations of brain neurons.
[0038] In an exemplary embodiment of the present invention, determining the second activation map corresponding to each neuron based on the activation information includes: The activation information of each neuron is sorted in descending order, and a candidate activation map set for each neuron is determined based on the result of the descending order. In each of the candidate activation image sets, a first similarity between the activation images is calculated; The second activation map set is determined based on the first similarity.
[0039] In this embodiment of the invention, in order to better reflect the functional tendencies of neurons in performing visual tasks, a second activation map with high activation corresponding to each neuron is determined based on activation information, thereby obtaining the neuron's... In a given The second activated image set below: ; Where A represents the criterion for selecting the second activation map set, that is, under certain conditions, the second activation map set of neuron k. Only the activated image whose activation information meets a certain judgment condition A is selected. Used as a concept label.
[0040] Specifically, the activation information of each neuron is sorted in descending order, and the top-K method is used to determine the candidate activation map set of each neuron from the results of the descending sort.
[0041] Because neurons in deep neural network models differ from manually constructed functional modules, they typically do not possess absolutely specific functions but rather exhibit ambiguity. This manifests in the candidate activation map set, where directly collected activation images cannot guarantee similar conceptual features. Since the ultimate output of neuron functional annotation is a concept, a secondary screening process is performed on the candidate activation map set to obtain semantically similar image samples.
[0042] Based on the candidate activation map set, the first similarity between the activation images in the same candidate activation map set is further calculated, and then the second activation map set is determined based on the first similarity.
[0043] Specifically, this is achieved through pre-trained image encoders containing semantic information, such as CLIP (Contrastive Language-Image Pre-Training). ), calculate the feature space embedding of the activation image in each candidate activation map set, and calculate the activation image using the following formula. With activation image Distance between Used as the first similarity: ; in, and These represent the activation images. With activation image Feature space embedding.
[0044] The second activation set is selected based on the first similarity score, meaning it is closer in relative distance and more similar in feature space. This ensures that the activated images exhibit semantic consistency, facilitating conceptual interpretation.
[0045] In an exemplary embodiment of the present invention, determining the functional description of each neuron based on the second activation map includes: Establish a visual question-answering task; wherein the visual question-answering task aims to generate a functional description; The activation images in the second activation set are input into the multimodal visual language large model to perform the visual question answering task and obtain the functional description of each neuron.
[0046] In this embodiment of the invention, the second activation map is used as an aid to understand the function of neurons. In addition to the manual interpretation and analysis of neurons, a model network that associates images and text is used to describe or annotate the obtained second activation map in a textual way, which can serve as an automated method for generating functional descriptions of neurons.
[0047] Specifically, for the second activation atlas It can be achieved by directly mapping the text concept set to the image set and outputting it to the second activation map set. The text concepts that are matched to the maximum extent by several activated images can also be used to generate natural language descriptions based on a multimodal visual language model.
[0048] This invention employs a multimodal visual language model (VLM) to generate text from a second activation map set. Compared to a predefined set of textual concepts, the VLM reduces human interference during concept definition and avoids limiting concept interpretation to the concept set. In the input of the VLM, the concept labeling problem is treated as a visual question answering (VQA) task. The VQA task aims to generate functional descriptions, requiring the VLM to answer the common visual concept of multiple activated images in the second activation map set, and using the obtained answer as the functional description of the neurons.
[0049] In another embodiment of the invention, since the process of generating a functional description based on the second activation graph does not include intermediate representation acquisition, it can also be applied to open-source or API-based models to generate functional descriptions. However, the model's output needs further optimization to obtain a conceptual representation of the target. To ensure that the acquired neuronal concepts can be applied to subsequent processes such as generation and verification, simply using prompts to constrain the model's responses will lead to unsatisfactory results and affect inference time. Therefore, a smaller LLM is used to summarize the model's output at the text level to obtain an output format that meets the requirements.
[0050] In an exemplary embodiment of the present invention, obtaining the activation response data of each neuron of the deep neural network when performing the target visual task includes: Extract the text concepts of the activated images in the second activation set, extract the text features of the text concepts using a text encoder, and extract the image features of the activated images in the second activation set using an image encoder; A second similarity is calculated based on the text features and image features corresponding to the same neuron; The required neurons are determined based on the second similarity, and the target visual task is performed based on the required neurons to obtain the activation response data.
[0051] In this embodiment of the invention, considering that the number of neurons in the deep neural network model is huge, and that some neurons have ambiguity at the functional level of feature recognition or deviate from human semantic understanding, i.e., some neurons cannot generate appropriate concept descriptions, in order to obtain voxel functional information more accurately, neurons with effective concept interpretation are selected for voxel interpretation in the next process.
[0052] Specifically, the textual concepts of the activated images in the second activation set are obtained through LLM. For activation images in the second activation map of neurons, a pre-trained text encoder is used. With image encoder The embeddings of the common feature space of the text concept and the activation image are obtained to acquire the text-image cosine similarity. The calculated cosine similarity is used as the second similarity, and the formula for calculating the second similarity is as follows: ; .
[0053] A similarity threshold is set to filter out neurons whose textual concepts match the semantics of the image; that is, neurons with a second similarity greater than or equal to the similarity threshold are designated as demand neurons. The target recognition task is then performed using these demand neurons to obtain activation response data.
[0054] In an exemplary embodiment of the present invention, constructing a voxel-encoded model based on the functional magnetic resonance imaging signal and the activation response data includes: The functional magnetic resonance imaging signal and the activation response data are fitted using the ridge regression algorithm to construct the voxel encoding model for each voxel.
[0055] In this embodiment of the invention, functional magnetic resonance imaging (fMRI) signals are recorded when a subject perceives stimuli or performs a target visual task. Subsequently, a set of features is extracted from the target visual task to form a feature space. Each feature space corresponds to a representation of the target visual task, and these feature spaces are used to investigate whether they are encoded in brain activity when constructing a voxel-based encoding model. To verify the hypotheses of the selected feature spaces, a regression model is trained using their features to predict brain activity. If the regression model can significantly predict the activity of a certain brain region, it is inferred that some features of the feature space are similar to the representation of brain activity. To maximize spatial resolution, a voxel-based encoding model is fitted individually for each voxel recorded for the fMRI signal, thereby constructing a voxel-level encoding model.
[0056] like Figure 2 As shown, voxel activity responses (i.e., functional magnetic resonance imaging signals) were collected when subjects viewed certain image data, and activation response data (i.e., signals from deep neural networks performing the same visual task) were also collected. Figure 2 The Activations in the model are fitted using the ridge regression algorithm to facilitate effective functional inference.
[0057] In this embodiment of the invention, the target feature space is set as the representation space for the deep neural network model to perform the target visual task. Specifically, for an image dataset with a sample size of N, fMRI signals of dimension voxel×N are collected from the subject, and the embedding neuron×N of the image encoding from the deep neural network model is obtained. Here, voxel and neuron represent the number of brain voxels collected from the subject and the number of neurons in the deep neural network model, respectively. Ridge regression is used to construct a voxel encoding model to ensure accuracy; the constructed voxel encoding model reflects the mapping between neurons and voxels.
[0058] In an exemplary embodiment of the present invention, determining the first target neuron corresponding to the voxel based on the voxel encoding model includes: Extract regression weight parameters from the voxel encoding model; Based on the regression weight parameters, the first target neuron corresponding to the voxel is determined.
[0059] In this embodiment of the invention, the constructed voxel encoding model is parsed, and the mapping parameters of each voxel encoding model are extracted, i.e., the extraction... The regression weight coefficients. For a specific voxel, its weight coefficient vector represents the strength and direction (positive or negative correlation) of the influence of different neurons on the voxel's activity. The larger the absolute value of a neuron's weight coefficient on a voxel, the stronger the functional coupling between the neuron and the voxel, meaning that this feature has a significant impact on the prediction results and drives voxel activity to a large extent.
[0060] Based on the regression weight parameters, the correspondence between neurons and selected brain voxels under the same task will be obtained, and then the first target neuron corresponding to the voxel will be determined. The functional description of the first target neuron will be used as the functional information of the voxel.
[0061] In an exemplary embodiment of the present invention, after determining the functional information of the voxel based on the functional description of the first target neuron, the method further includes: Obtain the target semantic concept and determine the second target neuron corresponding to the target semantic concept; Based on the voxel encoding model, the target voxel corresponding to the second target neuron is determined; The target voxel is mapped onto a planar image of the subject's brain.
[0062] In this embodiment of the invention, neurons are used For example, take The voxels-dimensional data will demonstrate the collected voxels affected by neurons. The influence of corresponding features. Assuming a neuron... A functional description with high effectiveness and clear semantics was obtained. Then describe the function. As a target semantic concept, PyCortex can be used to... Visualization to a planar view of the brain to demonstrate the functional descriptions of different voxels on the subject's cerebral cortex. Selectivity of representation.
[0063] The neuronal function profiling device provided by the present invention will be described below. The neuronal function profiling device described below can be referred to in correspondence with the neuronal function profiling method described above. It should be noted that the device provided in the following embodiments belongs to the same concept as the method provided in the above embodiments, and the specific way in which each module and unit performs its operation has been described in detail in the method embodiments, and will not be repeated here.
[0064] In one exemplary embodiment of the present invention, please refer to Figure 3 , Figure 3 This is an exemplary embodiment of a neuronal function profiling device, comprising the following modules.
[0065] The first acquisition module 310 is configured to acquire a first activation image set; wherein, the first activation image set includes multiple activation images; The input module 320 is configured to input multiple activation images into a deep neural network model respectively to obtain the activation information of each neuron in the deep neural network for each activation image; The first determining module 330 is configured to determine the second activation map corresponding to each neuron based on the activation information, and to determine the functional description of each neuron based on the second activation map. The second acquisition module 340 is configured to acquire the functional magnetic resonance imaging signal of the subject when performing the target visual task, and to acquire the activation response data of each neuron of the deep neural network when performing the target visual task. The construction module 350 is configured to construct a voxel encoding model based on the functional magnetic resonance imaging signal and the activation response data; wherein the voxel encoding model reflects the mapping relationship between the neuron and the voxel; The second determining module 360 is configured to determine the first target neuron corresponding to the voxel based on the voxel encoding model, and to determine the functional information of the voxel based on the functional description of the first target neuron.
[0066] In an exemplary embodiment of the present invention, the first determining module 330 includes: The first determining submodule is configured to sort the activation information of each neuron in descending order, and determine the candidate activation map set of each neuron based on the result of the descending order. The first calculation submodule is configured to calculate a first similarity between the activated images in each of the candidate activation image sets; The second determining submodule is configured to determine the second activated graph set based on the first similarity.
[0067] In an exemplary embodiment of the present invention, the first determining module 330 includes: A submodule is established and configured to create a visual question-answering task; wherein the visual question-answering task aims to generate a functional description. The input submodule is configured to input the activation images from the second activation set into the multimodal visual language large model to perform the visual question answering task and obtain the functional description of each neuron.
[0068] In an exemplary embodiment of the present invention, the second acquisition module 340 includes: The first extraction submodule is configured to extract the text concepts of the activated images in the second activation set, extract the text features of the text concepts through a text encoder, and extract the image features of the activated images in the second activation set through an image encoder. The second calculation submodule is configured to calculate a second similarity based on the text features and image features corresponding to the same neuron; The execution submodule is configured to determine the demand neuron based on the second similarity and perform the target visual task based on the demand neuron to obtain the activation response data.
[0069] In an exemplary embodiment of the present invention, the construction module 350 includes: A submodule is constructed and configured to fit the functional magnetic resonance imaging signal and the activation response data using the ridge regression algorithm, and to construct the voxel encoding model for each voxel.
[0070] In an exemplary embodiment of the present invention, the second determining module 360 includes: The second extraction submodule is configured to extract regression weight parameters from the voxel encoding model; The third determining submodule is configured to determine the first target neuron corresponding to the voxel based on the regression weight parameters.
[0071] In one exemplary embodiment of the present invention, the neuronal functional profiling device further includes: The third acquisition module is configured to acquire the target semantic concept and determine the second target neuron corresponding to the target semantic concept; The third determining module is configured to determine the target voxel corresponding to the second target neuron based on the voxel encoding model; The mapping module is configured to map the target voxel onto a planar view of the subject's brain.
[0072] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a neuronal functional profiling method, which includes: acquiring a first activation map set; wherein the first activation map set includes multiple activation images; Multiple activation images are input into a deep neural network model to obtain the activation information of each neuron in the deep neural network for each activation image. Based on the activation information, a second activation map corresponding to each neuron is determined, and based on the second activation map, a functional description of each neuron is determined; Acquire functional magnetic resonance imaging signals of the subject while performing the target visual task, and acquire activation response data of each neuron of the deep neural network while performing the target visual task; Based on the functional magnetic resonance imaging signal and the activation response data, a voxel encoding model is constructed; wherein, the voxel encoding model reflects the mapping relationship between the neuron and the voxel; Based on the voxel encoding model, a first target neuron corresponding to the voxel is determined, and based on the functional description of the first target neuron, the functional information of the voxel is determined.
[0073] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0074] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, the computer program being executed by a processor, the computer being able to execute the neuronal functional profiling method provided by the above methods, the method including: acquiring a first activation map set; wherein, the first activation map set includes multiple activation images; Multiple activation images are input into a deep neural network model to obtain the activation information of each neuron in the deep neural network for each activation image. Based on the activation information, a second activation map corresponding to each neuron is determined, and based on the second activation map, a functional description of each neuron is determined; Acquire functional magnetic resonance imaging signals of the subject while performing the target visual task, and acquire activation response data of each neuron of the deep neural network while performing the target visual task; Based on the functional magnetic resonance imaging signal and the activation response data, a voxel encoding model is constructed; wherein, the voxel encoding model reflects the mapping relationship between the neuron and the voxel; Based on the voxel encoding model, a first target neuron corresponding to the voxel is determined, and based on the functional description of the first target neuron, the functional information of the voxel is determined.
[0075] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the neuronal functional profiling method provided by the above methods, the method comprising: acquiring a first activation map set; wherein the first activation map set includes multiple activation images; Multiple activation images are input into a deep neural network model to obtain the activation information of each neuron in the deep neural network for each activation image. Based on the activation information, a second activation map corresponding to each neuron is determined, and based on the second activation map, a functional description of each neuron is determined; Acquire functional magnetic resonance imaging signals of the subject while performing the target visual task, and acquire activation response data of each neuron of the deep neural network while performing the target visual task; Based on the functional magnetic resonance imaging signal and the activation response data, a voxel encoding model is constructed; wherein, the voxel encoding model reflects the mapping relationship between the neuron and the voxel; Based on the voxel encoding model, a first target neuron corresponding to the voxel is determined, and based on the functional description of the first target neuron, the functional information of the voxel is determined.
[0076] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0077] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for analyzing neuronal function, characterized in that, include: Obtain a first activation image set; wherein, the first activation image set includes multiple activation images; Multiple activation images are input into a deep neural network model to obtain the activation information of each neuron in the deep neural network for each activation image. Based on the activation information, a second activation map corresponding to each neuron is determined, and based on the second activation map, a functional description of each neuron is determined; Acquire functional magnetic resonance imaging signals of the subject while performing the target visual task, and acquire activation response data of each neuron of the deep neural network while performing the target visual task; Based on the functional magnetic resonance imaging signal and the activation response data, a voxel encoding model is constructed; wherein, the voxel encoding model reflects the mapping relationship between the neuron and the voxel; Based on the voxel encoding model, a first target neuron corresponding to the voxel is determined, and based on the functional description of the first target neuron, the functional information of the voxel is determined.
2. The neuronal functional profiling method according to claim 1, characterized in that, The step of determining the second activation map corresponding to each neuron based on the activation information includes: The activation information of each neuron is sorted in descending order, and a candidate activation map set for each neuron is determined based on the result of the descending order. In each of the candidate activation image sets, a first similarity between the activation images is calculated; The second activation map set is determined based on the first similarity.
3. The neuronal functional profiling method according to claim 1, characterized in that, The determination of the functional description of each neuron based on the second activation map includes: Establish a visual question-answering task; wherein the visual question-answering task aims to generate a functional description; The activation images in the second activation set are input into the multimodal visual language large model to perform the visual question answering task and obtain the functional description of each neuron.
4. The neuronal functional profiling method according to claim 1, characterized in that, The step of acquiring the activation response data of each neuron in the deep neural network when performing the target visual task includes: Extract the text concepts of the activated images in the second activation set, extract the text features of the text concepts using a text encoder, and extract the image features of the activated images in the second activation set using an image encoder; A second similarity is calculated based on the text features and image features corresponding to the same neuron; The required neurons are determined based on the second similarity, and the target visual task is performed based on the required neurons to obtain the activation response data.
5. The neuronal functional profiling method according to claim 1, characterized in that, The construction of a voxel-encoded model based on the functional magnetic resonance imaging signal and the activation response data includes: The functional magnetic resonance imaging signal and the activation response data are fitted using the ridge regression algorithm to construct the voxel encoding model for each voxel.
6. The neuronal functional profiling method according to claim 5, characterized in that, The step of determining the first target neuron corresponding to the voxel based on the voxel encoding model includes: Extract regression weight parameters from the voxel encoding model; Based on the regression weight parameters, the first target neuron corresponding to the voxel is determined.
7. The method for neuronal functional profiling according to any one of claims 1 to 6, characterized in that, After determining the functional information of the voxel based on the functional description of the first target neuron, the method further includes: Obtain the target semantic concept and determine the second target neuron corresponding to the target semantic concept; Based on the voxel encoding model, the target voxel corresponding to the second target neuron is determined; The target voxel is mapped onto a planar image of the subject's brain.
8. A neuronal function analysis device, characterized in that, include: The first acquisition module is configured to acquire a first activation image set; wherein, the first activation image set includes multiple activation images; The input module is configured to input multiple activation images into a deep neural network model to obtain activation information of each neuron in the deep neural network for each activation image. The first determining module is configured to determine the second activation map corresponding to each neuron based on the activation information, and to determine the functional description of each neuron based on the second activation map. The second acquisition module is configured to acquire the functional magnetic resonance imaging signal of the subject when performing the target visual task, and to acquire the activation response data of each neuron of the deep neural network when performing the target visual task; The construction module is configured to construct a voxel encoding model based on the functional magnetic resonance imaging signal and the activation response data; wherein the voxel encoding model reflects the mapping relationship between the neuron and the voxel; The second determining module is configured to determine the first target neuron corresponding to the voxel based on the voxel encoding model, and to determine the functional information of the voxel based on the functional description of the first target neuron.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the neuronal functional profiling method as described in any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the neuronal function profiling method as described in any one of claims 1 to 7.