Neural network construction method and system based on visual search

Abstract

This invention provides a method and system for constructing a neural network based on visual search. The method includes: conducting visual search experiments on experimental subjects with and without background noise to obtain brain imaging data of the subjects; for each sub-experiment, based on the brain imaging data, determining the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli; based on the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli in each sub-experiment, performing biological brain neuron activation mechanism analysis to obtain the sparse activation mechanism, hierarchical activation mechanism, and activation distribution law of visual cortex neurons; and constructing a target neural network based on at least one of the sparse activation mechanism, hierarchical activation mechanism, and activation distribution law. This invention solves the representation problem of sparse activation mechanism, hierarchical activation mechanism, and activation distribution law, which can improve the robustness of neural networks and reduce power consumption.

Description

Technical Field

[0001] This invention relates to the field of visual search analysis technology, and in particular to a method and system for constructing a neural network based on visual search. Background Technology

[0002] The birth of neural networks (NNs) stems from the mathematical abstraction and simulation of the information processing mechanisms of the human brain. Their core idea is to construct a computational model composed of numerous interconnected simple processing units by mimicking the basic activation and transmission patterns of biological neurons, aiming to approximate and learn complex functions. Understanding how the neuronal activation mechanism was formalized and evolved into modern neural networks is key to grasping the trajectory of deep learning technology.

[0003] The emergence and development of real-time, high-resolution, and wide-field-of-view mesoscopic microscopy has made it possible to observe neurons in the biological brain, accelerated the research on the mechanisms of neuronal activity patterns, and further promoted the development of research on the neurocognitive mechanisms of visual search. The study of visual search mechanisms in rodents, especially mice, has always been a hot and challenging issue in the field of biological visual processing mechanisms.

[0004] However, different visual behavioral experiments employ different analytical methods. Some experiments typically use full-screen checkerboard or interference fringe stimuli, focusing on the physiological perception of visual brightness and color changes in normal or diseased mice. Others use visual stimuli involving differences in image shape or changes in position, focusing on memory and decision-making mechanisms. Because visual search is a fundamental visual ability in animals, visual behavioral experiments, except for those using full-screen stimuli, all involve the animal's visual search process. However, few experiments conduct in-depth, focused analysis of visual search results. The core reason is that their experimental protocols lack specific quantitative evaluation indicators from the perspective of visual search research.

[0005] Therefore, there is an urgent need for an effective solution to conduct in-depth analysis of visual search results in order to build artificial intelligence neural networks. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a method and system for constructing a neural network based on visual search.

[0007] This invention provides a method for constructing a neural network based on visual search, comprising: The experiment subjects were subjected to visual search experiments with and without background noise to obtain brain imaging data of the subjects. Both the visual search experiments with and without background noise included at least one round of experimental groups, and each experimental group included at least two sub-experiments. For each of the sub-experiments, based on the brain imaging data corresponding to the sub-experiment, the neurons contained in each visual cortex in the sub-experiment and the neurons in each visual cortex related to visual stimuli are determined. Based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, the activation mechanism of biological brain neurons is analyzed to obtain the sparse activation mechanism, hierarchical activation mechanism and activation distribution law of visual cortex neurons. The target neural network is constructed based on at least one of the sparse activation mechanism, the hierarchical activation mechanism, and the activation distribution law.

[0008] According to a method for constructing a neural network based on visual search provided by the present invention, the step of determining the neurons contained in each visual cortex of the sub-experiment and the neurons in each visual cortex related to visual stimuli based on brain imaging data corresponding to the sub-experiment includes: The brain imaging data corresponding to the sub-experiment is subjected to cell segmentation of biological brain neurons to extract activation trace data of the biological brain neurons, wherein the activation trace data represents the trace data of the activated biological brain neurons. Based on the activation trace data of the biological brain neurons, the neurons contained in each visual cortex in the sub-experiment and the neurons in each visual cortex related to visual stimuli were determined.

[0009] According to a method for constructing a neural network based on visual search provided by the present invention, the biological brain neurons of the experimental subject are fluorescently labeled with a calcium ion indicator, and the brain imaging data includes calcium ion imaging data. The step of segmenting the brain imaging data corresponding to the sub-experiment into biological brain neurons to extract activation trace data of the biological brain neurons includes: By using a supervised learning algorithm, cell location marking and identification are performed on the biological brain neurons in the calcium ion imaging data corresponding to the sub-experiment to obtain calcium transient traces; Calcium ion dynamics were estimated from the calcium transient traces to obtain activation trace data of the biological brain neurons.

[0010] According to a method for constructing a neural network based on visual search provided by the present invention, the step of determining the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli based on the activation trace data of the biological brain neurons includes: Correlation analysis and feature dimension reduction are performed on the activation trace data of the biological brain neurons to determine the neurons contained in each visual cortex and the neurons in each visual cortex that are related to visual stimuli.

[0011] According to a method for constructing a neural network based on visual search provided by the present invention, the step of performing correlation analysis and feature dimension reduction on the activation trace data of biological brain neurons to determine the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli includes: Based on the activation trace data of the biological brain neurons, the Pearson correlation coefficient of the average stimulus response intensity of each neuron was calculated. Principal component analysis is used to reduce the feature dimension of each neuron located in each of the visual cortexes to obtain the neurons contained in each of the visual cortexes. The neurons in each of the visual cortexes whose Pearson correlation coefficient represents a significant response are identified as neurons in each of the visual cortexes that are associated with visual stimuli.

[0012] According to a method for constructing a neural network based on visual search provided by the present invention, the activation mechanism of biological brain neurons is analyzed based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, to obtain the sparse activation mechanism, hierarchical activation mechanism and activation distribution law of visual cortex neurons, including: Based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex that are related to visual stimuli, a sparse activation analysis of neurons based on target stimulus relevance is performed to obtain the sparse activation mechanism. Based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, the neuronal activity pattern analysis of brain region functional hierarchy is performed to obtain the hierarchical activation mechanism. Based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex that are related to visual stimuli, the activation distribution pattern of neurons is analyzed based on the background noise control to obtain the activation distribution pattern.

[0013] According to a method for constructing a neural network based on visual search provided by the present invention, the method involves performing neuron activation analysis based on target stimulus relevance, based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, to obtain the sparse activation mechanism, including: Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each sub-experiment of the visual search experiment containing background noise, determine the first ratio of the number of neurons associated with visual stimuli to the number of neurons contained in each visual cortex when the stimulus image appears in all locations in all experimental groups included in the visual search experiment containing background noise. Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each of the sub-experiments included in the visual search experiment without background noise, determine a second ratio of the number of neurons associated with visual stimuli to the number of neurons contained in each visual cortex when the stimulus image appears at a single location in a single experimental group included in the visual search experiment with background noise. The sparse activation mechanism is determined based on the ratio of the second quantity ratio to the first quantity ratio.

[0014] According to a method for constructing a neural network based on visual search provided by the present invention, the neuronal activity pattern analysis based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli is performed to divide the brain regions into functional levels, thereby obtaining the hierarchical activation mechanism, including: Each of the aforementioned visual cortexes is divided into brain region functional levels to obtain at least one brain region functional level. Based on the neurons contained in each functional layer of the brain region and the neurons related to visual stimuli in each functional layer of the brain region in the sub-experiments of the visual search experiment containing background noise, determine the third ratio of the neurons related to visual stimuli to the neurons contained in each functional layer of the brain region when the stimulus image appears in all locations in all experimental groups of the visual search experiment containing background noise. Based on the neurons contained in each functional layer of the brain region and the neurons related to visual stimuli in each functional layer of the brain region in the sub-experiments of the visual search experiment without background noise, determine the fourth ratio of the neurons related to visual stimuli to the neurons contained in each functional layer of the brain region when the stimulus image appears in all locations in all experimental groups of the visual search experiment with background noise. The hierarchical activation mechanism is determined based on the third and fourth quantity ratios.

[0015] According to a method for constructing a neural network based on visual search provided by the present invention, the activation distribution pattern of neurons is analyzed based on the background noise control, based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, to obtain the activation distribution pattern, including: Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each sub-experiment of the visual search experiment containing background noise, determine the fifth ratio of the neurons associated with visual stimuli to the neurons contained in each visual cortex when the stimulus image appears in all locations in all experimental groups included in the visual search experiment containing background noise. Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each sub-experiment of the visual search experiment without background noise, determine the sixth ratio of the neurons associated with visual stimuli to the neurons contained in each visual cortex when the stimulus image appears in all locations in all experimental groups of the visual search experiment with background noise. The activation distribution pattern is determined based on the fifth and sixth quantity ratios.

[0016] According to a method for constructing a neural network based on visual search provided by the present invention, before determining the neurons contained in each visual cortex of the sub-experiment and the neurons in each visual cortex related to visual stimuli based on the brain imaging data corresponding to the sub-experiment, the method further includes: Non-rigid registration is performed on the brain imaging data to obtain registered brain imaging data; The step of determining the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli based on the brain imaging data corresponding to the sub-experiment includes: Based on the registered brain imaging data corresponding to the sub-experiment, the neurons contained in each visual cortex in the sub-experiment and the neurons in each visual cortex related to visual stimuli are determined.

[0017] The present invention also provides a neural network construction system based on visual search, comprising: The visual search experiment module is configured to perform visual search experiments with and without background noise on the experimental subjects to obtain brain imaging data of the experimental subjects. Both the visual search experiments with and without background noise include at least one round of experimental groups, and each experimental group includes at least two sub-experiments. The determination module is configured to, for each of the sub-experiments, determine the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli, based on the brain imaging data corresponding to the sub-experiment. The analysis module is configured to perform biological brain neuron activation mechanism analysis based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, to obtain the sparse activation mechanism, hierarchical activation mechanism and activation distribution law of visual cortex neurons. The building module is configured to build a target neural network based on at least one of the sparse activation mechanism, the hierarchical activation mechanism, and the activation distribution law.

[0018] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the neural network construction method based on visual search as described above.

[0019] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the neural network construction method based on visual search as described above.

[0020] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the neural network construction method based on visual search as described above.

[0021] The present invention provides a method and system for constructing a neural network based on visual search. This involves conducting visual search experiments with and without background noise on experimental subjects to obtain brain imaging data. Both the visual search experiments with and without background noise include at least one round of experimental groups, and each experimental group includes at least two sub-experiments. For each sub-experiment, based on the corresponding brain imaging data, the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli are determined. Based on the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli, the activation mechanism of biological brain neurons is analyzed to obtain the sparse activation mechanism, hierarchical activation mechanism, and activation distribution law of visual cortex neurons. A target neural network is constructed based on at least one of the sparse activation mechanism, the hierarchical activation mechanism, and the activation distribution law. This invention solves the problem of representing sparse activation mechanisms, hierarchical activation mechanisms, and activation distribution patterns. Based on this, at least one of the sparse activation mechanisms, hierarchical activation mechanisms, and activation distribution patterns is applied to the construction and optimization of neural networks in artificial intelligence. This not only improves the robustness of neural networks but also reduces their power consumption. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating the neural network construction method based on visual search provided by the present invention.

[0024] Figure 2 This is a schematic diagram of the structure of the neural network construction system based on visual search provided by the present invention.

[0025] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] The following is combined with Figures 1 to 3 The present invention describes a method and system for constructing a neural network based on visual search.

[0028] First, a brief description of the relevant content involved in this invention will be given.

[0029] The principle behind fluorescent labeling technology for neurons is the specific expression of fluorescent proteins by nerve cells, allowing researchers to observe fluorescence signals in specific wavelengths under a mesoscopic fluorescence microscope. By analyzing the intensity of the fluorescence signals, the strength of individual and collective neuronal activity can be indirectly determined.

[0030] Research on visual search mechanisms includes two aspects: the study of attention mechanisms and the study of neuronal activity patterns in the visual cortex. This invention mainly focuses on the study of neuronal activity patterns in the visual cortex, that is, the encoding mechanism of neuronal populations in the primary and higher visual cortex during the visual search process of experimental subjects for target objects / images.

[0031] Existing brain imaging analysis methods in visual behavioral experiments focus primarily on the basic cognitive functions of different brain regions under target visual stimuli, including brain regions or sub-brain regions such as the functional columns of the visual cortex, prefrontal cortex, hippocampus, dorsal striatum, and ventral striatum. They pay less attention to how mice find target stimuli through visual search, and the analysis of neuronal population activity patterns during the visual search process.

[0032] Existing theoretical research on brain atlases proposes the concept of functional levels in the visual cortex from the perspective of dendrites and axonal connections of visual neurons. This divides the visual cortex into four functional levels: the Primary Visual Cortex (V1), Lateromedial area (LM), Rostrolateral area (RL), Lateral Posterior nucleus (LP), Anterolateral area (AL), Posteromedial area (PM), and Anteromedial area (AM). These four functional levels are located in the feedforward and feedback layers of complex neural networks, precisely coordinating to complete the transmission and processing of visual information. However, current research on the activity patterns of visual cortical neurons in visual search treats different visual cortexes as isolated information islands, with limited research on the interconnected activation patterns between multiple visual cortexes. Furthermore, it rarely conducts grouped statistical studies of related brain regions of the visual cortex from a functional perspective, failing to uncover the activation patterns of different functional levels during the visual search process.

[0033] In addition, the real world environment is full of visual background noise, so it is valuable to simulate the real environment and study how model animals search for target visual stimuli under background noise interference. However, existing visual behavioral experiments usually only present target stimuli to model animals without background (usually a pure black background), so they cannot study the neurocognitive mechanisms under the influence of background noise.

[0034] It is evident that existing technologies suffer from the following problems: full-screen grating stimulation experiments cannot simulate the state of visual search in mice; some visual stimulation experiments involving changes in image shape or position cannot delve into the visual search process; their experimental protocols lack targeted quantitative evaluation indicators from the perspective of visual search research, and cannot study the neuronal population activity patterns during the visual search process; existing research on the activity patterns of neurons in the visual cortex treats different areas of the visual cortex as isolated information islands, rarely grouping and statistically studying related brain regions of the visual cortex from the perspective of functional levels of the visual cortex, and rarely studying the associated activation patterns of different functional levels, thus failing to discover the activation patterns of different functional levels during the visual search process; the real-world environment is full of visual background noise, and existing research has little to address how background noise affects the visual search efficiency of mice, and how it is represented at the level of neurocognitive mechanisms in the visual cortex.

[0035] Therefore, it is urgent to analyze the activity patterns of neurons in the mouse visual cortex. By analyzing the functional levels and association patterns of the primary and higher visual cortex during the mouse visual search process, we can study the activation mechanism of neurons in the visual cortex during this visual cognitive processing, and thus construct the target neural network.

[0036] Figure 1 This is a flowchart illustrating the neural network construction method based on visual search provided by the present invention, as shown below. Figure 1 As shown, the method includes: Step 101: Perform visual search experiments with and without background noise on the experimental subjects to obtain brain imaging data of the experimental subjects. Both the visual search experiments with and without background noise include at least one round of experimental groups, and each experimental group includes at least two sub-experiments. Step 102: For each sub-experiment, based on the brain imaging data corresponding to the sub-experiment, determine the neurons contained in each visual cortex in the sub-experiment and the neurons in each visual cortex related to visual stimuli; Step 103: Based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, perform biological brain neuron activation mechanism analysis to obtain the sparse activation mechanism, hierarchical activation mechanism and activation distribution law of visual cortex neurons. Step 104: Construct the target neural network based on at least one of the sparse activation mechanism, the hierarchical activation mechanism, and the activation distribution law.

[0037] Specifically, visual search experiments including background noise are represented by the Visual_E1 experiment, and visual search experiments without background noise are represented by the Visual_E2 experiment. The visual search experiment can be found in the "Visual Search Experiment Method for Head-Fixed Brain Imaging" provided by invention patent CN121195862A, which designs a visual search experiment method for mice, enabling stable brain imaging with the head fixed. This invention performs microscopic imaging and spike activity sequence analysis on neurons in the visual brain region during this visual search experiment and provides a method for constructing a neural network based on visual search. The data analysis involved in this invention is a follow-up operation to the visual search experiment mentioned in the aforementioned patent, mainly aimed at mining all neurons related to the target stimulus and the neuronal activity patterns under the influence of background noise, and does not involve patent conflict.

[0038] To avoid experimental bias, each visual search experiment is typically repeated at least once. For example, a visual search experiment might consist of five rounds, with each round containing 18 trials (sub-experiments). These 18 sub-experiments include nine trials with valid visual stimuli (with a target image) and nine trials with invalid visual stimuli (without a target image). It's important to note that the position of the target image on the screen is not fixed each time.

[0039] Specifically, the experimental subjects, or model animals, can be mice.

[0040] Before conducting the visual search experiment, the subjects needed to undergo the following procedures: they were placed under gas anesthesia, a cranial flap above the cortical tissue was removed to create a cranial window, a glass coverslip was fixed to the skull using cyanoacrylate adhesive, and a matching headpiece was glued to the skull using dental cement. Brain imaging studies, i.e., the visual search experiment, began two weeks after the surgery to ensure its successful execution.

[0041] For example, mice (e.g., male, 8-12 week old mice, numbered TIT2L-GC6s-ICL-tTA2, Ai162) were anesthetized with isoflurane. The mice were then fixed in place under a microscope (e.g., Olympus 25× / 1.05 NA). A cranial window was created by removing a 10 mm diameter cranial flap (with AP-5.2 as the center and ML-7.1 as the median) above the visual cortex (Interaural distance 0.2 mm, Bregma 5.2 mm) and surrounding cortical tissue, leaving the dura mater intact. The exposed cortical surface was covered with a 12mm diameter glass coverslip (No. 1 thickness, 64-0700, CS-5R, Warner Instruments), which was then fixed to the skull using UV-cured cyanoacrylate adhesive (Loctite, 4305). A matching metal headpiece was then attached to the skull using dental cement acrylic to secure the imaging instrument interface to the head. Neurons were fluorescently labeled with GCaMP6f as a calcium ion indicator. Brain imaging studies, specifically visual search experiments, began at least 14 days post-surgery.

[0042] Throughout the visual stimulation experiment (visual search experiment), calcium ion imaging of the cerebral cortex of the experimental subjects was continuously performed, i.e., brain imaging data was acquired. Specifically, single-photon or two-photon mesoscopic microscopy equipment could be used to observe the cerebral cortex of the experimental subjects. When neurotransmitters in neurons are active, the movement of calcium ions within the nerve cells is also active. At this time, a laser of a specific wavelength will cause fluorescent proteins to bind to calcium ions, thereby emitting fluorescence of a specific wavelength, which is captured by the microscopic imaging equipment. This allows for real-time observation of neuronal activity in the mouse brain, thus obtaining neuronal activation image data, i.e., brain imaging data.

[0043] Specifically, each visual cortex may include at least one of V1, LM, RL, LP, AL, PM, AM, medial mammary nucleus with a negative patch to PM (MMP), medial mammary nucleus with a positive patch to AM (MMA), retrosplenial cortex (RS), lateral intermediate area (LI), and lateral anterior area (LLA).

[0044] In practical applications, for the brain imaging data corresponding to each sub-experiment, the constrained nonnegative matrix factorization (CNMF) algorithm and the calcium imaging analysis (CaImAn) algorithm can be used to decompose the spatiotemporal activity of neurons in the brain imaging data into spatial components with local structure and temporal components of calcium fluorescence dynamics. The estimated value of neural activity can be extracted from the calcium transient trace, and neuronal cell segmentation and peak activity sequence extraction can be performed to determine the neurons contained in each visual cortex in the sub-experiment and the neurons in each visual cortex that are related to visual stimuli.

[0045] In the analysis, a state-space model can be used to represent the dynamic system of neuronal responses during biological visual search. From the level of individual neurons to network cognitive models, the information processing and encoding mechanisms of spike activity in visual cortex neurons are studied, revealing the complex dynamic characteristics of neuronal populations. The impulse response distribution of each neuron to visual stimulus signals is obtained, and the responses of all neurons within the population are combined to form an overall representation of the visual stimulus. Brain network analysis based on neuronal population encoding is then performed.

[0046] This study analyzes the complex spike-like firing activity signals of visual cortical neurons in experimental subjects under different states, including saccades, fixation, and non-visual attention. It examines the significant differences in neuronal activity patterns between task-oriented and resting-state states during biological visual search experiments, and infers the frequency and temporal encoding patterns of neurons during biological visual retrieval. Simultaneously, considering the influence of image features of different target stimuli and background noise intensity, clustering and correlation analysis are performed on different neuronal groups formed by population encoding to investigate the mechanisms of selective inhibition and sparse activation of neuronal populations. This leads to the understanding of the sparse activation mechanism, hierarchical activation mechanism, and activation distribution patterns of visual cortical neurons.

[0047] After obtaining the sparse activation mechanism, hierarchical activation mechanism, and activation distribution law of neurons in the visual cortex, at least one of them can be applied to the construction of the target neural network. The constructed target neural network can be an efficient and biologically plausible neural network.

[0048] For example, we will illustrate this by constructing a target neural network based on sparse activation and hierarchical activation mechanisms. We will use sparse activation as the core guiding principle and combine it with hierarchical activation to design the network architecture. We will use an image classification task as an example: First, a convolutional neural network for image classification is constructed. While ensuring classification accuracy, the computational overhead is reduced and the feature selectivity is improved through a sparse activation mechanism.

[0049] Then, employing the concept of hierarchical activation, a network with three convolutional stages is constructed, each containing multiple convolutional layers to progressively extract features from low to high levels. Simultaneously, a sparse activation mechanism is introduced after each nonlinear transformation. In hierarchical activation, convolutional layers in different stages are responsible for features at different levels of abstraction, gradually building semantic understanding of the image through layer-by-layer abstraction. In sparse activation, all convolutional layers use Rectified Linear Units (ReLU) as activation functions, whose one-sided inhibition naturally produces sparse activation (negative values ​​are set to zero). To further enhance sparsity, an L1 regularization penalty for activation values ​​can be added to the loss function.

[0050] While ReLU can produce some sparsity in standard networks, the degree of sparsity is uncontrollable. Sparse activation can be explicitly encouraged through activation function selection and regularization of the loss term.

[0051] Next, the constructed neural network is trained by setting training parameters, and the activation distribution is monitored and analyzed. In order to verify the effectiveness of the sparse activation mechanism, the distribution of activation values ​​of each layer is statistically analyzed periodically during the training process, such as sparsity and activation histogram.

[0052] Finally, post-training evaluation and adjustments are performed. After training, the impact of the sparse activation mechanism is analyzed from at least one aspect, such as final accuracy, activation sparsity, inference efficiency, and visualization. For example, if excessive sparsity leads to a significant decrease in accuracy, the sparsity regularization coefficient can be appropriately reduced; if sparsity is insufficient, the sparsity regularization coefficient can be increased or other sparsity-inducing methods can be tried.

[0053] Through the above process, a neural network based on sparse activation mechanism and hierarchical activation mechanism can be constructed.

[0054] It should be noted that the process of constructing a target neural network based on at least one of the sparse activation mechanism, hierarchical activation mechanism, and activation distribution law is similar, and will not be elaborated here.

[0055] The present invention provides a method for constructing a neural network based on visual search. This method obtains brain imaging data of the experimental subjects by conducting visual search experiments with and without background noise. Both the visual search experiments with and without background noise include at least one round of experimental groups, and each experimental group includes at least two sub-experiments. For each sub-experiment, based on the corresponding brain imaging data, the method determines the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli. Neurons and neurons in the visual cortex associated with visual stimuli are analyzed using biological brain neuron activation mechanisms to obtain sparse activation mechanisms, hierarchical activation mechanisms, and activation distribution patterns of visual cortex neurons. A target neural network is constructed based on at least one of these mechanisms. The representation problems of sparse activation mechanisms, hierarchical activation mechanisms, and activation distribution patterns are solved. Furthermore, applying at least one of these mechanisms to the construction and optimization of artificial intelligence neural networks can not only improve the robustness of the neural network but also reduce its power consumption.

[0056] Optionally, before determining the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli based on the brain imaging data corresponding to the sub-experiment, the method further includes: Non-rigid registration is performed on the brain imaging data to obtain registered brain imaging data; The step of determining the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli based on the brain imaging data corresponding to the sub-experiment includes: Based on the registered brain imaging data corresponding to the sub-experiment, the neurons contained in each visual cortex in the sub-experiment and the neurons in each visual cortex related to visual stimuli are determined.

[0057] In practical applications, the natural movement of the experimental subject can cause mechanical displacement between the imaging instrument and the biological brain, resulting in motion artifacts. Therefore, image registration is necessary.

[0058] Specifically, for non-rigid motion artifacts caused by the natural movement of the experimental object, non-rigid motion correction (NoRMCorre) is used for image registration to eliminate image jitter caused by the movement of the experimental object or the vibration of the equipment.

[0059] NoRMCorre overcomes the drawbacks of motion alignment and has low computational cost, making it suitable for online real-time registration of large volumes of streaming data. It works by splitting the field of view (FOV) into overlapping patches, registering each patch separately, and then merging them using smooth interpolation. Compared to other available non-rigid registration methods, NoRMCorre can account for non-uniform artifacts along all axes, enabling it to capture non-rigid brain motion well.

[0060] After image registration, determining the neurons contained in each visual cortex in the sub-experiment and the neurons in each visual cortex related to visual stimuli can ensure the accuracy of experimental neuron identification and experimental analysis.

[0061] Optionally, determining the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli based on the brain imaging data corresponding to the sub-experiment includes: The brain imaging data corresponding to the sub-experiment is subjected to cell segmentation of biological brain neurons to extract activation trace data of the biological brain neurons, wherein the activation trace data represents the trace data of the activated biological brain neurons. Based on the activation trace data of the biological brain neurons, the neurons contained in each visual cortex in the sub-experiment and the neurons in each visual cortex related to visual stimuli were determined.

[0062] In practical applications, (registered) microscopic imaging data, i.e. brain imaging data, can be converted into trace data of neuronal activation, which is a real-number time series: first, neuronal cells are labeled and identified through supervised learning algorithms, and then the signal-to-noise ratio and calcium ion dynamics of the image are estimated, thereby realizing trace extraction.

[0063] Furthermore, the neurons contained in each visual cortex in this sub-experiment, as well as the neurons in each visual cortex that are associated with visual stimuli, were identified from the trace.

[0064] In this embodiment of the invention, activation trace data is extracted by cell segmentation to identify each neuron, which can improve the accuracy of the identification.

[0065] Optionally, the biological brain neurons of the experimental subject are fluorescently labeled with a calcium ion indicator, and the brain imaging data includes calcium ion imaging data; The step of segmenting the brain imaging data corresponding to the sub-experiment into biological brain neurons to extract activation trace data of the biological brain neurons includes: By using a supervised learning algorithm, cell location marking and identification are performed on the biological brain neurons in the calcium ion imaging data corresponding to the sub-experiment to obtain calcium transient traces; In practical applications, the (registered) calcium ion imaging data can be used to identify cell locations and deconvolve their neural activity, extracting estimates of neural activity from fluorescence traces, and then extracting traces.

[0066] Specifically, the CNMF algorithm is used to decompose the spatiotemporal activity of neurons reflected in calcium ion imaging data into spatial components (x, y) with local structure and temporal components (t) simulating calcium dynamics. In the low signal-to-noise ratio (SNR) region, by applying a structured model of calcium index dynamics, a significant improvement in the spatial trace of observed neurons can be obtained.

[0067] In this embodiment of the invention, a structured model of calcium fluorescence dynamics is used to improve the extraction of neuronal spike sequences in low signal-to-noise ratio (SNR) regions, thereby improving the accuracy and reliability of traces.

[0068] Optionally, determining the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli based on the activation trace data of the biological brain neurons includes: Correlation analysis and feature dimension reduction are performed on the activation trace data of the biological brain neurons to determine the neurons contained in each visual cortex and the neurons in each visual cortex that are related to visual stimuli.

[0069] In practical applications, the effectiveness of target stimuli in visual search experiments is tested. Factors such as strain differences in experimental subjects, screen size, and the distance between the screen and the mouse's eyeballs all affect the visual search. The controlled variable method requires eliminating irrelevant variables, which need to be eliminated through multiple rounds of fine-tuning. Therefore, dimensionality reduction and clustering analysis algorithms such as t-distributed stochastic neighbor embedding (tSNE), principal component analysis (PCA), and K-means clustering can be used to test the effectiveness of visual stimuli. If neurons can be well divided into N... trail ×N time This class demonstrates the effectiveness of the visual stimulus, ensuring the validity of subsequent analyses of neuronal activation mechanisms. It also yields the neurons contained in each visual cortex within the activation trace data and the neurons in each visual cortex associated with the visual stimulus. Among these, N...trai N represents the number of repeated stimuli in a single round (the number of sub-experiments in the experimental group). time The duration of a single stimulus (the duration of a sub-experiment).

[0070] Optionally, the step of performing correlation analysis and feature dimension reduction on the activation trace data of the biological brain neurons to determine the neurons contained in each of the visual cortexes and the neurons in each of the visual cortexes related to visual stimuli includes: Based on the activation trace data of the biological brain neurons, the Pearson correlation coefficient of the average stimulus response intensity of each neuron was calculated. Principal component analysis is used to reduce the feature dimension of each neuron located in each of the visual cortexes to obtain the neurons contained in each of the visual cortexes. The neurons in each of the visual cortexes whose Pearson correlation coefficient represents a significant response are identified as neurons in each of the visual cortexes that are associated with visual stimuli.

[0071] Specifically, to analyze neural responses to visual stimuli, correlation analysis and feature dimension reduction were performed on the trace to screen out neurons associated with the visual stimuli. Neurons observed at specific locations during each trial were used.

[0072] To obtain the correlation map between different rounds, the central tendency of the entire population is captured by calculating the Pearson correlation coefficient of the average stimulus response intensity of each neuron. That is, the Pearson correlation coefficient can reflect the clustering trend.

[0073] Furthermore, principal component analysis (PCA) is used to reduce the dimensionality of neurons (i.e., after visual stimulation is generated, a set number of core neurons are selected from the trace neurons according to the stimulation intensity, which are then used as the required neurons). Then, neurons that show significant responses during stimulation are grouped, resulting in the neurons and groups output by the PCA method. For N... trial ×N time Data within a time window is filtered using tSNE to select the most relevant N. relation Individual neurons were visualized and analyzed. The average stimulus response intensity was obtained by averaging based on the time component and trace.

[0074] Furthermore, to compare changes in neural activity intensity, the curves were normalized using the mean in each round. The trace trajectory can be divided into two phases (during the period without background stimulation and during the period with background stimulation), and the standard deviation of the stimulus response intensity in each phase is used as a feature of each neuron. Then, K-means clustering is used to group neurons into clusters based on these features, and the percentage of each cluster is calculated.

[0075] For example, the trace is divided into two phases (during the period without background stimulation and during the period with background stimulation). The z-score is normalized, and the standard deviation of each phase is used as the feature of each neuron. Sequential Minimal Optimization (SMO) is used to optimize the performance of the Support Vector Machine (SVM) classifier, obtaining estimates of the overall linearly decodable information in multiple neuron populations, and exploring the sparsity principle of biological neuron population encoding during visual target search. The results show that the percentage of neurons encoding a specific target stimulus is less than 40% of all relevant neurons, reaching statistical significance.

[0076] In this embodiment of the invention, Pearson correlation statistical analysis was performed on the Visual_E1 and Visual_E2 experiments, and false positive results were corrected by false discovery rate (FDR) correction, thereby quantifying the sparse activation mechanism of neurons in the visual cortex. Optionally, based on the neurons contained in each visual cortex in each of the sub-experiments and the neurons in each visual cortex related to visual stimuli, the biological brain neuron activation mechanism analysis is performed to obtain the sparse activation mechanism, hierarchical activation mechanism, and activation distribution pattern of visual cortex neurons, including: Based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex that are related to visual stimuli, a sparse activation analysis of neurons based on target stimulus relevance is performed to obtain the sparse activation mechanism. Based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, the neuronal activity pattern analysis of brain region functional hierarchy is performed to obtain the hierarchical activation mechanism. Based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex that are related to visual stimuli, the activation distribution pattern of neurons is analyzed based on the background noise control to obtain the activation distribution pattern.

[0077] In practical applications, based on the acquisition of neurons contained in each visual cortex in each sub-experiment and neurons in each visual cortex related to visual stimuli, functional hierarchical divisions are performed for multiple brain regions in each visual cortex. Statistical analysis methods such as T-tests and Cohen's d-value effect size analysis are used to analyze the significant differences in functional hierarchical differences, thereby quantifying the impact of background noise on visual search and analyzing the hierarchical activation mechanism of visual cortex neurons. Specifically, a neuronal activation analysis method based on target stimulus relevance is used to determine the sparse activation mechanism of visual cortex neurons; a neuronal activity pattern analysis method based on the functional hierarchical division of visual brain regions is used to determine the hierarchical activation mechanism of visual cortex neurons; and a control experiment introducing background noise into visual stimuli is used to determine the activation distribution pattern of visual cortex neurons under the influence of background stimulus noise.

[0078] In the implementation of this invention, different methods are used to analyze different biological brain neuron activation mechanisms, ensuring the reliability of each biological brain neuron activation mechanism.

[0079] Optionally, the sparse activation mechanism is obtained by performing neuronal activation analysis based on target stimulus relevance, based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, including: Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each sub-experiment of the visual search experiment containing background noise, determine the first ratio of the number of neurons associated with visual stimuli to the number of neurons contained in each visual cortex when the stimulus image appears in all locations in all experimental groups included in the visual search experiment containing background noise. Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each of the sub-experiments included in the visual search experiment without background noise, determine a second ratio of the number of neurons associated with visual stimuli to the number of neurons contained in each visual cortex when the stimulus image appears at a single location in a single experimental group included in the visual search experiment with background noise. The sparse activation mechanism is determined based on the ratio of the second quantity ratio to the first quantity ratio.

[0080] Specifically, in the Visual_E1 experiment, the screen was divided into 3×3 squares, and the target stimulus image appeared randomly in one of the squares with equal probability. The subject visually searched the screen for the target stimulus image. A correct response was recorded when the target stimulus appeared on the screen and the subject licked the spout. The Visual_E2 experiment differed from the Visual_E1 experiment in that it incorporated background image noise.

[0081] To study the activation patterns of biological brain neurons for visual search from three levels, this invention conducted three sets of visual search experiments (experiment on sparse activation patterns of neurons, experiment on hierarchical activation patterns of neurons, and experiment on activation patterns of biological brain neurons under background noise interference), employing three statistical strategies respectively.

[0082] For the experiment on the sparse activation pattern of neurons, according to the brain region division rules, in the Visual_E1 experiment, the proportion of significantly relevant neurons to the total number of extracted neurons when the stimulus image appeared in all locations in all rounds was counted (first proportion); in the Visual_E2 experiment, only the response of neurons when visual stimuli appeared in a single round and a specific location (single location) was considered, and the proportion of neurons activated when the stimulus image appeared only in a certain location in that round to the total number of extracted neurons was counted (second proportion).

[0083] For example, in the Visual_E1 experiment, statistical analysis revealed the following: A total of 2957 neurons were extracted from the V1 brain region, of which 2547 neurons (86.1%) were significantly activated; 460 neurons were extracted from the PM brain region, of which 416 neurons (90.4%) were significantly activated; 452 neurons were extracted from the AM brain region, of which 324 neurons (71.7%) were significantly activated; 463 neurons were extracted from the RL brain region, of which 407 neurons (87.9%) were significantly activated; 309 neurons were extracted from the AL brain region, of which 260 neurons (84.1%) were significantly activated; and the LM brain region… A total of 347 neurons were extracted, of which 300 (86.5%) were significantly activated. 250 neurons were extracted from the MMP region, of which 229 (91.6%) were significantly activated. 327 neurons were extracted from the RS region, of which 255 (78.0%) were significantly activated. 169 neurons were extracted from the MMA region, of which 78 (46.2%) were significantly activated. 130 neurons were extracted from the LI region, of which 104 (80.0%) were significantly activated. 167 neurons were extracted from the LLA region, of which 130 (77.8%) were significantly activated. To find all relevant neurons, a total of 6031 neurons were extracted in the Visual_E1 experiment, of which 5050 (83.7%) were significantly activated.

[0084] In the Visual_E2 experiment, neurons activated under background interference were identified (partial neuronal activation). Statistical analysis revealed the following: 3011 neurons were extracted from the V1 brain region, of which 1016 were significantly activated (33.7%); 649 neurons were extracted from the PM brain region, of which 206 were significantly activated (31.7%); 575 neurons were extracted from the AM brain region, of which 173 were significantly activated (30.1%); 272 neurons were extracted from the RL brain region, of which 98 were significantly activated (36.0%); and 201 neurons were extracted from the AL brain region, of which 89 were significantly activated (4%). 4.4%; 261 neurons were extracted from the LM brain region, of which 93 were significantly activated, accounting for 35.6%; 302 neurons were extracted from the MMP brain region, of which 108 were significantly activated, accounting for 35.8%; 898 neurons were extracted from the RS brain region, of which 211 were significantly activated, accounting for 23.5%; 451 neurons were extracted from the MMA brain region, of which 162 were significantly activated, accounting for 35.9%; 19 neurons were extracted from the LI brain region, of which 11 were significantly activated, accounting for 57.9%; 90 neurons were extracted from the LLA brain region, of which 26 were significantly activated, accounting for 28.9%. To find all relevant neurons, a total of 6729 neurons were extracted in the Visual_E2 experiment, of which 2193 were significantly activated, accounting for 32.6%.

[0085] Define the sparse activation ratio as SAR=N Visual_E2 / N Visual_E1 Where SAR is the sparse activation ratio, and N is the sparse activation ratio. Visual_E2 N is the second quantity ratio. Visual_E1 This represents the first quantity proportion.

[0086] Calculations show that neurons in the biological visual cortex use sparse activation for visual searching, with a single activation rate of only 32.6 / 83.7=38.9%.

[0087] Optionally, the neuronal activity pattern analysis based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, performing brain region functional hierarchy division, yields the hierarchical activation mechanism, including: Each of the aforementioned visual cortexes is divided into brain region functional levels to obtain at least one brain region functional level. Based on the neurons contained in each functional layer of the brain region and the neurons related to visual stimuli in each functional layer of the brain region in the sub-experiments of the visual search experiment containing background noise, determine the third ratio of the neurons related to visual stimuli to the neurons contained in each functional layer of the brain region when the stimulus image appears in all locations in all experimental groups of the visual search experiment containing background noise. Based on the neurons contained in each functional layer of the brain region and the neurons related to visual stimuli in each functional layer of the brain region in the sub-experiments of the visual search experiment without background noise, determine the fourth ratio of the neurons related to visual stimuli to the neurons contained in each functional layer of the brain region when the stimulus image appears in all locations in all experimental groups of the visual search experiment with background noise. The hierarchical activation mechanism is determined based on the third and fourth quantity ratios.

[0088] In practical applications, there are many brain regions related to visual function. However, according to relevant studies on the functional hierarchy of brain maps, the visual cortex can be divided into 4 layers. Corresponding to experiments on the hierarchical activation of neurons, all related brain regions are divided into 4 levels: Level I (including V1), Level II (including LM and RL), Level III (including LP and AL), and Level IV (including AM and PM).

[0089] According to the hierarchical division rules, in the Visual_E1 experiment, the proportion of all saliency-related neurons at each level to the total number of extracted neurons was counted when the stimulus images appeared at all positions in all rounds. Similarly, in the Visual_E2 experiment, the proportion of all saliency-related neurons at each level to the total number of extracted neurons was counted when the stimulus images appeared at all positions in all rounds.

[0090] Specifically, to investigate the neuronal activation mechanisms at different functional levels of the cortex, neurons extracted from the Visual_E1 and Visual_E2 experiments were grouped and statistically analyzed according to functional level. In the Visual_E1 experiment, the neuronal activation proportions were as follows: I: 2579 / 3011 = 85.7%, II: 370 / 533 = 69.4%, III: 143 / 220 = 65%, IV: 709 / 1226 = 57.8%. In the Visual_E2 experiment, the neuronal activation proportions were as follows: I: 2071 / 2530 = 81.9%, II: 550 / 754 = 72.9%, III: 131 / 185 = 70.8%, IV: 847 / 1251 = 67.7%.

[0091] Longitudinal comparisons reveal that as functional levels increase, the activation rate of higher-level neurons decreases compared to lower-level neurons. Horizontal comparisons show that, due to the inclusion of background image noise in the Visual_E2 experiment compared to the Visual_E1 experiment, the activation rate of neurons at the same level is higher in Visual_E2 than in Visual_E1. This indirectly confirms that the visual noise in the experiment effectively elicited a response in the mouse visual cortex.

[0092] Optionally, based on the neurons contained in each visual cortex in each of the sub-experiments and the neurons in each visual cortex related to visual stimuli, the neuronal activation distribution pattern analysis based on the background noise control is performed to obtain the activation distribution pattern, including: Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each sub-experiment of the visual search experiment containing background noise, determine the fifth ratio of the neurons associated with visual stimuli to the neurons contained in each visual cortex when the stimulus image appears in all locations in all experimental groups included in the visual search experiment containing background noise. Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each sub-experiment of the visual search experiment without background noise, determine the sixth ratio of the neurons associated with visual stimuli to the neurons contained in each visual cortex when the stimulus image appears in all locations in all experimental groups of the visual search experiment with background noise. The activation distribution pattern is determined based on the fifth and sixth quantity ratios.

[0093] In practical applications, for experiments on the activation patterns of biological brain neurons under background noise interference, no division is made. In the Visual_E1 experiment, the proportion of all significantly related neurons to the total number of extracted neurons was counted when the stimulus image appeared in all positions in all rounds. Similarly, in the Visual_E2 experiment, the proportion of all significantly related neurons to the total number of extracted neurons was counted when the stimulus image appeared in all positions in all rounds.

[0094] Specifically, to further analyze the impact mechanism of interference noise on visual search, the proportion of all neurons related to the target stimulus to all extracted neurons was analyzed: In the Visual_E1 experiment, the proportion of all related neurons to all extracted neurons was: 4327 / 6031 = 70.7% ± 0.113; In the Visual_E2 experiment, the percentage of all relevant neurons to all extracted neurons was: Visual_E_2 = 5570 / 6676 = 78.4% ±0.096. Among them, " "Represents significance (Sig.)," "The more, the stronger the significance, specifically..." Sig. < 0.001; Sig. < 0.01; Sig. < 0.05.

[0095] This invention addresses the representation problem of sparse activation mechanisms in visual cortex neurons based on target stimulus relevance neuronal activation analysis. It also proposes a neuronal activity pattern analysis method based on functional hierarchical division of mouse visual brain regions, resolving the representation problem of hierarchical activation mechanisms in visual cortex neurons. Furthermore, by introducing background noise into the visual stimulus as a control experiment, it solves the problem of representing the activation distribution patterns of visual cortex neurons under the influence of background stimulus noise. Based on this, applying at least one of the sparse activation mechanism, hierarchical activation mechanism, and activation distribution patterns to the construction and optimization of artificial intelligence neural networks can not only improve the robustness of neural networks but also reduce their power consumption.

[0096] Grouped statistical analysis revealed that the activation rate of neurons under background noise interference was significantly higher than that without background noise, and this was observed at different levels.

[0097] The visual search-based neural network construction system provided by this invention will be described below. The visual search-based neural network construction system described below can be referred to in correspondence with the visual search-based neural network construction method described above.

[0098] Figure 2 This is a schematic diagram of the structure of the neural network construction system based on visual search provided by the present invention, as shown below. Figure 2 As shown, the system includes the following: The visual search experiment module 201 is configured to perform visual search experiments with and without background noise on the experimental subject to obtain brain imaging data of the experimental subject. Both the visual search experiments with and without background noise include at least one round of experimental groups, and each experimental group includes at least two sub-experiments. The determination module 202 is configured to, for each of the sub-experiments, determine the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli based on the brain imaging data corresponding to the sub-experiment. Analysis module 203 is configured to perform biological brain neuron activation mechanism analysis based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, to obtain the sparse activation mechanism, hierarchical activation mechanism and activation distribution law of visual cortex neurons. The construction module 204 is configured to construct a target neural network based on at least one of the sparse activation mechanism, the hierarchical activation mechanism, and the activation distribution law.

[0099] The present invention provides a neural network construction system based on visual search. This system obtains brain imaging data of experimental subjects by conducting visual search experiments with and without background noise. Both the visual search experiments with and without background noise include at least one experimental group, and each experimental group includes at least two sub-experiments. For each sub-experiment, based on the corresponding brain imaging data, the system determines the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli. Neurons and neurons in the visual cortex associated with visual stimuli are analyzed using biological brain neuron activation mechanisms to obtain sparse activation mechanisms, hierarchical activation mechanisms, and activation distribution patterns of visual cortex neurons. A target neural network is constructed based on at least one of these mechanisms. The representation problems of sparse activation mechanisms, hierarchical activation mechanisms, and activation distribution patterns are solved. Furthermore, applying at least one of these mechanisms to the construction and optimization of artificial intelligence neural networks can not only improve the robustness of the neural network but also reduce its power consumption.

[0100] Optionally, the determining module 202 is specifically configured as follows: The brain imaging data corresponding to the sub-experiment is subjected to cell segmentation of biological brain neurons to extract activation trace data of the biological brain neurons, wherein the activation trace data represents the trace data of the activated biological brain neurons. Based on the activation trace data of the biological brain neurons, the neurons contained in each visual cortex in the sub-experiment and the neurons in each visual cortex related to visual stimuli were determined.

[0101] Optionally, the biological brain neurons of the experimental subject are fluorescently labeled with a calcium ion indicator, and the brain imaging data includes calcium ion imaging data; The determining module 202 is specifically configured as follows: By using a supervised learning algorithm, cell location marking and identification are performed on the biological brain neurons in the calcium ion imaging data corresponding to the sub-experiment to obtain calcium transient traces; Calcium ion dynamics were estimated from the calcium transient traces to obtain activation trace data of the biological brain neurons.

[0102] Optionally, the determining module 202 is specifically configured as follows: Correlation analysis and feature dimension reduction are performed on the activation trace data of the biological brain neurons to determine the neurons contained in each visual cortex and the neurons in each visual cortex that are related to visual stimuli.

[0103] Optionally, the determining module 202 is specifically configured as follows: Based on the activation trace data of the biological brain neurons, the Pearson correlation coefficient of the average stimulus response intensity of each neuron was calculated. Principal component analysis is used to reduce the feature dimension of each neuron located in each of the visual cortexes to obtain the neurons contained in each of the visual cortexes. The neurons in each of the visual cortexes whose Pearson correlation coefficient represents a significant response are identified as neurons in each of the visual cortexes that are associated with visual stimuli.

[0104] Optionally, the analysis module 203 includes: The sparse activation analysis module is configured to perform neuronal sparse activation analysis based on target stimulus relevance based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex that are related to visual stimuli, so as to obtain the sparse activation mechanism. The hierarchical activation analysis module is configured to perform neuronal activity pattern analysis based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, and to obtain the hierarchical activation mechanism. The activation distribution pattern analysis module is configured to perform neuronal activation distribution pattern analysis based on the background noise control, based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, to obtain the activation distribution pattern.

[0105] Optionally, the sparse activation analysis module is specifically configured as follows: Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each sub-experiment of the visual search experiment containing background noise, determine the first ratio of the number of neurons associated with visual stimuli to the number of neurons contained in each visual cortex when the stimulus image appears in all locations in all experimental groups included in the visual search experiment containing background noise. Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each of the sub-experiments included in the visual search experiment without background noise, determine a second ratio of the number of neurons associated with visual stimuli to the number of neurons contained in each visual cortex when the stimulus image appears at a single location in a single experimental group included in the visual search experiment with background noise. The sparse activation mechanism is determined based on the ratio of the second quantity ratio to the first quantity ratio.

[0106] Optionally, the hierarchical activation analysis module is specifically configured as follows: Each of the aforementioned visual cortexes is divided into brain region functional levels to obtain at least one brain region functional level. Based on the neurons contained in each functional layer of the brain region and the neurons related to visual stimuli in each functional layer of the brain region in the sub-experiments of the visual search experiment containing background noise, determine the third ratio of the neurons related to visual stimuli to the neurons contained in each functional layer of the brain region when the stimulus image appears in all locations in all experimental groups of the visual search experiment containing background noise. Based on the neurons contained in each functional layer of the brain region and the neurons related to visual stimuli in each functional layer of the brain region in the sub-experiments of the visual search experiment without background noise, determine the fourth ratio of the neurons related to visual stimuli to the neurons contained in each functional layer of the brain region when the stimulus image appears in all locations in all experimental groups of the visual search experiment with background noise. The hierarchical activation mechanism is determined based on the third and fourth quantity ratios.

[0107] Optionally, the activation distribution pattern analysis module is specifically configured as follows: Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each sub-experiment of the visual search experiment containing background noise, determine the fifth ratio of the neurons associated with visual stimuli to the neurons contained in each visual cortex when the stimulus image appears in all locations in all experimental groups included in the visual search experiment containing background noise. Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each sub-experiment of the visual search experiment without background noise, determine the sixth ratio of the neurons associated with visual stimuli to the neurons contained in each visual cortex when the stimulus image appears in all locations in all experimental groups of the visual search experiment with background noise. The activation distribution pattern is determined based on the fifth and sixth quantity ratios.

[0108] Optionally, the visual search-based neural network construction system further includes a registration module, configured as follows: Non-rigid registration is performed on the brain imaging data to obtain registered brain imaging data; The determining module 202 is specifically configured as follows: Based on the registered brain imaging data corresponding to the sub-experiment, the neurons contained in each visual cortex in the sub-experiment and the neurons in each visual cortex related to visual stimuli are determined.

[0109] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 3As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communications bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other through the communications bus 340. The processor 310 can call logical instructions in the memory 330 to execute a visual search-based neural network construction method. This method includes: conducting visual search experiments with and without background noise on an experimental subject to obtain brain imaging data of the subject; each of the visual search experiments with and without background noise includes at least one round of experimental groups, and each experimental group includes at least two sub-experiments; for each sub-experiment, based on the corresponding brain imaging data, determining the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli; based on the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli, performing biological brain neuron activation mechanism analysis to obtain the sparse activation mechanism, hierarchical activation mechanism, and activation distribution law of visual cortex neurons; and constructing a target neural network based on at least one of the sparse activation mechanism, the hierarchical activation mechanism, and the activation distribution law.

[0110] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0111] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the visual search-based neural network construction method provided by the above methods. The method includes: conducting visual search experiments with and without background noise on an experimental subject to obtain brain imaging data of the experimental subject. Both the visual search experiments with and without background noise include at least one round of experimental groups, and each experimental group includes at least two sub-experiments. For each sub-experiment, based on the brain imaging data corresponding to the sub-experiment, determining the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli. Based on the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli, performing biological brain neuron activation mechanism analysis to obtain the sparse activation mechanism, hierarchical activation mechanism, and activation distribution law of visual cortex neurons. Based on at least one of the sparse activation mechanism, hierarchical activation mechanism, and activation distribution law, constructing a target neural network.

[0112] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program is implemented to perform the visual search-based neural network construction method provided by the above methods. The method includes: conducting visual search experiments with and without background noise on an experimental subject to obtain brain imaging data of the experimental subject. Both the visual search experiments with and without background noise include at least one round of experimental groups, and each experimental group includes at least two sub-experiments. For each sub-experiment, based on the brain imaging data corresponding to the sub-experiment, determining the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli. Based on the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli, performing biological brain neuron activation mechanism analysis to obtain the sparse activation mechanism, hierarchical activation mechanism, and activation distribution law of visual cortex neurons. Based on at least one of the sparse activation mechanism, the hierarchical activation mechanism, and the activation distribution law, constructing a target neural network.

[0113] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0114] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a neural network based on visual search, characterized in that, include: The experiment subjects were subjected to visual search experiments with and without background noise to obtain brain imaging data of the subjects. Both the visual search experiments with and without background noise included at least one round of experimental groups, and each experimental group included at least two sub-experiments. For each of the sub-experiments, based on the brain imaging data corresponding to the sub-experiment, the neurons contained in each visual cortex in the sub-experiment and the neurons in each visual cortex related to visual stimuli are determined. Based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, the activation mechanism of biological brain neurons is analyzed to obtain the sparse activation mechanism, hierarchical activation mechanism and activation distribution law of visual cortex neurons. The target neural network is constructed based on at least one of the sparse activation mechanism, the hierarchical activation mechanism, and the activation distribution law.

2. The method for constructing a neural network based on visual search according to claim 1, characterized in that, The step of determining the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli based on the brain imaging data corresponding to the sub-experiment includes: The brain imaging data corresponding to the sub-experiment is subjected to cell segmentation of biological brain neurons to extract activation trace data of the biological brain neurons, wherein the activation trace data represents the trace data of the activated biological brain neurons. Based on the activation trace data of the biological brain neurons, the neurons contained in each visual cortex in the sub-experiment and the neurons in each visual cortex related to visual stimuli were determined.

3. The method for constructing a neural network based on visual search according to claim 2, characterized in that, The biological brain neurons of the experimental subjects were fluorescently labeled with calcium ion indicators, and the brain imaging data included calcium ion imaging data. The step of segmenting the brain imaging data corresponding to the sub-experiment into biological brain neurons to extract activation trace data of the biological brain neurons includes: By using a supervised learning algorithm, cell location marking and identification are performed on the biological brain neurons in the calcium ion imaging data corresponding to the sub-experiment to obtain calcium transient traces; Calcium ion dynamics were estimated from the calcium transient traces to obtain activation trace data of the biological brain neurons.

4. The method for constructing a neural network based on visual search according to claim 2, characterized in that, The determination of the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli based on the activation trace data of the biological brain neurons in the sub-experiment includes: Correlation analysis and feature dimension reduction are performed on the activation trace data of the biological brain neurons to determine the neurons contained in each visual cortex and the neurons in each visual cortex that are related to visual stimuli.

5. The method for constructing a neural network based on visual search according to claim 1, characterized in that, Based on the neurons contained in each visual cortex in each of the sub-experiments and the neurons in each visual cortex related to visual stimuli, the activation mechanism of biological brain neurons is analyzed to obtain the sparse activation mechanism, hierarchical activation mechanism, and activation distribution pattern of visual cortex neurons, including: Based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex that are related to visual stimuli, a sparse activation analysis of neurons based on target stimulus relevance is performed to obtain the sparse activation mechanism. Based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, the neuronal activity pattern analysis of brain region functional hierarchy is performed to obtain the hierarchical activation mechanism. Based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex that are related to visual stimuli, the activation distribution pattern of neurons is analyzed based on the background noise control to obtain the activation distribution pattern.

6. The method for constructing a neural network based on visual search according to claim 5, characterized in that, The sparse activation mechanism is obtained by performing neuron activation analysis based on target stimulus relevance, based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex that are related to visual stimuli, including: Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each sub-experiment of the visual search experiment containing background noise, determine the first ratio of the number of neurons associated with visual stimuli to the number of neurons contained in each visual cortex when the stimulus image appears in all locations in all experimental groups included in the visual search experiment containing background noise. Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each of the sub-experiments included in the visual search experiment without background noise, determine a second ratio of the number of neurons associated with visual stimuli to the number of neurons contained in each visual cortex when the stimulus image appears at a single location in a single experimental group included in the visual search experiment with background noise. The sparse activation mechanism is determined based on the ratio of the second quantity ratio to the first quantity ratio.

7. The method for constructing a neural network based on visual search according to claim 5, characterized in that, The neuronal activity pattern analysis, based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, performs brain region functional hierarchy division, and obtains the hierarchical activation mechanism, including: Each of the aforementioned visual cortexes is divided into brain region functional levels to obtain at least one brain region functional level. Based on the neurons contained in each functional layer of the brain region and the neurons related to visual stimuli in each functional layer of the brain region in the sub-experiments of the visual search experiment containing background noise, determine the third ratio of the neurons related to visual stimuli to the neurons contained in each functional layer of the brain region when the stimulus image appears in all locations in all experimental groups of the visual search experiment containing background noise. Based on the neurons contained in each functional layer of the brain region and the neurons related to visual stimuli in each functional layer of the brain region in the sub-experiments of the visual search experiment without background noise, determine the fourth ratio of the neurons related to visual stimuli to the neurons contained in each functional layer of the brain region when the stimulus image appears in all locations in all experimental groups of the visual search experiment with background noise. The hierarchical activation mechanism is determined based on the third and fourth quantity ratios.

8. The method for constructing a neural network based on visual search according to claim 5, characterized in that, Based on the neurons contained in each visual cortex in each of the sub-experiments and the neurons in each visual cortex related to visual stimuli, an analysis of the neuronal activation distribution pattern based on the background noise control is performed to obtain the activation distribution pattern, including: Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each sub-experiment of the visual search experiment containing background noise, determine the fifth ratio of the neurons associated with visual stimuli to the neurons contained in each visual cortex when the stimulus image appears in all locations in all experimental groups included in the visual search experiment containing background noise. Based on the neurons contained in each visual cortex and the neurons in each visual cortex associated with visual stimuli in each sub-experiment of the visual search experiment without background noise, determine the sixth ratio of the neurons associated with visual stimuli to the neurons contained in each visual cortex when the stimulus image appears in all locations in all experimental groups of the visual search experiment with background noise. The activation distribution pattern is determined based on the fifth and sixth quantity ratios.

9. The method for constructing a neural network based on visual search according to any one of claims 1-8, characterized in that, Before determining the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli based on the brain imaging data corresponding to the sub-experiment, the method further includes: Non-rigid registration is performed on the brain imaging data to obtain registered brain imaging data; The step of determining the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli based on the brain imaging data corresponding to the sub-experiment includes: Based on the registered brain imaging data corresponding to the sub-experiment, the neurons contained in each visual cortex in the sub-experiment and the neurons in each visual cortex related to visual stimuli are determined.

10. A neural network construction system based on visual search, characterized in that, include: The visual search experiment module is configured to perform visual search experiments with and without background noise on the experimental subjects to obtain brain imaging data of the experimental subjects. Both the visual search experiments with and without background noise include at least one round of experimental groups, and each experimental group includes at least two sub-experiments. The determination module is configured to, for each of the sub-experiments, determine the neurons contained in each visual cortex and the neurons in each visual cortex related to visual stimuli, based on the brain imaging data corresponding to the sub-experiment. The analysis module is configured to perform biological brain neuron activation mechanism analysis based on the neurons contained in each visual cortex in each sub-experiment and the neurons in each visual cortex related to visual stimuli, to obtain the sparse activation mechanism, hierarchical activation mechanism and activation distribution law of visual cortex neurons. The building module is configured to build a target neural network based on at least one of the sparse activation mechanism, the hierarchical activation mechanism, and the activation distribution law.

Images