Whole cerebral cortex nerve photoelectric information processing method and device under memory mechanism

Abstract

The invention provides a whole cerebral cortex nerve photoelectric information processing method and device under a memory mechanism. The method comprises the following steps: collecting whole cerebral cortex calcium imaging signals and hippocampus sharp ripple signals generated by an experimental animal aiming at each group of visual stimulation sequences; under the same time step, phase-locked correlation analysis is carried out on the whole cerebral cortex calcium imaging signal and the hippocampal sharp ripple signal to obtain a neural activity association relationship, and the neural activity association relationship comprises a corresponding relationship between different visual stimuli and different signal combinations. The neural photoelectric information incidence relation between the whole cerebral cortex and the deep brain nuclei under the memory mechanism can be analyzed in the visual cognition prediction process, namely visual prediction is carried out by analyzing the whole cerebral cortex activity and the hippocampus sharp ripple discharge, and a basis is provided for constructing a visual prediction model based on the neural activity incidence relation; the attention mechanism about the neural activity association relationship is added for constructing the visual prediction model, so that the visual prediction accuracy of the visual prediction model can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of visual memory prediction, and in particular to a method and device for processing whole-cerebral cortex neural photoelectric information under a recall mechanism. Background Art

[0002] In the study of neural visual cognition mechanisms, visual cognition is not a simple process of information input and feedback, but rather a multi-level dynamic prediction process based on cortical-hippocampal circuits. This process not only involves specific neural responses in the primary and higher visual cortices but is also closely related to the memory reconstruction function of deep brain nuclei such as the hippocampus. Therefore, studying the correlation between photoelectric information from the entire cerebral cortex and deep brain nuclei during visual cognitive prediction is of great significance to the study of neural visual cognition mechanisms.

[0003] Therefore, finding a neural photoelectric information processing method that can analyze the neural photoelectric information correlation between the whole cerebral cortex and deep brain nuclei under the recall mechanism during visual cognitive prediction has become a research hotspot. Summary of the Invention

[0004] The present invention provides a method and device for processing neural photoelectric information of the whole cerebral cortex under the recall mechanism, which can analyze the neural photoelectric information correlation relationship between the whole cerebral cortex and deep brain nuclei under the recall mechanism during the visual cognitive prediction process, thereby laying the foundation for studying the neural visual cognitive mechanism.

[0005] The present invention provides a method for processing whole-brain cortical neural photoelectric information under a recall mechanism, the method comprising: collecting whole-brain cortical calcium imaging signals and hippocampal sharp wave ripple signals generated by experimental animals in response to each group of visual stimulation sequences, wherein the visual stimulation sequences are multiple groups of visual stimulations having a visual logical relationship; performing phase-locked correlation analysis on the whole-brain cortical calcium imaging signals and the hippocampal sharp wave ripple signals at the same time step to obtain a neural activity correlation relationship, wherein the neural activity correlation relationship includes a correspondence between different visual stimuli and different signal combinations, and the signal combination includes the whole-brain cortical calcium imaging signals and the hippocampal sharp wave ripple signals at the same time step.

[0006] According to a whole-brain cortex neural photoelectric information processing method under a recall mechanism provided by the present invention, at the same time step, the whole-brain cortex calcium imaging signal and the hippocampal sharp wave ripple signal are subjected to phase-locked correlation analysis to obtain a neural activity correlation relationship, specifically comprising: at the same time step, respectively extracting the calcium imaging signal features of the whole-brain cortex calcium imaging signal and extracting the ripple signal features of the hippocampal sharp wave ripple signal; performing phase-locked correlation analysis based on the calcium imaging signal features and the ripple signal features to obtain a neural activity correlation relationship.

[0007] According to a method for processing whole-brain cortical neural photoelectric information under a recall mechanism provided by the present invention, the calcium imaging signal characteristics include a first response amplitude of the calcium imaging signal; the ripple signal characteristics include a second response amplitude and a response frequency band of the hippocampal sharp wave ripple signal; the phase-locked correlation analysis based on the calcium imaging signal characteristics and the ripple signal characteristics is performed to obtain a neural activity correlation relationship, specifically including: obtaining the biomarker sharp wave ripple occurrence time based on the second response amplitude and response frequency band of the hippocampal sharp wave ripple signal; obtaining the first response amplitude of the calcium imaging signal at the biomarker sharp wave ripple occurrence time based on the biomarker sharp wave ripple occurrence time; obtaining the neural activity correlation relationship based on the first response amplitude of the calcium imaging signal at the biomarker sharp wave ripple occurrence time, and the second response amplitude and response frequency band of the hippocampal sharp wave ripple signal.

[0008] According to a method for processing whole-brain cortical neural photoelectric information under a recall mechanism provided by the present invention, after obtaining the neural activity association relationship, the method further includes: stimulating the experimental animal again based on the visual stimulation sequence, and collecting the current whole-brain cortical calcium imaging signal and the current hippocampal sharp wave ripple signal of the experimental animal; determining the current signal combination based on the current whole-brain cortical calcium imaging signal and the current hippocampal sharp wave ripple signal; obtaining the current predicted visual stimulation corresponding to the current signal combination based on the corresponding relationship in the neural activity association relationship and the current signal combination; obtaining the real current visual stimulation determined by the experimental animal based on the visual stimulation sequence; obtaining the target neural activity association relationship based on the current predicted visual stimulation, the real current visual stimulation, and the neural activity association relationship, and taking the target neural activity association relationship as the final neural activity association relationship.

[0009] According to a whole-brain cortex neural photoelectric information processing method under a recall mechanism provided by the present invention, the target neural activity association relationship is obtained based on the current predicted visual stimulus, the real current visual stimulus, and the neural activity association relationship, specifically including: when the difference between the current predicted visual stimulus and the real current visual stimulus is greater than a difference threshold, the neural activity association relationship is corrected based on the real current visual stimulus to obtain the target neural activity association relationship; when the difference between the current predicted visual stimulus and the real current visual stimulus is less than or equal to the difference threshold, the neural activity association relationship is used as the target neural activity association relationship.

[0010] According to a method for processing whole-brain cortex neural photoelectric information under a recall mechanism provided by the present invention, after obtaining the neural activity correlation relationship, the method further includes: analyzing the neural visual cognitive mechanism based on the neural activity correlation relationship.

[0011] According to a method for processing whole-brain cortical neural photoelectric information under a recall mechanism provided by the present invention, the hippocampal sharp wave ripple signal is obtained in the following manner: calling a micro-nano electrode array, implanting the micro-nano electrode array into the deep brain hippocampus area of the experimental animal, and obtaining the hippocampal sharp wave ripple signal, wherein the micro-nano electrode array is an implantable electrode array for collecting brain electrophysiological activity signals.

[0012] The present invention also provides a whole-brain cortex neural photoelectric information processing device under the recall mechanism, the device comprising: an acquisition module for acquiring whole-brain cortical calcium imaging signals and hippocampal sharp wave ripple signals generated by experimental animals in response to each group of visual stimulation sequences, wherein the visual stimulation sequences are multiple groups of visual stimulations with visual logical relationships; a processing module for performing phase-locked correlation analysis on the whole-brain cortical calcium imaging signals and the hippocampal sharp wave ripple signals at the same time step to obtain a neural activity correlation relationship, wherein the neural activity correlation relationship includes a correspondence between different visual stimuli and different signal combinations, and the signal combination includes whole-brain cortical calcium imaging signals and hippocampal sharp wave ripple signals at the same time step.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for processing whole-cerebral cortical neural photoelectric information under any of the above-described recall mechanisms is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for processing whole-cerebral cortical neural photoelectric information under any of the above-mentioned recall mechanisms is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the whole-cerebral cortex neural optoelectronic information processing method under any of the above-mentioned recall mechanisms.

[0016] The present invention provides a method and device for processing whole-brain cortical neural photoelectric information under a recall mechanism. The method comprises: collecting whole-brain cortical calcium imaging signals and hippocampal sharp wave ripple signals generated by experimental animals in response to each group of visual stimulation sequences, wherein the visual stimulation sequences are multiple groups of visual stimulations with visual logical relationships; performing phase-locked correlation analysis on the whole-brain cortical calcium imaging signals and the hippocampal sharp wave ripple signals at the same time step to obtain neural activity correlation relationships. This achieves the ability to analyze the neural photoelectric information correlation relationship between the whole-brain cortex and deep brain nuclei during visual cognitive prediction, that is, to perform visual prediction by analyzing whole-brain cortical activity and hippocampal sharp wave ripple discharges, thereby laying the foundation for studying neural visual cognitive mechanisms. It also provides a basis for constructing a visual prediction model based on the analyzed neural activity correlation relationship. By adding an attention mechanism on the neural activity correlation relationship to the construction of the visual prediction model, the accuracy of the visual prediction model in performing visual prediction can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is one of the flow charts of the whole-cerebral cortex neural photoelectric information processing method under the recall mechanism provided by the present invention.

[0019] Figure 2 The present invention provides a flow chart for performing phase-locked correlation analysis on whole-brain cortical calcium imaging signals and hippocampal sharp wave ripple signals at the same time step to obtain the correlation relationship of neural activities.

[0020] Figure 3 This is the second flow chart of the whole-brain cortex neural photoelectric information processing method under the recall mechanism provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of the whole-cerebral cortex neural photoelectric information processing device under the recall mechanism provided by the present invention.

[0022] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0024] The present invention provides a whole-cortical neural photoelectric information processing method under the recall mechanism. It combines visual cortical calcium signals (corresponding to whole-cortical calcium imaging signals) and neural oscillations (deep-brain electrophysiological activity signals) to extract and evaluate visual prediction coding features. In other words, it extracts the neural activity correlation formed by whole-cortical calcium imaging signals and hippocampal sharp wave ripple signals. This provides a new research approach for establishing visual prediction models based on real biological data, laying the foundation for the study of neural visual cognitive mechanisms. In one application scenario, it can provide a basis for constructing a visual prediction model based on the analyzed neural activity correlation. By adding an attention mechanism regarding the neural activity correlation to the construction of the visual prediction model, the accuracy of the visual prediction model in making visual predictions can be improved.

[0025] Figure 1 This is one of the flow charts of the whole-cerebral cortex neural photoelectric information processing method under the recall mechanism provided by the present invention.

[0026] The following will be combined Figure 1 The process of the whole-cerebral cortex neural photoelectric information processing method under the recall mechanism provided by the present invention is explained.

[0027] In an exemplary embodiment of the present invention, Figure 1 It can be seen that the whole-brain cortex neural photoelectric information processing method under the recall mechanism can include step 110 and step 120, and each step will be introduced below.

[0028] In step 110, whole-brain cortical calcium imaging signals and hippocampal sharp wave ripple signals generated by the experimental animal in response to each set of visual stimulation sequences are collected, wherein the visual stimulation sequences are multiple sets of visual stimulations with visual logical relationships; In step 120, at the same time step, phase-locked correlation analysis is performed on the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal to obtain a neural activity correlation relationship, wherein the neural activity correlation relationship includes the correspondence between different visual stimuli and different signal combinations, and the signal combination includes the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal at the same time step.

[0029] In one embodiment, a visual stimulation sequence can be output to an experimental animal, and when the experimental animal receives the visual stimulation sequence, the whole-brain cortical calcium imaging signal and hippocampal sharp wave ripple signal of the experimental animal are collected. The visual stimulation sequence is a plurality of groups of visual stimuli having a visual logical relationship. In one example, the visual stimulation sequence can be black and white stripes with continuously changing angles or a natural image with spatiotemporal continuity, for example, black and white stripes in different directions (0°→90°→180°→270°) can be presented in sequence, or it can be a dynamically changing moving light spot (acceleration→constant speed→deceleration). In this embodiment, the visual stimulation sequence is not specifically limited.

[0030] In another exemplary embodiment of the present invention, the hippocampal sharp wave ripple signal can be obtained in the following manner: The micro-nano electrode array is called and implanted into the deep brain hippocampus area of the experimental animal to obtain the hippocampal sharp wave ripple signal. The micro-nano electrode array is an implantable electrode array for collecting brain electrophysiological activity signals.

[0031] In one embodiment, the electrophysiological characteristic encoding of the visual circuit or learning and cognitive brain area in visual prediction activities can be obtained in real time based on an implantable neural micro-nano electrode array (corresponding to a micro-nano electrode array), that is, the hippocampal sharp wave ripple signal is obtained. It should be noted that since the hippocampal sharp wave ripple signal is a specific signal formed when the animal produces memories, the collection of the hippocampal sharp wave ripple signal can be used to analyze the correlation between the neural photoelectric information of the whole brain cortex and the deep brain nuclei under the recall mechanism, thereby laying the foundation for studying the neural visual cognitive mechanism.

[0032] In another embodiment, the implantable neural micro-nanoelectrode can have multiple detection sites and can be simultaneously implanted into multiple brain regions related to visual encoding, such as the lateral geniculate nucleus (LGN), superior colliculus (SC), and hippocampus (HC). The lateral geniculate nucleus (LGN) and superior colliculus (SC) are respectively upstream and downstream nuclei connected to the visual cortex; the hippocampus (HC) is a key nucleus for episodic memory. It is understood that deep brain regions include at least the lateral geniculate nucleus (LGN), superior colliculus (SC), and hippocampus (HC).

[0033] In another embodiment, whole-cortical calcium imaging signals can be acquired from experimental animals based on whole-cortical neuronal calcium activity imaging. During acquisition of whole-cortical calcium imaging signals from experimental animals, surgery can be performed on the experimental animals to remove a large area of skull or to make the skull transparent, thereby ensuring imaging of cortical neuronal calcium signals.

[0034] In another embodiment, the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal can be subjected to phase-locked correlation analysis at the same time step, so as to obtain a neural activity correlation relationship, wherein the neural activity correlation relationship includes the correspondence between different visual stimuli and different signal combinations. It should be noted that the corresponding visual stimulation and signal combination in the neural activity correlation relationship are obtained at the same time step, that is, they can be considered to be obtained at the same moment or within the same time period. Through this embodiment, the correlation relationship between the neural photoelectric information of the whole-brain cortex and the deep brain nuclei under the recall mechanism can be analyzed, thereby laying the foundation for studying the neural visual cognitive mechanism.

[0035] Figure 2 The present invention provides a flow chart for performing phase-locked correlation analysis on whole-brain cortical calcium imaging signals and hippocampal sharp wave ripple signals at the same time step to obtain the correlation relationship of neural activities.

[0036] The following will be combined Figure 2 The process of performing phase-locked correlation analysis on whole-brain cortical calcium imaging signals and hippocampal sharp wave ripple signals at the same time step to obtain the correlation relationship between neural activities is explained.

[0037] In an exemplary embodiment of the present invention, Figure 2 It can be seen that performing phase-locked correlation analysis on the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal at the same time step to obtain the neural activity correlation relationship may include steps 210 and 220. Each step will be described below.

[0038] In step 210, at the same time step, calcium imaging signal features of the whole-brain cortex calcium imaging signal and ripple signal features of the hippocampal sharp wave ripple signal are extracted respectively; In step 220, a phase-locked correlation analysis is performed based on the calcium imaging signal characteristics and the ripple signal characteristics to obtain a correlation relationship of neural activities.

[0039] In one embodiment, calcium imaging signal features of the whole-cortical calcium imaging signal and ripple signal features of the hippocampal sharp wave ripple signal can be extracted at the same time step. The calcium imaging signal features can include the response time and response amplitude of the whole-cortical calcium imaging signal (corresponding to the first response amplitude described below); the ripple signal features can include the response time, response frequency band, and response amplitude (corresponding to the second response amplitude described below).

[0040] Furthermore, phase-locked correlation analysis can be performed based on the calcium imaging signal characteristics and ripple signal characteristics to obtain the correlation relationship of neural activity.

[0041] In another embodiment, the response time and response amplitude of the neural cortical calcium signal can be correlated with different types of visual stimulation images (corresponding to different visual stimuli), and then based on the collected neural oscillation waveform (corresponding to the hippocampal sharp wave ripple signal), a phase-locked analysis is performed through the response time, response frequency band, response amplitude, and the response time and response amplitude of the neural cortical calcium signal, so that the correlation relationship of neural activity can be obtained.

[0042] In another exemplary embodiment of the present invention, the above-mentioned embodiment is continued as an example for description, wherein the calcium imaging signal feature may include a first response amplitude of the calcium imaging signal; the ripple signal feature may include a second response amplitude and a response frequency band of the hippocampal sharp wave ripple signal; Phase-locked correlation analysis based on calcium imaging signal characteristics and ripple signal characteristics can be performed to obtain the correlation relationship between neural activities. This can be achieved in the following ways: Based on the second response amplitude and response frequency band of the hippocampal sharp wave ripple signal, the appearance time of the biomarker sharp wave ripple is obtained; Based on the biomarker sharp wave ripple appearance time, obtaining the first response amplitude of the calcium imaging signal at the biomarker sharp wave ripple appearance time; Phase-locked correlation analysis was performed based on the first response amplitude of the calcium imaging signal at the moment when the biomarker sharp wave ripple appeared, and the second response amplitude and response frequency band of the hippocampal sharp wave ripple signal to obtain the correlation relationship of neural activity.

[0043] In one embodiment, the time of biomarker sharp wave ripple appearance can be determined based on the second response amplitude and response frequency of the hippocampal sharp wave ripple signal. It is understood that the time of biomarker sharp wave ripple appearance can be considered to correspond to the time of neuronal activity generated by the experimental animal during the visual memory prediction process, that is, the time corresponding to the animal's recall. Furthermore, based on the time of biomarker sharp wave ripple appearance, the first response amplitude of the calcium imaging signal at the time of biomarker sharp wave ripple appearance can be determined. Phase-locked correlation analysis can then be performed based on the first response amplitude of the calcium imaging signal at the time of biomarker sharp wave ripple appearance, as well as the second response amplitude and response frequency of the hippocampal sharp wave ripple signal, to determine a neural activity correlation. In other words, a neural activity correlation can be constructed based on the signal combination corresponding to the time of biomarker sharp wave ripple appearance and the visual stimulus. It is understood that in the neural activity correlation, the signal combination present is the signal combination corresponding to the time of biomarker sharp wave ripple appearance, and the visual stimulus is the visual stimulus corresponding to the time of biomarker sharp wave ripple appearance.

[0044] Figure 3 This is the second flow chart of the whole-brain cortex neural photoelectric information processing method under the recall mechanism provided by the present invention.

[0045] The following will be combined Figure 3 The process of the whole-cortical neural photoelectric information processing method under another recall mechanism is explained.

[0046] In an exemplary embodiment of the present invention, Figure 3 It can be seen that another method for processing whole-brain cortical neural optoelectronic information under a recall mechanism may include steps 310 to 350, and each step will be introduced below.

[0047] In step 310 , the experimental animal is stimulated again based on the visual stimulation sequence, and the current whole-brain cortex calcium imaging signal and the current hippocampal sharp wave ripple signal of the experimental animal are collected.

[0048] In step 320 , a current signal combination is determined based on the current whole-brain cortex calcium imaging signal and the current hippocampal sharp wave ripple signal.

[0049] In step 330 , based on the corresponding relationship in the neural activity association relationship and the current signal combination, a current predicted visual stimulus corresponding to the current signal combination is obtained.

[0050] In one embodiment, the experimental animal can be stimulated again based on the visual stimulation sequence, and the current whole-brain cortical calcium imaging signal and current hippocampal sharp wave ripple signal of the experimental animal can be collected. During application, the current whole-brain cortical calcium imaging signal and current hippocampal sharp wave ripple signal can be considered as the whole-brain cortical calcium imaging signal and hippocampal sharp wave ripple signal formed after the experimental animal receives the visual stimulation sequence again.

[0051] Furthermore, the current whole-brain cortical calcium imaging signal and the current hippocampal sharp wave ripple signal are used as the current signal combination, and based on the corresponding relationship in the neural activity association relationship and the current signal combination, the current predicted visual stimulus corresponding to the current signal combination is obtained. It can be understood that the current predicted visual stimulus is the visual stimulus about the current signal combination predicted based on the corresponding relationship in the neural activity association relationship. It can be understood that if the corresponding relationship in the neural activity association relationship is accurate, then the current predicted visual stimulus is close to the actual current visual stimulus.

[0052] In step 340 , the actual current visual stimulus determined by the experimental animal based on the visual stimulus sequence is obtained.

[0053] In step 350 , a target neural activity association relationship is obtained based on the current predicted visual stimulus, the actual current visual stimulus, and the neural activity association relationship, and the target neural activity association relationship is used as the final neural activity association relationship.

[0054] In another embodiment, the actual current visual stimulus corresponding to the current signal combination determined by the experimental animal based on the visual stimulus sequence can be obtained. Furthermore, the neural activity correlation relationship can be continuously corrected based on the current predicted visual stimulus and the actual current visual stimulus to obtain a more optimal target neural activity correlation relationship, and the target neural activity correlation relationship can be used as the final neural activity correlation relationship.

[0055] During the application process, the current visual stimulus can be predicted in real time (corresponding to the current predicted visual stimulus) based on the real-time changes in fluorescent calcium activity in multiple brain regions of the entire cortex and the synchronous and coordinated changes in characteristic neural oscillations (such as sharp wave ripples) of deep brain nuclei. By comparing it with the current real stimulus (corresponding to the real current visual stimulus), error feedback correction is continuously performed to form a closed-loop multimodal visual cognitive prediction system, that is, to obtain the target neural activity correlation, so that more rigorous neural visual cognitive mechanism research can be carried out based on the target neural activity correlation.

[0056] In another exemplary embodiment of the present invention, continuing with the above-mentioned embodiment as an example, based on the current predicted visual stimulus, the actual current visual stimulus, and the neural activity correlation relationship, the target neural activity correlation relationship is obtained, which can be achieved in the following manner: When the difference between the current predicted visual stimulus and the actual current visual stimulus is greater than a difference threshold, the neural activity correlation is corrected based on the actual current visual stimulus to obtain a target neural activity correlation; When the difference between the current predicted visual stimulus and the actual current visual stimulus is less than or equal to the difference threshold, the neural activity association relationship is used as the target neural activity association relationship.

[0057] In one embodiment, when the difference between the current predicted visual stimulus and the actual current visual stimulus is greater than a difference threshold, it indicates that there is an error in the neural activity correlation relationship, and thus the neural activity correlation relationship can be corrected based on the actual current visual stimulus, thereby obtaining a target neural activity correlation relationship. When the difference between the current predicted visual stimulus and the actual current visual stimulus is less than or equal to the difference threshold, it indicates that there is no error in the neural activity correlation relationship, and thus the neural activity correlation relationship can be directly used as the target neural activity correlation relationship. The difference threshold can be adjusted according to actual conditions, and the difference threshold is not specifically limited in this embodiment.

[0058] In another exemplary embodiment of the present invention, after obtaining the neural activity correlation relationship, the whole-cerebral cortical neural photoelectric information processing method under the recall mechanism may further include: Based on the correlation between neural activities, the neural visual cognitive mechanism is analyzed.

[0059] In one embodiment, the neural visual cognition mechanism can be analyzed based on the analyzed neural photoelectric information correlation between the whole brain cortex and deep brain nuclei under the recall mechanism, that is, the neural activity correlation, thereby laying the foundation for a deeper understanding of the process of neural visual cognition.

[0060] As described above, the whole-cortical neural photoelectric information processing method under the recall mechanism provided by this invention provides a technical means for obtaining neural activity correlations between real biological neural information in visual prediction experimental paradigms, thereby laying a foundation for studying the neural visual cognition mechanism. It also provides a basis for constructing visual prediction models based on the analyzed neural activity correlations. By adding an attention mechanism to the neural activity correlations to the construction of visual prediction models, the accuracy of visual prediction can be improved.

[0061] Figure 4 It is a structural schematic diagram of the whole-cerebral cortex neural photoelectric information processing device under the recall mechanism provided by the present invention.

[0062] The whole-brain cortex neural photoelectric information processing device under the recall mechanism provided by the present invention is described below. The whole-brain cortex neural photoelectric information processing device under the recall mechanism described below and the whole-brain cortex neural photoelectric information processing method under the recall mechanism described above can be referenced to each other.

[0063] In an exemplary embodiment of the present invention, Figure 4 It can be seen that the whole-brain cortex neural optoelectronic information processing device under the recall mechanism can include an acquisition module 410 and a processing module 420. Each module will be introduced below.

[0064] The acquisition module 410 may be configured to acquire whole-brain cortical calcium imaging signals and hippocampal sharp wave ripple signals generated by the experimental animal in response to each set of visual stimulus sequences, wherein the visual stimulus sequences are multiple sets of visual stimuli having a visual logical relationship; The processing module 420 can be configured to perform phase-locked correlation analysis on the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal at the same time step to obtain a neural activity correlation relationship, wherein the neural activity correlation relationship includes a correspondence between different visual stimuli and different signal combinations, and the signal combination includes the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal at the same time step.

[0065] In another exemplary embodiment of the present invention, the processing module 420 may perform phase-locked correlation analysis on the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal at the same time step to obtain a neural activity correlation relationship in the following manner: At the same time step, respectively extracting calcium imaging signal features of the whole-brain cortex calcium imaging signal and extracting ripple signal features of the hippocampal sharp wave ripple signal; Phase-locked correlation analysis was performed based on the calcium imaging signal characteristics and ripple signal characteristics to obtain the correlation between neural activities.

[0066] In another exemplary embodiment of the present invention, the calcium imaging signal feature includes a first response amplitude of the calcium imaging signal; the ripple signal feature includes a second response amplitude and a response frequency band of the hippocampal sharp wave ripple signal; the processing module 420 can perform phase-locked correlation analysis based on the calcium imaging signal feature and the ripple signal feature to obtain a neural activity correlation relationship in the following manner: Obtaining a biomarker sharp wave ripple appearance time based on a second response amplitude and a response frequency band of the hippocampal sharp wave ripple signal; Based on the biomarker sharp wave ripple appearance time, obtaining the first response amplitude of the calcium imaging signal at the biomarker sharp wave ripple appearance time; Based on the first response amplitude of the calcium imaging signal at the moment when the biomarker sharp wave ripple appears, and the second response amplitude and response frequency band of the hippocampal sharp wave ripple signal, a neural activity correlation relationship is obtained.

[0067] In yet another exemplary embodiment of the present invention, the processing module 420 may be further configured to: performing stimulation processing on the experimental animal again based on the visual stimulation sequence, and collecting the current whole-brain cortex calcium imaging signal and the current hippocampal sharp wave ripple signal of the experimental animal; Determining a current signal combination based on the current whole-brain cortex calcium imaging signal and the current hippocampal sharp wave ripple signal; Based on the corresponding relationship in the neural activity association relationship and the current signal combination, obtaining a current predicted visual stimulus corresponding to the current signal combination; obtaining a real current visual stimulus determined by the experimental animal based on the visual stimulus sequence; Based on the current predicted visual stimulus, the real current visual stimulus, and the neural activity association relationship, a target neural activity association relationship is obtained, and the target neural activity association relationship is used as the final neural activity association relationship.

[0068] In another exemplary embodiment of the present invention, the processing module 420 may obtain the target neural activity association relationship based on the current predicted visual stimulus, the actual current visual stimulus, and the neural activity association relationship in the following manner: When the difference between the current predicted visual stimulus and the real current visual stimulus is greater than a difference threshold, performing correction processing on the neural activity correlation relationship based on the real current visual stimulus to obtain the target neural activity correlation relationship; In a case where the difference between the current predicted visual stimulus and the real current visual stimulus is less than or equal to a difference threshold, the neural activity association relationship is used as the target neural activity association relationship.

[0069] In an exemplary embodiment of the present invention, the processing module 420 may also be configured to: Based on the neural activity correlation, the neural visual cognition mechanism is analyzed.

[0070] In an exemplary embodiment of the present invention, the acquisition module 410 may acquire the hippocampal sharp wave ripple signal in the following manner: A micro-nano electrode array is called and implanted into the deep brain hippocampus region of the experimental animal to obtain the hippocampal sharp wave ripple signal, wherein the micro-nano electrode array is an implantable electrode array for collecting brain electrophysiological activity signals.

[0071] Figure 5 An example of a physical structure diagram of an electronic device is shown below. Figure 5 As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other via the communications bus 540. The processor 510 may call logic instructions in the memory 530 to execute a whole-brain cortical neural photoelectric information processing method under a recall mechanism, the method comprising: collecting whole-brain cortical calcium imaging signals and hippocampal sharp wave ripple signals generated by an experimental animal in response to each set of visual stimulus sequences, wherein the visual stimulus sequences are multiple sets of visual stimuli having a visual logical relationship; performing phase-locked correlation analysis on the whole-brain cortical calcium imaging signals and the hippocampal sharp wave ripple signals at the same time step to obtain a neural activity correlation relationship, wherein the neural activity correlation relationship includes a correspondence between different visual stimuli and different signal combinations, wherein the signal combination includes the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal at the same time step.

[0072] Furthermore, the logic instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0073] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the whole-brain cortex neural photoelectric information processing method under the recall mechanism provided by the above-mentioned methods, the method including: collecting the whole-brain cortical calcium imaging signal and hippocampal sharp wave ripple signal generated by the experimental animal for each group of visual stimulation sequences, wherein the visual stimulation sequence is a plurality of groups of visual stimulations with a visual logical relationship; at the same time step, performing phase-locked correlation analysis on the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal to obtain a neural activity correlation relationship, wherein the neural activity correlation relationship includes a correspondence between different visual stimuli and different signal combinations, and the signal combination includes the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal at the same time step.

[0074] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the whole-brain cortex neural photoelectric information processing method under the recall mechanism provided by the above-mentioned methods, the method comprising: collecting the whole-brain cortex calcium imaging signal and hippocampal sharp wave ripple signal generated by the experimental animal for each group of visual stimulation sequences, wherein the visual stimulation sequence is a plurality of groups of visual stimulations having a visual logical relationship; at the same time step, performing phase-locked correlation analysis on the whole-brain cortex calcium imaging signal and the hippocampal sharp wave ripple signal to obtain a neural activity correlation relationship, wherein the neural activity correlation relationship includes a correspondence between different visual stimuli and different signal combinations, and the signal combination includes the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal at the same time step.

[0075] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0076] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for processing whole-brain cortical neural photoelectric information under a recall mechanism, characterized in that: The method comprises: collecting whole-brain cortical calcium imaging signals and hippocampal sharp wave ripple signals generated by the experimental animals in response to each set of visual stimulation sequences, wherein the visual stimulation sequences are multiple sets of visual stimulations with visual logical relationships; At the same time step, phase-locked correlation analysis is performed on the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal to obtain a neural activity correlation relationship, wherein the neural activity correlation relationship includes the correspondence between different visual stimuli and different signal combinations, and the signal combination includes the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal at the same time step.

2. The whole-brain cortex neural photoelectric information processing method under the recall mechanism according to claim 1 is characterized in that: The phase-locked correlation analysis is performed on the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal at the same time step to obtain a neural activity correlation relationship, specifically including: At the same time step, respectively extracting calcium imaging signal features of the whole-brain cortex calcium imaging signal and extracting ripple signal features of the hippocampal sharp wave ripple signal; Phase-locked correlation analysis is performed based on the calcium imaging signal characteristics and the ripple signal characteristics to obtain a correlation relationship between neural activities.

3. The whole-brain cortex neural photoelectric information processing method under the recall mechanism according to claim 2 is characterized in that: The calcium imaging signal feature includes a first response amplitude of the calcium imaging signal; the ripple signal feature includes a second response amplitude and a response frequency band of the hippocampal sharp wave ripple signal; and performing phase-locked correlation analysis based on the calcium imaging signal feature and the ripple signal feature to obtain a neural activity correlation relationship specifically includes: Obtaining a biomarker sharp wave ripple appearance time based on a second response amplitude and a response frequency band of the hippocampal sharp wave ripple signal; Based on the biomarker sharp wave ripple appearance time, obtaining the first response amplitude of the calcium imaging signal at the biomarker sharp wave ripple appearance time; Based on the first response amplitude of the calcium imaging signal at the moment when the biomarker sharp wave ripple appears, and the second response amplitude and response frequency band of the hippocampal sharp wave ripple signal, a neural activity correlation relationship is obtained.

4. The whole-brain cortex neural photoelectric information processing method under the recall mechanism according to claim 1 is characterized in that: After obtaining the neural activity correlation relationship, the method further includes: performing stimulation processing on the experimental animal again based on the visual stimulation sequence, and collecting the current whole-brain cortex calcium imaging signal and the current hippocampal sharp wave ripple signal of the experimental animal; Determining a current signal combination based on the current whole-brain cortex calcium imaging signal and the current hippocampal sharp wave ripple signal; Based on the corresponding relationship in the neural activity association relationship and the current signal combination, obtaining a current predicted visual stimulus corresponding to the current signal combination; obtaining a real current visual stimulus determined by the experimental animal based on the visual stimulus sequence; Based on the current predicted visual stimulus, the real current visual stimulus, and the neural activity association relationship, a target neural activity association relationship is obtained, and the target neural activity association relationship is used as the final neural activity association relationship.

5. The whole-brain cortex neural photoelectric information processing method under the recall mechanism according to claim 4 is characterized in that: The step of obtaining a target neural activity association relationship based on the current predicted visual stimulus, the real current visual stimulus, and the neural activity association relationship specifically includes: When the difference between the current predicted visual stimulus and the real current visual stimulus is greater than a difference threshold, performing correction processing on the neural activity correlation relationship based on the real current visual stimulus to obtain the target neural activity correlation relationship; In a case where the difference between the current predicted visual stimulus and the real current visual stimulus is less than or equal to a difference threshold, the neural activity association relationship is used as the target neural activity association relationship.

6. The method for processing whole-brain cortical neural photoelectric information under the recall mechanism according to any one of claims 1 to 5, characterized in that: After obtaining the neural activity correlation relationship, the method further includes: Based on the neural activity correlation, the neural visual cognition mechanism is analyzed.

7. The method for processing whole-brain cortical neural photoelectric information under the recall mechanism according to any one of claims 1 to 5, characterized in that: The hippocampal sharp wave ripple signal is obtained in the following manner: A micro-nano electrode array is called and implanted into the deep brain hippocampus region of the experimental animal to obtain the hippocampal sharp wave ripple signal, wherein the micro-nano electrode array is an implantable electrode array for collecting brain electrophysiological activity signals.

8. A whole-brain cortex neural optoelectronic information processing device under the recall mechanism, characterized in that: The device comprises: an acquisition module, configured to acquire whole-brain cortical calcium imaging signals and hippocampal sharp wave ripple signals generated by the experimental animals in response to each set of visual stimulation sequences, wherein the visual stimulation sequences are multiple sets of visual stimulations having a visual logical relationship; A processing module is used to perform phase-locked correlation analysis on the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal at the same time step to obtain a neural activity correlation relationship, wherein the neural activity correlation relationship includes a correspondence between different visual stimuli and different signal combinations, and the signal combination includes the whole-brain cortical calcium imaging signal and the hippocampal sharp wave ripple signal at the same time step.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the whole-cerebral cortical neural photoelectric information processing method under the recall mechanism as described in any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the whole-cerebral cortical neural photoelectric information processing method under the recall mechanism as described in any one of claims 1 to 7 is implemented.

Images