Single photon imaging data and physiological behavior data analysis method and system

Abstract

The invention provides a single-photon imaging data and physiological behavior data analysis method and system, and the system comprises a single-photon imaging module which is used for obtaining a fluorescence signal excited by the brain of a target object based on a single-photon detector, and converting the fluorescence signal into single-photon imaging data; the behavior monitoring module is used for acquiring behavior data of the target object; the synchronous control module is used for realizing time synchronization of the single-photon imaging data and the behavior data; and the data processing and storage module is used for processing and storing the single-photon imaging data and the behavior data in real time.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and in particular, to a method and system for analyzing single - photon imaging data and physiological behavior data. Background Art

[0002] With the development of biomedical imaging technology, single - photon imaging technology has received extensive attention due to its high sensitivity and high spatial resolution. This technology can capture fluorescence signals in living organisms at the microscopic level, providing structural and functional information of cells and tissues. However, single - photon imaging is usually static and lacks real - time monitoring of the dynamic behavior of organisms, which limits its application in neuroscience and behavioral research.

[0003] In the field of physiological behavior research, the progress of motion monitoring technology enables researchers to obtain the motion state and behavior patterns of individuals in real time. These technologies usually rely on motion sensors and video monitoring systems, which can provide rich behavioral data. However, existing methods for collecting physiological behavior data are often independent of the acquisition of imaging data, making it difficult to achieve temporal synchronization during data analysis, and thus affecting the in - depth understanding of the relationship between biological behavior and physiological responses.

[0004] In addition, existing single - photon imaging and behavior monitoring systems lack effective data processing and storage mechanisms, making it difficult to achieve real - time analysis and storage of large amounts of data, which restricts the utilization efficiency of data and the in - depth of research.

[0005] Therefore, there is an urgent need for a new method that can effectively combine single - photon imaging data and physiological behavior data, and through temporal synchronization and real - time processing, improve the understanding of the internal mechanisms of organisms. This method can not only improve the analysis efficiency of data, but also provide a more comprehensive perspective for research in fields such as neuroscience and psychology.

[0006] The present invention aims to solve the above problems and provides a method for analyzing single - photon imaging data and physiological behavior data, which realizes real - time synchronization, processing, and storage of data through modular design, thereby promoting the research progress in related fields. Summary of the Invention

[0007] Aiming at the deficiencies in the prior art, the purpose of the present invention is to provide a system and method for analyzing single - photon imaging data and physiological behavior data.

[0008] According to a system for analyzing single - photon imaging data and physiological behavior data provided by the present invention, it includes:

[0009] A single - photon imaging module, configured to obtain fluorescence signals excited in the brain of a target object based on a single - photon detector, and convert the fluorescence signals into single - photon imaging data;

[0010] The behavior monitoring module is used to obtain the behavior data of the target object;

[0011] The synchronization control module is used to achieve the time synchronization of the single-photon imaging data and the behavior data;

[0012] The data processing and storage module is used to perform real-time processing and storage on the single-photon imaging data and the behavior data.

[0013] Preferably, the single-photon imaging module includes:

[0014] Module M1.1: It is used to obtain the fluorescence signal excited by the brain of the target object based on a single-photon detector and convert the obtained fluorescence signal into a visual image;

[0015] Module M1.2: It is used to preprocess the visual image, including: noise reduction and contrast enhancement processing.

[0016] Preferably, the behavior monitoring module includes:

[0017] Module M2.1: It is used to obtain the motion state of the target object in real time based on a motion sensor, including: the moving speed, direction and position information of the target object;

[0018] Module M2.2: It is used to obtain the video data containing the target object based on a camera;

[0019] Module M2.3: It is used to extract behavior feature data from the video data through deep learning and computer vision technologies and identify the behavior pattern based on the behavior feature data.

[0020] Preferably, the module M2.3 includes: obtaining the behavior data and the background environment data through the SlowFast video behavior feature extraction algorithm, and performing feature extraction respectively; or based on the video data through a 3D convolutional network, after 3D convolutional operations, outputting a 3D feature map of K*l*h*w to extract the behavior features in the video.

[0021] Preferably, the module M2.3 includes:

[0022] The extracted behavior features are input into a machine learning model, including a convolutional neural network CNN or a recurrent neural network RNN, and the machine learning model is used to identify different behavior patterns.

[0023] Preferably, the synchronization control module includes: achieving the time synchronization of the single-photon imaging data and the behavior data through a clock synchronization unit and a data synchronization interface.

[0024] Preferably, the clock synchronization unit provides a unified time reference through clock synchronization;

[0025] The data synchronization interface aligns the single-photon imaging data and the behavior data in time to ensure data synchronization.

[0026] Preferably, the data processing and storage module includes a processor, a memory, and a data transmission interface;

[0027] The processor is used to process the single-photon imaging data and the behavior data in real time;

[0028] The memory is used to store the original data and the processed analysis results;

[0029] The data transmission interface is used to implement the data export function based on multiple data transmission protocols.

[0030] A method for analyzing single-photon imaging data and physiological behavior data provided by the present invention includes:

[0031] Step S1: Using the single-photon imaging module to obtain the fluorescence signal excited by the brain of the target object based on a single-photon detector, and converting the fluorescence signal into single-photon imaging data;

[0032] Step S2: Using the behavior monitoring module to obtain the behavior data of the target object;

[0033] Step S3: Using the synchronization control module to achieve the time synchronization of the single-photon imaging data and the behavior data;

[0034] Step S4: Using the data processing and storage module to perform real-time processing and storage on the single-photon imaging data and the behavior data.

[0035] Preferably, the step S1 includes:

[0036] Step S1.1: Used to obtain the fluorescence signal excited by the brain of the target object based on a single-photon detector, and convert the obtained fluorescence signal into a visual image;

[0037] Step S1.2: Used to preprocess the visual image, including: noise reduction and contrast enhancement processing;

[0038] The step S2 includes:

[0039] Step S2.1: Based on the motion sensor, real-time obtain the motion state of the target object, including: the moving speed, direction, and position information of the target object;

[0040] Step S2.2: Based on the camera, obtain the video data containing the target object;

[0041] Step S2.3: Based on the video data, extract the behavior feature data through deep learning and computer vision technologies, and identify the behavior pattern based on the behavior feature data;

[0042] Step S3 includes: realizing the time synchronization of single-photon imaging data and behavior data through a clock synchronization unit and a data synchronization interface.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. The present invention can provide detailed physiological behavior data to help researchers better understand the behavior patterns of organisms;

[0045] 2. The present invention has a faster signal processing ability in single-photon in-vivo monitoring, thus achieving stronger real-time performance;

[0046] 3. In the real-time monitoring of physiological behaviors, the present invention integrates more advanced sensors and data transmission technologies. For example, when monitoring the movement behaviors of animals, it can transmit the physiological signals acquired by the sensors to the analysis system in real time and quickly process and give feedback;

[0047] 4. The present invention obtains physiological behavior information through non-contact sensors, making the monitoring results more capable of reflecting the physiological behavior characteristics in the natural state. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] By reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings, other features, objects, and advantages of the present invention will become more apparent:

[0049] Figure 1 is the system working process.

[0050] Figure 2 is the behavior control research and synchronization mode diagram.

[0051] Figure 3 is the schematic diagram of the structure of the head-mounted microscope.

[0052] Figure 4 is the schematic diagram of the system framework.

[0053] Figure 5 is the structure diagram of the data collector.

[0054] Figure 6 is the schematic diagram of data transmission and connection.

[0055] Figure 7 is the data processing architecture.

[0056] Figure 8 is the lower computer data diagram.

[0057] Figure 9 is the upper computer data diagram.

[0058] Figure 10 is the schematic diagram of actual signal acquisition. Detailed implementation manners

[0059] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that those of ordinary skill in the art can make several changes and improvements without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0060] Embodiment 1

[0061] A method and system for analyzing single-photon imaging data and physiological behavior data provided by the present invention include:

[0062] High-sensitivity single-photon imaging technology: Using high-sensitivity single-photon detectors and miniaturized optical path units, high-precision single-photon imaging can be achieved under free movement conditions;

[0063] Real-time behavior monitoring: Including a behavior data analysis method based on deep learning and a high-definition camera, which can record the behavior activities of the experimental object in real time and provide refined behavior data analysis.

[0064] High-precision synchronous control: Equipped with a synchronous control module, through the clock synchronization unit and data synchronization interface, high-precision time synchronization of single-photon imaging data and behavior data is achieved to ensure the time consistency of the data.

[0065] Data processing and storage: Integrating a high-performance processor and a large-capacity memory, it can process and store single-photon imaging data and behavior data in real time, and support further analysis and research of the data.

[0066] Miniaturized design: The system components are miniaturized, which is convenient for carrying and operation, suitable for small animals or other experimental objects in free movement, and expands the application scope under natural behavior conditions.

[0067] Multifunctional expandability: The system can be expanded for monitoring other types of physiological signals, such as electrocardiogram, electroencephalogram, etc., and has strong adaptability and flexibility.

[0068] In this embodiment, through precise single-photon imaging technology and real-time behavior monitoring technology, high-precision synchronous acquisition and refined analysis of single-photon imaging and physiological behavior data under in vivo monitoring conditions are achieved; the miniaturized design and multifunctional expandability make it have a wide application prospect in biomedical research.

[0069] The single-photon imaging data and physiological behavior data analysis system includes:

[0070] A single-photon imaging module is used to transmit a laser beam to the brain of an experimental subject through a micro-optical fiber, excite specific fluorescent molecules, and the generated single-photon fluorescence signal is transmitted back to the detector through an optical prism for real-time imaging;

[0071] A behavior monitoring module is used to record the behavioral activities of an experimental subject through a camera and motion sensors;

[0072] A synchronization control module is used to ensure the time consistency of single-photon imaging data and behavioral data;

[0073] A data processing and storage module is used to process, analyze, and store all data.

[0074] In this embodiment, the single-photon imaging module includes: a laser source, an optical system, a single-photon detector, and an imaging processing circuit; the laser source transmits the laser to the brain of the experimental subject through an optical fiber bundle; the fluorescence signal is returned through the optical system, transmitted back through the optical fiber bundle, and received by the single-photon detector; the single-photon detector transmits the photon signal to the imaging processing circuit for image processing.

[0075] More specifically, the laser source: emits a laser with a specific wavelength for exciting the fluorescent molecules in the brain of the experimental subject. These fluorescent molecules will emit light during neuronal activity, reflecting the dynamic activity of neurons.

[0076] The optical system: transmits the laser to the brain of the experimental subject and transmits the returned fluorescence signal back to the detector. The miniaturized design of the optical system ensures the comfort and flexibility of the experimental subject in a free movement state.

[0077] The single-photon detector: a high-sensitivity single-photon detector is used to detect the fluorescence signal returned from the brain of the experimental subject. The detector can identify and count individual photons to ensure high-precision imaging effects.

[0078] The imaging processing circuit: converts the detected photon signal into a visual image and performs preliminary processing, such as noise reduction and contrast enhancement, for subsequent data analysis.

[0079] In this embodiment, the behavior monitoring module includes: motion sensors, a high-definition camera, and behavior analysis software; the motion sensors are fixed on the experimental subject or in the experimental environment to record motion data in real time; the high-definition camera is installed in the experimental environment to cover the activity area of the experimental subject and capture its behavior video. The captured video data is transmitted to the behavior analysis software for behavior feature analysis.

[0080] More specifically, based on the motion sensors, the motion state of the experimental subject is monitored in real time, and information such as its moving speed, direction, and position is recorded. These data can reflect the behavioral activities and physiological states of the experimental subject;

[0081] Synchronously record the behavioral activities of the experimental subjects based on high-definition cameras; the video data captured by the cameras provides rich behavioral feature information, which can be used to analyze in detail the behavioral patterns of the experimental subjects in different environments;

[0082] Analyze the video data captured by the cameras based on behavioral analysis software to extract behavioral feature data. The behavioral analysis software can identify specific behavioral patterns, such as exploration, eating, resting, etc.

[0083] In this embodiment, feature extraction based on deep learning and computer vision technologies: In a video, the features of animal behavior usually include aspects such as posture, action, and position. Through technical means such as deep learning and computer vision, these behavioral features can be extracted from the video. For example, the SlowFast video behavioral feature extraction algorithm will respectively obtain high-frequency (capturing behavior) and low-frequency image (capturing background environment) data, then perform feature extraction respectively, and finally perform operations such as feature fusion and prediction.

[0084] The C3D network (3D convolutional network) is also an effective network for video feature extraction. Since 2D convolution cannot capture temporal information well, 3D convolution was proposed and used in fields such as behavior recognition. The C3D network takes a video segment as input (with size c*l*h*w, where c is the number of image channels, l is the length of the video sequence, and h and w are the width and height of the video respectively). After 3D convolutional operations (such as 3D convolution with kernel size of 3*3*3, stride of 1, padding = True, and the number of filters being K), the output is a 3D feature map of K*l*h*w, which can effectively extract the behavioral features in the video.

[0085] Use machine learning models for pattern recognition and classification

[0086] The extracted behavioral features will be input into machine learning models, such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), etc. These models can identify different behavioral patterns, such as walking, running, playing ball, etc. Taking the SlowFast algorithm as an example, after extracting the key action features (containing temporal and spatial information), the yolov3 model can receive these features, perform object detection to determine whether there are people in the frame and their positions, and finally classify the actions performed by humans or other objects in the video through operations such as the combination of slow and fast paths.

[0087] For C3D network, in terms of behavior recognition, when it is combined with support vector machine (SVM) (C3D+SVM), behavior patterns can be recognized. For example, in the test on UCF101 database, the result of C3D+SVM reached 85.2% (test result in 2015). Of course, with the development of technology, the recognition accuracy has been further improved.

[0088] In a machine learning model, the model builds classification rules by learning a large amount of labeled behavioral feature data (training data). For example, for a task of identifying walking and running behaviors, the model will learn the commonalities in the walking behavior feature data (such as the value range of features such as step frequency and body sway amplitude, etc.) and the commonalities in the running behavior feature data (such as faster step frequency, greater body undulations, etc.). When new behavioral feature data is input, the model classifies it into the corresponding behavioral pattern according to the learned classification rules. At the same time, in order to improve the accuracy of the model, the model parameters will be continuously optimized, such as by adjusting the weights in the neural network, etc., to adapt to the behavioral pattern recognition needs in different scenarios.

[0089] In this embodiment, the synchronization control module includes: a clock synchronization unit and a data synchronization interface; the clock synchronization unit is connected to the single-photon imaging module and the behavior monitoring module to provide a unified time reference. The data synchronization interface is connected to the data output end of each module to time-align the single-photon imaging data with the behavior data to ensure data synchronization.

[0090] More specifically, the clock synchronization unit ensures the time consistency of single-photon imaging data and behavioral data. Through precise clock synchronization, the synchronization of data acquisition in each module is guaranteed, so that data from different sources can be correlated and analyzed.

[0091] The data synchronization interface is responsible for synchronizing the data from the single-photon imaging module and the behavior monitoring module. This interface aligns the data with different time tags to ensure that the time tags in the data processing and storage modules are consistent.

[0092] In this embodiment, the data processing and storage module includes: a high-performance processor, a large-capacity memory, and a data transmission interface; the high-performance processor is connected to the single-photon imaging module and the behavior monitoring module to receive and process data in real time. The processed data is stored in the large-capacity memory to ensure data security and integrity. The data transmission interface is connected to an external device or storage medium to support data export and further analysis.

[0093] More specifically, the high-performance processor: processes single-photon imaging data and behavioral data in real time. The processor runs complex data processing algorithms, including image processing, data fusion, statistical analysis, etc., to ensure the accuracy and integrity of the data.

[0094] The large-capacity memory: is used to store the original data and the processed analysis results. The memory has sufficient capacity to handle the storage requirements of large-scale data and supports fast read and write operations of the data.

[0095] The data transmission interface: provides a data export function, supporting the export of the processed data to external devices or storage media for further analysis and research. This interface is compatible with multiple data transmission protocols to ensure the security and integrity of the data.

[0096] A method for analyzing single-photon imaging data and physiological behavior data provided by the present invention includes:

[0097] Experiment preparation: Install the optical prism of the single-photon imaging module and the motion sensor and camera of the behavior monitoring module on the experimental subject. Adjust the position of the device to ensure that the laser can accurately irradiate the target brain area and the camera's field of view covers the activity area of the experimental subject.

[0098] Data acquisition: Start the laser source and single-photon detector to start single-photon imaging. At the same time, the motion sensor and camera start recording the behavioral activities of the experimental subject. The clock synchronization unit ensures the time consistency of the single-photon imaging data and behavioral data.

[0099] Data processing: The high-performance processor receives and processes the single-photon imaging data and behavioral data in real time. The imaging processing circuit performs image processing on the single-photon signal, and the behavior analysis software extracts behavioral features from the video data.

[0100] Data synchronization: The data synchronization interface aligns the single-photon imaging data and behavioral data in time to ensure that the time tags of all data are consistent.

[0101] Data storage: The processed single-photon imaging data and behavioral data are stored in the large-capacity memory. Both the original data and the processed analysis results are saved for subsequent analysis and research.

[0102] Data analysis and export: Researchers can export the data to external devices or storage media through the data transmission interface for further analysis and research. Through detailed data analysis, the neural activity laws and physiological behavior patterns of the experimental subject under natural behavior conditions can be revealed.

[0103] Embodiment 2

[0104] Embodiment 2 is a preferred example of Embodiment 1

[0105] A method for analyzing single-photon imaging data and physiological behavior data provided by the present invention is as follows Figures 1 to 10 as shown

[0106] Step 101: In order to achieve high-precision calcium imaging in in-vivo animals, a calcium indicator virus (such as GCaMP) is introduced into the neurons of the target brain region. This operation is usually completed by virus-mediated gene transduction technology. The following equipment is required: calcium indicator virus (such as AAV-GCaMP), surgical instruments (microsurgical tools, syringes, etc.), anesthetic drugs and injection equipment, disinfection equipment and materials, microscopes and operating microscopes, temperature control equipment, suture materials, and other equipment for assistance.

[0107] Step 102: In order to achieve high-precision calcium imaging of deep / superficial brain tissues in in-vivo animals, different implanted lens bodies need to be selected, and appropriate focusing prisms or glass slides need to be selected to obtain clear imaging results after the operation. After the injection of the AAV vector, the expression of the calcium indicator usually takes about 3 weeks. During this period, the calcium indicator will be fully expressed in the target neurons, preparing for the calcium imaging experiment; the focusing prism or glass slide is accurately placed in the target brain region, and its position and angle are ensured to be accurate through micromanipulation. The focusing prism or glass slide is fixed on the skull using glass ionomer cement to ensure its stability. During the operation, accurately arranging and fixing the focusing prism or glass slide is a key step for successful calcium imaging. The 3-week expression cycle after the injection of the AAV vector ensures the full expression of the calcium indicator in the neurons, providing a guarantee for subsequent imaging experiments.

[0108] Step 103: In order to perform calcium imaging observations under long-term conditions, we need to fix the Headstage substrate on the experimental animal to ensure the stability of the calcium indicator expression and the continuity of the imaging data. A high-quality Headstage substrate is used to connect the imaging system and the recording device. In the weeks after the substrate is fixed (usually 3 weeks), observe the expression of the calcium indicator (such as GCaMP). Use a microscope for preliminary imaging to confirm that the calcium indicator has been fully expressed in the neurons of the target brain region. Conduct pre-acquisition before the formal experiment, record the preliminary calcium signal data to calibrate the imaging system and confirm the data quality. The pre-acquired data can help determine the optimal settings of the imaging parameters and behavior synchronization. Synchronously record the behavior data of the experimental animal to ensure that the calcium signal can be correlated with the behavioral characteristics for analysis. Process and analyze the collected calcium signal data to study the relationship between neuronal activity and behavior. Adjust the imaging parameters and experimental design as needed to obtain more accurate and detailed research results.

[0109] Step 104: By determining appropriate behavioral paradigms, wearing a head-mounted microscope to collect neuronal signals in specific behavioral environments, and using a multi-camera system for behavioral monitoring, high-precision neuroscience research can be achieved. These steps help to reveal the relationship between neuronal activity and behavior, providing important experimental data for understanding brain function.

[0110] Select appropriate behavioral paradigms: Determine the main research objectives, such as exploring learning and memory, reward systems, motor control, etc. Select appropriate behavioral paradigms, such as the water maze experiment, treadmill experiment, free movement experiment, etc., to study the relationship between neuronal activity and specific behaviors. Before the experiment begins, conduct behavioral training on the experimental animals to make them familiar with the experimental environment and task requirements. Ensure that the animals exhibit stable and reproducible behavioral patterns in the behavioral paradigm.

[0111] Head-mounted microscope preparation: Select a lightweight head-mounted microscope suitable for animals to wear, ensuring that it does not affect the natural behavior of the animals. The head-mounted microscope should have high-resolution imaging capabilities to clearly record neuronal activity. Microscope fixation: Fix the head-mounted microscope on the Headstage substrate of the animal's head to ensure its stability and comfort. Adjust the position and angle of the microscope to obtain the best imaging effect.

[0112] Behavioral experimental environment: Set up an experimental environment that meets the requirements of the behavioral paradigm, such as a maze, treadmill, open field, etc. Ensure that there are no interfering factors in the environment to guarantee the naturalness of the animal behavior and the accuracy of the experimental data. Neuronal signal collection: When the animals perform specific behavioral tasks, collect neuronal calcium signals in real time through the head-mounted microscope. Ensure that the data acquisition system synchronously records neuronal activity and behavioral data.

[0113] Multi-camera monitoring: Install 3 cameras in the experimental environment to monitor the behavior of the animals in real time from different angles. The camera positions should cover the key areas of the experimental environment to ensure the comprehensiveness and accuracy of the behavioral data. Data synchronization and analysis: Synchronize the behavioral data recorded by the cameras with the neuronal signal data to ensure the time matching of the two. Use behavioral analysis software to process the recorded video data, extract behavioral features, and perform correlation analysis with the neuronal activity data.

[0114] Step 105: The acquisition software can perform frame alignment on the behavioral videos recorded by multiple cameras to ensure precise matching of the videos captured by different cameras on the time axis. Through the frame alignment function, it can be ensured that each frame in the video is synchronized with the corresponding frame recorded by other cameras, making the time points of the behavioral data consistent. The acquisition software can synchronize the neuronal calcium signals with the behavioral videos to ensure that the data of both are recorded and analyzed on the same time axis. Through the signal synchronization function, the neuronal activities can be precisely correlated with specific behavioral events, providing a basis for data analysis.

[0115] The acquisition software provides a logging function that can record key events and operations during the experiment in real time, such as the start and end times of the experiment, data acquisition status, device connection status, etc. Logging can help researchers track and review the experimental process to ensure data integrity and traceability. The software can set timed reminders and event reminders, such as device inspections, data backups, experimental operation reminders, etc. The log reminder function can help researchers stay alert during the experiment to prevent missed or incorrect operations.

[0116] The acquisition software allows researchers to preset optical acquisition parameters, including light source intensity, irradiation time, irradiation frequency, etc. The preset optical information can be customized according to experimental requirements to ensure the accuracy and consistency of optical acquisition under different experimental conditions. The acquisition software can automatically control the light source according to the preset optical information to precisely irradiate the experimental samples. Automated irradiation control can reduce manual intervention and improve experimental efficiency and data quality.

[0117] Step 106: In the calcium imaging experiment, neurons are labeled with a fluorescent indicator (such as GCaMP), and fluorescence is emitted when neurons are active. When the excitation light irradiates the neurons, the fluorescence signal emitted by the fluorescent indicator is captured by the CMOS sensor. The CMOS sensor converts the fluorescence signal into a digital image signal. These image signals contain the spatial and temporal information of neuronal activities. The acquisition software performs synchronous processing on the fluorescence images captured in real time.

[0118] These images can be time-synchronized with other behavioral data to ensure the consistency between neuronal activities and animal behavioral data. The images to be processed can be further analyzed, such as extracting information on fluorescence intensity, spatial distribution, and temporal changes, etc., to study the patterns and laws of neuronal activities. Through the CMOS sensor, the reflection information of the biological tissue sample can be translated into electrical signals of neuronal activities, and at the same time, the fluorescence signal is converted into images to be processed synchronously. These steps enable researchers to accurately capture and analyze neuronal activities and correlate them with the behavioral data of animals, providing important technical support and data basis for neuroscience research.

[0119] Step 107: Use a fluorescence microscope to record the calcium signal changes of neurons and generate time-series image data. These image data contain the fluorescence intensity changes of neuron activities. Use the CNMF-E algorithm to process the calcium imaging data, decompose the image matrix, and extract the calcium signals of individual neurons. CNMF-E can effectively remove noise and improve the accuracy and resolution of the signals. The extracted calcium signal data include the activity intensities of each neuron at different time points.

[0120] Synchronize the behavioral data obtained from deep learning analysis with the calcium signal data extracted by CNMF-E in time. Ensure that behavioral events and neuron activities are accurately matched on the time axis. Analyze the changes in neuron activities when the animal performs different behaviors. Identify the relationships between specific behaviors and specific neuron activity patterns. Use the analysis results to construct a behavior-neural network map to show the associations between behavioral events and neuron activities. The neural network map can show which neurons are activated during specific behaviors and the functional connections between these neurons.

[0121] The CNMF-E algorithm is an improvement based on NMF, adding spatial and temporal constraint conditions to improve the accuracy and robustness of signal extraction. CNMF-E combines the information of spatial resolution (image pixels) and temporal resolution (time series) and can more effectively separate neuron signals and background noise.

[0122] Steps of the CNMF-E algorithm: Conduct preliminary preprocessing on the calcium imaging data, including removing noise, normalizing, and correcting drift, etc. The preprocessed data is more suitable for subsequent matrix decomposition; Initialize two matrices: a spatial matrix (representing the spatial distribution of neurons) and a temporal matrix (representing the activities of neurons over time). The initial values can be obtained through simple K-means clustering or other methods; Alternately update the spatial matrix and the temporal matrix to minimize the reconstruction error. Use constraint conditions to ensure the non-negativity and sparsity of the decomposition results; During the update process of the spatial matrix, add spatial constraint conditions to ensure that the extracted neuron signals are local and sparse in space. For example, by setting the neighborhood range, limit the spatial expansion range of each neuron signal; During the update process of the temporal matrix, add temporal constraint conditions to ensure the coherence and sparsity of the signals over time. For example, use a sliding window or other methods to ensure the continuity and sparsity of the extracted neuron signals over time; Introduce background modeling to separate the background signal and the neuron activity signal. The background signal can be modeled and removed through a low-rank matrix or other methods; The iterative process continues until the reconstruction error reaches the convergence condition or the number of iterations reaches the preset upper limit.

[0123] In refined behavior analysis, deep learning methods play an important role. Through convolutional neural networks (CNNs), recurrent neural networks (RNNs), temporal convolutional networks (TCNs), 3D convolutional neural networks (3D-CNNs), multimodal deep learning, graph convolutional networks (GCNs), and attention mechanisms, efficient and accurate behavior recognition and analysis can be achieved. These methods not only improve the automation of analysis but also provide researchers with new perspectives and tools, promoting the in-depth development of behavioral science research.

[0124] A high-performance computer (shown as 201) is used to connect to a data acquisition box (shown as 205) to provide power supply requirements, communication services, data interaction, optical parameter settings, and instruction settings. The high-performance computer (shown as 201) is equipped with a software system for collecting software for in-vivo monitoring of single photons and refined analysis of physiological behaviors (shown as 202). This software system includes a microscope camera configuration module; an ethology camera configuration module; an experimental information configuration module; a trigger and input configuration module; an ethology tracking configuration module; a collection and recording module; a microscope camera parameter design module; a running log panel; a microscope real-time control module; a microscope status display module; a microscope image display module; a parameter configuration module, etc. Among them, the high-performance computer (shown as 201) is connected to the data acquisition box (shown as 205) through a data transmission and power supply interface (shown as 203). A synchronization interface (shown as 204) is used to connect to a head-mounted microscope (shown as 208) and an ethology camera (shown as 206). The ethology paradigm maze (shown as 207) can be connected to the data acquisition box (shown as 205) in a wired manner, but it depends on the type of maze and the experimental design. Experimental animals (shown as 209) are not limited to mice, rats, primates, rabbits, etc.

[0125] The high-performance computer (201) ensures the efficient operation of the data acquisition box (205) by providing power supply requirements, communication services, data interaction, optical parameter settings, and instruction settings. It plays a key role in various application scenarios, significantly improving the efficiency and accuracy of data processing and control.

[0126] Power supply requirements:

[0127] The high-performance computer provides stable power supply to the data acquisition box through a power interface to ensure its normal operation. The power management system monitors the power supply to prevent damage to the equipment caused by voltage fluctuations or power outages.

[0128] Communication services:

[0129] The computer communicates with the data acquisition box through a high-speed network interface (such as Ethernet, InfiniBand). Using the TCP / IP protocol or a dedicated protocol, efficient data transmission and command control are achieved.

[0130] Data Interaction:

[0131] The collected data is transmitted through the communication interface to a high-performance computer for processing and storage. The computer utilizes its powerful processing capabilities to perform real-time analysis, filtering, and compression of the data.

[0132] Optical Parameter Setting:

[0133] The user sets optical parameters such as light source intensity, exposure time, filter selection, etc. through the computer interface (GUI or CLI). The computer transmits these parameters to the data acquisition box through the communication interface to ensure precise optical control.

[0134] Instruction Setting:

[0135] The high-performance computer, acting as the central control unit, receives user instructions and transmits them to the data acquisition box for execution. Instructions include starting or stopping data acquisition, adjusting the acquisition frequency, switching working modes, etc.

[0136] Working Principle

[0137] Initialization and Configuration: After power-on, the high-performance computer initializes the system configuration and loads necessary driver programs and control software. The user inputs initial settings through the software interface, including power management, communication protocol, optical parameters, etc.

[0138] Power Supply and Startup: The computer powers the data acquisition box through the power interface to ensure its normal startup and operation. The power management system monitors the power status and gives an alarm in case of anomalies.

[0139] Data Acquisition and Transmission: The data acquisition box collects data through sensors or other input devices and transmits the data to the high-performance computer through a high-speed network interface. After receiving the data, the computer performs real-time processing, including data cleaning, format conversion, preliminary analysis, etc.

[0140] Optical Control and Parameter Adjustment: The user adjusts optical parameters through the computer interface, and the computer sends the setting instructions to the data acquisition box. The data acquisition box adjusts the optical device according to the instructions to ensure that the collected images or data meet the preset requirements.

[0141] Instruction Transfer and Execution: The high-performance computer receives user instructions or preset program instructions and transmits them to the data acquisition box through the communication interface. After receiving the instructions, the data acquisition box performs corresponding operations, such as starting / stopping acquisition, adjusting the acquisition mode, etc.

[0142] Data Storage and Analysis: The computer performs in-depth analysis on the collected data and utilizes its high computing power to complete complex calculation tasks. The analysis results can be displayed in real-time or stored in the computer's storage system for subsequent use.

[0143] The single-photon system microscope integrated circuit control board (shown as 301) is one of the core components of the single-photon microscopy imaging system. It is mainly used to control and manage various operations of the microscope, including functions such as light source control, image acquisition, data processing, and transmission. Power management module: Provides a stable power supply to ensure the normal operation of the control board and connected devices. It includes functions such as voltage regulation, overcurrent protection, and short-circuit protection; Processor module: Embeds a high-performance processor for real-time processing and analysis of image data. It supports multi-threaded parallel computing to improve data processing efficiency; Communication interface module: Provides multiple high-speed communication interfaces, such as USB, serial port, etc., to achieve data transmission and device control; Image sensor interface module: Connects to the image sensor of the microscope to achieve image data acquisition and transmission. It supports various types of image sensors, such as CMOS photosensitive units (shown as 302), etc.; Light source control module: Controls parameters such as the on / off, intensity, and wavelength of the light source to achieve precise optical control. It supports various types of light sources, such as built-in LED units or external laser units (shown as 303).

[0144] The control board can adjust the intensity and wavelength of the light source to adapt to different experimental requirements. By precisely controlling the pulse frequency and duration of the light source, the quality of single-photon imaging is optimized. The control board connects to the image sensor of the microscope (such as the CMOS photosensitive unit (shown as 302)) to achieve real-time image acquisition. It supports various image acquisition modes, such as continuous acquisition, timed acquisition, and trigger acquisition, etc. It embeds a high-performance processor that can perform real-time processing and analysis on the acquired image data. It supports various image processing algorithms, such as noise reduction, enhancement, filtering, etc., to improve the quality and resolution of the image. The control board transmits the processed image data to a high-performance computer through a high-speed interface (such as USB3.0) for further analysis. It supports various data transmission protocols to ensure the stability and efficiency of data transmission. Users can set the optical parameters of the microscope through the control board interface (high-performance computer (201)). These include the wavelength, intensity, exposure time, filter selection, etc. of the light source. At the same time, it receives and executes instructions from the high-performance computer or users, such as starting / stopping image acquisition, adjusting optical settings, etc. It provides logging and status monitoring functions for convenient experiment management and troubleshooting.

[0145] Electronic zoom (shown as 304) adjusts the relative position of the optical lens through the microscope integrated circuit control board (shown as 301), changes the lens focal length, and thus realizes the magnification or reduction of the object to be photographed. This zoom does not reduce the resolution and quality of the image, so it has important applications in high-precision imaging. Adjust the position of the movable lens group to make the focal length of the lens system shorter or longer, so as to realize the magnification or reduction of the object to be photographed. The longer the focal length, the greater the magnification; the shorter the focal length, the smaller the magnification. During the zoom process, the lens system focuses simultaneously to ensure clear images, and uses an autofocus system (AF) or manual focus (MF) for precise focusing.

[0146] The connection fixing unit (shown as 305) is directly connected to the substrate to ensure imaging stability in a freely moving and strongly changing environment.

[0147] The information interface unit (shown as 306) is connected to the data acquisition box (shown as 205) for functions such as power supply, information interaction, and command transmission.

[0148] The single-photon imaging system is applied in biomedical research to observe and analyze the microscopic structure and dynamic processes of freely moving biological tissues. The imaging application program based on the single-photon system usually includes acquisition software and analysis software, and these two software modules work together to achieve efficient and accurate data acquisition and in-depth data analysis.

[0149] I. Acquisition software

[0150] The acquisition software is responsible for obtaining image data from the single-photon microscope system and ensuring the high quality and synchronization of the data. Its main functions include:

[0151] Image acquisition: Obtain the image data of the microscope system in real time, supporting multiple image formats. Provide high-resolution and high-frame-rate image acquisition capabilities to ensure capturing subtle biological dynamic changes.

[0152] Optical parameter setting: Configure the optical parameters of the microscope, such as excitation light wavelength, light intensity, exposure time, etc. Support preset parameter configuration and real-time adjustment to adapt to different experimental requirements.

[0153] Synchronous acquisition: Ensure the synchronous acquisition of image data and other physiological signals (such as behavior monitoring data). Provide accurate timestamps and frame alignment functions to ensure the consistency of multi-source data.

[0154] Log reminder: Record important events and parameter changes during the acquisition process. Provide a log reminder function to help researchers monitor and adjust the experiment in real time.

[0155] II. Analysis software

[0156] The analysis software is used to process and analyze the collected image data and extract valuable biological information. Its main functions include:

[0157] Image preprocessing: Denoise, correct, and enhance the collected original images. Improve the clarity and contrast of the images for subsequent analysis.

[0158] Signal extraction: Use advanced algorithms (such as CNMF-E) to extract calcium signals and other biological signals. Achieve precise detection and quantitative analysis of neuronal activities.

[0159] Behavior analysis: Based on deep learning technology, analyze the behavior patterns and characteristics of animals. Construct an association map between behavior and neural signals to reveal neural mechanisms.

[0160] Data visualization: Provide various data visualization tools, such as heat maps, time series graphs, 3D reconstructions, etc. Help researchers intuitively understand and display experimental results.

[0161] Statistical analysis: Perform statistical analysis on the extracted signal and behavior data, calculate indicators such as correlation and significance. Support multiple statistical models and methods and provide detailed analysis reports.

[0162] III. System Architecture

[0163] Imaging applications based on single-photon systems usually adopt a modular design to ensure the flexibility and scalability of the software. Its system architecture includes the following components:

[0164] User interface (UI): Provide a friendly and intuitive operation interface for researchers to configure and operate conveniently. Support multiple views and interaction methods to improve the user experience.

[0165] Data management module: Responsible for the storage, management, and retrieval of image and signal data. Provide an efficient data compression and transmission mechanism to ensure data security and fast access.

[0166] Algorithm library: Integrate a variety of image processing and analysis algorithms and support plug-in expansion. Provide an open API interface for researchers to develop and customize algorithms.

[0167] System integration module: Connect the microscope hardware and external devices to achieve data acquisition and control. Provide device driver and communication protocol support to ensure the compatibility and stability of the system.

[0168] Those skilled in the art know that, in addition to implementing the system and its various devices, modules, and units provided by the present invention in the form of pure computer-readable program code, the method steps can be logically programmed to enable the system and its various devices, modules, and units provided by the present invention to be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers, etc. to achieve the same functions. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a kind of hardware component, and the devices, modules, and units included therein for implementing various functions can also be regarded as the structures within the hardware component; the devices, modules, and units for implementing various functions can also be regarded as either software modules for implementing the method or structures within the hardware component.

[0169] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. Without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A single-photon imaging data and physiological behavior data analysis system, characterized in that: include: A single-photon imaging module, used for acquiring the fluorescence signal excited by the brain of the target object based on a single-photon detector, and converting the fluorescence signal into single-photon imaging data; Behavior monitoring module, used to obtain the behavior data of the target object; A synchronization control module, used to achieve time synchronization between single-photon imaging data and behavioral data; The data processing and storage module is used for real-time processing and storage of single-photon imaging data and behavioral data.

2. The single-photon imaging data and physiological behavior data analysis system according to claim 1, characterized in that: The single-photon imaging module comprises: Module M1.1: used to obtain the fluorescence signal stimulated by the brain of the target object based on a single-photon detector, and convert the obtained fluorescence signal into a visual image; Module M1.2: used to pre-process the visualization image, including noise reduction and contrast enhancement.

3. The single-photon imaging data and physiological behavior data analysis system according to claim 1, characterized in that: The behavior monitoring module includes: Module M2.1: used to obtain the motion state of the target object in real time based on the motion sensor, including: the moving speed, direction and position information of the target object; Module M2.2: used to obtain video data containing a target object based on a camera; Module M2.3: Used to extract behavioral feature data based on video data through deep learning and computer vision technology, and identify behavioral patterns based on the behavioral feature data.

4. The single-photon imaging data and physiological behavior data analysis system according to claim 3, characterized in that: The module M2.3 includes: obtaining behavior data and background environment data through the SlowFast video behavior feature extraction algorithm, and performing feature extraction respectively; or using a 3D convolution network based on video data, after a 3D convolution operation, outputting a K*l*h*w 3D feature map to extract behavior features in the video.

5. The single-photon imaging data and physiological behavior data analysis system according to claim 3, characterized in that: The module M2.3 includes: The extracted behavioral features are input into a machine learning model, including a convolutional neural network (CNN) or a recurrent neural network (RNN), and the machine learning model is used to identify different behavioral patterns.

6. The single-photon imaging data and physiological behavior data analysis system according to claim 1, characterized in that: The synchronization control module includes: realizing time synchronization of single-photon imaging data and behavior data through a clock synchronization unit and a data synchronization interface.

7. The single-photon imaging data and physiological behavior data analysis system according to claim 6, characterized in that: The clock synchronization unit provides a unified time reference through clock synchronization; The data synchronization interface time-aligns the single-photon imaging data with the behavioral data to ensure data synchronization.

8. The single-photon imaging data and physiological behavior data analysis system according to claim 1, characterized in that: The data processing and storage module includes a processor, a memory and a data transmission interface; The processor is used to process single-photon imaging data and behavioral data in real time; The memory is used to store raw data and processed analysis results; The data transmission interface is used to implement a data export function based on a variety of data transmission protocols.

9. A method for analyzing single-photon imaging data and physiological behavior data, characterized in that: include: Step S1: using a single-photon imaging module to acquire a fluorescence signal excited by the brain of a target object based on a single-photon detector, and converting the fluorescence signal into single-photon imaging data; Step S2: using a behavior monitoring module to obtain behavior data of the target object; Step S3: using a synchronization control module to achieve time synchronization between single-photon imaging data and behavioral data; Step S4: Using the data processing and storage module to process and store the single-photon imaging data and behavior data in real time.

10. The single photon imaging data and physiological behavior data analysis method according to claim 9, characterized in that: The step S1 comprises: Step S1.1: used to acquire the fluorescence signal excited by the brain of the target object based on the single-photon detector, and convert the acquired fluorescence signal into a visual image; Step S1.2: used to pre-process the visualization image, including: noise reduction and contrast enhancement processing; The step S2 comprises: Step S2.1: acquiring the motion state of the target object in real time based on the motion sensor, including: the moving speed, direction and position information of the target object; Step S2.2: Acquire video data containing the target object based on the camera; Step S2.3: extracting behavior feature data based on the video data through deep learning and computer vision technology, and identifying the behavior pattern based on the behavior feature data; The step S3 includes: achieving time synchronization of single-photon imaging data and behavior data through a clock synchronization unit and a data synchronization interface.

Images