A system and method for neuron signal recovery and extraction in four-dimensional spacetime data

Abstract

This invention belongs to the field of biological signal processing and discloses a system and method for recovering and extracting neuronal signals from four-dimensional spatiotemporal data. The system includes a biological data acquisition module, a signal processing module, and a signal extraction module. The biological data acquisition module is connected to the signal processing module, and the signal processing module is connected to the signal extraction module. The biological data acquisition module is used to acquire image sequences of biological samples; the signal processing module is used to process the acquired biological image sequences; and the signal extraction module extracts the neuronal signal intensity from the data. Users can use this system to quickly and efficiently acquire, identify, extract, and analyze biological four-dimensional spatiotemporal data, significantly shortening the biological signal processing cycle and improving its efficiency. This invention flexibly utilizes various machine learning algorithms and designs a complete and reliable deployment scheme, obtaining highly accurate sample data while reducing the cost of manual intervention.

Description

Technical Field

[0001] This invention belongs to the field of biological signal processing, and in particular relates to a system and method for recovering and extracting neuronal signals from four-dimensional spatiotemporal data. Background Technology

[0002] In studying the neural-behavioral correspondences of organisms, it is necessary to collect neuronal calcium signals. For the collected calcium and behavioral signals, key information is typically identified visually, and then further extracted using image and video analysis software to determine indicators reflecting biological behavior. However, for large-scale datasets, this process requires extensive repetitive manual annotation and faces problems such as inaccurate annotations, transfer difficulties, and a lack of visualization tools. A complete solution is particularly lacking for the acquisition, processing, and extraction of four-dimensional spatiotemporal data. Summary of the Invention

[0003] The purpose of this invention is to provide a system and method for recovering and extracting neuronal signals from four-dimensional spatiotemporal data, so as to solve the above-mentioned technical problems.

[0004] To address the aforementioned technical problems, the specific technical solution of the system and method for neuron signal recovery and extraction from four-dimensional spatiotemporal data according to the present invention is as follows:

[0005] A system for recovering and extracting neuronal signals from four-dimensional spatiotemporal data includes a biological data acquisition module, a signal processing module, and a signal extraction module. The biological data acquisition module is connected to the signal processing module, and the signal processing module is connected to the signal extraction module. The biological data acquisition module is used to acquire image sequences of biological samples; the signal processing module is used to process the acquired biological image sequences; and the signal extraction module extracts the neuronal signal intensity from the data.

[0006] Furthermore, the biological data acquisition module includes an illumination unit, an imaging unit, and a piezoelectric drive unit, with the illumination unit and imaging unit placed perpendicular to each other; the illumination unit is built on the left and right sides of the piezoelectric drive unit; the illumination unit is used to illuminate the sample and excite biological fluorescence signals; the imaging unit is used to detect and record the biological fluorescence signals excited by the illumination unit, and adjust the image capture speed by setting the camera frame rate; the piezoelectric drive unit is used to adjust the speed and distance of sample movement.

[0007] Furthermore, the illumination unit is composed of a laser and multiple optical elements, used to form light sheets with a thickness of about 8 micrometers on each side, and the two beams of light are adjusted to completely overlap through optical alignment.

[0008] Furthermore, the optical elements include a collimating head, a galvanometer, a scanning lens, a sleeve lens, and an illumination objective.

[0009] Furthermore, the imaging unit consists of an imaging objective, a fluorescence filter, an inverted biological microscope, and a camera.

[0010] Furthermore, the piezoelectric drive unit includes a sample clamping device and a piezoelectric actuator. The piezoelectric actuator is used to drive the biological sample to reciprocate within a certain z-axis height range. The speed and distance of sample movement are adjusted by setting the piezo frequency and the z-axis height.

[0011] Furthermore, the signal processing module includes a volume recovery unit, a noise reduction unit, and an isotropic recovery unit; the volume recovery unit constructs the sequence into multiple volumes using the starting point of the original sequence capture, camera frame rate, piezo frequency, and total length along the z-axis, and the time of the nth volume. The volume represents the temporal information; each volume is composed of several superimposed images, used to represent the three-dimensional shape of the target at a given moment. The m-th image has a height of [value missing] on the z-axis. The denoising unit consists of a noise fitting unit and a noise filtering unit, used to detect and filter noise in the volume. The noise fitting and filtering are implemented using a deep neural network or a noise2noise model. The denoised volume is input to the isotropic recovery unit. The isotropic recovery unit consists of a deep neural network, used to restore neurons that are stretched and deformed in the z-axis direction due to low resolution to a standard shape.

[0012] Furthermore, the signal extraction module includes an interactive labeling unit, a neuron 3D tracking unit, and a grayscale detection unit. The interactive labeling unit includes a client capable of interacting with volume information and a labeling tool. Interactive functions include dragging, zooming in, zooming out, playing, pausing, and coordinate updating. The labeling content includes region labels and point labels, represented by polygon boxes and points. The neuron 3D tracking unit includes a pre-trained SiamRPN network for object detection and a 3D high-resolution network for keypoint tracking. The 3D high-resolution network is an extension of a general high-resolution network for processing 3D data; its convolutional units, batch-normalization, and fusion units are all extended to 3D to process the input 3D data and identify keypoint information in the 3D data. The grayscale detection unit extracts the grayscale values ​​of the neuron signals based on the neuron position information, forming a grayscale change curve.

[0013] A method for recovering and extracting neuronal signals from four-dimensional spatiotemporal data includes the following steps:

[0014] Step 1: Data acquisition. The data acquisition module uses the illumination unit to form an 8-micrometer-thick light sheet to illuminate the sample, excite fluorescence signals, and uses the piezoelectric drive unit to drive the sample to move back and forth rapidly within a set height range. Then, the imaging unit detects and records the captured image sequence in real time.

[0015] Step 2: Signal processing. The signal processing module reads the captured image sequence, uses the volume restoration unit to construct a volume from the image sequence, then uses the denoising unit to detect noise in the volume, extracts and filters it to improve the signal-to-noise ratio of the volume, and then uses the isotropic restoration unit to restore the low-resolution planes in the volume to ensure that the morphology of neurons in each plane is complete, similar and consistent with reality.

[0016] Step 3: Grayscale extraction. The experimenter manually marks the approximate area to be tracked using the interactive labeling unit, and then uses the SiamRPN network in the neuron 3D tracking unit to track key areas. In the results of the automatic area, the interactive labeling unit is used to mark the neurons to be tracked. The number of marked volumes is about 5%-10% of the total volume. Then, the key points of the neurons are tracked through the neuron 3D tracking unit. After verifying that the neuron tracking position is correct, the experimenter uses the grayscale detection unit to extract the grayscale values ​​of the neurons and generate a grayscale curve.

[0017] Furthermore, step one includes the following specific steps:

[0018] The left optical path emits a 488nm laser beam. The light passes sequentially through a collimator, XY and Z mirrors, a scanning lens, a sleeve lens, and a 4x illumination objective. These optical components are arranged according to optical principles to ensure the light remains horizontal. The right optical path is identical to the left, and the two paths are aligned perfectly using optical components. When the XY mirror scans at high speed while the Z mirror remains stationary, a light sheet with a thickness of approximately 8 micrometers is formed. The sample is then illuminated by this light sheet, exciting the biofluorescence signal. The excited fluorescence signal passes through the 4x imaging objective and a fluorescence filter in the imaging unit, and finally... Finally, the image is detected and recorded by the camera. The camera frame rate, i.e., the image acquisition speed, is adjusted by setting the exposure time. The exposure time and the image acquisition speed are inversely proportional. The longer the exposure time, the slower the image acquisition speed and the higher the signal strength. Conversely, the shorter the exposure time, the faster the image acquisition speed and the lower the signal strength. The appropriate exposure time is selected according to the signal quality. The sample is fixed on the sample holder and connected to the piezo actuator. The frequency and z-axis movement height of the piezo are set so that the piezo drives the sample to move back and forth within the z-axis height range, thereby acquiring the three-dimensional imaging information of the sample.

[0019] The system and method for neuronal signal recovery and extraction from four-dimensional spatiotemporal data disclosed in this invention have the following advantages: Users can quickly and efficiently collect, identify, extract, and analyze biological four-dimensional spatiotemporal data through the system of this invention, greatly shortening the biological signal processing cycle and improving the efficiency of biological signal processing. This invention flexibly utilizes various machine learning algorithms and designs a complete and reliable deployment scheme, obtaining highly accurate sample data while reducing the cost of manual intervention. Attached Figure Description

[0020] Figure 1 This is a system architecture diagram of neuron signal recovery and extraction in four-dimensional spatiotemporal data according to the present invention;

[0021] Figure 2 This is a schematic diagram of the biological data acquisition module of the present invention;

[0022] Figure 3 This is a flowchart of the signal processing module of the present invention;

[0023] Figure 4 This is a flowchart of the signal extraction module of the present invention;

[0024] Figure 5 This is a schematic diagram of the dual-sided illumination of the present invention;

[0025] Figure 6 This is a flowchart of the automatic neuron detection and tracking process of the present invention;

[0026] Figure 7 This is a flowchart of the method for recovering and extracting neuronal signals from four-dimensional spatiotemporal data according to the present invention. Detailed Implementation

[0027] To better understand the purpose, structure, and function of this invention, the following detailed description, in conjunction with the accompanying drawings, provides a system and method for neuron signal recovery and extraction from four-dimensional spatiotemporal data.

[0028] like Figure 1 As shown, the present invention discloses a system for recovering and extracting neuronal signals from four-dimensional spatiotemporal data, comprising a biological data acquisition module, a signal processing module, and a signal extraction module. The biological data acquisition module is connected to the signal processing module, and the signal processing module is connected to the signal extraction module. The biological data acquisition module is used to acquire image sequences of biological samples; the signal processing module is used to process the acquired biological image sequences; and the signal extraction module extracts the neuronal signal intensity from the volume.

[0029] like Figure 2As shown, the biological data acquisition module includes an illumination unit, an imaging unit, and a piezoelectric drive unit. The illumination and imaging units are placed perpendicular to each other. The illumination unit is built on the left and right sides of the piezoelectric drive unit, forming light sheets approximately 8 micrometers thick on each side. Optical alignment is used to ensure complete overlap of the two light beams, thereby illuminating the sample and exciting biofluorescence signals. The illumination unit consists of a laser and multiple optical components, including a collimator, galvanometer, scanning lens, cannulated lens, and illumination objective. The imaging unit consists of an imaging objective, a fluorescence filter, an inverted biological microscope, and a camera, used to detect and record the biofluorescence signals excited by the illumination unit. The image capture speed can be adjusted by setting the camera frame rate. The piezoelectric drive unit includes a sample clamping device and a piezoelectric actuator (piezo), used to drive the biological sample to reciprocate within a certain z-axis height range. The speed and distance of sample movement can be adjusted by setting the piezo frequency and the z-axis height.

[0030] The illumination unit employs a dual-sided illumination method, which can enhance the biosignal intensity of the sample and avoid insufficient illumination on the contralateral side caused by unilateral illumination. For example... Figure 5 As shown, the left optical path emits a 488nm laser beam. The light passes sequentially through a collimator, galvanometer, scanning lens, sleeve lens, and a 4x illumination objective. The galvanometers include XY and Z mirrors. These optical components are arranged according to optical principles to ensure the light remains horizontal. The right optical path is identical to the left, with optical elements used to ensure perfect alignment. When the XY mirror scans at high speed while the Z mirror remains stationary, a light sheet approximately 8 micrometers thick is formed. The sample is then illuminated by this light sheet, exciting the biofluorescence signal. The excited fluorescence signal passes through the 4x imaging objective and a fluorescence filter in the imaging unit before being detected and recorded by the camera. The camera frame rate, i.e., the image acquisition speed, can be adjusted by setting the exposure time. Exposure time and image acquisition speed are inversely proportional; a longer exposure time results in a slower image acquisition speed and a higher signal strength, while a shorter exposure time results in a faster image acquisition speed and a lower signal strength. In the experiment, an appropriate exposure time must be selected based on the signal quality. The sample is fixed on the sample holder and connected to the piezo actuator. The frequency and z-axis movement height of the piezo can be set, so that the piezo drives the sample to move back and forth within the z-axis height range, thereby acquiring the three-dimensional imaging information of the sample.

[0031] like Figure 3 As shown, the signal processing module includes a volume recovery unit, a noise reduction unit, and an isotropic recovery unit. The volume recovery unit constructs the sequence into multiple volumes using the starting point of the original sequence capture, the camera frame rate, the piezo frequency, and the total length along the z-axis. The time of the nth volume... The volume represents the temporal information; each volume is composed of several superimposed images, used to represent the three-dimensional shape of the target at a given moment. The m-th image has a height of [value missing] on the z-axis. The denoising unit consists of a noise fitting unit and a noise filtering unit, used to detect and filter noise in the volume. Noise fitting and filtering can be implemented using a deep neural network or a noise2noise model. The denoised volume is then input to the isotropic restoration unit. The isotropic restoration unit consists of a deep neural network, used to restore neurons that have been stretched and deformed in the z-axis direction due to low resolution to their standard shape.

[0032] like Figure 4 As shown, the signal extraction module includes an interactive labeling unit, a neuron 3D tracking unit, and a grayscale detection unit. The interactive labeling unit includes a client capable of interacting with volume information and a labeling tool; interactive functions include dragging, zooming in, zooming out, playing, pausing, and coordinate updating; the labeling content includes region labels and point labels, represented by polygon boxes and points. For example... Figure 6 As shown, the neuron 3D tracking unit includes a pre-trained SiamRPN network for object detection and a 3D high-resolution network for keypoint tracking. The 3D high-resolution network is an extension of general high-resolution networks for processing 3D data; its convolutional units, batch-normalization, and fusion units are all extended to 3D to process the input 3D data and identify keypoint information within it. The grayscale detection unit extracts the grayscale values ​​of the neuron signals based on the neuron's position information, forming a grayscale change curve.

[0033] like Figure 7 As shown, a method for recovering and extracting neuronal signals from four-dimensional spatiotemporal data includes the following steps:

[0034] Step 1: Data acquisition. The data acquisition module uses the illumination unit to form an 8-micrometer-thick light sheet to illuminate the sample, excite fluorescence signals, and uses the piezoelectric drive unit to drive the sample to move back and forth rapidly within a set height range. Then, the imaging unit detects and records the captured image sequence in real time.

[0035] Step 2: Signal processing. The signal processing module reads the captured image sequence, uses the volume restoration unit to construct a volume from the image sequence, then uses the denoising unit to detect noise in the volume, extracts and filters it to improve the signal-to-noise ratio of the volume, and then uses the isotropic restoration unit to restore the low-resolution planes in the volume to ensure that the morphology of neurons in each plane is complete, similar and consistent with reality.

[0036] Step 3: Grayscale extraction. The experimenter manually marks the approximate area to be tracked using the interactive labeling unit, and then uses the SiamRPN network in the neuron 3D tracking unit to track key areas. In the results of the automatic area, the interactive labeling unit is used to mark the neurons to be tracked. The number of marked volumes is about 5%-10% of the total volume. Then, the key points of the neurons are tracked through the neuron 3D tracking unit. After verifying that the neuron tracking position is correct, the experimenter uses the grayscale detection unit to extract the grayscale values ​​of the neurons and generate a grayscale curve.

[0037] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A system for recovering and extracting neuronal signals from four-dimensional spatiotemporal data, characterized in that, The system includes a biological data acquisition module, a signal processing module, and a signal extraction module. The biological data acquisition module is connected to the signal processing module, and the signal processing module is connected to the signal extraction module. The biological data acquisition module is used to acquire image sequences of biological samples. The signal processing module is used to process the acquired biological image sequences. The signal extraction module extracts the neuronal signal intensity from the volumes. The signal processing module includes a volume recovery unit, a denoising unit, and an isotropic recovery unit. The volume recovery unit constructs the sequence into multiple volumes based on the starting point of the original sequence capture, the camera frame rate, the piezo frequency, and the total length in the z-axis direction. The time of the nth volume is... The volume represents the temporal information; each volume is composed of several superimposed images, used to represent the three-dimensional shape of the target at a given moment. The m-th image has a height of [value missing] on the z-axis. The denoising unit consists of a noise fitting unit and a noise filtering unit, used to detect and filter noise in the volume. The noise fitting and filtering are implemented using a deep neural network or a noise2noise model. The denoised volume is input to the isotropic recovery unit. The isotropic recovery unit consists of a deep neural network, used to restore neurons that are stretched and deformed in the z-axis direction due to low resolution to a standard shape.

2. The system for recovering and extracting neuron signals from four-dimensional spatiotemporal data according to claim 1, characterized in that, The biological data acquisition module includes an illumination unit, an imaging unit, and a piezoelectric drive unit, with the illumination unit and imaging unit placed perpendicular to each other. The illumination unit is built on the left and right sides of the piezoelectric drive unit. The illumination unit is used to illuminate the sample and excite biological fluorescence signals. The imaging unit is used to detect and record the biological fluorescence signals excited by the illumination unit and adjust the image capture speed by setting the camera frame rate. The piezoelectric drive unit is used to adjust the speed and distance of sample movement.

3. The system for recovering and extracting neuron signals from four-dimensional spatiotemporal data according to claim 2, characterized in that, The illumination unit consists of a laser and multiple optical elements, used to form light sheets with a thickness of 8 micrometers on each side, and the two beams of light are adjusted to completely overlap through optical alignment.

4. The system for recovering and extracting neuron signals from four-dimensional spatiotemporal data according to claim 3, characterized in that, The optical elements include a collimator, a galvanometer, a scanning lens, a sleeve lens, and an illumination objective.

5. The system for recovering and extracting neuron signals from four-dimensional spatiotemporal data according to claim 2, characterized in that, The imaging unit consists of an imaging objective, a fluorescence filter, an inverted biological microscope, and a camera.

6. The system for recovering and extracting neuron signals from four-dimensional spatiotemporal data according to claim 2, characterized in that, The piezoelectric drive unit includes a sample clamping device and a piezoelectric actuator. The piezoelectric actuator is used to drive the biological sample to move back and forth within a certain z-axis height range. The speed and distance of sample movement are adjusted by setting the piezo frequency and the z-axis height.

7. The system for recovering and extracting neuron signals from four-dimensional spatiotemporal data according to claim 1, characterized in that, The signal extraction module includes an interactive labeling unit, a neuron 3D tracking unit, and a grayscale detection unit. The interactive labeling unit includes a client capable of interacting with volume information and a labeling tool. Interactive functions include dragging, zooming in, zooming out, playing, pausing, and coordinate updating. Labeling content includes region labels and point labels, represented by polygon boxes and points. The neuron 3D tracking unit includes a pre-trained SiamRPN network for object detection and a 3D high-resolution network for keypoint tracking. The 3D high-resolution network is an extension of a general high-resolution network for processing 3D data; its convolutional units, batch-normalization, and fusion units are all extended to 3D for processing input 3D data and identifying keypoint information in the 3D data. The grayscale detection unit extracts the grayscale values ​​of the neuron signals based on the neuron position information, forming a grayscale change curve.

8. A method for neuron signal recovery and extraction using a system for neuron signal recovery and extraction from four-dimensional spatiotemporal data as described in any one of claims 1-7, characterized in that, Includes the following steps: Step 1: Data acquisition. The data acquisition module uses the illumination unit to form an 8-micrometer-thick light sheet to illuminate the sample, excite fluorescence signals, and uses the piezoelectric drive unit to drive the sample to move back and forth rapidly within a set height range. Then, the imaging unit detects and records the captured image sequence in real time. Step 2: Signal processing. The signal processing module reads the captured image sequence, uses the volume restoration unit to construct a volume from the image sequence, then uses the denoising unit to detect noise in the volume, extracts and filters it to improve the signal-to-noise ratio of the volume, and then uses the isotropic restoration unit to restore the low-resolution planes in the volume to ensure that the morphology of neurons in each plane is complete, similar and consistent with reality. Step 3: Grayscale extraction. The experimenter manually marks the area to be tracked using the interactive labeling unit, and then uses the SiamRPN network in the neuron 3D tracking unit to track key areas. In the automatic area results, the interactive labeling unit is used to mark the neurons to be tracked. The number of marked volumes is 5%-10% of the total volume. Then, the key points of the neurons are tracked through the neuron 3D tracking unit. After verifying that the neuron tracking position is correct, the experimenter uses the grayscale detection unit to extract the grayscale values ​​of the neurons and generate a grayscale curve.

9. The method for recovering and extracting neuron signals from four-dimensional spatiotemporal data according to claim 8, characterized in that, Step one includes the following specific steps: The left optical path emits a 488nm laser beam. The light passes sequentially through a collimator, XY and Z mirrors, a scanning lens, a sleeve lens, and a 4x illumination objective. These optical components are arranged according to optical principles to ensure the light remains horizontal. The right optical path is identical to the left, and the two paths are aligned perfectly using optical components. When the XY mirror scans at high speed while the Z mirror remains stationary, an 8-micrometer-thick light sheet is formed. The sample is then illuminated by this light sheet, exciting the biofluorescence signal. The excited fluorescence signal passes through the 4x imaging objective and a fluorescence filter in the imaging unit, ultimately... The image is captured and recorded by a camera. The camera frame rate, or image acquisition speed, is adjusted by setting the exposure time. Exposure time and image acquisition speed are inversely proportional. The longer the exposure time, the slower the image acquisition speed and the higher the signal strength. Conversely, the shorter the exposure time, the faster the image acquisition speed and the lower the signal strength. An appropriate exposure time is selected based on the signal quality. The sample is fixed on a sample holder and connected to a piezoelectric actuator. The frequency and z-axis movement height of the piezo are set so that the piezo drives the sample to move back and forth within the z-axis height range, thereby acquiring the three-dimensional imaging information of the sample.

Images