Single-photon kilohertz calcium imaging system

Abstract

The invention relates to the technical field of scientific imaging, in particular to a single-photon kilohertz calcium imaging system. The system comprises a microscopic imaging module, a data processing module and an upper processing module, wherein the microscopic imaging module is used for acquiring high-speed image data for calcium imaging through a microscopic imaging device comprising a single photon avalanche diode array sensor; the data processing module is used for acquiring the high-speed image data, processing the high-speed image data and outputting a calcium imaging image; and the upper computer module is used for controlling the image acquisition process of the microscopic imaging module, controlling the image processing process of the data processing module and displaying the calcium imaging image. According to the system provided by the invention, the image output frame rate is higher, the neural activity can be quantitatively detected, and more accurate information is provided for neurology research.

Description

Technical Field

[0001] The invention relates to the technical field of scientific imaging, and in particular to a single-photon kilohertz calcium imaging system. Background Art

[0002] High-precision spatiotemporal measurement of neural activity is closely related to the accuracy of brain analysis. The calcium imaging that is widely used at present is based on fluorescent proteins. The calcium imaging activity recorded is the cumulative result of multiple neural activities. The brightness change cycle of each neuron is long, and the strength of neuronal activity is determined by the brightness change. Although this method can judge the strength of neural activity by the change of fluorescence brightness, it cannot accurately obtain the cycle, rate, density, etc. of neural activity. Genetically encoded voltage indicators (GEVIs) are a fast voltage-sensitive staining method that can analyze neural activity with millisecond resolution. This rate indicator can display each action potential of a neuron in the form of light. Compared with traditional fluorescent gene-encoded calcium indicators, this gene-encoded voltage indicator makes it possible to analyze neural activity with high spatiotemporal precision.

[0003] To date, electron multiplying CCD (emCCD) and scientific CMOS (sCMOS) cameras have been commonly used to image calcium in neurons. However, while these sensors can achieve image recording at up to 100 frames per second, the speed of exposure and refresh severely limits the acquisition of images at higher frame rates. At higher frame rates, the camera may amplify the noise in the electronic pathway while increasing the frame rate, resulting in excessive noise in the image and hiding the real signal. State-of-the-art emCCD cameras have relatively low readout noise and can achieve single-photon sensitivity. However, the limited frame rate of emCCD cameras (about 100 frames per second) does not allow kilohertz frame rate imaging, and the amplification process also introduces noise, which actually reduces the signal-to-noise ratio and their quantum efficiency. State-of-the-art sCMOS have difficulty distinguishing weak signals generated by a limited number of incident photons from stray noise events at high frame rates due to inherent dark noise and readout noise. However, with the high speed of GEVIs, traditional CMOS image recording methods have become difficult to adapt to the needs of high-precision neural activity analysis. Summary of the invention

[0004] The present invention provides a single-photon kilohertz calcium imaging system, which aims to solve the problem that the existing CMOS image recording method is difficult to meet the needs of high-precision neural activity analysis.

[0005] In order to solve the above technical problems, the present invention provides a single-photon kilohertz calcium imaging system, comprising a microscopic imaging module, a data processing module and an upper processing module:

[0006] The microscopic imaging module is used to collect high-speed image data for calcium imaging through a microscopic imaging device including a single-photon avalanche diode array sensor;

[0007] The data processing module is used to acquire the high-speed image data, process the high-speed image data, and output a calcium imaging image;

[0008] The host computer module is used to control the image acquisition process of the microscopic imaging module, control the image processing process of the data processing module, and display the calcium imaging image.

[0009] Furthermore, the data processing module is specifically used for:

[0010] Determining the total number of photons in the signal frame included in the high-speed image data;

[0011] Determining the number of synthesized photons used to synthesize different signal frames in the high-speed image data according to the total number of frames by a time binning method;

[0012] According to the number of synthesized photons, synthesizing the high-speed image data in the order of frame signals to obtain synthesized image data;

[0013] The synthesized image data is subjected to noise removal using a preset noise removal method to obtain the calcium imaging image.

[0014] Furthermore, the data processing module determines the number of synthesized photons for synthesizing different signal frames in the high-speed image data according to the total number of photons by a time binning method, satisfying the following photon counting equation:

[0015]

[0016] Wherein, t represents the frame duration of the high-speed image data in the time bin, N(t) represents the number of synthesized photons, represents the number of photons at each point in each frame signal of the high-speed image data, and M(t) represents the Martingale noise term of the photon counting equation.

[0017] Furthermore, the number of photons at each point in each frame signal of the high-speed image data is It is determined by the flux detection equation ρ(τ), which satisfies the following expression:

[0018]

[0019] Where ρ represents the sum of photon counts within the current time bin (0, t), M p(t) represents the Martingale noise term of the flux detection equation.

[0020] Furthermore, the preset denoising method is at least one of mean filtering, standard deviation filtering, Gaussian filtering, and adaptive filtering.

[0021] Furthermore, the data processing module is also used for:

[0022] The calcium imaging images were converted to RGB format.

[0023] Furthermore, the microscopic imaging device includes an excitation light source, an excitation light filter, a beam splitter, an imaging achromatic lens, an objective lens group, a fluorescence filter and a liquid lens, wherein:

[0024] The excitation light source is used to emit excitation light to excite the indicator of the target object to produce fluorescence;

[0025] The excitation light filter is used to transmit the first light band of the excitation light;

[0026] The beam splitter is used to reflect the excitation light and transmit the fluorescence;

[0027] The imaging achromatic lens is used to adjust the spherical aberration and aberration of the microscopic imaging device;

[0028] The objective lens group is used to realize the imaging channel of the microscopic imaging device;

[0029] The fluorescence filter is used to transmit the second light band of the fluorescence;

[0030] The liquid lens is used to adjust the focal length of the microscopic imaging device;

[0031] The single photon avalanche diode array sensor receives the fluorescence to perform imaging and obtain the high-speed image data.

[0032] Furthermore, the indicator is a genetically encoded voltage indicator GEVI s virus;

[0033] The excitation light filter is a blue light filter, and the first light band is a blue light band of 450-490nm;

[0034] The fluorescent filter is a green light filter, and the second light band is a green light band of 510-550 nm.

[0035] The beneficial effect achieved by the present invention is that a single-photon kilohertz calcium imaging system is proposed. The system utilizes a single-photon avalanche diode sensor array to achieve high-speed image acquisition, adjusts the output frame rate through a time-binning processing method, and combines photon counting and flux detection to achieve image noise processing. Compared with the prior art, the system image output frame rate proposed by the present invention is higher, neural activity can be quantitatively detected, and more accurate information can be provided for neurological research. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a schematic structural diagram of a single-photon kilohertz calcium imaging system provided by an embodiment of the present invention;

[0037] Figure 2 is a schematic diagram of output frame rate synthesis of a single-photon kilohertz calcium imaging system provided by an embodiment of the present invention;

[0038] Figure 3 Schematic diagram of the Martingale noise term of the photon counting equation of the single-photon kilohertz calcium imaging system provided by an embodiment of the present invention;

[0039] Figure 4 It is a schematic diagram of the optical path of a microscopic imaging device of a single-photon kilohertz calcium imaging system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0041] Please refer to Figure 1 , Figure 1 : is a schematic diagram of the structure of a single-photon kilohertz calcium imaging system provided in an embodiment of the present invention. The single-photon kilohertz calcium imaging system 100 includes a microscopic imaging module 101, a data processing module 102 and an upper processing module 103:

[0042] The microscopic imaging module 101 is used to collect high-speed image data for calcium imaging through a microscopic imaging device 1011 including a single photon avalanche diode array sensor 1012 (SPAD);

[0043] The data processing module 102 is used to acquire the high-speed image data, process the high-speed image data, and output a calcium imaging image;

[0044] The host computer module 103 is used to control the image acquisition process of the microscopic imaging module 101 and the image processing process of the data processing module 102 , and to display the calcium imaging image.

[0045] Furthermore, the data processing module 102 is specifically used for:

[0046] Determining the total number of photons in the signal frame included in the high-speed image data;

[0047] Determining the number of synthesized photons used to synthesize different signal frames in the high-speed image data according to the total number of frames by a time binning method;

[0048] According to the number of synthesized photons, synthesizing the high-speed image data in the order of frame signals to obtain synthesized image data;

[0049] The synthesized image data is subjected to noise removal using a preset noise removal method to obtain the calcium imaging image.

[0050] During the implementation of the present invention, according to the photon count of the original single-photon avalanche diode array sensor 1012, there will be 10,000 photon counts in one second, that is, the single-photon avalanche diode array sensor collects 10,000 original bit plane data and 10,000 frames of image data in one second.

[0051] However, since the array data of a single original bit plane is only the photons received by the single photon avalanche diode array sensor 1012 at the current moment, its readout shot noise is too large to accurately identify the image. Therefore, describing this frame of image by the average number of photons of several frames of bit planes will effectively reduce the shot noise. Figure 2 As shown, Figure 2 A in the middle shows the photon count of the original bit plane. Figure 2 Figure B shows that 5 frames of original bit plane data are averaged to form a composite image. In this case, the system output frame rate is actually 2000 frames per second. In the implementation process, the required frame rate will be different according to different application scenarios. Therefore, the size of the time bin can be adjusted according to the scenario. For example, Figure 2 The C in the figure shows that 20 frames of signals are synthesized into 1 frame, so the system output frame rate is 500 Hz. In the implementation process, the single-photon kilohertz calcium imaging system 100 described in the embodiment of the present invention is used to shoot GCaMP fluorescent indicator, and the output frame rate can be controlled to 10-60 frames; if voltage-sensitive dye (a biological indicator) is to be shot, the output frame rate can be controlled to 200 frames.

[0052] Furthermore, the data processing module determines the number of synthesized photons for synthesizing different signal frames in the high-speed image data according to the total number of photons by a time binning method, satisfying the following photon counting equation:

[0053]

[0054] Wherein, t represents the frame duration of the high-speed image data in the time bin, N(t) represents the number of synthesized photons, represents the number of photons at each point in each frame signal of the high-speed image data, and M(t) represents the Martingale noise term of the photon counting equation. Figure 3 As shown, the Martingale noise term of the photon counting equation in the embodiment of the present invention is determined by the error between the photon count in each frame and the expected photon count in the frame, and the expected photon count is determined by the fitting straight line of the count growth.

[0055] The photon counting equation can be used to separate shot noise from effective information, but the existing method still uses total mean processing to extract effective information. This method obviously cannot express the characteristics of the number of photons per frame. Therefore, the embodiment of the present invention adds a flux detection equation based on photon counting. The photon count of each point in each frame is first determined by the flux detection equation, and then put into the photon counting equation for total calculation. Furthermore, the number of photons at each point in each frame signal of the high-speed image data is It is determined by the flux detection equation ρ(τ), which satisfies the following expression:

[0056]

[0057] Where ρ represents the sum of photon counts within the current time bin (0, t), M p (t) represents the Martingale noise term of the flux detection equation.

[0058] The total number of photons in each time bin calculated through the above process reflects the discrete and noise measurement of the time-varying flux. Shot noise is also eliminated as needed during reconstruction, laying the foundation for the accuracy of signal extraction.

[0059] Since each pixel in the image is a point in the array of the single-photon avalanche diode array sensor 1012, the brightness of each point depends on the number of photons received by the point. For time binning, the smaller the time binning is set, the smaller the total number of photons. However, for each frame of the image, if the number of photons is too small, the shot noise generated will also be large. Therefore, in addition to achieving brightness setting by controlling the photon count of the frame data, the embodiment of the present invention also uses the preset denoising method to reduce the shot noise in the image.

[0060] Furthermore, the preset denoising method is at least one of mean filtering, standard deviation filtering, Gaussian filtering, and adaptive filtering. Among them, mean filtering can be expressed as:

[0061]

[0062] Where N j is the time bin T j The number of data points in x i is the value of the ith point, is the time box T j The average of the data points in .

[0063] Standard deviation filtering can be expressed as:

[0064]

[0065] Among them, the variance is Var j and the standard deviation is σ j .

[0066] Gaussian filtering can be expressed as:

[0067]

[0068] where x i is the value of the i-th point, y i Represents the value of the i-th position in the data sequence after Gaussian filtering. The weight is determined by the Gaussian function G(j). Generally, Where σ is the standard deviation, which controls the strength and range of the filtering.

[0069] Adaptive filtering can be expressed as:

[0070]

[0071] in is the mean of the neighborhood, and α is a coefficient adjusted by the local variance. The above preset denoising method can help effectively remove the shot noise of photons while temporally binning, and retain more image details in a mathematically reasonable way.

[0072] Furthermore, the data processing module 102 is also used for:

[0073] The calcium imaging images were converted to RGB format.

[0074] Exemplarily, the optical path schematic diagram of the microscopic imaging device 1011 in the embodiment of the present invention is as follows: Figure 4 As shown, the microscopic imaging device 1011 includes an excitation light source 201, an excitation light filter 202, a beam splitter 203, an imaging achromatic lens 204, an objective lens group 205, a fluorescence filter 206 and a liquid lens 207, wherein:

[0075] The excitation light source 201 is used to emit excitation light to excite the indicator of the target object to produce fluorescence;

[0076] The excitation light filter 202 is used to transmit the first light band of the excitation light;

[0077] The beam splitter 203 is used to reflect the excitation light and transmit the fluorescence;

[0078] The imaging achromatic lens 204 is used to adjust the spherical aberration and aberration of the microscopic imaging device 1011;

[0079] The objective lens group 205 is used to realize the imaging channel of the microscopic imaging device 1011;

[0080] The fluorescence filter 206 is used to transmit the second light band of the fluorescence;

[0081] The liquid lens 207 is used to adjust the focal length of the microscopic imaging device 1011;

[0082] The single photon avalanche diode array sensor 1012 receives the fluorescence for imaging to obtain the high-speed image data.

[0083] Furthermore, the indicator is a genetically encoded voltage indicator GEVIs virus; correspondingly, the excitation light source 201 is an excitation light source for the genetically encoded voltage indicator virus to produce fluorescence, emitting light in the blue light band, and the blue light excites the genetically encoded voltage indicator virus to produce green fluorescence. Since the genetically encoded voltage indicator virus can respond to the spike activity and subthreshold activity of neurons, during the implementation process, the driving response control of the excitation light source 201 needs to be within 0.1 milliseconds.

[0084] The excitation light filter 202 is a blue light filter, and the first light band is a blue light band of 450-490nm, in order to prevent the remaining light bands except the blue light from entering the optical path system;

[0085] The fluorescent filter 206 is a green light filter, and the second light band is a green light band of 510-550 nm, in order to prevent light waves other than the fluorescent band from entering the single photon avalanche diode array sensor 1012 to interfere with imaging.

[0086] During the implementation, preferably, the data processing module 102 is composed of two FPGA chips and one single-chip microcomputer chip, wherein one FPGA chip is mainly used for time binning processing of the single-photon avalanche diode array sensor 1012 signal, and image denoising and data conversion within the time bin; the other FPGA chip is mainly used to implement the data flow channel from RGB to PCIE, where the data flow channel is configured with DDR3 memory to achieve 1G bandwidth, and the FPGA is also responsible for parsing the frame flip signal and forming high and low levels to send out; the single-chip microcomputer is responsible for the parameter setting of the sensor, the light source drive control, the control of the liquid lens, etc.

[0087] Correspondingly, the single-photon avalanche diode array sensor in the microscopic imaging module 101 transmits the array data to the data processing module 102 through two pairs of differential serial port signals; the brightness and start-up signals of the excitation light source are directly connected to the driving circuit in the data processing module 102 through a cluster of extremely fine coaxial lines; the liquid lens module contains an HV892 driver chip, and its control signal is connected to the single-chip microcomputer on the data processing module 102 through an I2C signal;

[0088] The host computer module 103 is implemented by a computer device, mainly controlling the functions of the microscopic imaging module 101 and the image data processing module 102, and displaying the final calcium imaging image to the user, and also adjusting the corresponding parameters. The functions of the host computer module 103 include but are not limited to:

[0089] 1. Set the time box during initial configuration;

[0090] 2. Display image data, and select some details for zooming in;

[0091] 3. Storage of image data in AVI format, which can be decomposed into multiple sub-videos for storage;

[0092] 4. Configure various settings for image storage, such as path, name, etc.;

[0093] 5. Real-time control of excitation light source and liquid lens;

[0094] 6. Display of general input and output ports, frame flip signal display, etc.

[0095] The beneficial effect achieved by the present invention is that a single-photon kilohertz calcium imaging system is proposed. The system utilizes a single-photon avalanche diode sensor array to achieve high-speed image acquisition, adjusts the output frame rate through a time-binning processing method, and combines photon counting and flux detection to achieve image noise processing. Compared with the prior art, the system image output frame rate proposed by the present invention is higher, neural activity can be quantitatively detected, and more accurate information can be provided for neurological research.

[0096] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0097] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0098] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0099] The embodiments of the present invention are described above in conjunction with the accompanying drawings. What is disclosed is only the preferred embodiment of the present invention. However, the present invention is not limited to the above-mentioned specific implementation manner. The above-mentioned specific implementation manner is only illustrative rather than restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms and equivalent changes without departing from the scope of protection of the purpose of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A single-photon kilohertz calcium imaging system, characterized in that: Including microscopic imaging module, data processing module and upper processing module: The microscopic imaging module is used to collect high-speed image data for calcium imaging through a microscopic imaging device including a single-photon avalanche diode array sensor; The data processing module is used to acquire the high-speed image data, process the high-speed image data, and output a calcium imaging image; The host computer module is used to control the image acquisition process of the microscopic imaging module, control the image processing process of the data processing module, and display the calcium imaging image.

2. The single-photon kilohertz calcium imaging system according to claim 1, characterized in that: The data processing module is specifically used for: Determining the total number of photons in the signal frame included in the high-speed image data; Determining the number of synthesized photons used to synthesize different signal frames in the high-speed image data according to the total number of frames by a time binning method; According to the number of synthesized photons, synthesizing the high-speed image data in the order of frame signals to obtain synthesized image data; The synthesized image data is subjected to noise removal using a preset noise removal method to obtain the calcium imaging image.

3. The single-photon kilohertz calcium imaging system according to claim 2, characterized in that: The data processing module determines the number of synthesized photons for synthesizing different signal frames in the high-speed image data according to the total number of photons by a time binning method, satisfying the following photon counting equation: Wherein, t represents the frame duration of the high-speed image data in the time bin, N(t) represents the number of synthesized photons, represents the number of photons at each point in each frame signal of the high-speed image data, and M(t) represents the Martingale noise term of the photon counting equation.

4. The single-photon kilohertz calcium imaging system according to claim 3, characterized in that: The number of photons at each point in each frame signal of the high-speed image data It is determined by the flux detection equation ρ(τ), which satisfies the following expression: Where ρ represents the sum of photon counts within the current time bin (0, t), M p (t) represents the Martingale noise term of the flux detection equation.

5. The single-photon kilohertz calcium imaging system according to claim 2, characterized in that: The preset denoising method is at least one of mean filtering, standard deviation filtering, Gaussian filtering, and adaptive filtering.

6. The single-photon kilohertz calcium imaging system according to claim 1, characterized in that: The data processing module is also used for: The calcium imaging images were converted to RGB format.

7. The single-photon kilohertz calcium imaging system according to claim 1, characterized in that: The microscopic imaging device comprises an excitation light source, an excitation light filter, a beam splitter, an imaging achromatic lens, an objective lens group, a fluorescence filter and a liquid lens, wherein: The excitation light source is used to emit excitation light to excite the indicator of the target object to produce fluorescence; The excitation light filter is used to transmit the first light band of the excitation light; The beam splitter is used to reflect the excitation light and transmit the fluorescence; The imaging achromatic lens is used to adjust the spherical aberration and aberration of the microscopic imaging device; The objective lens group is used to realize the imaging channel of the microscopic imaging device; The fluorescence filter is used to transmit the second light band of the fluorescence; The liquid lens is used to adjust the focal length of the microscopic imaging device; The single photon avalanche diode array sensor receives the fluorescence to perform imaging and obtain the high-speed image data.

8. The single-photon kilohertz calcium imaging system according to claim 7, characterized in that: The indicator is a genetically encoded voltage indicator GEVIs virus; The excitation light filter is a blue light filter, and the first light band is a blue light band of 450-490 nm; The fluorescent filter is a green light filter, and the second light band is a green light band of 510-550 nm.

Images