Single-photon kilohertz calcium imaging system

Abstract

The application relates to the technical field of scientific imaging, in particular to a single-photon kilohertz calcium imaging system, which comprises a microscopic imaging module, a data processing module and an upper processing module; the microscopic imaging module is used for collecting high-speed image data for calcium imaging through a microscopic imaging device containing a single-photon avalanche diode array sensor; the data processing module is used for acquiring the high-speed image data, processing the high-speed image data and outputting a calcium imaging image; the upper computer module is used for controlling the image collection process of the microscopic imaging module, controlling the image processing process of the data processing module and displaying the calcium imaging image. The system proposed in the application has a higher image output frame rate, can quantitatively detect neural activity and provides more accurate information for the research of neurology.

Description

Technical Field

[0001] This invention relates to the field of scientific imaging technology, and in particular to a single-photon kilohertz calcium imaging system. Background Technology

[0002] High-precision spatiotemporal measurement of neural activity is closely related to the accuracy of brain analysis. Currently widely used calcium imaging relies on fluorescent proteins, recording calcium activity as a cumulative result of multiple neural activities. The brightness changes of each neuron have long cycles, and the intensity of neuronal activity is determined by these brightness variations. While this method can determine the intensity of neural activity through changes in fluorescence brightness, it cannot accurately capture the cycle, rate, and density of neural activity. Gene-encoded voltage indicators (GEVIs) are a rapid voltage-sensitive staining method capable of resolving neural activity at millisecond resolution. These rate indicators can display each action potential of a neuron in light form. Compared to traditional fluorescent gene-encoded calcium indicators, these gene-encoded voltage indicators offer the possibility of high spatiotemporal precision in resolving neural activity.

[0003] To date, electron-multiplying CCD (emCCD) and scientific CMOS (sCMOS) cameras have been commonly used for imaging neuronal calcium. However, while these sensors can achieve image recording rates up to 100 frames per second, the exposure and refresh rates severely limit the acquisition of images at higher frame rates. At higher frame rate requirements, the camera may amplify noise in the electronic pathways, resulting in excessive noise in the image and obscuring the true signal. State-of-the-art emCCD cameras have relatively low readout noise and can achieve single-photon sensitivity; however, the limited frame rate of emCCD cameras (approximately 100 frames per second) does not allow for kilohertz frame rate imaging, and the amplification process also introduces noise, effectively reducing the signal-to-noise ratio and their quantum efficiency. State-of-the-art sCMOS, due to inherent dark noise and readout noise, struggles to distinguish weak signals generated by a limited number of incident photons from stray noise events at high frame rates. Furthermore, with the high speed of GEVIs, traditional CMOS image recording methods are increasingly inadequate for meeting the demands of high-precision neural activity analysis. Summary of the Invention

[0004] This invention provides a single-photon kilohertz calcium imaging system, which aims to solve the problem that existing CMOS image recording methods cannot meet the requirements for high-precision neural activity analysis.

[0005] To address the aforementioned technical problems, this invention provides a single-photon kilohertz calcium imaging system, comprising a microscopic imaging module, a data processing module, and a host processing module:

[0006] The microscopic imaging module is used to acquire high-speed image data for calcium imaging through a microscopic imaging device containing a single-photon avalanche diode array sensor.

[0007] The data processing module is used to acquire the high-speed image data, process the high-speed image data, and output a calcium imaging image.

[0008] The host computer module is used to control the image acquisition process of the microscopic imaging module, control the image processing process of the data processing module, and display the calcium imaging image.

[0009] Furthermore, the data processing module is specifically used for:

[0010] Determine the total number of photons in the signal frames contained in the high-speed image data;

[0011] The number of synthetic photons used to synthesize different signal frames in the high-speed image data is determined based on the total number of frames using a time-binding method.

[0012] Based on the number of synthesized photons, the high-speed image data is synthesized in the order of the frame signals to obtain synthesized image data;

[0013] The synthesized image data is subjected to noise removal using a preset denoising method to obtain the calcium imaging image.

[0014] Furthermore, the data processing module determines the number of synthesized photons used to synthesize different signal frames in the high-speed image data based on the total number of photons using a time-binding method, satisfying the following photon counting equation:

[0015]

[0016] Where t represents the frame duration of the high-speed image data within a time bin, and N(t) represents the number of synthesized photons. M(t) represents the number of photons at each point in each frame of the high-speed image data, and M(t) represents the Martingale noise term in the photon counting equation.

[0017] Furthermore, the number of photons at each point in each frame of the high-speed image data signal. The flux detection equation ρ(τ) is used to determine the following expression:

[0018]

[0019] Where ρ represents the sum of photon counts within the current time bin (0, t), and M p(t) represents the Martingale noise term in the flux detection equation.

[0020] Furthermore, the preset denoising method is at least one of mean filtering, standard deviation filtering, Gaussian filtering, and adaptive filtering.

[0021] Furthermore, the data processing module is also used for:

[0022] The calcium imaging image is converted to RGB format.

[0023] Furthermore, the microscopic imaging device includes an excitation light source, an excitation light filter, a beam splitter, an imaging achromatic lens, an objective lens group, a fluorescence filter, and a liquid lens, wherein:

[0024] The excitation light source is used to emit excitation light to excite the indicator of the target object to produce fluorescence;

[0025] The excitation light filter is used to transmit the first optical band of the excitation light;

[0026] The beam splitter is used to reflect the excitation light and transmit the fluorescence;

[0027] The imaging achromatic lens is used to adjust the spherical aberration and aberration of the microscopic imaging device;

[0028] The objective lens group is used to realize the imaging channel of the microscopic imaging device;

[0029] The fluorescent filter is used to transmit the second wavelength of the fluorescence;

[0030] The liquid lens is used to adjust the focal length of the microscopic imaging device;

[0031] The single-photon avalanche diode array sensor obtains the high-speed image data by receiving the fluorescence.

[0032] Furthermore, the indicator is a gene-encoded voltage indicator GEVI virus;

[0033] The excitation light filter is a blue light filter, and the first light band is the blue light 450-490nm band;

[0034] The fluorescent filter is a green light filter, and the second light band is the green light 510-550nm band.

[0035] The beneficial effects achieved by this invention lie in proposing a single-photon kilohertz calcium imaging system. This system utilizes a single-photon avalanche diode sensor array to achieve high-speed image acquisition and adjusts the output frame rate through time-binning processing. Combined with photon counting and flux detection, it achieves image noise processing. Compared with the prior art, the system proposed in this invention has a higher image output frame rate, which can quantitatively detect neural activity and provide more accurate information for neurological research. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the structure of the single-photon kilohertz calcium imaging system provided in an embodiment of the present invention;

[0037] Figure 2 This is a schematic diagram of the output frame rate synthesis of the single-photon kilohertz calcium imaging system provided in an embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of the Martingale noise term in the photon counting equation of the single-photon kilohertz calcium imaging system provided in this embodiment of the invention;

[0039] Figure 4 This is a schematic diagram of the optical path of the microscopic imaging device of the single-photon kilohertz calcium imaging system provided in an embodiment of the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0041] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the structure of a single-photon kilohertz calcium imaging system provided in an embodiment of the present invention. The single-photon kilohertz calcium imaging system 100 includes a microscopic imaging module 101, a data processing module 102, and a host processing module 103.

[0042] The microscopic imaging module 101 is used to acquire high-speed image data for calcium imaging through a microscopic imaging device 1011 containing a single-photon avalanche diode array sensor 1012 (SPAD).

[0043] The data processing module 102 is used to acquire the high-speed image data, process the high-speed image data, and output a calcium imaging image.

[0044] The host computer module 103 is used to control the image acquisition process of the microscopic imaging module 101, control the image processing process of the data processing module 102, and display the calcium imaging image.

[0045] Furthermore, the data processing module 102 is specifically used for:

[0046] Determine the total number of photons in the signal frames contained in the high-speed image data;

[0047] The number of synthetic photons used to synthesize different signal frames in the high-speed image data is determined based on the total number of frames using a time-binding method.

[0048] Based on the number of synthesized photons, the high-speed image data is synthesized in the order of the frame signals to obtain synthesized image data;

[0049] The synthesized image data is subjected to noise removal using a preset denoising method to obtain the calcium imaging image.

[0050] During the implementation of this invention, based on the photon count of the original single-photon avalanche diode array sensor 1012, there will be 10,000 photon counts in one second, that is, the single-photon avalanche diode array sensor collects 10,000 original bit plane data and 10,000 frames of image data in one second.

[0051] However, since the array data of a single raw bit plane only represents the photons received by the single-photon avalanche diode array sensor 1012 at the current moment, its readout shot noise is too large to accurately identify the image. Therefore, describing the image frame by the average number of photons across several bit planes effectively reduces shot noise. Figure 2 As shown, Figure 2 Image A shows the photon count of the original bit plane. Figure 2 Image B shows the result of averaging five frames of raw bit-plane data to form a composite image. In this case, the system's actual output frame rate is 2000 frames per second. During implementation, the required frame rate will vary depending on the application scenario. Therefore, the size of the time bins can be adjusted according to the scenario. For example... Figure 2 The image shown in Figure C illustrates the case where 20 frames of signal are combined into 1 frame; therefore, the system output frame rate is 500Hz. During implementation, when using the single-photon kilohertz calcium imaging system 100 described in this embodiment of the invention to image GCaMP fluorescent indicators, the output frame rate can be controlled to 10-60 frames; if imaging a voltage-sensitive dye (a biological indicator) is required, the output frame rate can be controlled to 200 frames.

[0052] Furthermore, the data processing module determines the number of synthesized photons used to synthesize different signal frames in the high-speed image data based on the total number of photons using a time-binding method, satisfying the following photon counting equation:

[0053]

[0054] Where t represents the frame duration of the high-speed image data within a time bin, and N(t) represents the number of synthesized photons. Let M(t) represent the number of photons at each point in each frame of the high-speed image data, and M(t) represent the Martingale noise term in the photon counting equation. Wherein, as... Figure 3 As shown, in this embodiment of the invention, the Martingale noise term of the photon counting equation is determined by the error between the photon count in each frame and the expected photon count in that frame, and the expected photon count is determined by a fitted straight line of count growth.

[0055] Photon counting equations can be used to separate shot noise from effective information. However, existing methods still use total average processing to extract effective information, which obviously cannot express the characteristics of the number of photons in each frame. Therefore, this embodiment of the invention adds a flux detection equation to the photon counting process. The photon count at each point in each frame is first determined by the flux detection equation, and then fed into the photon counting equation for total calculation. Furthermore, the number of photons at each point in each frame of the high-speed image data... The flux detection equation ρ(τ) is used to determine the following expression:

[0056]

[0057] Where ρ represents the sum of photon counts within the current time bin (0, t), and M p (t) represents the Martingale noise term in the flux detection equation.

[0058] The total number of photons in each time box calculated through the above process reflects the discreteness of time-varying flux and noise measurement. It also helps to remove shot noise as needed during reconstruction, laying the foundation for the accuracy of signal extraction.

[0059] Since each pixel in the image is a point in the single-photon avalanche diode array sensor 1012, the brightness of each point depends on the number of photons received at that point. For time bins, the smaller the time bin setting, the fewer the total number of photons. However, for each frame of the image, if the number of photons is too small, the resulting shot noise will be large. Therefore, in addition to controlling the photon count of the frame data to achieve brightness setting, this embodiment of the invention also uses the aforementioned preset denoising method to reduce shot noise in the image.

[0060] Furthermore, the preset denoising method is at least one of mean filtering, standard deviation filtering, Gaussian filtering, and adaptive filtering. Mean filtering can be expressed as:

[0061]

[0062] Where N j It is a time-divided box T j The number of data points in the middle, x i It is the value of the i-th point. It is a time box T j The average value of the data points.

[0063] Standard deviation filtering can be expressed as:

[0064]

[0065] Wherein, the variance is Var j The sum and standard deviation are σ j .

[0066] Gaussian filtering can be expressed as:

[0067]

[0068] Where x i It is the value of the i-th point, y i This represents the value at the i-th position in the data sequence after Gaussian filtering. The weight is determined by the Gaussian function G(j), generally... Where σ is the standard deviation, which controls the strength and range of the filter.

[0069] Adaptive filtering can be expressed as:

[0070]

[0071] in α is the mean of the neighborhood, and α is a coefficient adjusted by the local variance. The above-mentioned preset denoising method can help effectively remove shot noise from photons while performing temporal binning, and preserve more image details in a mathematically sound manner.

[0072] Furthermore, the data processing module 102 is also used for:

[0073] The calcium imaging image is converted to RGB format.

[0074] For example, the optical path diagram of the microscopic imaging device 1011 in this embodiment of the invention is as follows: Figure 4 As shown, the microscopic imaging device 1011 includes an excitation light source 201, an excitation light filter 202, a beam splitter 203, an imaging achromatic lens 204, an objective lens group 205, a fluorescence filter 206, and a liquid lens 207, wherein:

[0075] The excitation light source 201 is used to emit excitation light to excite the indicator of the target object to produce fluorescence;

[0076] The excitation light filter 202 is used to transmit the first optical band of the excitation light;

[0077] The beam splitter 203 is used to reflect the excitation light and transmit the fluorescence;

[0078] The imaging achromatic lens 204 is used to adjust the spherical aberration and aberration of the microscopic imaging device 1011;

[0079] The objective lens group 205 is used to realize the imaging channel of the microscopic imaging device 1011;

[0080] The fluorescent filter 206 is used to transmit the second wavelength of the fluorescence;

[0081] The liquid lens 207 is used to adjust the focal length of the microscopic imaging device 1011;

[0082] The single-photon avalanche diode array sensor 1012 obtains the high-speed image data by receiving the fluorescence.

[0083] Furthermore, the indicator is the gene-encoded voltage indicator (GEVIs) virus; correspondingly, the excitation light source 201 is the excitation light source for the GEVIs virus to produce fluorescence, emitting light in the blue light band, which excites the GEVIs virus to produce green fluorescence. Since the GEVIs virus can respond to the spike activity and subthreshold activity of neurons, the driving response control of the excitation light source 201 needs to be within 0.1 milliseconds during implementation.

[0084] The excitation light filter 202 is a blue light filter, and the first light band is the blue light 450-490nm band. This is to prevent other light bands besides blue light from entering the optical path system.

[0085] The fluorescent filter 206 is a green light filter, and the second light band is the green light 510-550nm band. In order to prevent light waves other than the fluorescent band from entering the single-photon avalanche diode array sensor 1012 and interfering with imaging.

[0086] In practice, the preferred data processing module 102 consists of two FPGA chips and one microcontroller chip. One FPGA chip mainly performs time-binning processing of the signal from the single-photon avalanche diode array sensor 1012, and also performs image denoising and data conversion within the time-bin. The other FPGA chip mainly implements the RGB to PCIe data stream channel, which is configured with DDR3 memory to achieve a bandwidth of 1G. This FPGA is also responsible for parsing the frame flip signal and generating high and low levels for external transmission. The microcontroller is responsible for setting the sensor parameters, controlling the light source drive, and controlling the liquid lens.

[0087] Correspondingly, the single-photon avalanche diode array sensor in the microscopic imaging module 101 transmits array data to the data processing module 102 through two pairs of differential serial port signals; the brightness and start-up signals of the excitation light source are directly connected to the drive circuit in the data processing module 102 through a cluster of ultra-fine coaxial cables; the liquid lens module contains an HV892 driver chip, whose control signals are connected to the microcontroller on the data processing module 102 through I2C signals.

[0088] The host computer module 103, implemented via a computer, primarily controls the functions of the microscopic imaging module 101 and the image data processing module 102, displays the final calcium imaging image to the user, and adjusts the corresponding parameters. The functions of the host computer module 103 include, but are not limited to:

[0089] 1. Set the timebox during initial configuration;

[0090] 2. Displays image data, allowing users to select and zoom in on specific details;

[0091] 3. Image data is stored in AVI format and can be broken down into multiple sub-videos for storage;

[0092] 4. Configure various settings for image storage, such as path and name;

[0093] 5. Real-time control of the excitation source and liquid lens;

[0094] 6. Display of general input / output ports, display of frame toggle signals, etc.

[0095] The beneficial effects achieved by this invention lie in proposing a single-photon kilohertz calcium imaging system. This system utilizes a single-photon avalanche diode sensor array to achieve high-speed image acquisition and adjusts the output frame rate through time-binning processing. Combined with photon counting and flux detection, it achieves image noise processing. Compared with the prior art, the system proposed in this invention has a higher image output frame rate, which can quantitatively detect neural activity and provide more accurate information for neurological research.

[0096] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0097] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0099] The embodiments of the present invention have been described above with reference to the accompanying drawings. The disclosed embodiments are merely preferred embodiments of the present invention. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many equivalent changes in form without departing from the spirit and scope of the claims of the present invention, and all such changes are within the protection scope of the present invention.

Claims

1. A single-photon kilohertz calcium imaging system, characterized in that, It includes a microscopic imaging module, a data processing module, and a host processing module: The microscopic imaging module is used to acquire high-speed image data for calcium imaging through a microscopic imaging device containing a single-photon avalanche diode array sensor. The data processing module is used to acquire the high-speed image data, process the high-speed image data, and output a calcium imaging image. The host processing module is used to control the image acquisition process of the microscopic imaging module, control the image processing process of the data processing module, and display the calcium imaging image. The data processing module is specifically used for: Determine the total number of photons in the signal frames contained in the high-speed image data; The number of synthesized photons used to synthesize different signal frames in the high-speed image data is determined based on the total number of photons by using a time-binding method. Based on the number of synthesized photons, the high-speed image data is synthesized in the order of the frame signals to obtain synthesized image data; The synthesized image data is subjected to noise removal using a preset denoising method to obtain the calcium imaging image; The data processing module determines the number of synthesized photons used to synthesize different signal frames in the high-speed image data based on the total number of photons using a time-binding method, satisfying the following photon counting equation: ； in, This indicates the frame duration of the high-speed image data within a time bin. This indicates the number of synthesized photons. This represents the number of photons at each point in each frame of the high-speed image data. Represents the Martingale noise term in the photon counting equation; The number of photons at each point in each frame of the high-speed image data Through flux detection equation The flux detection equation is determined. Satisfy the following expression: ； in, Indicates the current time bin (0, The sum of photon counts within a time period. This represents the Martingale noise term in the flux detection equation.

2. The single-photon kilohertz calcium imaging system according to claim 1, characterized in that, The preset denoising method is at least one of mean filtering, standard deviation filtering, Gaussian filtering, and adaptive filtering.

3. The single-photon kilohertz calcium imaging system according to claim 1, characterized in that, The data processing module is also used for: The calcium imaging image is converted to RGB format.

4. The single-photon kilohertz calcium imaging system according to claim 1, characterized in that, The microscopic imaging device includes an excitation source, an excitation light filter, a beam splitter, an imaging achromatic lens, an objective lens group, a fluorescence filter, and a liquid lens, wherein: The excitation light source is used to emit excitation light to excite the indicator of the target object to produce fluorescence; The excitation light filter is used to transmit the first optical band of the excitation light; The beam splitter is used to reflect the excitation light and transmit the fluorescence; The imaging achromatic lens is used to adjust the spherical aberration and aberration of the microscopic imaging device; The objective lens group is used to realize the imaging channel of the microscopic imaging device; The fluorescent filter is used to transmit the second wavelength of the fluorescence; The liquid lens is used to adjust the focal length of the microscopic imaging device; The single-photon avalanche diode array sensor obtains the high-speed image data by receiving the fluorescence.

5. The single-photon kilohertz calcium imaging system according to claim 4, characterized in that, The indicator is the gene-encoded voltage indicator GEVIs virus; The excitation light filter is a blue light filter, and the first light band is the blue light 450-490nm band; The fluorescent filter is a green light filter, and the second light band is the green light 510-550nm band.

Images