A light sheet microscopy imaging system and method based on snapshot time compression

Abstract

The present invention discloses a light-sheet microscopy imaging system and method based on snapshot time compression, comprising: a light-sheet fluorescence signal excitation module for generating two non-diffracting Bessel-like light sheets with a focal depth range greater than the Rayleigh distance of a Gaussian beam, collimating them and illuminating both sides of a sample in opposite directions to excite the sample to generate fluorescence; a fluorescence signal encoding module for encoding and modulating the fluorescence signal generated by the light-sheet fluorescence signal excitation module; a fluorescence signal detection module for matching pixels at the single-pixel level and obtaining a fluorescence compressed snapshot imaging pattern according to multiple encoded fluorescence signals collected within a single-frame exposure time; and a fluorescence signal decoding module for decoding the collected fluorescence compressed snapshot imaging pattern to obtain a high-resolution and high-frame-rate fluorescence video image to be restored. The present invention solves the problems of low signal-to-noise ratio caused by short exposure time and the need for a large data bandwidth in high-speed and high-resolution microscopic transient scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of fluorescence microscopy imaging, and particularly relates to a light-sheet microscopy imaging system and method based on snapshot time compression. Background Art

[0002] Light-sheet fluorescence microscopy uses a special design in which the excitation light path and the detection light path are perpendicular to each other. That is, a thin-sheet light beam is used to excite the sample from the side, and a fluorescence image of the illuminated plane is obtained through a microscope objective and a detector perpendicular to the light-sheet direction, thereby avoiding the interference of fluorescence signals in non-imaging planes. It has the characteristics of low background noise, high resolution, large observation field of view, low photobleaching and low phototoxicity, and is suitable for high-quality and long-term dynamic observation of relatively large living biological samples. At present, light-sheet microscopy imaging has been widely used in scientific research such as cell biology, developmental biology, and neurobiology. In addition to imaging performance such as spatial resolution and observation field of view, the imaging speed (two-dimensional imaging speed and three-dimensional imaging speed) is another very important indicator for measuring the performance of light-sheet fluorescence microscopy. Limited by the camera acquisition speed and the short exposure time required for high-speed imaging, which reduces the image signal-to-noise ratio, it is difficult for existing light-sheet microscopy imaging technologies to detect high-speed and high-resolution microscopic transient scene images. With the development of compressive sensing technology, snapshot time compression imaging technology has become a low-cost and effective method to break through the camera acquisition speed and capture high-speed and high-resolution scenes.

[0003] Snapshot time compression imaging technology is divided into optical hardware encoding and software decoding. Optical hardware encoding refers to generating N random masks using a digital micromirror device or other modulators such as a spatial light modulator within the single exposure time of a two-dimensional detector, and then spatially encoding N rapidly changing scenes, that is, encoding and compressing N scenes into a single-frame measurement image. Software decoding refers to reconstructing the encoded N original scene images from a single measurement image using a corresponding image reconstruction method. Therefore, snapshot compression imaging technology can increase the image acquisition speed by N times, breaking through the limitation of the camera acquisition speed, and has the advantages of low bandwidth, high signal-to-noise ratio, fast acquisition speed, low cost, and low power consumption.

[0004] Applying snapshot time compression imaging technology to the field of light sheet fluorescence microscopy mainly faces two technical challenges. One is the design of the optical hardware encoding optical path, and the other is the design of the software decoding method. In terms of the design of the optical hardware encoding optical path, the pixel mapping relationship between the modulator and the detector is the primary factor affecting the resolution of the decoded image. Existing video snapshot compression imaging technologies do not perform pixel matching between the modulator and the detector, which easily leads to phenomena such as cross-talk and blurred images in the decoded microscopic images. In terms of software decoding design, the time required for decoding and the quality of the decoded image are the main factors limiting the wide application of snapshot time compression imaging technology. Existing decoding algorithms mainly include iterative decoding methods, end-to-end deep learning decoding methods, deep image prior decoding methods without training, and plug-and-play decoding methods (PnP) that combine iterative methods with pre-trained deep denoisers. The iterative decoding method based on generalized alternating projection of total variation (GAP-TV) has a fast decoding speed and can retain image details, but the decoded image has high noise. The end-to-end deep learning decoding method (E2E) is the fastest and has high decoded image quality among existing decoding methods. However, in addition to requiring a large amount of training data (which is not easy to obtain for high-speed microscopes), this method is not flexible because whenever the sensing matrix (encoding mask) changes, a new network must be retrained. The deep image prior (DIP) decoding method without training can obtain clean and clear decoded images, but the deep image prior still requires (usually thousands of times) iterations to optimize the parameters in the network, so the decoding speed of this method is slow. The decoding method using a pre-trained deep denoiser in the iterative optimization framework has a fast decoding speed and high decoded image quality. However, these deep denoisers are usually trained from natural images and perform poorly in microscopic image decoding.

[0005] Therefore, the existing technology still needs to be improved. Summary of the Invention

[0006] The technical problem to be solved by the present invention is that, aiming at the defects of the existing technology, the present invention provides a light sheet microscopy system and method based on snapshot time compression to solve the problem of low signal-to-noise ratio caused by short exposure time and the need for a large data bandwidth in high-speed and high-resolution microscopic transient scenarios.

[0007] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0008] In the first aspect, the present invention provides a light sheet microscopy system based on snapshot time compression, including:

[0009] A light sheet fluorescence signal excitation module for generating two non-diffracting Bessel-like light sheets with a focal depth range greater than the Rayleigh distance of a Gaussian beam, collimating them and illuminating both sides of the sample in opposite directions to excite the sample to generate fluorescence;

[0010] A fluorescence signal encoding module, which is used to encode and modulate the fluorescence signal generated by the light sheet fluorescence signal excitation module;

[0011] A fluorescence signal detection module, which is used to match pixels at the single pixel level, and obtain a fluorescence compressed snapshot imaging pattern according to multiple encoded fluorescence signals collected within a single frame exposure time;

[0012] A fluorescence signal decoding module, which is used to decode the collected fluorescence compressed snapshot imaging pattern to obtain a high-resolution and high-frame-rate fluorescence video image to be restored;

[0013] The fluorescence signal encoding module is connected to the light sheet fluorescence signal excitation module, and the fluorescence signal detection module is connected to the fluorescence signal encoding module; the light sheet fluorescence signal excitation module, the fluorescence signal encoding module, and the fluorescence signal detection module are respectively connected to the fluorescence signal decoding module.

[0014] In one implementation, the light sheet fluorescence signal excitation module includes: a laser, a beam expander and collimator unit, a first mirror, a beam splitter, a first optical path component, a second optical path component, and a sample cell;

[0015] The laser beam generated by the laser is expanded and collimated by the beam expander and collimator unit, and then the expanded and collimated laser beam is split into a first Gaussian beam and a second Gaussian beam via the first mirror and the beam splitter;

[0016] The first Gaussian beam forms a first type of non-diffracting Bessel beam through the first optical path component, and the second Gaussian beam forms a second type of non-diffracting Bessel beam through the second optical path component. The first type of non-diffracting Bessel beam and the second type of non-diffracting Bessel beam respectively excite the sample in the sample cell to generate corresponding fluorescence.

[0017] In one implementation, the first optical path component includes: a first multi-slit optical mask, a first cylindrical lens, and a first objective lens;

[0018] The first Gaussian beam forms a first type of non-diffracting Bessel beam with a focal depth range greater than the Rayleigh distance of the Gaussian beam through the first optical multi-slit mask, the first cylindrical lens, and the first objective lens;

[0019] The second optical path component includes: a second mirror, a third mirror, a fourth mirror, a second multi-slit optical mask, a second cylindrical lens, and a second objective lens;

[0020] The second Gaussian beam forms a second non-diffracting Bessel beam with a depth of focus greater than the Rayleigh distance of the Gaussian beam and propagating in the opposite direction to that of the first type of non-diffracting Bessel beam after passing through the second mirror, the third mirror, the fourth mirror, the second multi-slit optical mask, the second cylindrical lens, and the second objective lens.

[0021] In one implementation, the fluorescence signal encoding module includes: a third objective lens, a tube lens, a fifth mirror, and a digital micromirror device;

[0022] The dynamic fluorescence signal excited by the fluorescence excitation module is collected by the third objective lens and the tube lens, reflected by the fifth mirror to the panel of the digital micromirror device, and spatially encoded by the random mask loaded by the digital micromirror device.

[0023] In one implementation, the fluorescence signal detection module includes: a filter, a zoom lens group, a detector, and a four-dimensional fine adjustment platform;

[0024] The detector is arranged on the four-dimensional fine adjustment platform. The magnification of the four-dimensional fine adjustment platform and the zoom lens group is adjusted to achieve pixel matching at the single-pixel level between the detector and the digital micromirror device;

[0025] After pixel matching at the single-pixel level between the digital micromirror device and the detector, within the single-frame exposure time of the detector, multiple encoded fluorescence signals are collected by the detector after passing through the filter and the zoom lens group, and a compressed snapshot imaging pattern is obtained.

[0026] In one implementation, the fluorescence signal decoding module includes: a projection decoding unit and a multi-stage joint noise reduction unit;

[0027] The projection decoding unit and the multi-stage joint noise reduction unit are used to substitute the compressed snapshot imaging picture and the random mask pattern into the projection formula for decoding to obtain multiple initial fluorescence signal images, and a high-frame-rate fluorescence video image to be measured is reconstructed based on the multiple initial fluorescence signal images.

[0028] In one implementation, it further includes:

[0029] System control module: a four-dimensional electric displacement stage, a NI acquisition card, and a control terminal;

[0030] The four-dimensional electric displacement stage is connected to the control terminal through a data cable, and the four-dimensional electric displacement stage is used to control the movement of the sample in the sample cell;

[0031] The NI acquisition card is respectively connected to the digital micromirror device, the detector, and the control terminal to realize synchronization between the digital micromirror device and the detector;

[0032] The control terminal is connected to the digital micromirror device. The control terminal is used to load a pre-designed random mask onto the digital micromirror device. The control terminal is connected to the detector through a data line, and the control terminal is used to transmit the image collected by the detector to a computer.

[0033] In a second aspect, the present invention further provides a light sheet microscopy imaging method based on snapshot time compression, which is applied to the light sheet microscopy imaging system based on snapshot time compression as described in the first aspect, and includes:

[0034] Convert a Gaussian beam into a non-diffracting Bessel-like light sheet with a depth of focus greater than the Rayleigh distance of the Gaussian beam, and project the non-diffracting Bessel-like light sheet onto the sample surface to excite the sample to generate a fluorescence signal;

[0035] Perform spatial encoding on the fluorescence signal;

[0036] Perform pixel matching at the single-pixel level between the digital micromirror device and the detector, and within the single-frame exposure time of the detector, collect multiple encoded fluorescence signals to obtain a compressed snapshot imaging pattern;

[0037] Decode the compressed snapshot imaging pattern to obtain a high-frame-rate fluorescence video image to be restored.

[0038] In one implementation, the decoding of the compressed snapshot imaging pattern includes:

[0039] Substitute the collected fluorescence compressed snapshot imaging pattern and the random mask pattern into the projection formula to decode and obtain an initial high-speed fluorescence video frame to be restored;

[0040] The projection formula is:

[0041] x (k+1) = v (k) + H T (HH T ) -1 (y - Hv (k) ),

[0042] where H represents the random mask pattern collected by the detector, y represents the fluorescence compressed snapshot imaging pattern, x represents the initial high-speed fluorescence video frame to be restored, and k represents the current iteration number.

[0043] In one implementation, the obtaining of the high-frame-rate fluorescence video image to be restored includes:

[0044] Perform multi-level joint noise reduction processing on the initial high-speed fluorescence video frame to be restored to reconstruct the high-frame-rate fluorescence video image to be restored.

[0045] The present invention adopting the above technical solution has the following effects:

[0046] The present invention realizes the pixel matching process at the single pixel level between the digital micromirror device and the detector in the fluorescence signal detection module, solves the problem of mutual crosstalk and blurred images in the decoded microscopic images, and also realizes fast decoding and improves the resolution and signal-to-noise ratio of the decoded images in the fluorescence signal decoding module, making it have the advantages of fast decoding speed, high signal-to-noise ratio, small bandwidth, low cost, simple system, high frame rate, etc., overcomes the defects of the existing light-sheet microscopy system that cannot break through the limitation of the camera acquisition speed during high-speed imaging, has low signal-to-noise ratio due to short exposure time and requires a large data bandwidth, thereby realizing the capture of high-speed and high-resolution microscopic transient scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the structures shown in these drawings.

[0048] Figure 1 is a schematic diagram of a light-sheet microscopy system based on snapshot time compression in one implementation manner of the present invention.

[0049] Figure 2 is a schematic diagram of the optical path of the fluorescence signal encoding and detection module of the light-sheet microscopy based on snapshot time compression in one implementation manner of the present invention.

[0050] Figure 3 is a flowchart of a light-sheet microscopy method based on snapshot time compression in one implementation manner of the present invention.

[0051] In the figure:

[0052] 1. Laser; 2. First lens; 3. Second lens; 4. First mirror; 5. Beam splitter; 6. First multi-slit optical mask; 7. First cylindrical lens; 8. First objective lens; 9. Second mirror; 10. Third mirror; 11. Fourth mirror; 12. Second multi-slit optical mask; 13. Second cylindrical lens; 14. Second objective lens; 15. Sample cell; 16. Third objective lens; 17. Tube lens; 18. Fifth mirror; 19. Digital micromirror device; 20. Filter; 21. Third lens; 22. Fourth lens; 23. Detector; 24. Four-dimensional fine adjustment platform; 25. Four-dimensional electric displacement stage; 26. NI acquisition card; 27. Control terminal.

[0053] The realization, functional features, and advantages of the present invention will be further described in conjunction with embodiments and with reference to the accompanying drawings. Detailed Embodiment

[0054] To make the objectives, technical solutions, and advantages of the present invention clearer and more explicit, the following further elaborates on the present invention by way of examples with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0055] Exemplary System

[0056] An embodiment of the present invention provides a light-sheet microscopy imaging system based on snapshot time compression, including:

[0057] A light-sheet fluorescence signal excitation module, a fluorescence signal encoding module, a fluorescence signal detection module, and a fluorescence signal decoding module; the fluorescence signal encoding module is connected to the light-sheet fluorescence signal excitation module, and the fluorescence signal detection module is connected to the fluorescence signal encoding module; the light-sheet fluorescence signal excitation module, the fluorescence signal encoding module, and the fluorescence signal detection module are respectively connected to the fluorescence signal decoding module.

[0058] Among them, the light-sheet fluorescence signal excitation module is used to generate two non-diffracting Bessel light-sheets with a depth of focus greater than the Rayleigh distance of a Gaussian beam, which are collimated and illuminated on both sides of the sample in opposite directions to excite the sample to generate fluorescence; the fluorescence signal encoding module is used to encode and modulate the fluorescence signal generated by the light-sheet fluorescence signal excitation module; the fluorescence signal detection module is used to match pixels at the single-pixel level and obtain a fluorescence compressed snapshot imaging pattern based on multiple encoded fluorescence signals collected within a single-frame exposure time; the fluorescence signal decoding module is used to decode the collected fluorescence compressed snapshot imaging pattern to obtain a high-resolution and high-frame-rate fluorescence video image to be restored.

[0059] As Figure 1 shown, the light-sheet fluorescence signal excitation module includes:

[0060] Laser 1, a beam expansion and collimation unit, a first mirror 4, a beam splitter 5, a first multi-slit optical mask 6, a first cylindrical lens 7, a first objective lens 8, a second mirror 9, a third mirror 10, a fourth mirror 11, a second multi-slit optical mask 12, a second cylindrical lens 13, a second objective lens 14, and a sample cell 15; among them, the beam expansion and collimation unit includes: a first lens 2 and a second lens 3.

[0061] Specifically, the process of exciting the fluorescence signal by the light-sheet fluorescence signal excitation module is as follows: A Gaussian beam is generated by Laser 1, which is expanded and collimated by the beam expansion and collimation unit, and then passes through the first mirror 4 and the beam splitter 5 to be divided into two Gaussian beams, namely the first Gaussian beam and the second Gaussian beam.

[0062] Among them, after the first Gaussian beam passes through the first multi-slit optical mask 6, the first cylindrical lens 7, and the first objective lens 8, a first type of non-diffracting Bessel light with a depth of focus range greater than the Rayleigh distance of the Gaussian beam (i.e., the first type of non-diffracting Bessel light) is formed. By projecting this type of non-diffracting Bessel light onto the sample surface, the sample can be excited to generate fluorescence, thereby reducing the thickness of the illumination light sheet, expanding the depth of focus of the illumination light sheet, eliminating the side lobes of the Bessel beam, reducing background noise, and achieving the effect of simultaneously improving the imaging field of view and resolution.

[0063] After the second Gaussian beam passes through the second mirror 9, the third mirror 10, the fourth mirror 11, the second multi-slit optical mask 12, the second cylindrical lens 13, and the second objective lens 14, a second type of non-diffracting Bessel light with a depth of focus range greater than the Rayleigh distance of the Gaussian beam (i.e., the second type of non-diffracting Bessel light) is formed. The propagation direction of this second type of non-diffracting Bessel light is opposite to that of the first type of non-diffracting Bessel light.

[0064] In this embodiment, by projecting two types of non-diffracting Bessel lights with opposite propagation directions on both sides of the sample to excite the sample to generate fluorescence signals, the problems of image dark lines and artifacts caused by low light sheet energy utilization rate and uneven intensity in the propagation direction of the light sheet when the Gaussian beam is focused by the cylindrical lens can be solved.

[0065] In the light sheet fluorescence signal excitation module, the front focal plane of the first multi-slit optical mask 6 coincides with the front focal plane of the first cylindrical lens 7, and the rear focal plane of the first cylindrical lens 7 coincides with the rear focal plane of the first objective lens 8; the front focal plane of the second multi-slit optical mask 12 coincides with the front focal plane of the second cylindrical lens 13, and the rear focal plane of the second cylindrical lens 13 coincides with the rear focal plane of the second objective lens 14. The sample cell 15 is used to place the sample.

[0066] As Figure 1 shown, the fluorescence signal encoding module includes:

[0067] The third objective lens 16, the tube lens 17, the fifth mirror 18, and the digital micromirror device 19.

[0068] Specifically, the encoding process of the fluorescence signal encoding module is as follows: The dynamic fluorescence signal excited by the light sheet fluorescence signal excitation module is collected by the third objective lens 16 and the tube lens 17, and then reflected by the fifth mirror 18 at an angle of 24° with respect to the normal of the panel of the digital micromirror device 19 onto the panel of the digital micromirror device 19, and is spatially encoded by the random mask loaded by the digital micromirror device 19.

[0069] Among them, the rear focal plane of the tube lens 17 coincides with the micromirror panel of the digital micromirror device 19; the digital micromirror device 19 needs to be placed by rotating 45° around the normal of its micromirror panel so that the light emitted from the digital micromirror device 19 can be parallel to the optical platform.

[0070] As shown Figure 1 in the figure, the fluorescence signal detection module includes:

[0071] a filter 20, a variable magnification lens group, a detector 23, and a four-dimensional fine adjustment platform 24; among them, the variable magnification lens group includes: a third lens 21 and a fourth lens 22; the four-dimensional fine adjustment platform 24 is Figure 1 the XYZ-θ four-dimensional fine adjustment platform shown in the figure.

[0072] Specifically, as shown Figure 2 in the figure, the detection process of the fluorescence signal detection module is as follows: after achieving pixel matching at the single-pixel level between the digital micromirror device 19 and the detector 23, within the single-frame exposure time of the detector 23, multiple encoded fluorescence signals are collected by the detector 23 after passing through the filter 20 and the variable magnification lens group, thereby obtaining a compressed snapshot imaging pattern.

[0073] Among them, in order to achieve pixel matching at the single-pixel level between the digital micromirror device 19 and the detector 23, first, the pixel array of the detector 23 needs to be aligned with the pixel array of the digital micromirror device 19. Then, the detector 23 needs to be tilted by 45° and installed on the four-dimensional fine adjustment platform 24. This four-dimensional fine adjustment platform 24 can be finely adjusted in the directions of XYZ-θ. Then, the magnification of the variable magnification lens group needs to be adjusted to achieve the process of pixel matching at the single-pixel level between the digital micromirror device 19 and the detector 23.

[0074] Furthermore, the fluorescence signal decoding module includes:

[0075] a projection decoding unit and a multi-stage joint denoising unit, which are used to substitute the compressed snapshot imaging picture and the random mask pattern into the projection formula to decode and obtain multiple initial fluorescence signal images, and reconstruct a high-frame-rate fluorescence video image to be measured based on the multiple initial fluorescence signal images.

[0076] Among them, the fluorescence decoding module is further used to perform multi-stage joint denoising processing on the initial video frames to reconstruct a high-frame-rate fluorescence video image to be measured.

[0077] It should be noted that in order to recover the high-frame-rate fluorescence video image to be measured from the collected compressed snapshot imaging picture in this embodiment, a supporting decoding and reconstruction algorithm is designed. This algorithm is based on the plug-and-play PnP-GAP iterative solution framework and introduces a joint total variation and deep network denoising unit. Compared with the GAP-TV algorithm in the related technology or the PnP-FFDNet algorithm with a single network denoising module, the algorithm in this embodiment has a better reconstruction effect.

[0078] It is understandable that the compressed snapshot imaging pattern collected by the imaging system in this embodiment and the random mask pattern used in the encoding process are used as the input of the decoding and reconstruction algorithm. The algorithm iteratively performs projection and denoising, and finally restores the high-frame-rate fluorescence video image to be measured.

[0079] Specifically, the input of the algorithm is the random mask pattern H and the compressed snapshot imaging pattern y. v and x are intermediate variables, and the remaining quantities are the weight parameters and iteration conditions of the algorithm. The algorithm is iteratively performed in two steps:

[0080] In the first step, a projection problem is solved to obtain a preliminary estimate x of the high-speed fluorescence video frame;

[0081] In the second step, a denoising problem is solved to optimize the estimated x to obtain v.

[0082] The above two steps are alternately iterated, and finally a high-quality high-frame-rate fluorescence image video to be measured can be restored. Compared with the traditional PnP iterative solution algorithm, in this embodiment, a multi-stage joint denoising strategy is introduced in the denoising environment of the second step, that is, by jointly denoising the total variation denoiser and the deep learning network denoiser pre-trained from the original noise image dataset and running on the GPU, the speed of the overall microscopic video image reconstruction is faster and the result is better.

[0083] Among them, the specific process of the decoding and reconstruction algorithm set in the decoding module is as follows:

[0084] (1) Input the encoding pattern H and the compressed snapshot imaging picture y;

[0085] (2) Initialize the parameters v0, α (0 ≤ α ≤ 1), k = 0, k1, k Max ; v0 is the initial estimate of v, that is, the initial estimate of the high-speed fluorescence video frame to be restored. α is the weight parameter for joint noise reduction of the total variation denoiser and the deep learning network denoiser. k represents the current iteration number, k1 represents the number of iterations required for the first stage (denoising only based on the total variation model denoiser), and k Max represents the total maximum number of iterations;

[0086] (3) If k ≤ k Max , and it does not converge, then execute the iterative loop for updating x and v, otherwise end the iterative loop, where x and v are both intermediate variables and both represent the high-speed fluorescence video frame to be restored, that is, the fluorescence video frame to be reconstructed.

[0087] Update x:

[0088] x (k+1) = v (k) + H T (HH T ) -1(y - Hv (k) ),

[0089] where H represents the random mask pattern collected by the detector, and y represents the fluorescence compressed snapshot imaging pattern.

[0090] Update v:

[0091] If k ≤ k1, then v (k+1) = D TV (x (k+1) ), where D TV () represents a noise reducer based on the total variation model; otherwise, v a = D TV (x (k+1) ), v b = D IDR (x (k+1) ), v (k+1) = αv a + (1 - α)v b , where D IDR () represents a deep network noise reducer trained based on the iterative data refinement method, and v a and v b respectively represent the high-speed fluorescence video frames to be restored after being denoised by the total variation and IDR noise reducers.

[0092] For example, in this embodiment, 20 high-speed microscopic video frames can be encoded to obtain a compressed snapshot imaging image, and 20 original high-speed microscopic video frames can be restored by decoding and reconstructing the compressed snapshot imaging image, so that the original high-speed microscopic video frames can be restored with high quality from a single encoded and compressed image.

[0093] As Figure 1 shown, the light sheet microscopy imaging system based on snapshot time compression in this embodiment further includes: a system control module;

[0094] The system control module includes: a four-dimensional electric displacement stage 25, an NI acquisition card 26, and a control terminal 27.

[0095] The specific control process of the system control module is as follows: The four-dimensional electric displacement stage 25 is connected to the control terminal 27 through a data cable. The four-dimensional electric displacement stage 25 is used to control the movement of the sample in the sample cell 15. The NI acquisition card 26 is respectively connected to the digital micromirror device 19, the detector 23, and the control terminal 27. The NI acquisition card 26 is used to achieve synchronization between the digital micromirror device 19 and the detector 23. The control terminal 27 is connected to the digital micromirror device 19, and the control terminal 27 is also used to load the designed random mask onto the digital micromirror device 19. The control terminal 27 is connected to the detector 23 using a data cable, and the control terminal 27 is also used to transmit the images collected by the detector 23 to the computer.

[0096] The present embodiment achieves the following technical effects through the above technical solutions:

[0097] This embodiment can use the snapshot time-compressed light-sheet microscopy system and the supporting algorithm to achieve high-speed and high-resolution video acquisition at about 1000 fps (when the detector operating frequency is 50 fps), overcoming the disadvantages of low signal-to-noise ratio, large data bandwidth, high cost, and complex system in the related high-speed and high-resolution imaging systems. Moreover, this embodiment realizes pixel matching at the single-pixel level between the modulator and the detector, solving the problems of mutual crosstalk and blurred images in the decoded microscopic video images. In order to decode and reconstruct the data collected by the snapshot time-compressed light-sheet microscopy system, this embodiment realizes an efficient reconstruction algorithm, which can complete the decoding and reconstruction of the collected video at a relatively fast speed and high quality. Based on the snapshot time-compressed light-sheet microscopy system, this embodiment not only realizes pixel matching at the single-pixel level between the modulator and the detector, solving the problems of mutual crosstalk and blurred images in the decoded microscopic images, but also designs an iterative decoding and reconstruction algorithm containing a joint denoising module of total variation and deep network, which can complete high-throughput and high-quality microscopic video acquisition with the advantages of low bandwidth and low cost.

[0098] Exemplary method

[0099] As Figure 3 shown, the embodiment of the present invention provides a snapshot time-compressed light-sheet microscopy method, including the following steps:

[0100] Step S100, converting a Gaussian beam into a non-diffracting Bessel-like light sheet with a depth of focus range greater than the Rayleigh distance of the Gaussian beam, and projecting the non-diffracting Bessel-like light sheet onto the sample surface to excite the sample to generate a fluorescence signal;

[0101] Step S200, performing spatial encoding on the fluorescence signal;

[0102] Step S300: Perform pixel matching between the digital micromirror device and the detector at the single-pixel level. During the single-frame exposure time of the detector, collect multiple encoded fluorescence signals to obtain a compressed snapshot imaging pattern.

[0103] Step S400: Decode the compressed snapshot imaging pattern to obtain a high-frame-rate fluorescence video image to be restored.

[0104] Specifically, in one implementation of this embodiment, step S400 includes the following steps:

[0105] Step S401: Substitute the collected fluorescence compressed snapshot imaging pattern and the random mask pattern into the projection formula to decode and obtain an initial high-speed fluorescence video frame to be restored.

[0106] In this embodiment, the projection formula is:

[0107] x (k+1) =v (k) +H T (HH T ) -1 (y - Hv (k) ),

[0108] where H represents the random mask pattern collected by the detector, y represents the fluorescence compressed snapshot imaging pattern, x represents the initial high-speed fluorescence video frame to be restored, and k represents the current iteration number.

[0109] In this embodiment, decoding is performed by a fluorescence signal decoding module. Among them, the fluorescence signal decoding module includes:

[0110] A projection decoding unit and a multi-level joint denoising unit, which are used to substitute the compressed snapshot imaging picture and the random mask pattern into the projection formula respectively to decode and obtain multiple initial fluorescence signal images, and reconstruct a high-frame-rate fluorescence video image to be measured based on the multiple initial fluorescence signal images. Among them, the multi-level joint denoising unit is further used to perform multi-level joint denoising processing on the initial video frame to reconstruct a high-frame-rate fluorescence video image to be measured.

[0111] It should be noted that in order to restore the high-frame-rate fluorescence video image to be measured from the collected compressed snapshot imaging picture in this embodiment, a supporting decoding and reconstruction algorithm is designed. This algorithm is based on the plug-and-play PnP-GAP iterative solution framework and introduces a total variation and deep network joint denoising unit. Compared with the GAP-TV algorithm in the related technology or the PnP-FFDNet algorithm with a single network denoising module, the algorithm in this embodiment has a better reconstruction effect.

[0112] It is understandable that the compressed snapshot imaging pattern collected by the imaging system in this embodiment and the random mask pattern used in the encoding process are used as the input of the decoding and reconstruction algorithm. The algorithm iteratively projects and denoises, and finally restores the high-frame-rate fluorescence video image to be measured.

[0113] Specifically, the input of the algorithm is the random mask pattern H and the compressed snapshot imaging pattern y. v and x are intermediate variables, and the remaining quantities are the weight parameters and iteration conditions of the algorithm. The algorithm iteratively proceeds in two steps:

[0114] In the first step, a projection problem is solved to obtain a preliminary estimate x of the high-speed fluorescence video frame;

[0115] In the second step, a denoising problem is solved to optimize the estimated x to obtain v.

[0116] The above two steps are alternately iterated, and finally a high-quality high-frame-rate fluorescence image video to be measured can be restored. Compared with the traditional PnP iterative solution algorithm, in this embodiment, a multi-stage joint denoising strategy is introduced in the denoising environment of the second step, that is, by jointly denoising the total variation denoiser and the deep learning network denoiser pre-trained from the original noise image dataset and running on the GPU, the overall speed of microscopic video image reconstruction is faster and the result is better.

[0117] In this embodiment, the fluorescence signal decoding module is provided with a decoding and reconstruction algorithm, and the specific process of this decoding and reconstruction algorithm is as follows:

[0118] (1) Input the encoding pattern H and the compressed snapshot imaging picture y;

[0119] (2) Initialize the parameters v0, α (0 ≤ α ≤ 1), k = 0, k1, k Max ; v0 is the initial estimate of v, that is, the initial estimate of the high-speed fluorescence video frame to be restored. α is the weight parameter for joint noise reduction of the total variation denoiser and the deep learning network denoiser. k represents the current iteration number, k1 represents the number of iterations required for the first stage (denoising only based on the total variation model denoiser), and k Max represents the total maximum number of iterations;

[0120] (3) If k ≤ k Max , and it does not converge, then execute the iterative loop for updating x and v, otherwise end the iterative loop, where both x and v are intermediate variables and both represent the high-speed fluorescence video frame to be restored, that is, the fluorescence video frame to be reconstructed.

[0121] Specifically, in one implementation manner of this embodiment, step S400 includes the following steps:

[0122] Step S402: Perform multi-level joint noise reduction processing on the initial high-speed fluorescence video frame to be restored, so as to reconstruct the fluorescence video image of the high frame rate to be restored.

[0123] In this embodiment, the process of the multi-level joint noise reduction is as follows: Determine whether k is less than k1, where k1 represents the number of iterations of noise reduction by the total variation model denoiser; if k ≤ k1, then v (k+1) = D TV (x (k+1) ), where v represents the high-speed fluorescence video frame to be restored, and D TV () represents the denoiser based on the total variation model; otherwise, v a = D TV (x (k+1) ), v b = D IDR (x (k+1) ), v (k+1) = αv a + (1 - α)v b , where D IDR () represents the deep network denoiser trained by the iterative data refinement method, v a and v b respectively represent the high-speed fluorescence video frames to be restored after noise reduction by the total variation and IDR denoisers, α represents the weight parameter, and 0 ≤ α ≤ 1.

[0124] Further, let the total number of decoding and reconstruction iterations be k Max , if k ≤ k Max , then perform projection decoding and multi-level joint noise reduction processing on the multiple fluorescence signal images after the multi-level joint noise reduction processing, and finally obtain the high-speed fluorescence video image to be restored.

[0125] This embodiment achieves the following technical effects through the above technical solutions:

[0126] The imaging method of this embodiment not only realizes pixel matching at the single-pixel level between the digital micromirror device and the detector in the fluorescence signal detection module, solves the problems of mutual crosstalk and image blurring in the decoded microscopic images, but also realizes fast decoding and improves the resolution and signal-to-noise ratio of the decoded images in the fluorescence signal decoding module, making this method have the advantages of fast decoding speed, high signal-to-noise ratio, small bandwidth, low cost, simple system, and high frame rate, overcoming the defects that the existing light sheet microscopy system cannot break through the limitation of the camera acquisition speed during high-speed imaging, the low signal-to-noise ratio caused by short exposure time and the need for a large data bandwidth, thereby realizing the capture of high-speed and high-resolution microscopic transient scenes.

[0127] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided by the present invention can include non-volatile and / or volatile memories.

[0128] In summary, the present invention provides a light-sheet microscopy imaging system and method based on snapshot time compression, including: a light-sheet fluorescence signal excitation module for generating two non-diffracting Bessel-like light sheets with a depth of focus greater than the Rayleigh distance of a Gaussian beam, collimating them and illuminating both sides of the sample in opposite directions to excite the sample to generate fluorescence; a fluorescence signal encoding module for encoding and modulating the fluorescence signals generated by the light-sheet fluorescence signal excitation module; a fluorescence signal detection module for matching pixels at the single-pixel level and obtaining a fluorescence compressed snapshot imaging pattern according to multiple encoded fluorescence signals collected within a single-frame exposure time; and a fluorescence signal decoding module for decoding the collected fluorescence compressed snapshot imaging pattern to obtain a high-resolution and high-frame-rate fluorescence video image to be restored. The present invention solves the problem of low signal-to-noise ratio caused by short exposure time and the need for a large data bandwidth in high-speed and high-resolution microscopic transient scenarios.

[0129] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description, and all such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. A light sheet microscopy imaging system based on snapshot time compression, characterized in that, Comprising: A light-sheet fluorescence signal excitation module, configured to generate two non-diffracting Bessel-like light-sheets with a depth of focus greater than the Rayleigh distance of a Gaussian beam, which are collimated and illuminated on both sides of a sample in opposite directions to excite the sample to generate fluorescence; A fluorescence signal encoding module, configured to perform encoding modulation on the fluorescence signals generated by the light-sheet fluorescence signal excitation module; A fluorescence signal detection module, configured to match pixels at the single-pixel level, and obtain a fluorescence compressed snapshot imaging pattern based on multiple encoded fluorescence signals collected within a single-frame exposure time; A fluorescence signal decoding module, configured to decode the collected fluorescence compressed snapshot imaging pattern to obtain a high-resolution and high-frame-rate fluorescence video image to be restored; The fluorescence signal encoding module is connected to the light-sheet fluorescence signal excitation module, and the fluorescence signal detection module is connected to the fluorescence signal encoding module; the light-sheet fluorescence signal excitation module, the fluorescence signal encoding module, and the fluorescence signal detection module are respectively connected to the fluorescence signal decoding module; The fluorescence signal encoding module includes: a third objective lens, a tube lens, a fifth mirror, and a digital micromirror device; the rear focal plane of the tube lens coincides with the micromirror panel of the digital micromirror device; The dynamic fluorescence signals excited by the light-sheet fluorescence signal excitation module are collected by the third objective lens and the tube lens, and are reflected by the fifth mirror to the micromirror panel of the digital micromirror device at an angle of 24° with respect to the normal of the micromirror panel of the digital micromirror device, and are spatially encoded by the random mask loaded by the digital micromirror device; The fluorescence signal detection module includes: a filter, a zoom lens group, a detector, and a four-dimensional fine adjustment platform; The detector is disposed on the four-dimensional fine adjustment platform, and the four-dimensional fine adjustment platform and the magnification of the zoom lens group are adjusted to achieve pixel matching at the single-pixel level between the detector and the digital micromirror device; After pixel matching at the single-pixel level between the digital micromirror device and the detector, within the single-frame exposure time of the detector, multiple encoded fluorescence signals are collected by the detector after passing through the filter and the zoom lens group to obtain a compressed snapshot imaging pattern.

2. The light sheet microscopy imaging system based on snapshot time compression according to claim 1, wherein The light-sheet fluorescence signal excitation module includes: a laser, a beam expander and collimator unit, a first mirror, a beam splitter, a first optical path component, a second optical path component, and a sample cell; The laser beam generated by the laser is expanded and collimated by the beam expander and collimator unit, and the expanded and collimated laser beam is split into a first Gaussian beam and a second Gaussian beam via the first mirror and the beam splitter; The first Gaussian beam forms a first non-diffracting Bessel-like light through the first optical path component, and the second Gaussian beam forms a second non-diffracting Bessel-like light through the second optical path component. The first non-diffracting Bessel-like light and the second non-diffracting Bessel-like light respectively excite the sample in the sample cell to generate corresponding fluorescence.

3. The light sheet microscopy imaging system based on snapshot time compression according to claim 2, wherein, The first optical path component includes: a first multi-slit optical mask, a first cylindrical lens, and a first objective lens; The first Gaussian beam passes through the first multi-slit optical mask, the first cylindrical lens, and the first objective lens to form a first type of non-diffracting Bessel light with a focal depth range greater than the Rayleigh distance of the Gaussian beam; The second optical path component includes: a second mirror, a third mirror, a fourth mirror, a second multi-slit optical mask, a second cylindrical lens, and a second objective lens; The second Gaussian beam passes through the second mirror, the third mirror, the fourth mirror, the second multi-slit optical mask, the second cylindrical lens, and the second objective lens to form a second type of non-diffracting Bessel light with a focal depth range greater than the Rayleigh distance of the Gaussian beam and propagating in the opposite direction to the first type of non-diffracting Bessel light.

4. The light sheet microscopy imaging system based on snapshot time compression according to claim 1, wherein The fluorescence signal decoding module includes: a projection decoding unit and a multi-stage joint noise reduction unit; The projection decoding unit and the multi-stage joint noise reduction unit are used to substitute the compressed snapshot imaging picture and the random mask pattern into the projection formula respectively to decode multiple initial fluorescence signal images, and reconstruct a high-frame-rate fluorescence video image to be measured based on the multiple initial fluorescence signal images.

5. The light sheet microscopy imaging system based on snapshot time compression according to claim 1, wherein It further includes: System control module: a four-dimensional electric displacement stage, an NI acquisition card, and a control terminal; The four-dimensional electric displacement stage is connected to the control terminal through a data cable, and the four-dimensional electric displacement stage is used to control the movement of the sample in the sample cell; The NI acquisition card is connected to the digital micromirror device, the detector, and the control terminal respectively to synchronize the digital micromirror device and the detector; The control terminal is connected to the digital micromirror device, and the control terminal is used to load a pre-designed random mask onto the digital micromirror device. The control terminal is connected to the detector through a data cable, and the control terminal is used to transmit the image collected by the detector to a computer.

6. A light-sheet microscopy imaging method based on snapshot time compression, which uses the light-sheet microscopy imaging system based on snapshot time compression as described in any one of claims 1 to 5, and is characterized in that It includes: Converting a Gaussian beam into a non-diffracting Bessel light sheet with a focal depth range greater than the Rayleigh distance of the Gaussian beam, and projecting the non-diffracting Bessel light sheet onto the sample surface to excite the sample to generate fluorescence signals; Performing spatial encoding on the fluorescence signals; Performing pixel matching at the single-pixel level between the digital micromirror device and the detector, and collecting multiple encoded fluorescence signals within the single-frame exposure time of the detector to obtain a compressed snapshot imaging pattern; Decoding the compressed snapshot imaging pattern to obtain a high-frame-rate fluorescence video image to be restored.

7. The light sheet microscopy imaging method based on snapshot time compression according to claim 6, wherein The decoding of the compressed snapshot imaging pattern includes: Substituting the collected fluorescence compressed snapshot imaging pattern and the random mask pattern into the projection formula to decode an initial high-speed fluorescence video frame to be restored; The projection formula is: where H represents the random mask pattern collected by the detector, y represents the fluorescence compressed snapshot imaging pattern, x represents the initial high-speed fluorescence video frame to be restored, and k represents the current iteration number.

8. The light sheet microscopy imaging method based on snapshot time compression according to claim 7, characterized in that, The obtaining of the high-frame-rate fluorescence video image to be restored includes: Performing multi-stage joint noise reduction processing on the initial high-speed fluorescence video frame to be restored to reconstruct the high-frame-rate fluorescence video image to be restored.

Images