A reconstruction method and system for three-dimensional fluorescence imaging of structured light microscopy

Abstract

The present invention discloses a reconstruction method and system for three-dimensional fluorescence imaging using a structured light microscope. The present invention first uses the HT algorithm to decode and obtain optical slices OS of different sections of the sample. HT , and calculate the wide field image WF; then OS HT Gaussian low-pass filtering is performed on both WF and WF to obtain their low-frequency images OS lp 、WF lp and wide-field high-frequency images WF hp ; then by OS lp and WF hp The combination produces a high signal-to-noise ratio slice image. The present invention obtains low-frequency information from the optical slice image and high-frequency information from the wide-field image through spatial filtering and recombines them. This can significantly improve the signal-to-noise ratio of the optical slice image, approximately double its imaging depth, and effectively save the required processing time.

Description

TECHNICAL FIELD

[0001] The present application relates to an image reconstruction algorithm of a structured light microscope, which can realize fast three-dimensional fluorescence imaging with high signal-to-noise ratio and can be widely applied to research in the fields of biology, medicine and the like. BACKGROUND

[0002] Three-dimensional fluorescence imaging has become an indispensable tool in biomedical research, which provides the ability to observe complex biological tissue structures such as organoids, neurons and nerve fibers. In order to clearly observe the inside of thicker tissues and understand biological processes and their functions, many advanced three-dimensional imaging techniques have been developed.

[0003] As a commonly used three-dimensional fluorescence imaging technique, confocal laser scanning microscopy uses a pinhole to eliminate out-of-focus light, thereby obtaining optical sections with background removed. However, it has the problem of photobleaching, and the imaging speed is also slow, which is not ideal for live cell imaging. Two-photon microscopy uses near-infrared light to excite fluorescence, has a deeper penetration depth and less photobleaching, but usually at the cost of longer acquisition time and photodamage caused by high-intensity illumination. Light sheet microscopy uses a thin light sheet to illuminate a plane of the sample, which can quickly acquire images with less photodamage, but its resolution in the deep region is limited and may be affected by uneven illumination.

[0004] As a wide-field imaging method, structured illumination microscopy (SIM) has the advantages of high spatial resolution, fast imaging speed and low phototoxicity. SIM was originally invented by Neil et al. by projecting a sinusoidal fringe pattern onto the sample surface through an objective lens. Only the focal plane is modulated by the fringe, while the out-of-focus background decays to uniform illumination. Through the root mean square (RMS) decoding algorithm, the in-focus information with background removed can be extracted, thereby obtaining optical sections (OS). The structured illumination can also be replaced by a speckle pattern in HiLo microscopy, in which the high-frequency information of the wide-field image and the low-frequency information of the structured illumination image are combined to obtain optical sections. In 2015, Zhou et al. proposed a light section decoding algorithm based on Hilbert transform (HT), which reduced the image acquisition amount by 1 / 3 compared to the RMS algorithm, thereby improving the imaging speed of SIM. In 2020, Qian et al. applied the HT algorithm to three-dimensional color imaging of pollen grains and tiger beetle samples, verifying the advantage of fast imaging of the algorithm. It should be noted that they are all imaging of self-fluorescence or reflective surfaces, and the signal and modulation contrast are very strong.

[0005] However, for three-dimensional fluorescence imaging of biological samples, OS-SIM faces the problem of low signal-to-noise ratio (SNR), especially in thick tissues (generally with a thickness of more than tens of microns, which is considered as thick tissues). Before the image becomes blurred due to the scattering of light, the structured illumination has been degraded into uniform illumination with the increase of depth, resulting in the loss of optical sectioning capability of SIM. Even in the shallow region, the weak modulation degree of the stripe and the image acquisition noise can cause significant noise in the decoded optical section. In 2022, Dang et al. proposed a Fourier domain-based optical sectioning image reconstruction algorithm to suppress noise, but its effect was not significant, and it caused loss of high-frequency information, and the Fourier transform also reduced the processing speed. Qiao et al. designed a channel attention generative adversarial network (caGAN) for 3D-SIM for denoising, but the training of the neural network is time-consuming, and its applicability to different samples cannot be determined. SUMMARY

[0006] In view of the defects or deficiencies of the prior art, the present application provides a reconstruction method for three-dimensional fluorescence imaging of a structured light microscope.

[0007] To this end, the reconstruction method for three-dimensional fluorescence imaging of a structured light microscope provided by the present application comprises the following steps:

[0008] Step 1: Acquire structured illumination images:

[0009] Under the sinusoidal stripe structured illumination light field, two structured illumination images I1 and I2 of the current section of the sample to be imaged are acquired, wherein the phase shift difference of the stripes of I1 and I2 is π, and:

[0010]

[0011] Or:

[0012]

[0013] In formula (1-1) and (1-2), I out (x, y, z) represents the out-of-focus background part of the image, I in (x, y, z) represents the in-focus part of the image, m represents the stripe modulation intensity, v represents the spatial frequency of the stripe, represents the initial phase;

[0014] Step 2: Calculate the optical section of the current section

[0015] Step 2.1: Perform decoding operation on the two structured illumination images I1 and I2 acquired in step 1 using Hilbert transform algorithm to obtain the HT-decoded optical section OS HT :

[0016]

[0017] or,

[0018]

[0019] In formula (2-1), HT x represents one-dimensional Hilbert transform operator along x direction, and I1 and I2 are calculated by formula (1-1);

[0020] In formula (2-2), HT y represents one-dimensional Hilbert transform operator along y direction, and I1 and I2 are calculated by formula (1-2);

[0021] Step 2.1, average two structured illumination images I1 and I2 collected in step 1 to obtain a wide-field image WF of the current section without stripes, WF = (I1 + I2) / 2;

[0022] Step 2.2, respectively, Gaussian low-pass filter OS HT and WF to obtain the low-frequency component OS lp and the high-frequency component WF hp of the wide-field image WF of the current section without stripes:

[0023]

[0024] In formula (2-3) and (2-4), represents convolution operator, and σ represents standard deviation of Gaussian kernel; WF lp is the low-frequency component of the wide-field image of the current section without stripes;

[0025] Step 2.3, recombine OS lp and WF hp obtained in step 2.2 to obtain the optical section OS HT-SHiLo of the current section:

[0026] OS HT-SHiLo = OS lp + ηWF hp (2-5)

[0027] In formula (2-5), η is a scaling factor;

[0028] Step 3, perform step 1 to sample the remaining sections of the sample along the z-axis axial distribution respectively, and calculate the optical section of the corresponding section by using the method of step 2 for each layer section collected image; the total number of sections of the sample along the z-axis axial distribution is not less than 2;

[0029] Step 4, stack the optical sections of all layer sections to form a three-dimensional image of the sample.

[0030] The present application can significantly inhibit the noise of the OS-SIM decoded slice, greatly improve the signal-to-noise ratio of the slice image and the three-dimensional image, and meanwhile maintain the spatial resolution. Moreover, compared with the traditional frequency domain HiLo algorithm, the present application saves the time-consuming operation of Fourier transform in the spatial domain operation, greatly improves the processing speed; meanwhile, the HT algorithm only needs to collect two original images to obtain a slice, and has higher imaging efficiency compared with the RMS algorithm. In addition, due to the improvement of the signal-to-noise ratio, the imaging depth of the OS-SIM can be extended by nearly one time, which has important significance for the deep layer imaging of biological tissues.

[0031] Optionally, the filter radius of the Gaussian low-pass filter in step 2.2 is The standard deviation of the Gaussian kernel

[0032] Optionally, in step 2.3, η is the OS of all sections of the sample along the axial distribution of the z axis. lp The maximum value in the ratio of WF lp to WF

[0033]

[0034] Correspondingly, the present application also provides a related structured light microscope system, which adopts the method of the present application to obtain the three-dimensional image of the sample.

[0035] A storage medium related to the method of the present application, which stores computer programs / instructions, and the computer programs / instructions are executed by a processor to realize the steps of the above method. And a software product related to the present application, which includes computer programs / instructions, and the computer programs / instructions are executed by a processor to realize the steps of the above method. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 The flow chart of the HT-SHiLo algorithm of the present application;

[0037] Figure 2 The simulation results of the HT, HiLo and HT-SHiLo algorithms under different modulation factors in the embodiment; (a)-(d) are the decoded images of each algorithm under the stripe modulation factors m=0.4, 0.2, 0.1 and 0.05, respectively; (e) is the peak signal-to-noise ratio (PSNR) of the decoded images of each algorithm under different modulation factors; and (f) is the operation time of each algorithm for different image sizes.

[0038] Figure 3Three-dimensional imaging results of COS-7 cells in the example; Fig. a is a maximum intensity projection (MIP) image of COS-7 cells; b, c are local contrast images of decoded images of HT algorithm and HT-SHiLo algorithm, respectively; d, e are three-axis projection images and three-dimensional images of mitochondria, respectively.

[0039] Figure 4 Three-dimensional imaging results of Thy1 mouse brain nerve in the example; Figs. a, b are three-dimensional images obtained by HT algorithm and HT-SHiLo algorithm, respectively; c-e are contrast of optical sections decoded by HT algorithm and HT-SHiLo algorithm at depths z = 40, 180, 320 μm, respectively. DETAILED DESCRIPTION

[0040] Unless otherwise defined, scientific and technical terms used herein are understood according to the common meanings of the terms as understood by one of ordinary skill in the relevant art.

[0041] The sample to be imaged in the method of the present application includes, but is not limited to, cells, cell contents, microorganisms, biological tissues, protein molecules and other biological materials. Before imaging by the method of the present application, the sample can be subjected to fluorescent staining, fixation and transparentization and the like. In a specific scheme, the distance of each section in the sample along the z axis or the appropriate interlayer spacing is determined according to the depth of field of the objective lens used, for example, about half of the depth of field of the objective lens.

[0042] In order to solve the problem of noise in decoding sections at low modulation, the present application adopts an algorithm (HT-SHiLo) for recombining high and low frequency information of optical sections in spatial domain, the flow chart of which is shown in Fig. 2. Figure 1 First, the HT algorithm is used to decode to obtain optical sections OS of different sections of the sample HT , and to calculate a wide-field image WF; then, both OS HT and WF are subjected to Gaussian low-pass filtering (GLPF) to obtain low-frequency images OS lp , WF lp and high-frequency images WF hp of the wide field; because the high-frequency information in the wide-field image only comes from the focal plane, it can be used to make up the high-frequency information of the section image; thus, the section image with high signal-to-noise ratio can be obtained by combining OS lp and WF hp , i.e. OS HT-SKHiLo = OS lp + ηWF hp .

[0043] In a specific scheme, the scaling factor or proportion factor η is determined by ensuring seamless connection of high and low frequency components at the cut-off frequency. In order to avoid time-consuming Fourier transform, in a preferred scheme, the scaling factor or proportion factor is used to ensure that the low-frequency component and the high-frequency component are combined in the corresponding proportion.

[0044] The filter radius of the Gaussian filter (or the size of the Gaussian kernel) determines the final reconstructed slice from OS HT The denoising effect will be improved with the increase of the filter radius, but the slicing effect will be weakened. Therefore, in the preferred embodiment, the present invention adopts 1 / v as the filter radius, and the standard deviation of the corresponding Gaussian kernel is

[0045] The present invention will be further explained below with reference to specific embodiments, but the present invention is not limited thereto. The structured light microscope used in the following embodiments is the structure system disclosed in "ZL201110448980.8 A High-Speed ​​Structured Illumination Optical Microscopy System and Method Based on Digital Micromirror Device".

[0046] Example 1:

[0047] In order to simulate the influence of fringe modulation on SIM decoding slices and the processing effect of the algorithm, fringes with different modulation intensities m are added to an initial image, and the HT decoding algorithm, the traditional HiLo algorithm and the HT-SHiLo algorithm are used respectively (the HT-SHiLo algorithm of this embodiment specifically adopts step 2 of the present invention. The two images in the processing process are generated by adding two fringes with a phase shift difference of π to the initial image, where The filter radius of the Gaussian low-pass filter is η Take all slice OS lp With WF lp The maximum value of the ratio is used to calculate the optical section of the initial image.

[0048] Figure 2 Results are shown for modulation intensities m = 0.4, 0.2, 0.1, and 0.05. It is found that as the modulation intensity decreases, the noise in the HT-decoded image increases significantly, while the images calculated by the HiLo and HT-SHiLo algorithms are less affected.

[0049] Figure 2 (e) The peak signal-to-noise ratio (PSNR) of the images calculated by each algorithm at various modulation intensities is shown. The results show that the PSNR of HT-decoded images decreases rapidly as the modulation intensity decreases, while the PSNR of images processed by the HT-SHiLo algorithm remains high. The PSNR of images processed by the HiLo algorithm lies somewhere in between.

[0050] Combine Figure 2(f) The running time of each algorithm on the GPU (RTX3070) was measured multiple times by MATLAB (R2023a) and the results showed that the HT-SHiLo algorithm only needed an additional 3 ms of processing time compared to the HT algorithm, while the HiLo algorithm needed more processing time and increased exponentially with the increase of image size. At 2048x2048 pixels, the processing speed of the HT-SHiLo algorithm was about 32 times faster than that of the HiLo algorithm.

[0051] Example 2:

[0052] In this example, the HT algorithm and the method of the present application were used to perform three-dimensional imaging of the nuclei (DAPI-labeled, 405 nm excitation) and mitochondria (Mito-Tracker Green-labeled, 488 nm excitation) of fluorescently labeled COS-7 cells, respectively.

[0053] In the method of this example, m, v, The appropriate layer spacing values are: m is about 0.2, v is 0.91 μm-1, The layer spacing is randomly set by the system to be 0.2 μm; the sample generates 36 optical sections; The filter radius of the Gaussian low-pass filter is η takes all the sections OS lp The WF lp The maximum value of the ratio.

[0054] The imaging results of this example are shown in Figure 3 where cyan represents the nuclei and yellow represents the mitochondria. As can be seen from Figure 3 The comparison between the HT algorithm and the method of the present application (HT-SHiLo algorithm) in (b) and (c) shows that the method of the present application effectively removes noise and residual stripes from the image; Figure 3 (d) and (e) show the three-dimensional morphology of the local mitochondria, indicating that the method of the present application has good spatial resolution.

[0055] Example 3:

[0056] In this example, the HT algorithm and the method of the present application were used to perform three-dimensional imaging of the Thy1-EGFP (488 nm excitation) labeled mouse brain nerve.

[0057] In the method of this example, m, v, The appropriate layer spacing values are: m gradually decreases from shallow to deep, about 0.2 to 0.05, and v is 0.18 μm-1, The layer spacing is randomly set by the system to be 4 μm; the sample generates 143 optical sections; The filter radius of the Gaussian low-pass filter is η Take all slice OS lp With WF lp The maximum value of the ratio.

[0058] Figure 4 The results of the HT algorithm of this embodiment and the method of the present invention (HT-SHiLo algorithm) are compared. It can be seen that the HT algorithm loses a lot of information as the imaging depth increases, while the method of the present invention can maintain a high signal-to-noise ratio and can clearly image the deep part of the sample. Figure 4 As can be seen from (a) and (b), the imaging depth of the method of the present invention is nearly doubled compared with the HT algorithm.

Claims

1. A reconstruction method for three-dimensional fluorescence imaging with structured light microscopy, characterized in that, The method comprises the following steps: Step 1, collecting structured illumination images Two structured illumination images I1 and I2 of the current section of the sample to be imaged are collected under a sinusoidal stripe structured illumination light field, wherein the phase shift difference between I1 and I2 is π, and: Or: In formulas (1-1) and (1-2), I out (x, y, z) represents an out-of-focus background portion of the image, I in (x, y, z) represents an in-focus portion of the image, m represents a fringe modulation intensity, and v represents a spatial frequency of the fringe, represents an initial phase; Step 2, calculating the optical section of the current section Step 2.1, decode the two structured illumination images I1 and I2 acquired in step 1 using a Hilbert transform algorithm to obtain the HT-decoded optical section OS HT : Or, In formula (2-1), HT x {} represents a one-dimensional Hilbert transform operator in the x direction, and I1 and I2 are calculated using formula (1-1); In formula (2-2), HT y {} represents a one-dimensional Hilbert transform operator in the y direction, and I1and I2are calculated using formula (1-2); The wide-field image WF of the current section without stripes is obtained by averaging the two structured illumination images I1 and I2 collected in step 1, WF=(I1+I2) / 2; Step 2.2, apply Gaussian low-pass filter to OS HT and WF, respectively, to obtain low-frequency component OS lp and high-frequency component WF hp of the current facet-stripped wide-field image WF in formulas (2-3) and (2-4), represents a convolution operator, and σ represents a standard deviation of a Gaussian kernel; WF lp removes low-frequency components of the wide-field image of the current slice from the streaks Step 2.

3. The OS obtained in step 2.2 is recombined to the current slice lp and WF hp The optical slice OS of the current slice is recombined HT-SHiLo : OS HT-SHiLo = OS lp + ηWF hp (2-5) In formula (2-5), η is a scaling factor; Step 3, performing step 1 to sample the remaining sections of the sample along the z-axis axial distribution, and using the method of step 2 to calculate the optical section of each layer section; Step 4, stacking the optical sections of all layer sections to form a three-dimensional image of the sample.

2. The reconstruction method for three-dimensional fluorescence imaging of structured light microscopy according to claim 1, characterized in that, The filter radius of the Gaussian low-pass filtering in step 2.2 is Standard deviation of the Gaussian kernel 3. The reconstruction method for three-dimensional fluorescence imaging of structured light microscopy according to claim 1, characterized in that: OS of all cross sections of the sample along the z-axis axial distribution in step 2.3 lp WF lp Maximum in the ratio:

4. A structured light microscope system, characterized in that: The system uses the method of claim 1, 2 or 3 to obtain a three-dimensional image of the sample.

5. A storage medium, characterized by A computer program / instruction is stored thereon, and when the computer program / instruction is executed by a processor, the steps of the method of any one of claims 1-3 are implemented.

6. A software product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by a processor to implement the steps of the method of any one of claims 1-3.

Images