Reconstruction method and system for structured light microscope three-dimensional fluorescence imaging

Abstract

The invention discloses a reconstruction method and system for three-dimensional fluorescence imaging of a structured light microscope, and the method comprises the steps: carrying out the decoding through employing an HT algorithm, obtaining optical slices OSHT of different sections of a sample, and calculating a wide-field image WF; gaussian low-pass filtering is carried out on the OSHT and the WF to obtain low-frequency images OSlp and WFlp and a wide-field high-frequency image WFhp; and then the OSlp and the WFhp are combined to obtain a slice image with a high signal-to-noise ratio. According to the method, the low-frequency information of the optical slice image and the high-frequency information of the wide-field image are obtained through spatial filtering and are recombined, so that the signal-to-noise ratio of the optical slice image can be remarkably improved, the imaging depth of the optical slice image can be doubled, and meanwhile, the required processing time is effectively saved.

Description

Technical Field

[0001] The present invention relates to an image reconstruction algorithm for a structured light microscope, which can realize fast three-dimensional fluorescence imaging with a high signal-to-noise ratio and can be widely used in research in the fields of biology, medicine, etc. Background Art

[0002] Three-dimensional fluorescence imaging has become an indispensable tool in biomedical research, providing the ability to observe complex biological tissue structures such as organoids, neurons and nerve fibers. In order to clearly observe the inside of thicker tissues and understand biological processes and their functions, many advanced three-dimensional imaging technologies have been developed.

[0003] As a commonly used three-dimensional fluorescence imaging technique, confocal laser scanning microscopy uses a pinhole to eliminate out-of-focus light to obtain optical sections with background removed. However, it has photobleaching problems and slow imaging speed, which is not ideal for living cell imaging. Two-photon microscopy uses near-infrared light to excite fluorescence, which has a deeper penetration depth and less photobleaching, but usually at the expense of longer acquisition time and photodamage caused by high-intensity illumination. Light sheet microscopy uses a thin beam of light to illuminate a plane of the sample, which can quickly acquire images with less photodamage, but its resolution in deep areas is limited and may be affected by uneven illumination.

[0004] As a widefield imaging method, structured illumination microscopy (SIM) has the advantages of high spatial resolution, fast imaging speed and low phototoxicity. SIM was originally invented by Neil et al., who projected a sinusoidal fringe pattern onto the sample surface through an objective lens. Only the focal plane will be modulated by the fringe, while the out-of-focus background decays to uniform illumination. Through the root mean square (RMS) decoding algorithm, the in-focus information with the background removed can be extracted to obtain optical sections (OS). Structured illumination can also be replaced by the speckle pattern in HiLo microscopy, where the high-frequency information of the widefield image and the low-frequency information of the structured illumination image are combined to obtain optical sections. In 2015, Zhou Xing et al. proposed an optical section decoding algorithm based on Hilbert transform (HT), which reduced the image acquisition by 1 / 3 compared with the RMS algorithm and improved the imaging speed of SIM. In 2020, Qian Jia et al. applied the HT algorithm to three-dimensional color imaging of samples such as pollen grains and tiger beetles, verifying the fast imaging advantage of the algorithm. It should be noted that both are imaging of autofluorescence or reflective surfaces, with strong signal and modulation contrast.

[0005] However, for three-dimensional fluorescence imaging of biological samples, OS-SIM faces the problem of low signal-to-noise ratio (SNR), especially in thick tissues (thickness is generally more than tens of microns, which is considered thick tissue). Before the image becomes blurred due to light scattering, the structured illumination has degenerated into uniform illumination with increasing depth, causing SIM to lose its optical sectioning ability. Even in shallow areas, the weak modulation degree of the fringes and image acquisition noise may cause significant noise in the decoded optical sections. In 2022, Dang Shipei et al. proposed a Fourier domain-based optical sectioning image reconstruction algorithm to suppress noise, but the effect was not significant and it would lead to the loss of high-frequency information. The Fourier transform also reduced the processing speed. Qiao et al. designed a channel attention generative adversarial network (caGAN) for 3D-SIM for denoising, but the training of the neural network took a long time and its applicability to different samples could not be determined. Summary of the invention

[0006] In view of the defects or shortcomings of the prior art, the present invention provides a reconstruction method for three-dimensional fluorescence imaging of a structured light microscope.

[0007] To this end, the reconstruction method for three-dimensional fluorescence imaging of a structured light microscope provided by the present invention comprises the following steps:

[0008] Step 1: Collect structured illumination images:

[0009] Under the sinusoidal fringe structured illumination light field, two structured illumination images I1 and I2 of the current section of the sample to be imaged are collected, where the fringe phase shift difference between I1 and I2 is π, and:

[0010]

[0011] or:

[0012]

[0013] In formula (1-1) and (1-2), I out (x, y, z) represents the out-of-focus background part of the image, I in (x, y, z) represents the focused part of the image, m represents the fringe modulation intensity, v represents the spatial frequency of the fringe, Represents the initial phase;

[0014] Step 2: Calculate the optical section of the current section

[0015] Step 2.1: Use the Hilbert transform algorithm to decode the two structured illumination images I1 and I2 acquired in step 1 to obtain the HT-decoded light slice OS. HT :

[0016]

[0017] or,

[0018]

[0019] In formula (2-1), HT x {} represents the one-dimensional Hilbert transform operator along the x direction, and I1 and I2 are calculated using formula (1-1);

[0020] In formula (2-2), HT y {} represents the one-dimensional Hilbert transform operator along the y direction, and I1 and I2 are calculated using formula (1-2);

[0021] The two structured illumination images I1 and I2 acquired in step 1 are averaged to obtain a wide-field image WF of the current section with stripes removed, WF = (I1 + I2) / 2;

[0022] Step 2.2, respectively for OS HT Perform Gaussian low-pass filtering with WF to obtain the low-frequency component OS of the current section lp and the high-frequency component WF of the wide-field image WF without fringes on the current slice hp :

[0023]

[0024] In formulas (2-3) and (2-4), represents the convolution operator, σ represents the standard deviation of the Gaussian kernel; WF lp Remove the low-frequency components of the wide-field image of the current slice from the stripes;

[0025] Step 2.3: Convert the OS obtained in step 2.2 to lp and WF hp Reconstruct the optical section OS of the current section HT-SHiLo :

[0026] OS HT-SHiLo =OS lp +ηWF hp (2-5)

[0027] In formula (2-5), η is the scaling factor;

[0028] Step 3, executing step 1 to sample the remaining sections of the sample distributed along the z-axis, and using the method in step 2 to calculate the optical section of the corresponding section for the image collected from each section; the total number of sections of the sample distributed along the z-axis is not less than 2;

[0029] Step 4: stack the optical slices from all the slices to form a three-dimensional image of the sample.

[0030] The present invention can significantly suppress the noise of OS-SIM decoded slices, greatly improve the signal-to-noise ratio of slice images and three-dimensional images, and maintain spatial resolution. Moreover, compared with the traditional frequency domain HiLo algorithm, the present invention saves the time-consuming Fourier transform operation in the spatial domain and greatly improves the processing speed; at the same time, the HT algorithm only needs to collect two original images to obtain a slice, which has higher imaging efficiency than the RMS algorithm. In addition, due to the improvement of the signal-to-noise ratio, the imaging depth of OS-SIM can be expanded by nearly one-fold, which is of great significance for deep volume imaging of biological tissues.

[0031] An optional solution is that the filter radius of the Gaussian low-pass filter in step 2.2 is Standard deviation of the Gaussian kernel

[0032] An alternative solution is to take the OS of all sections of the sample along the z-axis in step 2.3. lp With WF lp The maximum value among the ratios:

[0033]

[0034] Correspondingly, the present invention also provides a related structured light microscope system, which uses the above method of the claim to obtain a three-dimensional image of the sample.

[0035] A storage medium related to the method of the present invention stores a computer program / instruction thereon, which implements the steps of the above method when the computer program / instruction is executed by a processor. And a software product related to the present invention includes a computer program / instruction, which implements the steps of the above method when the computer program / instruction is executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Flow chart of HT-SHiLo algorithm of the present invention;

[0037] Figure 2 These are the simulation results of the HT, HiLo and HT-SHiLo algorithms under different modulation indices in the embodiment; Figures (a)-(d) are the decoded images of each algorithm under the fringe modulation indices m=0.4, 0.2, 0.1, and 0.05, respectively; (e) is the peak signal-to-noise ratio (PSNR) of the decoded images of each algorithm under different modulation indices; (f) is the operation time of each algorithm for different image sizes.

[0038] Figure 33D imaging results of COS-7 cells in the embodiment; Figure a is the maximum projection (MIP) image of COS-7 cells; b and c are local comparison images of the decoded images of the HT algorithm and HT-SHiLo algorithm, respectively; d and e are the three-axis projection image and three-dimensional image of mitochondria, respectively.

[0039] Figure 4 3D imaging results of Thy1 mouse brain nerves in the embodiment; Figures a and b are 3D images processed by HT algorithm and HT-SHiLo algorithm respectively; ce are comparisons of optical slice images decoded by HT algorithm and HT-SHiLo algorithm at depths z=40, 180, and 320 μm respectively. DETAILED DESCRIPTION

[0040] Unless otherwise specified, the scientific and technical terms used herein are understood according to the knowledge of ordinary technicians in the relevant fields.

[0041] The sample to be imaged in the method of the present invention includes but is not limited to biological materials such as cells, cell contents, microorganisms, biological tissues, protein molecules, etc. Before imaging using the method of the present invention, the sample can be treated with fluorescent staining, fixation, and transparency. In a specific scheme, the distance of each section along the z axis or the appropriate inter-layer spacing in the sample is determined according to the depth of field of the objective lens used, for example, about half of the depth of field of the objective lens is taken.

[0042] In order to solve the noise problem of decoding slices under low-profile system, the present invention adopts an algorithm (HT-SHiLo) to reorganize the high and low frequency information of optical slices in the spatial domain, and its flow chart is as follows: Figure 1 First, the HT algorithm is used to decode and obtain the optical slices OS of different sections of the sample. HT , and calculate the wide field image WF; then OS HT Both WF and WF are subjected to Gaussian low-pass filtering (GLPF) to obtain their low-frequency images OS lp 、WF lp and wide field high frequency image WF hp ; Because the high-frequency information in the wide-field image comes only from the focal plane, it can be used to compensate for the high-frequency information in the slice image; thus, the slice image with a high signal-to-noise ratio can be obtained by OS lp and WF hp Combined, namely OS HT-SKHiLo =OS lp +ηWF hp .

[0043] In a specific embodiment, the scaling factor or proportionality factor η is determined by ensuring seamless connection of high and low frequency components at the cutoff frequency. In order to avoid time-consuming Fourier transform, in a preferred embodiment, the scaling factor or proportionality factor η is determined by ensuring seamless connection of high and low frequency components at the cutoff frequency. Used to ensure that low-frequency and high-frequency components are combined in appropriate proportions.

[0044] The filter radius of the Gaussian filter (or the Gaussian kernel size) determines the number of pixels from OS in the final reconstructed slice. HT The denoising effect will increase with the increase of the filter radius, but the slicing effect will decrease accordingly. Therefore, in the preferred embodiment, the present invention uses 1 / v as the filter radius, and the standard deviation of the corresponding Gaussian kernel is

[0045] The present invention is further explained below in conjunction with specific embodiments, but the present invention is not limited thereto. The structured light microscope used in the following embodiments is the structure system disclosed in “ZL201110448980.8 A high-speed structured illumination optical microscope system and method based on digital micromirror device”.

[0046] Embodiment 1:

[0047] In order to simulate the influence of fringe modulation on SIM decoding slices and the processing effect of the algorithm, fringe with different modulation intensity m is added to an initial image, and the HT decoding algorithm, the traditional HiLo algorithm and the HT-SHiLo algorithm are used respectively (the HT-SHiLo algorithm of this embodiment specifically adopts step 2 of the present invention, and the two images in the processing process are respectively generated by adding two fringe with a phase shift difference of π to the initial image, wherein The filter radius of the Gaussian low-pass filter is η Take all slice OS lp With WF lp The maximum value of the ratio) is used to calculate the optical sectioning of the initial image.

[0048] Figure 2 The results are shown for modulation strengths m = 0.4, 0.2, 0.1, and 0.05. It is found that as the modulation strength decreases, the image noise of HT decoding increases significantly, while the images calculated by HiLo and HT-SHiLo algorithms are less affected.

[0049] Figure 2 (e) The peak signal-to-noise ratio (PSNR) of the images calculated by each algorithm under different modulation intensities is given. The results show that as the modulation intensity decreases, the PSNR of the HT decoded image decreases rapidly, while the PSNR of the image processed by the HT-SHiLo algorithm can be maintained at a high level, and the PSNR of the image processed by the HiLo algorithm is between the two.

[0050] Combination Figure 2As shown in (f), the running time of each algorithm on the GPU (RTX3070) was measured multiple times by MATLAB (R2023a). The results show that the HT-SHiLo algorithm only needs an additional 3ms of processing time compared to the HT algorithm, while the HiLo algorithm requires more processing time, which increases exponentially with the increase of image size. At 2048×2048 pixels, the processing speed of the HT-SHiLo algorithm is about 32 times faster than that of the HiLo algorithm.

[0051] Embodiment 2:

[0052] This embodiment uses the HT algorithm and the method of the present invention to perform three-dimensional imaging of the cell nucleus (DAPI labeled, 405nm excitation) and mitochondria (Mito-Tracker Green labeled, 488nm excitation) of fluorescently stained COS-7 cells.

[0053] In the method of this embodiment, m, v, The appropriate interlayer spacing values ​​are: m is about 0.2, v is 0.91μm-1, The inter-layer spacing is randomly set by the system to be 0.2 μm; the sample generates 36 layers of optical slices; The filter radius of the Gaussian low-pass filter is η Take all slice OS lp With WF lp The maximum value of the ratio.

[0054] The imaging results of this embodiment are as follows Figure 3 As shown, cyan represents the nucleus and yellow represents the mitochondria. Figure 3 The comparison between the HT algorithm and the method of the present invention (HT-SHiLo algorithm) in (b) and (c) shows that the method of the present invention removes the noise in the image well and removes the residual stripes; Figure 3 (d) and (e) show the three-dimensional morphology of local mitochondria, indicating that the method of the present invention has good spatial resolution.

[0055] Embodiment 3:

[0056] This example uses the HT algorithm and the method of the present invention to perform three-dimensional imaging of rat brain nerves labeled with Thy1-EGFP (488 nm excitation).

[0057] The m, v, The appropriate interlayer spacing values ​​are: m decreases gradually from shallow to deep, about 0.2 to 0.05, v is 0.18μm-1, The layer spacing is randomly set by the system to be 4 μm; the sample generates 143 layers of optical slices; The filter radius of the Gaussian low-pass filter is η Take all slice OS lp With WF lp The maximum value of the ratio.

[0058] Figure 4 The results of the HT algorithm of this embodiment and the method of the present invention (HT-SHiLo algorithm) are compared. It can be seen that the HT algorithm loses a lot of information as the imaging depth increases, while the method of the present invention can maintain a high signal-to-noise ratio and can clearly image the deep part of the sample. Figure 4 It can be seen from (a) and (b) that the imaging depth of the method of the present invention is nearly doubled compared with the HT algorithm.

Claims

1. A reconstruction method for three-dimensional fluorescence imaging of a structured light microscope, characterized in that: The following steps are involved: Step 1: Collect structured illumination images: Under the sinusoidal fringe structured illumination light field, two structured illumination images I1 and I2 of the current section of the sample to be imaged are collected, where the fringe phase shift difference between I1 and I2 is π, and: or: In formula (1-1) and (1-2), I out (x, y, z) represents the out-of-focus background part of the image, I in (x, y, z) represents the focused part of the image, m represents the fringe modulation intensity, v represents the spatial frequency of the fringe, Represents the initial phase; Step 2: Calculate the optical section of the current section Step 2.1: Use the Hilbert transform algorithm to decode the two structured illumination images I1 and I2 acquired in step 1 to obtain the HT-decoded light slice OS. HT : or, In formula (2-1), HT x {} represents the one-dimensional Hilbert transform operator along the x direction, and I1 and I2 are calculated using formula (1-1); In formula (2-2), HT y {} represents the one-dimensional Hilbert transform operator along the y direction, and I1 and I2 are calculated using formula (1-2); The two structured illumination images I1 and I2 acquired in step 1 are averaged to obtain a wide-field image WF of the current section with stripes removed, WF = (I1 + I2) / 2; Step 2.2, respectively for OS HT Perform Gaussian low-pass filtering with WF to obtain the low-frequency component OS of the current section lp and the high-frequency component WF of the wide-field image WF without fringes on the current slice hp : In formulas (2-3) and (2-4), represents the convolution operator, σ represents the standard deviation of the Gaussian kernel; WF lp Remove the low-frequency components of the wide-field image of the current slice from the stripes; Step 2.3: Convert the OS obtained in step 2.2 to lp and WF hp Reconstruct the optical section OS of the current section HT-SHiLo : YOU HT-SHiLo =OS lp +ηWF hp (2-5) In formula (2-5), η is the scaling factor; Step 3, executing step 1 to sample the remaining sections of the sample distributed along the z-axis, and using the method in step 2 to calculate the optical section of the corresponding section for the image collected from each section; the total number of sections of the sample distributed along the z-axis is not less than 2; Step 4: stack the optical slices from all the slices to form a three-dimensional image of the sample.

2. The reconstruction method for three-dimensional fluorescence imaging of a structured light microscope according to claim 1, characterized in that: The filter radius of the Gaussian low-pass filter in step 2.2 is Standard deviation of the Gaussian kernel 3. The reconstruction method for three-dimensional fluorescence imaging of a structured light microscope according to claim 1, characterized in that: In step 2.3, η is the OS of all sections of the sample distributed along the z-axis. lp With WF lp The maximum value among the ratios:

4. A structured light microscope system, characterized in that: The system adopts the method described in claim 1, 2 or 3 to obtain a three-dimensional image of the sample.

5. A storage medium, characterized in that: A computer program / instruction is stored thereon, wherein the computer program / instruction, when executed by a processor, implements the steps of the method described in any one of claims 1 to 3.

6. A software product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the method described in any one of claims 1 to 3 are implemented.

Images