High-speed optical layer-cutting three-dimensional structure light illumination microscopic imaging method

Abstract

The application discloses a high-speed optical layer-cut three-dimensional structure light illumination microscopic imaging method. The application solves a phase correlation coefficient, obtains an initial three-dimensional super-resolution image by using an original image, and then performs optimization to obtain a final super-resolution image according to the optimized super-resolution image, a wide-field image and a three-dimensional optical layer-cut image. The application combines spatial domain reconstruction and optical layer-cut, improves the speed of traditional 3DSIM reconstruction by 2-3 orders of magnitude, greatly suppresses the reconstruction artifacts caused by defocus background, and has good reconstruction capability on low signal-to-noise ratio data. The application can greatly reduce the operation cost, has the potential of real-time imaging and high-throughput imaging, has obvious advantages in reconstruction speed, fidelity and thick sample reconstruction compared with other algorithms, and greatly promotes the application of 3DSIM in large field of view and real-time imaging.

Description

Technical Field

[0001] This invention relates to data processing of microscopic images, specifically to a high-speed optical layer-cutting method for illuminating three-dimensional structures for microscopic imaging. Background Technology

[0002] Three-dimensional structured illumination microscopy (3DSIM) technology simultaneously increases the three-dimensional spatial resolution to twice the diffraction limit through layer-by-layer acquisition and frequency domain shifting. However, most existing 3DSIM reconstruction techniques are based on traditional frequency domain reconstruction, in which frequency domain separation and order shifting operations are extremely time-consuming, and the reconstruction speed is usually on the order of minutes to hours, which greatly limits the processing of large field-of-view data and the real-time observation of living cells.

[0003] Furthermore, since 3DSIM is typically used for imaging thick samples, and the out-of-focus background of thick samples can significantly interfere with reconstruction, traditional 3DSIM reconstruction usually ignores or uses notch filters to suppress the out-of-focus background in thick sample imaging, failing to completely remove background interference or introducing various artifacts. Therefore, in order to improve reconstruction speed and reduce background interference, developing a high-speed reconstruction and optically tomographic 3DSIM reconstruction technique has become an urgent need in the field of fluorescence microscopy. Summary of the Invention

[0004] To address the issues of slow reconstruction speed and inability to eliminate background in the 3DSIM algorithm, this invention proposes a high-speed optical layer-cutting method for illuminating micro-imaging of three-dimensional structures.

[0005] The high-speed optical layer-cutting three-dimensional structure illumination micro-imaging method of the present invention includes the following steps:

[0006] 1) Obtain the original image stack and acquire the illumination fringe parameters:

[0007] Three-dimensional structured light is used to illuminate the sample, with M illumination angles, each with 5 phases; one raw image is acquired for each phase at each illumination angle. Where, θ i For the i-th illumination angle, Let be the i-th phase, and r and z represent the horizontal and vertical vectors in the cylindrical coordinate system, respectively. The sample is scanned layer by layer, with a total of T layers, resulting in M×5×T original images. The fringe parameters of the structured light are estimated by a cross-correlation iterative parameter estimation algorithm. The fringe parameters include the frequency ω of the xoy plane, the illumination angle, the modulation coefficients a1 and a2 of the first and second-order frequencies, and the phase; i = 1, ..., M, j = 1, ..., 5.

[0008] 2) Obtain the three-dimensional optical tomographic image:

[0009] The three-dimensional wide-field image at the i-th illumination angle is calculated by averaging the original images of the five phases at the i-th illumination angle of each layer. Then, the T layers in the original image and the 3D wide-field image are passed through high-pass and low-pass filters respectively to obtain the optical slice images of each layer. These optical slice images are then stacked to obtain the 3D optical slice image SR. OS (r, z);

[0010] 3) Solve for the phase correlation coefficient:

[0011] a) Obtain the phase solution equation:

[0012]

[0013] in, Let O(r, z) be the phase correlation coefficient of the j-th phase at the i-th illumination angle, and let O(r, z) be the true fluorescence distribution. 1+cos(ω z z)·cos(ωr)+cos(2ωr) is the modulation factor, H(r,z) is the system point spread function, and ω and ω z These are the frequencies of the xoy plane and the xoz plane, respectively;

[0014] b) Solving the equation by transforming the phase:

[0015]

[0016] Where I0 is the intensity amplitude of the structured light. Let r′ and r be the initial phases of the structured light. j These represent the horizontal vector and the horizontal translation vector after the phase change, respectively.

[0017] c) Solve for the phase correlation coefficient matrix A using the phase equation:

[0018]

[0019] 4) Obtain the initial super-resolution image:

[0020] Using the phase correlation coefficient obtained in step 3), the original images of the M illumination angles of each layer are stacked to obtain the initial three-dimensional super-resolution image SR0(r, z).

[0021] 5) Frequency domain optimization:

[0022] The spectral peaks of the initial 3D super-resolution image SR0(r,z) were notched using a 3D notch function, and the spectrum was optimized by multi-step Wiener filtering to obtain the optimized 3D super-resolution image SR1(r,z).

[0023] 6) Obtain the super-resolution image:

[0024] The initial super-resolution image contains a wide-field image, which has a strong defocused background. The optimized super-resolution image SR1(r,z) is used to subtract the mean of the wide-field image at each illumination angle and add the three-dimensional optical layer image SR0(r,z) obtained in step 2) to obtain the final super-resolution image.

[0025] In step 1), the number of illumination angles M of the three-dimensional structured light is 2 to 10. The interval between two adjacent phases is 2π / 5. The number of layers T ranges from 1 to 1000.

[0026] In step 2), the three-dimensional wide-field image of the i-th illumination angle 3D optical tomographic images Where Lo[·] and Hi[·] represent low-pass and high-pass filters, respectively.

[0027] In step 3)a), obtaining the phase solution equation includes the following steps:

[0028] Super-resolution image at the i-th illumination angle It has two representations: the first is a linear combination of the images of the five phases, i.e. in, The first is the phase correlation coefficient of the j-th phase at the i-th illumination angle; the second representation is the point spread function H(r,z) convolved by multiplying the modulation factor and the true fluorescence distribution O(r,z), where ω and ω z Let be the frequencies of the xoy plane and the xoz plane, respectively; from this, we obtain the phase solution equation for calculating the phase correlation coefficient:

[0029]

[0030] In step 3)b), the transformation phase solution equation includes the following steps:

[0031] Original image Represented as the emission fluorescence distribution O(r, z) multiplied by the structured light intensity distribution The expression for the point spread function H(r, z) of a convolutional system is, i.e.

[0032] in, Where a1 and a2 represent the modulation coefficients of the first and second frequencies, respectively, and I0 is the intensity amplitude of the structured light. This represents the initial phase of the structured light;

[0033] Apply the j-th phase to the structured light View this as a translation of the horizontal vector r, i.e., r = r′ - r j , where r′ and r j These represent the horizontal vector and horizontal translation vector after the phase change, respectively. Therefore, the structured light intensity distribution after transformation for:

[0034]

[0035] At the same time, the modulation factor is rewritten as 1+cos(ω z z)·cos(ωr′-ωr j )+cos(2ωr′-2ωr j Therefore, we obtain the following formula:

[0036]

[0037] The equation for solving the phase transformation is obtained based on the sum-to-product formula.

[0038] In step 3)c), solving for the phase correlation coefficient based on the phase equation includes the following steps:

[0039] according to and The orthogonality of the equations leads to the following equation:

[0040] Based on the above system of linear equations and We obtain the following formula:

[0041]

[0042] The phase correlation coefficient is expressed as: The phase correlation coefficient matrix can be obtained from the above formula.

[0043] In step 4), the initial super-resolution image

[0044] In step 6), the wide-field image at each illumination angle is: Final 3D super-resolution image

[0045] Advantages of this invention:

[0046] This invention improves the reconstruction speed of traditional 3DSIM by 2 to 3 orders of magnitude by combining spatial reconstruction with optical tomography, and greatly suppresses reconstruction artifacts caused by defocused backgrounds. It also has excellent reconstruction capabilities on low signal-to-noise ratio data. This invention can greatly reduce computational costs and has the potential for real-time and high-throughput imaging. It has significant advantages over other algorithms in reconstruction speed, fidelity, and thick sample reconstruction, thus greatly promoting the application of 3DSIM in large field of view and real-time imaging. Attached Figure Description

[0047] Figure 1 This is a flowchart of the high-speed optical layer-cutting three-dimensional structure illumination micro-imaging method of the present invention;

[0048] Figure 2 This is a comparison of the reconstruction speed of the high-speed optical layer-cutting three-dimensional structure illumination micro-imaging method of the present invention and the traditional reconstruction method;

[0049] Figure 3 This is a comparison of the reconstruction effects of the high-speed optical layer-cutting three-dimensional structure illumination micro-imaging method of the present invention and the traditional reconstruction method. Detailed Implementation

[0050] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0051] like Figure 1 As shown, the high-speed optical layer-cutting three-dimensional structure illumination micro-imaging method of this embodiment includes the following steps:

[0052] 1) Obtain the original image stack and acquire the illumination fringe parameters:

[0053] Three-dimensional structured light was used to illuminate the sample, with three illumination angles and five phases for each angle, with a 2π / 5 interval between adjacent phases. One raw image was acquired for each phase at each illumination angle. Where, θ i For the i-th illumination angle, Let r and z represent the horizontal and vertical vectors in the cylindrical coordinate system, respectively, for the j-th phase. The samples are scanned layer by layer, with a total of T layers scanned, resulting in 3×5×T original images. The fringe parameters of the structured light are estimated using a cross-correlation iterative parameter estimation algorithm. These parameters include the frequency ω in the xoy plane, the illumination angle, and the modulation coefficients a of the first and second-order frequencies. a 2 and phase; i = 1, ..., 3, j = 1, ..., 5;

[0054] 2) Solve for the optical slice image of each layer:

[0055] The three-dimensional wide-field image at the i-th illumination angle is calculated by averaging the original images of the five phases at the i-th illumination angle of each layer. Then, the T layers in the original image and the 3D wide-field image are passed through high-pass and low-pass filters respectively to obtain the optical slice images of each layer. These optical slice images are then stacked to obtain the 3D optical slice image SR. OS (r, z):

[0056] Where Lo[·] and Hi[·] represent low-pass and high-pass filters, respectively;

[0057] 3) Solve for the phase correlation coefficient:

[0058] a) Obtain the phase solution equation:

[0059] Super-resolution image at the i-th illumination angle It has two representations: the first is a linear combination of the images of the five phases, i.e. in, The first is the phase correlation coefficient of the j-th phase at the i-th illumination angle; the second representation is the point spread function H(r,z) convolved by multiplying the modulation factor and the true fluorescence distribution O(r,z), where ω and ω z Let be the frequencies of the xoy plane and the xoz plane, respectively; from this, we obtain the phase solution equation for calculating the phase correlation coefficient:

[0060]

[0061] b) Simplify the phase solution equation:

[0062] Original image Represented as the emission fluorescence distribution O(r, z) multiplied by the structured light intensity distribution The expression for the point spread function H(r, z) of a convolutional system is, i.e. in,

[0063] Where a1 and a2 represent the modulation coefficients of the first and second frequencies, respectively, and I0 is the intensity amplitude of the structured light. This represents the initial phase of the structured light;

[0064] Apply the j-th phase to the structured light View this as a translation of the horizontal vector r, i.e., r = r′ - r j , where r′ and r j These represent the horizontal vector and horizontal translation vector after the phase change, respectively. Therefore, the structured light intensity distribution after transformation for

[0065]

[0066] At the same time, the modulation factor is rewritten as 1+cos(ω z z)·cos(ωr′-ωr j )+cos(2ωr′-2ωr j Therefore, we obtain the following formula:

[0067]

[0068] According to the sum-to-product formula, we get:

[0069]

[0070] c The phase correlation coefficient is obtained by solving the phase equation:

[0071] according to and The orthogonality of the equations leads to the following equation:

[0072] Based on the above system of linear equations and We obtain the following formula:

[0073]

[0074] The phase correlation coefficient is expressed as: Phase correlation coefficient

[0075] 4) Obtain the initial super-resolution image:

[0076] Using the phase correlation coefficient obtained in step 3), the original images of the M illumination angles of each layer are stacked to obtain the initial 3D super-resolution image.

[0077] 5) Frequency domain optimization:

[0078] The spectral peaks of the initial 3D super-resolution image SR0(r,z) were notched using a 3D notch function, and the spectrum was optimized by multi-step Wiener filtering to obtain the optimized 3D super-resolution image SR1(r,z).

[0079] 6) Optical layer cutting:

[0080] The initial super-resolution image includes a wide-field image, which has a strong out-of-focus background. The optimized super-resolution image SR1(r,z) is then used, subtracting the mean of the wide-field image at each illumination angle. Adding the three-dimensional optical tomographic image SR0(r, z) obtained in step 2), the final super-resolution image is obtained:

[0081]

[0082] from Figure 2 As can be seen from the data, when the reconstruction time of the traditional reconstruction method using the open-source 3D structured illumination microscopy reconstruction method (Open-3DSIM) is statistically analyzed for images of different sizes and depths, it can be found that the reconstruction time of the present invention is 86.2 times to 855.7 times faster than that of the traditional reconstruction method. This proves that the method of the present invention has a fast reconstruction speed, can greatly reduce the computational cost, and has the potential for real-time imaging and high-throughput imaging.

[0083] like Figure 3 As shown, traditional reconstruction method 1 is a noise-suppressed 3D structured micro-imaging reconstruction method with obvious illumination, while traditional reconstruction method 2 is an open-source 3D structured micro-imaging reconstruction method with obvious illumination. Figure 3 As can be seen, the high-speed optical layer cutting three-dimensional structure illumination micro-imaging method of the present invention can effectively suppress the defocus background of the image and reduce the artifact problem caused by the defocus background in thick sample imaging.

[0084] Finally, it should be noted that the purpose of disclosing the embodiments is to help further understand the present invention. However, those skilled in the art will understand that various substitutions and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the embodiments, and the scope of protection of the present invention is defined by the claims.

Claims

1. A high-speed optical layer-cutting method for illuminating three-dimensional structures, characterized in that, The imaging method includes the following steps: 1) Obtain the original image stack and acquire the illumination fringe parameters: Three-dimensional structured light is used to illuminate the sample, with M illumination angles, each with 5 phases; one raw image is acquired for each phase at each illumination angle. Where, θ i For the i-th illumination angle, Let r and z represent the horizontal and vertical vectors in the cylindrical coordinate system, respectively, for the j-th phase. The sample is scanned layer by layer for a total of T layers, resulting in M×5×T original images. The fringe parameters of the structured light are estimated by a cross-correlation iterative parameter estimation algorithm. The fringe parameters include the frequency ω of the xoy plane, the illumination angle, the modulation coefficients a1 and a2 of the first and second-order frequencies, and the phase; i = 1, ..., M, j = 1, ..., 5. 2) Obtain the three-dimensional optical tomographic image: The three-dimensional wide-field image at the i-th illumination angle is calculated by averaging the original images of the five phases at the i-th illumination angle of each layer. Then, the T layers in the original image and the 3D wide-field image are passed through high-pass and low-pass filters respectively to obtain the optical slice images of each layer. These optical slice images are then stacked to obtain the 3D optical slice image SR. OS (r, z); 3) Solve for the phase correlation coefficient: a) Obtain the phase solution equation: in, Let O(r, z) be the phase correlation coefficient of the j-th phase at the i-th illumination angle, and let O(r, z) be the true fluorescence distribution. 1+cos(ω z z)·cos(ωr)+cos(2ωr) is the modulation factor, H(r,z) is the system point spread function, and ω and ω z These are the frequencies of the xoy plane and the xoz plane, respectively; b) Solving the equation by transforming the phase: Where I0 is the intensity amplitude of the structured light. Let r′ and r be the initial phases of the structured light. j These represent the horizontal vector and the horizontal translation vector after the phase change, respectively. c) Solve for the phase correlation coefficient matrix A using the phase solution equation: 4) Obtain the initial super-resolution image: Using the phase correlation coefficient obtained in step 3), the original images of the M illumination angles of each layer are stacked to obtain the initial three-dimensional super-resolution image SR0(r, z). 5) Frequency domain optimization: The spectral peaks of the initial 3D super-resolution image SR0(r,z) were notched using a 3D notch function, and the spectrum was optimized by multi-step Wiener filtering to obtain the optimized 3D super-resolution image SR1(r,z). 6) Obtain the super-resolution image: The initial super-resolution image contains a wide-field image, which has a strong defocused background. The optimized super-resolution image SR1(r,z) is used to subtract the mean of the wide-field image at each illumination angle and add the three-dimensional optical layer image SR0(r,z) obtained in step 2) to obtain the final super-resolution image.

2. The imaging method as described in claim 1, characterized in that, In step 1), the number M of the illumination angles of the three-dimensional structured light is 2 to 10.

3. The imaging method as described in claim 1, characterized in that, In step 2), the three-dimensional wide-field image of the i-th illumination angle 4. The imaging method as described in claim 1, characterized in that, In step 2), the three-dimensional optical tomographic image Where Lo[·] and Hi[·] represent low-pass and high-pass filters, respectively.

5. The imaging method as described in claim 1, characterized in that, In step 3)a), obtaining the phase solution equation includes the following steps: Super-resolution image at the i-th illumination angle It has two representations: the first is a linear combination of the images of the five phases, i.e. in, The first is the phase correlation coefficient of the j-th phase at the i-th illumination angle; the second representation is the point spread function H(r,z) convolved by multiplying the modulation factor and the true fluorescence distribution O(r,z), where ω and ω z Let be the frequencies of the xoy plane and the xoz plane, respectively; from this, we obtain the phase solution equation for calculating the phase correlation coefficient:

6. The imaging method as described in claim 1, characterized in that, In step 3)b), the transformation phase solution equation includes the following steps: Original image Represented as the emission fluorescence distribution O(r, z) multiplied by the structured light intensity distribution The expression for the point spread function H(r, z) of a convolutional system is, i.e. in, Where a1 and a2 represent the modulation coefficients of the first and second frequencies, respectively, and I0 is the intensity amplitude of the structured light. This represents the initial phase of the structured light; Apply the j-th phase to the structured light This can be viewed as a translation of the horizontal vector r, i.e., r = r′ - r j , where r′ and r j These represent the horizontal vector and horizontal translation vector after the phase change, respectively. Therefore, the structured light intensity distribution after transformation for: At the same time, the modulation factor is rewritten as 1+cos(ω z z)·cos(ωr′-ωr j )+cos(2ωr′-2ωr j Therefore, we obtain the following formula: The equation for solving the phase transformation is obtained based on the sum-to-product formula.

7. The imaging method as described in claim 1, characterized in that, In step 3)c), solving for the phase correlation coefficient based on the phase equation includes the following steps: according to and The orthogonality of the equations leads to the following equation: Based on the above system of linear equations and We obtain the following formula: The phase correlation coefficient is expressed as: The phase correlation coefficient matrix can be obtained from the above formula.

8. The imaging method as described in claim 1, characterized in that, In step 4), the initial super-resolution image 9. The imaging method as described in claim 1, characterized in that, In step 6), the wide-field image at each illumination angle is:

10. The imaging method as described in claim 1, characterized in that, In step 6), the final 3D super-resolution image

Images