A Fluorescence Microscopy Image Reconstruction Method Based on Noise Compensation Guided Filtering and Mismatched Projection Deconvolution

Abstract

This invention discloses a method for reconstructing low-quality fluorescence microscopy images based on noise-compensated guided filtering and mismatched projection deconvolution. This method achieves efficient detail preservation and background noise suppression through noise-compensated guided filtering, which is then used as input to the mismatched projection deconvolution method to remove blur caused by diffraction. The pre-denoising step effectively suppresses noise amplification during the deconvolution process. The entire process improves the signal-to-noise ratio and restores the resolution of the image. Therefore, this invention can simply and effectively improve the quality of images acquired under low-light-dose imaging conditions, which is beneficial for information acquisition in long-term fluorescence imaging of live cells.

Description

Technical Field

[0001] This invention belongs to the field of fluorescence microscopy imaging technology, specifically relating to a method for reconstructing low-quality fluorescence microscopy images based on noise-compensated guided filtering and mismatched projection deconvolution. Background Technology

[0002] Fluorescence microscopy is widely used in cell biology research. However, biological cells are highly sensitive to light; prolonged or high-intensity light exposure can cause phototoxicity, damaging cell structure and function. Furthermore, fluorescent dyes or proteins gradually lose their fluorescence ability under prolonged or high-intensity light exposure, limiting the imaging time of fluorescence imaging experiments. To achieve long-term live-cell fluorescence imaging, researchers need to reduce the intensity of the excitation light or shorten the exposure time, resulting in a lower number of photons per image and creating low-light-dose imaging conditions. Low light dose leads to weak signal intensity and a reduced signal-to-noise ratio. For diffraction-limited fluorescence microscopy systems, due to the diffraction properties of light, the acquired data will also be affected by blurring and degradation. Blurring and low signal-to-noise ratio are the main factors causing image quality degradation under low-light-dose imaging conditions.

[0003] To improve the quality of acquired images, algorithms can be used to reconstruct low-quality images. For data following a Poisson distribution, the Richardson-Lucy algorithm based on maximum likelihood estimation is a common method for image blur restoration. By generalizing the back projection process in the Richardson-Lucy algorithm, a mismatched projection deconvolution method can be constructed (such as the patent technology with application number CN202010263899.1), which greatly improves the deconvolution efficiency. However, this iterative algorithm will cause noise amplification under low signal-to-noise ratio conditions.

[0004] To achieve simultaneous denoising and deblurring, a regularization term can be introduced into the mismatched projection deconvolution method to suppress noise. However, since the mismatched projection deconvolution method requires very few iterations, the regularization term is difficult to be effective, and its computational burden is usually heavy, leading to a decrease in computational efficiency. Therefore, to preserve the iterative efficiency of the mismatched projection deconvolution method, low-quality data can be pre-denoised before deconvolution. Conventional denoising operations, such as Gaussian filters and wavelet filters, are essentially low-pass filters, making image details too blurry.

[0005] Currently, some advanced filters can preserve image details, such as block matching denoising (e.g., the patent technology in application number CN202111077118.0) and bilateral filters (e.g., the patent technology in application number CN202311436747.7). However, these advanced filters typically have a high computational burden and low efficiency. Therefore, how to balance the effectiveness and efficiency of the pre-denoising step and combine it with the mismatched projection deconvolution method to achieve low-quality image reconstruction under low light dose imaging conditions is of great value to the development of fluorescence live cell imaging. Summary of the Invention

[0006] In view of the above, the present invention provides a method for reconstructing low-quality fluorescence microscopy images based on noise-compensated guided filtering and mismatched projection deconvolution. By combining a pre-denoising step and a deconvolution algorithm, high-quality restoration of low-quality images under low light dose imaging conditions can be achieved.

[0007] A method for reconstructing low-quality fluorescence microscopy images based on noise-compensated guided filtering and mismatched projection deconvolution includes the following steps:

[0008] (1) The biological samples were fluorescently labeled, and the images of the biological samples were acquired using a fluorescence microscope under extremely low light dose imaging conditions. A series of acquired two-dimensional images were stacked to obtain a three-dimensional fluorescence microscopic image of the biological samples.

[0009] (2) Axial interpolation is performed on the three-dimensional fluorescence microscopy image to make the step size represented by each pixel in the axial and horizontal directions the same. Then, variance stabilization transformation is performed on the three-dimensional fluorescence microscopy image to obtain a low-quality image that conforms to the additive noise characteristics. At the same time, the point spread function of the fluorescence microscope is obtained by measurement or simulation based on the imaging parameters and normalized.

[0010] (3) Use filters to smooth low-quality images to obtain guide images and pre-estimated noisy images;

[0011] (4) Based on the guide image and the estimated noise image, a noise-compensated guided filtering algorithm is used to denoise the low-quality image to obtain a blurred image with a high signal-to-noise ratio.

[0012] (5) The normalized point spread function is used as the forward projection operator and its corresponding Wiener-Butterworth backward projection operator is calculated to construct the mismatch projection deconvolution algorithm. The blurred image is used as the input of the algorithm to obtain a high signal-to-noise ratio and high resolution reconstructed image through iteration.

[0013] Furthermore, the extremely low light dose imaging conditions in step (1) are low excitation light intensity or short exposure time, and the acquired two-dimensional image contains a large amount of noise (Poisson noise and Gaussian noise), with an extremely low signal-to-noise ratio; the point spread function is a three-dimensional matrix structure with a Gaussian distribution.

[0014] Furthermore, the degradation equation of the three-dimensional fluorescence micrograph obtained by image acquisition in step (1) is expressed as follows:

[0015]

[0016] Where: o(x,y,z) represents the biological sample with coordinates (x,y,z), p0(x,y,z) represents the pixel value of coordinates (x,y,z) in the three-dimensional fluorescence microscopy image, h(x,y,z) represents the point spread function of coordinates (x,y,z), n(x,y,z) represents the additive Gaussian noise of coordinates (x,y,z) during imaging, and P() represents the Poisson random process that introduces photon noise during imaging. This represents the convolution operator.

[0017] Furthermore, in step (2), the variance stabilization transformation of the three-dimensional fluorescence micrograph is performed using the following formula:

[0018]

[0019] Where p represents a low-quality image, and p0 represents a three-dimensional fluorescence microscopy image.

[0020] Furthermore, the calculation expressions for the guiding image and the pre-estimated noise image in step (3) are as follows:

[0021] I = Gau(p,σ)

[0022] N_est=pI

[0023] Where: I is the guiding image, Gau(p,σ) represents the three-dimensional Gaussian smoothing of the low-quality image p with variance σ, and N_est is the pre-estimated noisy image.

[0024] Further, the specific implementation of step (4) is as follows: First, the low-quality image is denoised using the guiding image and the pre-estimated noise image to obtain a smooth image with guided filtering. Then, the difference between the guiding image and the smooth image is calculated as residual noise, and high-frequency detail information is extracted from the residual noise. Then, the smooth image is summed and compensated using the high-frequency detail information to obtain the denoising result. Finally, the denoising result is subjected to variance stabilization inverse transform to obtain a blurred image with high signal-to-noise ratio.

[0025] Furthermore, the calculation expression for the smoothed image is as follows:

[0026]

[0027] Where: ω k This represents the filtering window centered at pixel k, where m is the filtering window ω. k For any pixel in , q′ m I represents the numerical value of pixel m in a smoothed image. m This represents the numerical value of pixel m in the guide image. and Corresponding to p k 、N_est k I k a k b k In the filter window ω k The average pixel value within the range, p k N_est represents the value of pixel k in a low-quality image. k I represents the estimated value of pixel k in the noisy image. k a represents the numerical value of pixel k in the guide image. k and b k The linear coefficient diagram of the guided filter. This indicates the calculation of covariance. This indicates variance calculation, where ε is the adjustment coefficient.

[0028] Furthermore, the calculation expression for the denoising result is as follows:

[0029] q=q′+ΔN

[0030] ΔN=γF(Iq′)

[0031] Where: q is the denoising result, q′ is the smoothed image, ΔN is the high-frequency detail information, I is the guiding image, F() represents the smoothing function with feature extraction mask, and γ is the gain coefficient of residual noise.

[0032] Furthermore, the calculation expression for the Wiener-Butterworth back projection operator in step (5) is as follows:

[0033] B = DFT -1 (B WB (k x ,k y ,k z ))

[0034]

[0035] Where: DFT() represents Discrete Fourier Transform, conj[] represents the complex conjugate operator, α is the parameter of the Wiener filter, and k x ky k z This corresponds to the spatial frequency components along the x, y, and z axes, k cx k cy k cz These correspond to the components of the cutoff frequency along the x, y, and z axes, where F is the forward projection operator, B is the Wiener-Butterworth backward projection operator, ∈ is the maximum bandwidth gain of the Butterworth filter, and n is the order of the Butterworth filter. WB (k x ,k y ,k z ) is the frequency domain representation of the Wiener-Butterworth back projection operator.

[0036] Furthermore, the iterative process of the mismatched projection deconvolution algorithm in step (5) is expressed as follows:

[0037]

[0038] Where: F is the forward projection operator, and B is the Wiener-Butterworth backward projection operator. This represents the convolution operator, where q1 is the blurred image, and e j+1 and e j These represent the reconstructed images of the biological sample after the (j+1)th and jth iterations, respectively.

[0039] This invention provides an efficient reconstruction method for low-quality images acquired through imaging with extremely low signal-to-noise ratios. By optimizing the denoising process of guided filtering, it efficiently achieves pre-denoising results that preserve details and suppress background noise. These results are then used as input to a deconvolution method based on mismatch operators to remove blur caused by diffraction. Through a single process, the signal-to-noise ratio and contrast are restored. This allows for the rapid and simple reconstruction of low-quality fluorescence microscopy images under low light dose imaging conditions with low computational burden, thereby assisting in the realization of long-term fluorescence microscopy imaging of live cells. Attached Figure Description

[0040] Figure 1 Image data obtained by imaging ERmoxGFP-labeled endoplasmic reticulum in live U2OS cells using transient structured light illumination microscopy under low light dose conditions.

[0041] Figure 2 This is a flowchart illustrating the low-quality fluorescence microscopy image reconstruction method based on noise-compensated guided filtering and mismatched projection deconvolution according to the present invention.

[0042] Figure 3 This invention presents the reconstructed image from low-quality data and its comparison with images reconstructed by other methods.

[0043] Figure 4This diagram illustrates the comparison of relevant metrics between reconstructed images and reference data using the present invention and other methods, where (a) represents structural similarity comparison, (b) represents peak signal-to-noise ratio comparison, and (c) represents runtime comparison. Detailed Implementation

[0044] To describe the present invention in more detail, the technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] In three-dimensional fluorescence microscopy, the excitation light path of a fluorescence microscope emits a laser, illuminating biological samples that have been fluorescently labeled or possess autofluorescence. This causes the biological sample to fluoresce, which is then collected by a detector. By continuously moving the imaging focal plane and traversing the entire biological sample, a three-dimensional volumetric image of the sample can be generated by stacking the two-dimensional images acquired each time. Biological samples are typically sensitive to light intensity; excessive light can damage cell structure and function. Furthermore, excessive light can cause photobleaching of fluorescent labels. Therefore, to extend the imaging time as much as possible for long-term imaging of live cells, it is necessary to reduce the excitation light intensity or exposure time for each acquisition. This results in a reduction in the number of photons per image. Combined with light diffraction during imaging, this leads to image degradation, including not only the inherent blurring caused by the microscopic system but also a significant amount of noise.

[0046] like Figure 1 The image shown is obtained by imaging the ERmoxGFP-labeled endoplasmic reticulum in live U2OS cells under low light dose conditions using transient structured light illumination microscopy. The endoplasmic reticulum signal is submerged in a large amount of noise, and its resolution is also low. To reconstruct such a low-quality image, denoising and deblurring are necessary. The Richardson-Lucy deconvolution method is a commonly used method for deblurring fluorescence microscopy images. It is based on the maximum likelihood function following a Poisson distribution and restores resolution iteratively. Through optimization, mismatched projection deconvolution was obtained, significantly improving the algorithm's efficiency. However, for data with extremely low signal-to-noise ratios, the mismatched projection deconvolution method inevitably amplifies noise. To retain the efficiency of deconvolution methods based on mismatched projection operators, this invention designs a low-quality fluorescence microscopy image reconstruction method based on noise-compensated guided filtering and mismatched projection deconvolution. Noise-compensated guided filtering efficiently and easily suppresses noise and preserves details, while mismatched projection deconvolution improves resolution. The entire process is as follows: Figure 2 As shown:

[0047] Step S1: Data Acquisition. Under low-dose imaging conditions, biological sample data is acquired, and under known imaging conditions, the point spread function of the system under the current imaging conditions is obtained through simulation or experimentation. Through data preprocessing, background intensity is removed from the acquired data, and the acquired data and point spread function are interpolated to ensure that their horizontal and axial pixel steps are consistent.

[0048] Step S2: VST Transformation. Because the noise in the acquired data is mainly Poisson noise under low light dose conditions, which is noise that varies with signal intensity and is difficult to process; this invention transforms additive noise into additive noise through variance-stabilizing transformations (VST), using the following formula:

[0049]

[0050] Step S3: Noise-compensated guided filtering. Assuming the total noise is the difference between the guided-filtered image q′ and the unfiltered image p, we can obtain:

[0051] N = pq'

[0052] Using the guiding image I as an intermediate parameter, the noise can be calculated as:

[0053] N=(pI)+(Iq′)=N_est+ΔN

[0054] q′+ΔN=p-N_est

[0055] This invention performs preliminary denoising on the VST-transformed image p using a smoothing filter to obtain the guiding image I and calculates the estimated noise N_est. A typical smoothing filter is a Gaussian filter. By minimizing the loss function using I and N_est, the corresponding guiding filter can be obtained. The minimization loss function is as follows:

[0056]

[0057] By minimizing the loss function, the linear coefficients of the guided filter can be obtained:

[0058]

[0059] Where ε is the adjustment coefficient; if ε is too small, it cannot effectively suppress background noise, and if ε is too large, the result of the guided filtering will become blurred. The setting needs to be adjusted according to the specific experiment. The result after guided filtering is expressed as follows:

[0060]

[0061] Noise compensation is obtained through Iq′. Since it is necessary to remove unnecessary background noise, this invention uses a smoothing function with a feature extraction mask to process the compensation noise. The specific formula is as follows:

[0062] ΔN=γF(Iq′)

[0063] Since the linear coefficients of the guided filter have the ability to extract details, a typical smoothing function with a feature extraction mask can be chosen as follows:

[0064]

[0065] Therefore, the final result of noise-compensated guided filtering is:

[0066] q=q′+ΔN

[0067] In the process of generating the guide image, the typical Gaussian blur variance is selected as 1.0. The lower the signal-to-noise ratio of the low-quality image, the larger the variance should be. In the noise compensation process, the typical Gaussian blur variance is selected as 1.5.

[0068] Step S4: Inverse VST Transform. Perform inverse variance stabilization transform (inverse VST) on the result of noise-compensated guided filtering to obtain high signal-to-noise ratio fuzzy data q1.

[0069] Step S5: Deconvolution of mismatched projections. Generate the forward projection operator and the corresponding backward projection operator based on the system point spread function. The formula for the backward projection operator is as follows:

[0070] B = DFT -1 (B WB (k x ,k y ,k z ))

[0071]

[0072] Where: α is the Wiener filter parameter; if α is too large, the convergence speed is slow, and if α is too small, noise amplification is likely to occur; n is the order of the Butterworth filter, and ∈ is the maximum bandwidth gain of the filter. Through research, the values ​​of α are between 0.001 and 0.05, the values ​​of n are between 10 and 12, and the values ​​of ∈ are between 500 and 100.

[0073] The forward / backward projection operator is introduced into the iteration of the mismatched projection deconvolution algorithm, and the high signal-to-noise ratio blurred data q1 is substituted into the iteration. The specific iteration formula is as follows:

[0074]

[0075] Step S6: Determine whether the iteration stopping condition is met, i.e. whether the expected number of iterations has been reached. If the condition is not met, continue to execute step S5. If the condition is met, the iteration stops. The image estimate obtained from the last iteration is used as the estimate of the final sample. The number of iterations is usually set to 1 to 5.

[0076] The effectiveness of this invention is verified using data obtained from imaging mEmerald-Tomm20-C-10 labeled mitochondria in live U2OS cells under low light dose conditions using transient structured light illumination microscopy. This data exhibits an extremely low signal-to-noise ratio. This experiment compares different pre-denoising algorithms (Gaussian filtering, wavelet filtering, block matching, bilateral filtering, and noise-free compensated guided filtering) combined with mismatched projection deconvolution, and compares this method with direct mismatched projection deconvolution. Deconvolution results from images acquired under high light dose conditions are used as a reference to compare the structural similarity, peak signal-to-noise ratio, and running time of different methods.

[0077] pass Figure 3 and Figure 4 The results show that, in terms of both visual effects and quantitative numerical analysis, the low-quality image reconstruction method based on noise-compensated guided filtering and mismatch operator deconvolution of this invention achieves the best results. It effectively suppresses background noise, preserves details better, and maintains mitochondrial shape well. Compared to other methods, this invention more closely approximates high-quality reference data and produces a cleaner background. Its runtime is significantly faster than advanced filters with detail-preserving capabilities (block matching and bilateral filtering), and its performance surpasses that of fast, simple filters (Gaussian filtering and wavelet filtering). This demonstrates the high efficiency and ease of operation of this invention. Typically, only the parameter ε of the guided filter needs experimental tuning; other parameters can be fixed empirically. It can easily and quickly restore low-quality images under low-light-dose conditions to high quality.

[0078] The above description of the embodiments is provided to enable those skilled in the art to understand and apply the present invention. Those skilled in the art can readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made to the present invention by those skilled in the art based on the disclosure thereof should be within the scope of protection of the present invention.

Claims

1. A method for reconstructing low-quality fluorescence microscopy images based on noise-compensated guided filtering and mismatched projection deconvolution, comprising the following steps: (1) The biological samples were fluorescently labeled, and the images of the biological samples were acquired using a fluorescence microscope under extremely low light dose imaging conditions. A series of acquired two-dimensional images were stacked to obtain a three-dimensional fluorescence microscopic image of the biological samples. (2) Axial interpolation is performed on the three-dimensional fluorescence microscopy image to make the step size represented by each pixel in the axial and horizontal directions the same. Then, variance stabilization transformation is performed on the three-dimensional fluorescence microscopy image to obtain a low-quality image that conforms to the additive noise characteristics. At the same time, the point spread function of the fluorescence microscope is obtained by measurement or simulation based on the imaging parameters and normalized. (3) Use filters to smooth low-quality images to obtain guide images and pre-estimated noisy images; (4) Based on the guide image and the estimated noise image, a noise-compensated guided filtering algorithm is used to denoise the low-quality image to obtain a blurred image with a high signal-to-noise ratio. (5) The normalized point spread function is used as the forward projection operator and its corresponding Wiener-Butterworth backward projection operator is calculated to construct the mismatch projection deconvolution algorithm. The blurred image is used as the input of the algorithm to obtain a high signal-to-noise ratio and high resolution reconstructed image through iteration.

2. The method for reconstructing low-quality fluorescence microscopic images according to claim 1, characterized in that: The extremely low light dose imaging conditions in step (1) are low excitation light intensity or short exposure time, and the acquired two-dimensional image contains a lot of noise with an extremely low signal-to-noise ratio. The point spread function is a three-dimensional matrix structure with a Gaussian distribution.

3. The method for reconstructing low-quality fluorescence microscopic images according to claim 1, characterized in that: The degradation equation of the three-dimensional fluorescence micrograph obtained by image acquisition in step (1) is expressed as follows: Where: o(x,y,z) represents the biological sample with coordinates (x,y,z), p0(x,y,z) represents the pixel value of coordinates (x,y,z) in the three-dimensional fluorescence microscopy image, h(x,y,z) represents the point spread function of coordinates (x,y,z), n(x,y,z) represents the additive Gaussian noise of coordinates (x,y,z) during imaging, and P() represents the Poisson random process that introduces photon noise during imaging. This represents the convolution operator.

4. The method for reconstructing low-quality fluorescence microscopic images according to claim 1, characterized in that: In step (2), the variance stabilization transformation of the three-dimensional fluorescence microscopy image is performed using the following formula: Where p represents a low-quality image, and p0 represents a three-dimensional fluorescence microscopy image.

5. The method for reconstructing low-quality fluorescence microscopic images according to claim 1, characterized in that: The calculation expressions for the guiding image and the pre-estimated noise image in step (3) are as follows: I = Gau(p,σ) N_est=pI Where: I is the guiding image, Gau(p,σ) represents the three-dimensional Gaussian smoothing of the low-quality image p with variance σ, and N_est is the pre-estimated noisy image.

6. The method for reconstructing low-quality fluorescence microscopic images according to claim 1, characterized in that: The specific implementation method of step (4) is as follows: First, the low-quality image is denoised using the guide image and the pre-estimated noise image to obtain a smooth image with guide filtering. Then, the difference between the guide image and the smooth image is calculated as residual noise, and high-frequency detail information is extracted from the residual noise. Then, the smooth image is summed and compensated using the high-frequency detail information to obtain the denoising result. Finally, the variance stabilization inverse transform is performed on the denoising result to obtain a blurred image with high signal-to-noise ratio.

7. The method for reconstructing low-quality fluorescence microscopic images according to claim 6, characterized in that: The calculation expression for the smoothed image is as follows: Where: ω k This represents the filtering window centered at pixel k, where m is the filtering window ω. k For any pixel in , q′ m I represents the numerical value of pixel m in a smoothed image. m This represents the numerical value of pixel m in the guide image. and Corresponding to p k 、N_est k I k a k b k In the filter window ω k The average pixel value within the range, p k N_est represents the value of pixel k in a low-quality image. k I represents the estimated value of pixel k in the noisy image. k a represents the numerical value of pixel k in the guide image. k and b k The linear coefficient diagram of the guided filter. This indicates the calculation of covariance. This indicates variance calculation, where ε is the adjustment coefficient.

8. The method for reconstructing low-quality fluorescence microscopic images according to claim 6, characterized in that: The calculation expression for the denoising result is as follows: q=q′+ΔN ΔN=γF(Iq′) Where: q is the denoising result, q′ is the smoothed image, ΔN is the high-frequency detail information, I is the guiding image, F() represents the smoothing function with feature extraction mask, and γ is the gain coefficient of residual noise.

9. The method for reconstructing low-quality fluorescence microscopic images according to claim 1, characterized in that: The calculation expression for the Wiener-Butterworth back projection operator in step (5) is as follows: B=DFT -1 (B WB (k x ,k y ,k z )) Where: DFT() represents Discrete Fourier Transform, conj[] represents the complex conjugate operator, α is the parameter of the Wiener filter, and k x k y k z This corresponds to the spatial frequency components along the x, y, and z axes, k cx k cy k cz These correspond to the components of the cutoff frequency along the x, y, and z axes, where F is the forward projection operator, B is the Wiener-Butterworth backward projection operator, ∈ is the maximum bandwidth gain of the Butterworth filter, and n is the order of the Butterworth filter. WB (k x ,k y ,k z ) is the frequency domain representation of the Wiener-Butterworth back projection operator.

10. The method for reconstructing low-quality fluorescence microscopic images according to claim 1, characterized in that: The iterative process of the mismatched projection deconvolution algorithm in step (5) is expressed as follows: Where: F is the forward projection operator, and B is the Wiener-Butterworth backward projection operator. This represents the convolution operator, where q1 is the blurred image, and e j+1 and e j These represent the reconstructed images of the biological sample after the (j+1)th and jth iterations, respectively.

Images