Fourier lamination microscopic preprocessing noise reduction method based on DnSwin

Abstract

The invention discloses a Fourier lamination microscopy preprocessing noise reduction method based on DnSwin. Noise exists in a low-resolution image obtained through a Fourier lamination microscopic optical path, and reconstruction quality is affected. The method comprises the following steps: firstly, calculating NAill and NAobj for a low-resolution image, dividing the low-resolution image into a bright field image and a dark field image, and adding Poisson Gaussian noise to the bright field image and the dark field image to construct a noise data set; and carrying out noise reduction on a noisy low-resolution image through the trained DnSwin network, and carrying out Fourier lamination microscopic reconstruction to obtain a high-resolution amplitude and a high-resolution phase. Experiments prove that the method achieves a good noise reduction effect in practical application, and the robustness of the Fourier lamination reconstruction system to noise is enhanced.

Description

(1) Technical Field

[0001] The present invention belongs to the field of Fourier ptychographic microscopy, and particularly relates to a Fourier ptychographic microscopy preprocessing noise reduction method based on DnSwin. (2) Background Art

[0002] Microscopy technology provides important optical image evidence for the detection of biological samples, the interaction between cell structures, and the occurrence mechanism of life activities. However, with the development of social technology, people's exploration of the microscopic world has gradually become diversified, and microscopy technology can no longer meet these diversified needs. Its traditional imaging mode also restricts the development of microscopy technology.

[0003] Fourier ptychographic microscopy imaging technology combines technical ideas such as phase retrieval, ptychography, and synthetic aperture, breaking the trade-off between the spatial resolution and imaging field of view of traditional microscopes, and realizing the reconstruction of high-resolution amplitude images and phase images while maintaining the original field of view size of the objective lens. During the Fourier ptychographic microscopy imaging process, the quality of the original image data, especially the background and related noise levels in the dark-field images, is particularly important. During the reconstruction process, the image information of the non-zero background is added to the spectrum, which will generate false signals at the positions corresponding to the illumination wave vectors. In the real space, this signal appears as "orange peel" artifacts superimposed on the reconstructed image. However, due to the inappropriate long image acquisition time, the exposure time of the camera usually remains below several hundred milliseconds, resulting in many dark-field images having a low signal-to-noise ratio (SNR). Therefore, improving the system's robustness to noise is an essential part of Fourier ptychographic microscopy imaging. Existing solutions include traditional threshold methods and neural network methods.

[0004] The traditional threshold method realizes noise reduction by calculating the background signal estimate and subtracting this estimate. This method often faces the trade-off between the noise reduction effect and the clarity of the picture structure, and it is impossible to retain the detailed information of the picture while achieving noise reduction.

[0005] The neural network method utilizes the learning ability and pattern recognition ability of the neural network to extract useful signals from the data containing noise and simultaneously reduce or eliminate the influence of noise. However, in the real world, the CNN denoising method trained based on the Poisson-Gaussian noise dataset performs poorly, and the same denoising strategy is adopted for bright-field and dark-field images, resulting in an inability to achieve a good noise reduction effect. (3) Summary of the Invention

[0006] The present invention proposes a Fourier ptychographic microscopy preprocessing noise reduction method based on DnSwin, aiming to solve the problem that the noise introduced during the image acquisition process affects the high-resolution reconstruction.

[0007] To achieve the above object, the present invention provides a Fourier ptychographic microscopy preprocessing noise reduction method based on DnSwin, and the method includes the following steps:

[0008] S1: Build a Fourier ptychographic microscopy optical path to sample the sample to be measured, and record the low-resolution image of the measured sample.

[0009] S2: Simulate the sampling process, simulate the low-resolution image, calculate the NA ill and NA obj of the simulated low-resolution image and classify it into bright-field images and dark-field images.

[0010] S3: Add Gaussian Poisson noise to the bright-field images and dark-field images to construct a noise data set.

[0011] S4: Use the noise data set to train the DnSwin noise reduction network.

[0012] S5: Denoise the real noisy low-resolution image through the trained DnSwin noise reduction network, and perform Fourier ptychographic microscopy reconstruction on the denoised image to obtain the amplitude and phase of the high resolution.

[0013] In the S1, the Fourier ptychographic microscopy optical path is a 4f system composed of an array of LED light sources, a microscope objective lens, a tube lens, and an imaging camera. A set of low-resolution images is obtained by illuminating LEDs at different positions.

[0014] In the S2, the formula for calculating NA ill and NA obj is:

[0015]

[0016]

[0017]

[0018]

[0019] where n is the refractive index of the medium in the illumination path, which is 1 in air, is the maximum half angle at which the objective lens can collect light, usually determined by the hardware device, (x center , y center ) represents the LED center coordinates, (x m , y m ) represents the coordinates of the m-th LED, and L represents the distance from the LED array to the sample.

[0020] In the low-resolution image, if NA ill >NA obj, it is classified as a dark-field image if NA ill ≤NA obj , it is classified as a bright-field image.

[0021] In the above S3, the noise introduced during the acquisition process includes the noise brought by the camera dark current, stray light, as well as the read noise and quantization noise. When constructing the data set, the formula for adding Gaussian-Poisson noise is:

[0022]

[0023] where y is the image after adding noise, k is the Poisson parameter, x * is the clean image, μ is the background signal value, and v is the standard deviation of Gaussian noise.

[0024] In the above S4, the two-channel input of the bright-field image and the dark-field image is used to replace the RGB three-channel input in DnSwin. The image is decomposed into sub-bands of different frequencies through wavelet transform, and the low-frequency and high-frequency information is processed separately. Combining the advantages of CNN and Transformer, the image features of different frequencies are effectively extracted and integrated to achieve a better denoising effect.

[0025] In the above S5, Fourier ptychography microscopy reconstruction fuses the information collected under different illumination angles in the frequency domain through an iterative algorithm, and reconstructs high-resolution amplitude images and phase images while maintaining the original field of view size of the objective lens.

[0026] The beneficial effects of the present invention are as follows:

[0027] The present invention designs a Fourier ptychography microscopy preprocessing denoising method based on DnSwin. Compared with the traditional method of directly adding Poisson-Gaussian noise to pictures for training neural networks, the present invention classifies low-resolution images into bright-field images and dark-field images by calculating and comparing NA ill and NA obj , adds Poisson-Gaussian noise to the bright-field images and dark-field images to construct a noise data set; denoises the noisy low-resolution images through the trained DnSwin network and obtains high-resolution amplitude and phase through Fourier ptychography microscopy reconstruction. Experimental results show that this method has a good denoising effect and enhances the robustness of the Fourier ptychography microscopy imaging system to noise. (4) Description of the Drawings

[0028] Figure 1 is a schematic flow chart of a Fourier ptychography microscopy preprocessing denoising method based on DnSwin of the present invention;

[0029] Figure 2 is a schematic diagram of the composition of the noise data set of the present invention, where the area within the red line is the bright-field image and the area outside the red line is the dark-field image.

[0030] Figure 3 It is a schematic diagram of the preprocessing noise reduction reconstruction result of the present invention. (V) Specific implementation manners

[0031] Next, in combination with the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. The examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0032] A Fourier ptychographic microscopy preprocessing noise reduction method based on DnSwin includes the following steps:

[0033] S1: Build a Fourier ptychographic microscopy optical path to sample the sample to be measured, and record the low-resolution image of the measured sample.

[0034] S2: Simulate the sampling process, simulate the low-resolution image, calculate the NA of the simulated low-resolution image ill and NA obj and classify it into bright-field images and dark-field images.

[0035] S3: Add Gaussian Poisson noise to the bright-field images and dark-field images to construct a noise data set.

[0036] S4: Use the noise data set to train the DnSwin noise reduction network.

[0037] S5: Denoise the real noisy low-resolution image through the trained DnSwin noise reduction network, and perform Fourier ptychographic microscopy reconstruction on the denoised image to obtain the amplitude and phase of the high resolution.

[0038] In the said S1, the Fourier ptychographic microscopy optical path is a 4f system composed of an array-type LED light source, a microscope objective lens, a tube lens, and an imaging camera. A group of low-resolution images are obtained by illuminating the LED at different positions.

[0039] In the said S2, the formula for calculating NA ill and NA obj is:

[0040]

[0041]

[0042]

[0043]

[0044] where n is the refractive index of the medium of the illumination path, which is 1 in air, is the maximum half-angle at which the objective lens can collect light, which is usually determined by the hardware device, (x center , y center ) represents the central coordinates of the LED, and (x m , y m ) represents the coordinates of the m-th LED, and L represents the distance from the LED array to the sample.

[0045] In a low-resolution image, if NA ill > NA obj , it is classified as a dark-field image. If NA ill ≤ NA obj , it is classified as a bright-field image.

[0046] In S3, the noise introduced during the acquisition process includes the noise brought by the camera dark current and stray light, as well as the read noise and quantization noise. During simulation, the formula for adding Gaussian-Poisson noise is:

[0047]

[0048] where y is the image after adding noise, k is the Poisson parameter, x * is the clean image, μ is the background signal value, and σ is the standard deviation of the Gaussian noise.

[0049] In S4, the two-channel input of the bright-field image and the dark-field image is used to replace the RGB three-channel input in DnSwin. The image is decomposed into sub-bands of different frequencies through wavelet transform, and the low-frequency and high-frequency information is processed separately. By combining the advantages of CNN and Transformer, the image features of different frequencies are effectively extracted and integrated to achieve a better denoising effect.

[0050] In S5, Fourier ptychography microscopy reconstruction fuses the information collected under different-angle illuminations in the frequency domain through an iterative algorithm, and reconstructs a high-resolution amplitude image and a phase image on the premise of maintaining the original field of view size of the objective lens.

[0051] The above-disclosed is only a preferred embodiment of the present invention. Of course, it cannot be used to limit the scope of the rights of the present invention. Those of ordinary skill in the art can understand all or part of the processes of implementing the above embodiments, and the equivalent changes made according to the claims of the present invention still fall within the scope covered by the invention.

Claims

1. A Fourier ptychography preprocessing noise reduction method based on DnSwin, characterized in that, The method includes the following steps: S1: Build a Fourier ptychography microscopy optical path to sample the sample to be measured, and record the low-resolution image of the measured sample. S2: Simulate the sampling process, simulate a low-resolution image, and calculate the NA of the simulated low-resolution image ill and NA obj and classify it into bright-field images and dark-field images. S3: Add Gaussian Poisson noise to the bright-field image and the dark-field image to construct a noise data set. S4: Use the noise data set to train the DnSwin denoising network. S5: Denoise the real noisy low-resolution image through the trained DnSwin denoising network, and perform Fourier ptychography microscopy reconstruction on the denoised image to obtain the high-resolution amplitude and phase.

2. A Fourier ptychographic microscopy preprocessing noise reduction method based on DnSwin according to claim 1, characterized in that: In the above S1, the Fourier ptychography microscopy optical path is a 4f system composed of an array-type LED light source, a microscope objective lens, a tube lens, and an imaging camera.

3. A Fourier ptychography preprocessing noise reduction method based on DnSwin according to claim 1, characterized in that: In the above S1, a group of low-resolution images are obtained by illuminating the LED at different positions.

4. A Fourier ptychographic microscopy preprocessing noise reduction method based on DnSwin according to claim 1, characterized in that: In S2, calculate NA iii and NA obj The formula is: where n is the refractive index of the medium of the illumination path, which is 1 in air, is the maximum half-angle at which the objective lens can collect light, usually determined by hardware devices, (x center , y center ) represents the center coordinates of the LED, (x m , y m ) represents the coordinates of the m-th LED, and L represents the distance from the LED array to the sample.

5. A Fourier ptychographic microscopy preprocessing noise reduction method based on DnSwin according to claim 1, characterized in that: In S2, in the low-resolution image, if NA ill >NA obj , it is classified as a dark-field image. If NA ill ≤NA obj , it is classified as a bright-field image.

6. A Fourier ptychography preprocessing noise reduction method based on DnSwin according to claim 1, characterized in that: In the above S3, the formula for adding Gaussian Poisson noise is: where y is the noisy image, k is the Poisson parameter, x * is the clean image, μ is the background signal value, and σ is the Gaussian noise standard deviation.

7. A Fourier ptychography preprocessing noise reduction method based on DnSwin according to claim 1, characterized in that: In the above S4, the DnSwin network decomposes the image into sub-bands of different frequencies through wavelet transform, processes the low-frequency and high-frequency information respectively, combines the advantages of CNN and Transformer, effectively extracts and integrates the image features of different frequencies, so as to achieve a better denoising effect.

8. A Fourier ptychography preprocessing noise reduction method based on DnSwin according to claim 1, characterized in that: In the above S4, the two-channel input of the bright-field image and the dark-field image is used to replace the RGB three-channel input in DnSwin.

9. A Fourier ptychography preprocessing noise reduction method based on DnSwin according to claim 1, characterized in that: In the above S5, the Fourier ptychography microscopy reconstruction fuses the information collected under different angle illuminations in the frequency domain through an iterative algorithm, so as to reconstruct a high-resolution amplitude image and a phase image on the premise of maintaining the original field of view size of the objective lens.

Images