Fast full-color imaging method based on Fourier lamination microscope

Abstract

The invention discloses a fast full-color imaging method based on a Fourier laminated microscope, which solves the problems of long time consumption and poor imaging quality of a full-color imaging mode of the existing FPM microscopic imaging technology, and specifically comprises the following steps: step 1, obtaining a low-resolution color image of a target through a Fourier laminated microscope system, selecting an optimal acquisition channel from three channels of the system; 2, collecting a target through the optimal collection channel, reconstructing a high-resolution grayscale image based on the collected FPM original data, and calculating a system coherent transfer function and a system incoherent transfer function; 3, calculating a deconvolution IHS color space variable corresponding to the low-resolution color image through a system incoherent transfer function; and step 4, calculating the high-resolution grayscale images of the other two channels through the high-resolution grayscale image of the optimal acquisition channel and the deconvolution IHS color space variable, and synthesizing the three high-resolution grayscale images into a high-resolution color image to complete rapid full-color imaging.

Description

Technical Field

[0001] The present invention relates to an imaging method, and particularly to a fast full-color imaging method based on Fourier ptychographic microscopy. Background Art

[0002] Traditional digital pathology often uses high-magnification objective lenses and scanning stitching methods to obtain large-field-of-view, high-resolution images. Components such as high-precision motorized stages, high-magnification objective lenses, and pulsed light sources are expensive, resulting in high imaging costs. A large number of mechanical movements also slow down the time efficiency of imaging. At the same time, problems such as narrow depth of field brought by high-magnification objective lenses and artifacts, ghosting, and failures brought by mechanical scanning stitching also reduce the imaging quality.

[0003] Fourier ptychographic microscopy (FPM) was invented in 2013. It uses a low-magnification objective lens to obtain a natural large field of view, and collects a set of low-resolution images through multi-angle scanning. It iteratively reconstructs high-resolution results in the frequency domain, and can obtain high-resolution, large-field-of-view images without mechanical scanning, effectively solving the quality problems of traditional scanning imaging, breaking through the contradictory relationship between resolution and field of view in traditional microscopic imaging, and making it possible to achieve high-throughput imaging in digital pathology.

[0004] In an environment where the demand for digital pathology section imaging is increasing day by day, the demand for high-end pathology instruments in China has increased rapidly, and optical microscopy instruments based on FPM technology have emerged as the times require. In the application scenario of pathology section imaging, full-color imaging is crucial. Color imaging can provide rich sample details and sufficient pathological features to support the needs of pathology research and detection. FPM relies on a narrowband monochromatic light source to achieve high-resolution image reconstruction. In a certain channel, it needs to collect up to hundreds of pieces of original data and realize image reconstruction through a phase retrieval algorithm. Therefore, the full-color imaging mode of traditional FPM needs to collect data and reconstruct in the R, G, and B channels respectively, and then synthesize a color image. This imaging process is very time-consuming and seriously affects the efficiency of FPM in the full-color imaging mode. In addition, due to the use of a narrowband light source for illumination in FPM, the traditional full-color imaging method has serious coherent color artifacts, seriously affecting the imaging quality.

[0005] Existing fast colorization techniques require manual selection of the color channels reconstructed by FPM, resulting in poor colorization quality, serious loss of color information, and long calculation time. The published paper (Gao, Y., Chen, J., Wang, A. et al. High-throughput fast full-color digital pathology based on Fourier ptychographic microscopy via color transfer. Sci. China Phys. Mech. Astron. 64, 114211 (2021)) proposed a fast colorization method for FPM based on color space histogram matching, but this method cannot achieve colorization of more than two colors and takes up to several hours to calculate on the CPU; the published paper (Jiurun Chen, Aiye Wang, An Pan, Guoan Zheng, Caiwen Ma, and Baoli Yao, "Rapid full-color Fourier ptychographic microscopy via spatially filtered color transfer," Photon. Res. 10, 2410-2421 (2022)) proposed a fast colorization method based on trilateral filtering and color space iteration, which still does not solve the problems of low colorization accuracy and long calculation time. Summary of the Invention

[0006] To solve the technical problems of long time consumption and poor imaging quality in the full-color imaging mode of existing FPM microscopy techniques, the present invention provides a fast full-color imaging method based on Fourier ptychographic microscopy.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A fast full-color imaging method based on Fourier ptychographic microscopy, characterized in that it includes the following steps:

[0009] Step 1: Obtain a low-resolution color image of the target through a Fourier ptychographic microscopy system, and select the best acquisition channel from the R, G, and B channels of the Fourier ptychographic microscopy system;

[0010] Step 2: Collect the target through the best acquisition channel, reconstruct the high-resolution grayscale image of the best acquisition channel based on the collected FPM raw data, and calculate the system coherent transfer function and the system incoherent transfer function;

[0011] Step 3: Calculate the deconvolution IHS color space variables corresponding to the low-resolution color map through the system incoherent transfer function and

[0012] Step 4: Calculate the high-resolution grayscale images of the two channels other than the best acquisition channel through the high-resolution grayscale image of the best acquisition channel and the deconvolution IHS color space variables and Calculate the high-resolution grayscale images of the three channels, and synthesize the high-resolution grayscale images of the three channels into a high-resolution color image to complete fast full-color imaging.

[0013] 2. The fast full-color imaging method based on Fourier ptychography according to claim 1, wherein step 1 specifically includes:

[0014] 1.1: Light up all the R, G, and B channel lamp beads on the LED array in the Fourier ptychography system respectively, collect the target based on the R, G, and B channels, and correspondingly obtain three low-resolution grayscale images, and synthesize the three low-resolution grayscale images into a low-resolution color map;

[0015] 1.2: Calculate the image entropy of the R, G, and B channels in the low-resolution color map respectively, and select the color channel with the largest entropy value, which is recorded as the best acquisition channel.

[0016] 3. The fast full-color imaging method based on Fourier ptychography according to claim 2, wherein between step 1.1 and step 1.2, it further includes:

[0017] Automatically adjust the white balance of the low-resolution color map by using the dynamic threshold method.

[0018] 4. The fast full-color imaging method based on Fourier ptychography according to claim 2 or 3, wherein step 2 specifically includes:

[0019] 2.1: Collect the target through the best acquisition channel to obtain FPM raw data;

[0020] 2.2: Reconstruct the FPM raw data to obtain the high-resolution grayscale image of the best acquisition channel and the system coherent transfer function;

[0021] 2.3: Calculate the autocorrelation of the system coherent transfer function to obtain the system incoherent transfer function.

[0022] 5. The fast full-color imaging method based on Fourier ptychography according to claim 4, wherein step 3 specifically includes:

[0023] 3.1. Calculate the IHS color space variables V1 and V2 corresponding to the low-resolution color image synthesized in Step 1.1:

[0024]

[0025] Wherein, R, G, and B are the gray values of the R, G, and B channels of the low-resolution color image respectively;

[0026] 3.2. Use the system incoherent transfer function obtained in Step 2.3 to perform deconvolution on the IHS color space variables V1 and V2 respectively to obtain the deconvolved IHS color space variables and

[0027]

[0028] Wherein, i = 1, 2, OTF represents the system incoherent transfer function, F and F -1 respectively represent two-dimensional Fourier transform and two-dimensional inverse Fourier transform, and σ is the regularization coefficient, and its value range is 10 -5 ~10 -1 .

[0029] 6. The fast full-color imaging method based on Fourier ptychography microscope according to claim 5, characterized in that, in Step 3.2:

[0030] The regularization coefficient σ is 0.01.

[0031] 7. The fast full-color imaging method based on Fourier ptychography microscope according to claim 6, characterized in that Step 4 specifically includes:

[0032] 4.1. Upsample the deconvolved IHS color space variables and to be the same size as the high-resolution gray image of the best acquisition channel;

[0033] 4.2. Use the closed-form colorization model to calculate the high-resolution color map (R HR , G HR , B HR );

[0034] If the best acquisition channel selected in Step 1.2 is the R channel, then the one reconstructed in Step 2.2 is the high-resolution gray image R HR , and calculate the high-resolution gray image G HR of the G channel and the high-resolution gray image B HR of the B channel respectively:

[0035]

[0036] If the best acquisition channel selected in step 1.2 is the G channel, the high-resolution grayscale image G is reconstructed in step 2.2 HR , and calculate the high-resolution grayscale image R of the R channel respectively HR and the high-resolution grayscale image B of the B channel HR :

[0037]

[0038] If the best acquisition channel selected in step 1.2 is the B channel, the high-resolution grayscale image B is reconstructed in step 2.2 HR , and calculate the high-resolution grayscale image R of the R channel respectively HR and the high-resolution grayscale image G of the G channel HR :

[0039]

[0040] Combine the high-resolution grayscale images of the R, G, and B channels into a high-resolution color image (R HR , G HR , B HR ), and complete the fast full-color imaging.

[0041] Advantages of the present invention:

[0042] 1. The fast full-color imaging method provided by the present invention performs deconvolution enhancement on low-resolution color images under incoherent illumination, can avoid coherent color artifacts while enhancing the contrast of color information, improve the imaging quality, and reduce the time consumption.

[0043] 2. The fast full-color imaging method provided by the present invention can adaptively judge the FPM raw data and the acquired color channels without manual selection, making the fast colorization scheme applicable to all microscopic imaging samples.

[0044] 3. The fast full-color imaging method provided by the present invention uses a closed-form colorization model to calculate the high-resolution color image, with fast calculation speed and small calculation amount. Description of the drawings

[0045] Figure 1 is a flowchart of an embodiment of a fast full-color imaging method based on Fourier ptychographic microscopy of the present invention;

[0046] Figure 2 is the low-resolution color image synthesized in step 1.1 in the embodiment of the present invention;

[0047] Figure 3 is the high-resolution grayscale image obtained through the G channel in step 2.2 in the embodiment of the present invention;

[0048] Figure 4 It is the high-resolution color image finally synthesized in step 4.2 of the embodiment of the present invention. Specific embodiments

[0049] Next, the technical solutions of the present invention will be clearly and completely described in conjunction with the accompanying drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0050] A fast full-color imaging method based on Fourier ptychography provided by an embodiment of the present invention, as Figure 1 shown, the target adopted in this embodiment is a stained section of cat gastric smooth muscle; the Fourier ptychography system adopted uses a 4× objective lens (Olmpus OPLN4×NA 0.10) and a CCD camera (ZWO ASI178MM, 3096×2080), and the LED array is equipped with three-channel RGB lamp beads with central wavelengths of 631.23 nm, 538.86 nm, and 456.70 nm, respectively.

[0051] The fast full-color imaging method includes the following steps:

[0052] Step 1, select the best acquisition channel;

[0053] 1.1. Light up all the R, G, and B channel lamp beads on the LED array in the Fourier ptychography system respectively, collect the target based on the R, G, and B channels, and correspondingly obtain three low-resolution grayscale images, as Figure 2 shown, and synthesize a low-resolution color image;

[0054] After that, the white balance of the low-resolution color image is automatically adjusted by using the dynamic threshold method;

[0055] 1.2. Calculate the image entropy of the R, G, and B channels in the low-resolution color image respectively, select the color channel with the largest entropy value, and record it as the best acquisition channel; in this embodiment, the selected best acquisition channel is the G channel;

[0056] The calculation formula of the image entropy is as follows:

[0057]

[0058] In the formula, the pixel size of the image I is M*N, and m and n are the pixel indexes in the length and width directions of the image I respectively;

[0059] Step 2, collect and process the FPM raw data;

[0060] 2.1. Collect the target through the selected best acquisition channel (G channel) to obtain the FPM raw data;

[0061] 2.2. Reconstruct the FPM raw data to obtain the high-resolution grayscale image and the system coherent transfer function as shown in the best acquisition channel (G channel); Figure 3 shown;

[0062] 2.3. Calculate the autocorrelation of the system coherent transfer function to obtain the system incoherent transfer function;

[0063] Step 3. Obtain the IHS color space variables and

[0064] 3.1. Calculate the IHS color space variables V1 and V2 corresponding to the synthesized low-resolution color image in Step 1.1:

[0065]

[0066] where R, G, and B are the grayscale values of the R, G, and B channels of the low-resolution color image respectively;

[0067] 3.2. Use the system incoherent transfer function obtained in Step 2.3 to perform deconvolution on the IHS color space variables V1 and V2 respectively to obtain the deconvolved IHS color space variables and

[0068]

[0069] where i = 1, 2, OTF represents the system incoherent transfer function, F and F -1 represent the two-dimensional Fourier transform and the two-dimensional inverse Fourier transform respectively, and the regularization coefficient σ can be set to 10 -5 ~10 -1 , and in this embodiment, σ = 0.01 is preferably selected;

[0070] Step 4. Synthesize the high-resolution color image;

[0071] 4.1. Upsample the deconvolved IHS color space variables and to the same size as the high-resolution grayscale image of the best acquisition channel obtained in Step 2.2;

[0072] 4.2. Use the closed-form colorization model to calculate the high-resolution color map (R HR , G HR , B HR );

[0073] If the best acquisition channel selected in step 1.2 is the R channel, then the high-resolution grayscale image R is reconstructed in step 2.2 HR , and calculate the high-resolution grayscale image G of the G channel respectively HR and the high-resolution grayscale image B of the B channel HR :

[0074]

[0075] If the best acquisition channel selected in step 1.2 is the B channel, then the high-resolution grayscale image B is reconstructed in step 2.2 HR , and calculate the high-resolution grayscale image R of the R channel respectively HR and the high-resolution grayscale image G of the G channel HR :

[0076]

[0077] If the best acquisition channel selected in step 1.2 is the G channel, then the high-resolution grayscale image G is reconstructed in step 2.2 HR , and calculate the high-resolution grayscale image R of the R channel respectively HR and the high-resolution grayscale image B of the B channel HR :

[0078]

[0079] In this embodiment, the best acquisition channel is the G channel. Therefore, the high-resolution grayscale image R of the R channel is finally calculated by this formula HR and the high-resolution grayscale image B of the B channel HR , and finally synthesize the high-resolution color map as shown in Figure 4 (R HR ,G HR ,B HR ). This high-resolution color map (R HR ,G HR ,B HR )) retains the rich color information of the low-resolution color map in step 1( Figure 2 ), and fuses the details of the single-channel high-resolution grayscale map in step 2( Figure 3 ), while presenting high-resolution and full-color imaging results

[0080] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims

Claims

1. A fast full-color imaging method based on Fourier stacking microscope, characterized in that: The following steps are involved: Step 1, obtaining a low-resolution color image of the target through a Fourier stacking microscope system, and selecting the best acquisition channel from the three channels of R, G, and B of the Fourier stacking microscope system; Step 2, collect the target through the best collection channel, reconstruct the high-resolution grayscale image of the best collection channel based on the collected FPM raw data, and calculate the system coherent transfer function and the system incoherent transfer function; Step 3: Calculate the deconvolution IHS color space variable corresponding to the low-resolution color image through the system incoherent transfer function and Step 4: High-resolution grayscale image and deconvolution of IHS color space variables through the optimal acquisition channel and The high-resolution grayscale images of the two channels other than the best acquisition channel are calculated, and the high-resolution grayscale images of the three channels are synthesized into a high-resolution color image to complete fast full-color imaging.

2. The fast full-color imaging method based on Fourier stacking microscope according to claim 1, characterized in that: Step 1 specifically includes: 1.

1. Light up all the R, G, and B three-channel lamp beads on the LED array of the Fourier stacking microscope system respectively, collect the target based on the three channels of R, G, and B, and obtain three low-resolution grayscale images accordingly, and synthesize a low-resolution color image through the three low-resolution grayscale images; 1.

2. Calculate the image entropy of the three channels R, G, and B in the low-resolution color image respectively, select the color channel with the largest entropy value, and record it as the best acquisition channel.

3. The fast full-color imaging method based on Fourier stacking microscope according to claim 2, characterized in that: Between step 1.1 and step 1.2 also include: A dynamic threshold method is used to automatically adjust the white balance of the low-resolution color image.

4. The fast full-color imaging method based on Fourier stacking microscope according to claim 2 or 3, characterized in that: Step 2 specifically includes: 2.

1. Collect the target through the optimal acquisition channel to obtain FPM raw data; 2.

2. Reconstruct the FPM raw data to obtain the high-resolution grayscale image and system coherence transfer function of the optimal acquisition channel; 2.

3. Calculate the autocorrelation of the system coherent transfer function and obtain the system incoherent transfer function.

5. The fast full-color imaging method based on Fourier stacking microscope according to claim 4, characterized in that: Step 3 specifically includes: 3.

1. Calculate the IHS color space variables V1 and V2 corresponding to the low-resolution color image synthesized in step 1.1: In the formula, R, G, and B are the grayscale values ​​of the R, G, and B channels of the low-resolution color image respectively; 3.

2. Use the system incoherent transfer function obtained in step 2.3 to deconvolve the IHS color space variables V1 and V2 respectively to obtain the deconvoluted IHS color space variables and Where i = 1, 2, OTF represents the incoherent transfer function of the system, F and F -1 They represent two-dimensional Fourier transform and two-dimensional inverse Fourier transform respectively, σ is the regularization coefficient, and its value range is 10 -5 ~10 -1 .

6. The fast full-color imaging method based on Fourier stacking microscope according to claim 5, characterized in that: In step 3.2: The regularization coefficient σ is 0.

01.

7. The fast full-color imaging method based on Fourier stacking microscope according to claim 6, characterized in that: Step 4 specifically includes: 4.

1. Deconvolute IHS color space variables and Upsample to the same size as the high-resolution grayscale image of the best acquisition channel; 4.

2. Computing high-resolution color images (R HR ,G HR ,B HR ); If the best acquisition channel selected in step 1.2 is the R channel, then the high-resolution grayscale image R reconstructed in step 2.2 is HR , respectively calculate the high-resolution grayscale image G of the G channel HR and the high-resolution grayscale image B of the B channel HR : If the best acquisition channel selected in step 1.2 is the G channel, then the high-resolution grayscale image G reconstructed in step 2.2 is HR , respectively calculate the high-resolution grayscale image R of the R channel HR and the high-resolution grayscale image B of the B channel HR : If the best acquisition channel selected in step 1.2 is channel B, then the high-resolution grayscale image B is reconstructed in step 2.

2. HR , respectively calculate the high-resolution grayscale image R of the R channel HR and G channel high-resolution grayscale image G HR : The high-resolution grayscale images of the R, G, and B channels are synthesized into a high-resolution color image (R HR ,G HR ,B HR ), to achieve fast full-color imaging.

Images