Large field of view high resolution microscopic imaging method based on partially coherent coded illumination

Abstract

The application discloses a kind of based on partial coherence encoding illumination large field of view high-resolution microscopic imaging method.The method uses programmable LED array as illumination light source, utilizes four non-symmetrical illumination patterns and n concentric circle sector illumination patterns, respectively obtains the bright-dark field intensity image of object under low numerical aperture;Using differential contrast algorithm, the complex amplitude of the object is initialized, the dark field intensity image is used as intensity constraint, and the high-frequency information of the object is recovered through multiple iteration updates, so that large field of view high-resolution microscopic imaging is realized.The application combines high-angle partial coherence dark field illumination with bright field illumination, which can record large field of view image without reducing spatial resolution, and provides a non-invasive, high-performance, low-cost, quantitative, simple and universal high-throughput imaging tool for life science and biomedical research.

Description

Technical Field

[0001] This invention belongs to the field of optical microscopy imaging technology, specifically a large field-of-view, high-resolution microscopy imaging method based on partially coherent coded illumination. Background Technology

[0002] Fourier ptychography microscopy (FPM) is a novel computational imaging technique proposed in recent years that effectively solves the resolution and field-of-view limitations of traditional microscopy. However, traditional FPM requires lighting up each LED unit individually and capturing hundreds of corresponding low-resolution images, resulting in excessively long sampling times and reduced overall imaging efficiency, which to some extent limits the further development and application of this technology.

[0003] To improve data acquisition efficiency, two methods are usually adopted: one is to reduce the total exposure time of each image, and the other is to reduce the total number of images acquired. In 2014, Dong et al. first applied the sparse sampling principle to Fourier layer imaging. By carefully designing a mask, they eliminated overexposed pixels and significantly reduced the total exposure time of each image. Subsequently, they introduced the idea of ​​information reuse in spatial layer imaging into Fourier layer imaging. By simultaneously lighting up multiple LED units, they used incoherent illumination to illuminate the sample and combined it with the subsequent iterative reconstruction process to decompose the incoherent state into a combination of multiple coherent states, thereby greatly reducing the number of images acquired. "Dong S, Shiradkar R, Nanda P et al. Spectral multiplexing and coherent-state decomposition in Fourierptychographic imaging[J]. Biomedical Optics Express,2014,5(6):1757–1767.

[0004] In the same year, Tian et al. proposed a similar Fourier layered imaging technique based on random multiplexing of illumination angles. By randomly multiplexing and coding the LED illumination units, they were able to reduce the number of images acquired by 4 to 8 times. This method significantly reduced the amount of data acquired and improved imaging efficiency without reducing the imaging quality. "Tian L, Li X, Ramchandran K et al. Multiplexed coded illumination for Fourier Ptychography with an LED array microscope[J]. Biomedical Optics Express, 2014, 5(7): 2376–2389.

[0005] In 2015, Tian et al. combined differential phase contrast imaging (DPC) with Fourier ptychographic microscopy using random multiplexing coding. They made full use of the coding characteristics of the programmable LED array illumination system, used differential phase contrast imaging to quickly recover the low-frequency components of the sample, and used Fourier ptychographic microscopy using random multiplexing coding to recover the high-frequency components of the sample, thereby further improving the image acquisition efficiency and realizing large field of view, high-resolution dynamic measurement of live cells. (Tian L, Liu Z, Yeh LH et al. Computational illumination for high-speed in vitro Fourier ptychographic microscopy[J].Optica,2015,2(10):904–911.)

[0006] The methods described above have achieved significant results in improving data acquisition efficiency, but they still have limitations. For example, Dong et al.'s method requires the use of masks to exclude overexposed pixels. Although this shortens the exposure time, it still reduces the amount of usable information, requiring a trade-off between exposure time and the number of images captured. While Tian et al.'s method can reduce the number of images captured by randomly illuminating multiple LEDs simultaneously, the LED multiplexing illumination pattern has not been optimized, and the impact of different illumination patterns on the reconstruction results in random mode is uncertain. Summary of the Invention

[0007] The purpose of this invention is to propose a large field-of-view, high-resolution microscopic imaging method based on partially coherent coded illumination, which solves the problems of excessively long sampling time and low imaging efficiency in traditional FPM.

[0008] The technical solution to achieve the objective of this invention is as follows: a large field-of-view, high-resolution microscopic imaging method based on partially coherent coded illumination, the specific steps of which are as follows:

[0009] Step 1: Use an LED array to generate semi-circular lighting patterns in four directions: top, bottom, left, and right. Sequentially photograph the bright field intensity of the object under the four lighting patterns.

[0010] Step 2: Use an LED array to generate n partially coherent illumination patterns in the dark field, and then take pictures of the dark field light intensity of the object under the n partially coherent illumination patterns in the dark field in sequence.

[0011] Step 3: Based on the bright field intensity maps of the object under the four lighting patterns, use the differential phase contrast imaging calculation formula to obtain the differential phase contrast intensity maps under different directions.

[0012] Step 4: Interpolate the differential phase contrast intensity map under bright field, then deconvolve it to obtain the complex amplitude information of the object, and perform Fourier transform on the complex amplitude information of the object to obtain the initial high-resolution spectrum of the object.

[0013] Step 5: Generate an estimated light intensity value based on the current complex amplitude of the sample in the positive direction;

[0014] Step 6: Calculate the complex amplitude and corresponding complex amplitude spectrum after light intensity constraint;

[0015] Step 7: Update the i-th sub-aperture in the high-resolution spectrum using the complex amplitude spectrum after light intensity constraint;

[0016] Step 8: Repeat steps 5-7 to update the corresponding sub-apertures of all LEDs in sequence, and repeat the iteration several times until the high-resolution complex amplitude distribution is stably converged.

[0017] Preferably, in step one, the illumination angle is sequentially illuminated at the objective lens numerical aperture NA. obj The LEDs in four semicircular regions, each containing j LEDs, are used to achieve a differential phase-contrast lighting mode. The luminance distribution function of the m-th bright-field incoherent lighting mode is given. Represented as:

[0018]

[0019] Wherein, δ(uu i1 Let be the Dirac function, and u be the spatial frequency coordinate. i1 Let θ be the illumination angle of the i-th LED in the m-th semicircular illumination. i1 The corresponding spatial frequency coordinates:

[0020]

[0021] Where λ is the wavelength of the illumination light.

[0022] Preferably, a ring-shaped photograph is radially divided into n equal parts, each part serving as a dark-field partial coherent illumination pattern. These n dark-field partial coherent illumination patterns are sequentially illuminated to form n dark-field partial coherent illumination patterns. Each illumination pattern contains k LEDs, and its inner radius corresponds to an illumination angle equal to the objective lens numerical aperture NA. obj The outer radius is determined by the objective lens numerical aperture and the required super-resolution magnification, and its range is within the NA range. obj ~5NA obj The illumination intensity distribution function of the m-th partially coherent illumination mode in the dark field It is represented as:

[0023]

[0024] In the formula, u i The illumination angle θ of the i-th LED in the m-th partial coherent illumination pattern of the dark field is... i The corresponding spatial frequency coordinates.

[0025] Preferably, in step three, the specific formula for obtaining bright-field differential phase-contrast images under different directions using the differential phase-contrast imaging calculation formula is as follows: In the formula, The images show the bright field intensity of objects under four different lighting patterns.

[0026] Preferably, in step four, the deconvolution obtains the complex amplitude information of the object, and the specific process of performing a Fourier transform on the complex amplitude information of the object to obtain the initial high-resolution spectrum of the object is as follows:

[0027] The specific formula for calculating the differential phase contrast transfer function is as follows:

[0028] H DPC1 (u)=∫S DPC1 (u0)[P(u-u0)-P(u+u0)]du0

[0029] H DPC2 (u)=∫S DPC2 (u0)[P(u-u0)-P(u+u0)]du0

[0030] Among them, H DPC1 (u) is the phase contrast transfer function in the left and right directions, H DPC2 (u) is the phase contrast transfer function in the vertical and horizontal directions, and u0 is the spatial frequency coordinate corresponding to the current lighting angle;

[0031]

[0032]

[0033] Let P(u) be the illumination intensity distribution function in the left, top, right, and bottom illumination directions, respectively, and let P(u) be the pupil function of the objective lens.

[0034] The phase is solved by deconvolution using the differential phase-contrast transfer function and the differential phase-contrast intensity map:

[0035]

[0036] Where α is the regularization parameter, F{·}, F -1 {·} represent the Fourier transform and inverse Fourier transform, respectively; using the solved phase image φ 0 (x) Initialize the sample complex amplitude o 0 (x):

[0037]

[0038] In the formula, NA obj This is the numerical aperture of the objective lens.

[0039] Preferably, the pupil function of the objective lens is as follows:

[0040]

[0041] Where u is the spatial frequency coordinate.

[0042] Preferably, step five generates a light intensity estimate based on the current complex amplitude of the sample in a positive direction. The calculation formula is:

[0043]

[0044] In the formula, O represents the illumination intensity distribution function of the m-th partially coherent illumination mode in the dark field. n (uu i ) represents the spectrum of the current sample, and P(u) represents the pupil function of the objective lens.

[0045] Preferably, the complex amplitude o after light intensity constraint update The formula for calculating (x) is:

[0046]

[0047] In the formula, This is the light intensity distribution in the actual photograph. P(u) is the light intensity estimate generated in the positive direction based on the current complex amplitude of the sample, where P(u) is the pupil function of the objective lens, and O is the light intensity estimate generated in the positive direction based on the current complex amplitude of the sample. n (uu i ) represents the spectrum of the current sample.

[0048] Preferably, in step seven, the i-th sub-aperture in the spectrum O(u) is updated using the complex amplitude spectrum after light intensity constraint, and the update formula is:

[0049]

[0050] Where n is the current iteration number, O(u) is the sample spectrum, is the Fourier transform of the complex amplitude o(x), β is the update step size, and P * (uu i ) is the conjugate of the pupil function, O update (u) represents the complex amplitude spectrum after light intensity constraint.

[0051] Preferably, the method for determining the stable convergence of the high-resolution complex amplitude distribution is as follows: when the following conditions are met:

[0052]

[0053] Then stop iterating; at this point, o n+1 (x) represents the reconstructed high-resolution complex amplitude of the sample, o n (x) represents the high-resolution complex amplitude of the current sample at the start of each reconstruction.

[0054] Compared with the prior art, the present invention has the following significant advantages: (1) Compared with traditional incoherent illumination, it has higher imaging resolution and better imaging effect. (2) It uses differential phase contrast imaging to quickly recover the low-frequency components of the sample and uses non-random multiplexing encoded Fourier stack imaging technology to recover the high-frequency components of the sample, thereby further improving the image acquisition efficiency.

[0055] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0056] Figure 1 This is a flowchart of a large field-of-view, high-resolution microscopic imaging method based on partially coherent coded illumination.

[0057] Figure 2 A schematic diagram of the LED illumination pattern during differential phase contrast imaging image acquisition, using a semi-circular illumination.

[0058] Figure 3 It is a partially coherent lighting pattern used when shooting dark-field images.

[0059] Figure 4 This is a comparison diagram of the effects of the present invention and the traditional method. Figure 4 (a) is the imaging result under conventional coherent illumination. Figure 4 (b) shows the imaging results using this method. Detailed Implementation

[0060] The present invention will now be described in further detail with reference to the accompanying drawings.

[0061] like Figure 1 As shown, a large field-of-view, high-resolution microscopic imaging method based on partially coherent coded illumination includes the following steps:

[0062] Step 1: Bright-field image acquisition: such as Figure 2 As shown, lights are sequentially illuminated with an illumination radius equal to the objective lens numerical aperture NA. obj The system uses four semicircular LED regions, each containing j LEDs. The angle between each semicircle is 90°, thus achieving a differential phase-contrast lighting mode. Triggered by the LED signal, the camera sequentially captures the bright-field intensity map of the object under the current lighting mode.

[0063] Furthermore, the illumination intensity distribution function of the m-th bright-field incoherent illumination mode Represented as:

[0064]

[0065] Wherein, δ(uu i1 Let be the Dirac function, and u be the spatial frequency coordinate. i1 Let θ be the illumination angle of the i-th LED in the m-th semicircular illumination. i1 Corresponding frequency coordinates:

[0066]

[0067] Where λ is the wavelength of the illumination light.

[0068] Step 2: Illuminate n coherent lighting patterns in the dark areas sequentially, such as... Figure 3 As shown, a circular photograph is radially divided into n equal parts, each part serving as the coherent illumination pattern for the dark field. The LED illumination angle for the dark field is greater than the objective lens numerical aperture NA. obj In some embodiments, n takes values ​​from 8 to 16. Each illumination pattern contains k LEDs, and its inner radius corresponds to the illumination angle of the objective lens numerical aperture NA. obj The outer radius is determined by the objective lens numerical aperture and the required super-resolution magnification, and its range is within the NA range. obj ~5NA obj The illumination intensity distribution function of the m-th part of the dark field coherent illumination mode. It is represented as:

[0069]

[0070] In the formula, u iThe illumination angle θi of the i-th LED in the m-th partial coherent illumination pattern of the dark field. i The corresponding frequency coordinates.

[0071] Triggered by an LED signal, the camera sequentially captures dark-field light intensity images of the object under n lighting modes.

[0072] Step 3: Differential Image Calculation: Using the differential phase-contrast imaging calculation formula, bright-field differential phase-contrast images under different directions are obtained.

[0073] Step 4: Differential phase contrast light intensity diagram I under bright field DPC1 Interpolation and deconvolution are performed to solve for the initial complex amplitude information of the object. 0 (x).

[0074] Furthermore, deconvolution is used to solve for the initial complex amplitude information of the object. 0 The specific process of (x) is as follows:

[0075] First, calculate the differential phase-contrast transfer function:

[0076] H DPC1 (u)=∫S DPC1 (u0)[P(u-u0)-P(u+u0)]du0

[0077] H DPC2 (u)=∫S DPC2 (u0)[P(u-u0)-P(u+u0)]du0

[0078] Among them, H DPC1 (u) Phase contrast transfer functions in the left and right directions, H DPC2 (u) is the phase contrast transfer function in both the vertical and horizontal directions, and u0 is the translation of the object's position in the spectrum due to the illumination at the current angle; in simpler terms, it is the spectrum translation. In the formula Let P(u) be the illumination intensity distribution function in the left, top, right, and bottom illumination directions, respectively, and let P(u) be the pupil function of the objective lens, which is determined by the objective lens and the illumination wavelength.

[0079]

[0080] Then, the phase is solved by deconvolution using the differential phase-contrast transfer function and the differential phase-contrast intensity map:

[0081]

[0082] Where α is the regularization parameter, and a value of 0.01 to 0.1 is recommended. F{·}, F -1{·} represent the Fourier transform and inverse Fourier transform, respectively. The phase image φ is obtained by solving. 0 (x) is used to initialize the sample complex amplitude. 0 (x):

[0083]

[0084] Step 5: Use the coherent illumination intensity map of the dark area captured. During the update of each sub-aperture in the high-resolution spectrum O(u), it is necessary to calculate the estimated intensity map of the m-th partially coherent illumination mode. Estimated light intensity value The calculation formula is:

[0085]

[0086] In the formula, O represents the illumination intensity distribution function of the m-th partially coherent illumination mode in the dark field. n (uu i ) represents the spectrum of the current sample, and P(u) represents the pupil function of the objective lens.

[0087] Step Six: Calculate the complex amplitude o after light intensity constraint update (x), the calculation formula is:

[0088]

[0089] In the formula, This is the light intensity distribution in the actual photograph. This is the light intensity estimate generated in the positive direction based on the current complex amplitude of the sample.

[0090] Complex amplitude o after light intensity constraint update (x) is subjected to Fourier transform to obtain the complex amplitude spectrum O after light intensity constraint. update (u).

[0091] Step 7: Update the i-th sub-aperture in the high-resolution spectrum using the complex amplitude spectrum after intensity constraint. The update formula is:

[0092]

[0093] Where n is the current iteration number, O(u) is the sample spectrum, which is the Fourier transform of the complex amplitude o(x), β is the update step size, and a value of 0.1 to 1 is recommended. * (uu i ) is the conjugate of the pupil function, O update (u) represents the complex amplitude spectrum after light intensity constraint. The updated i-th sub-aperture is one of the LEDs in the m-th partially coherent illumination mode in step five, which must satisfy...

[0094] Step 8: Update the sub-apertures of all LEDs sequentially according to the above formula, and repeat the iteration several times until the following convergence condition is met in the nth round:

[0095]

[0096] Then the iteration stops. At this point, o n+1 (x) represents the reconstructed high-resolution complex amplitude of the sample, o n (x) represents the high-resolution complex amplitude of the current sample at the start of each reconstruction.

[0097] Figure 2 This is a schematic diagram of the LED illumination pattern during differential phase contrast imaging image acquisition, using a semi-circular illumination.

[0098] Figure 3 It is a partially coherent lighting pattern used when shooting dark-field images.

[0099] Figure 4 This is a comparison chart of the effects of this method and the traditional method. Figure 4 (a) is the imaging result under conventional coherent illumination. Figure 4 (b) shows the imaging results using this method.

[0100] To verify the experimental results of this invention, Fourier transform film microscopy was performed using a USAF resolution plate as the sample to be tested. Figure 4 (a) shows the reconstruction result of traditional single-point scanning, while Figure 4 (b) Presents the results of the partially coherent coded illumination (FPM) technique proposed in this invention. Comparing the two results shows that their reconstruction resolution is consistent, and there is no significant reduction in image quality. However, this invention has a significant advantage in image acquisition. Compared to conventional methods, this invention reduces the number of images acquired to only 10 (4+6), thereby improving measurement efficiency by 20 times.

[0101] The high efficiency of this technology allows us to obtain imaging results comparable to traditional methods in a shorter time, greatly improving experimental efficiency. Therefore, the partially coherent coded illumination (FPM) technology of this invention has the potential to improve imaging speed and efficiency in practical applications and has had a positive impact on related research and applications in this field.

Claims

1. A large field-of-view, high-resolution microscopic imaging method based on partially coherent coded illumination, characterized in that, The specific steps are as follows: Step 1: Use an LED array to generate semi-circular lighting patterns in four directions: top, bottom, left, and right. Sequentially photograph the bright field intensity of the object under the four lighting patterns. Step 2: Use an LED array to generate n partially coherent illumination patterns in the dark field, and then take pictures of the dark field light intensity of the object under the n partially coherent illumination patterns in the dark field in sequence. Step 3: Based on the bright field intensity maps of the object under the four lighting patterns, use the differential phase contrast imaging calculation formula to obtain the differential phase contrast intensity maps under different directions. Step 4: Interpolate the differential phase contrast intensity map under bright field, then deconvolve it to obtain the complex amplitude information of the object, and perform Fourier transform on the complex amplitude information of the object to obtain the initial high-resolution spectrum of the object. Step 5: Generate an estimated light intensity value based on the current complex amplitude of the sample in the positive direction; Step 6: Calculate the complex amplitude and corresponding complex amplitude spectrum after light intensity constraint; Step 7: Update the i-th sub-aperture in the high-resolution spectrum using the complex amplitude spectrum after light intensity constraint; Step 8: Repeat steps 5-7 to update the corresponding sub-apertures of all LEDs in sequence, and repeat the iteration several times until the high-resolution complex amplitude distribution is stably converged.

2. The large field-of-view, high-resolution microscopic imaging method based on partially coherent coded illumination according to claim 1, characterized in that, Step 1: Illuminate the lens sequentially at an angle equal to the objective lens's numerical aperture (NA). obj The four semicircular regions of LEDs contain j LEDs each, and the rotation angle between each semicircle is 90°, thus realizing a differential phase contrast lighting mode. The illumination intensity distribution function of the m-th bright field incoherent lighting mode is... Represented as: Wherein, δ(uu i1 Let be the Dirac function, and u be the spatial frequency coordinate. i1 Let θ be the illumination angle of the i-th LED in the m-th semicircular illumination. i1 Corresponding frequency coordinates: Where λ is the wavelength of the illumination light.

3. The large field-of-view, high-resolution microscopic imaging method based on partially coherent coded illumination according to claim 1, characterized in that, A ring-shaped photograph is radially divided into n equal parts, each part serving as a dark-field coherent illumination pattern. These n dark-field coherent illumination patterns are sequentially illuminated, forming n dark-field coherent illumination patterns. Each illumination pattern contains k LEDs, and its inner radius corresponds to the illumination angle NA of the objective lens. obj The outer radius is determined by the objective lens numerical aperture and the required super-resolution magnification, and its range is within the NA range. obj ~5NA obj The illumination intensity distribution function of the m-th partially coherent illumination mode in the dark field It is represented as: In the formula, u i The illumination angle θ of the i-th LED in the m-th partial coherent illumination pattern of the dark field is... i The corresponding spatial frequency coordinates.

4. The large field-of-view, high-resolution microscopic imaging method based on partially coherent coded illumination according to claim 1, characterized in that, Step 3: Using the differential phase contrast imaging calculation formula, the specific formulas for obtaining bright-field differential phase contrast images under different directions are as follows: In the formula, The images show the bright field intensity of objects under four different lighting patterns.

5. The large field-of-view, high-resolution microscopic imaging method based on partially coherent coded illumination according to claim 1, characterized in that, In step four, deconvolution yields the complex amplitude information of the object. The specific process of performing a Fourier transform on the complex amplitude information of the object to obtain the initial high-resolution spectrum of the object is as follows: The specific formula for calculating the differential phase contrast transfer function is as follows: H DPC1 (u)=∫S DPC1 (u0)[P(u-u0)-P(u+u0)du0 H DPC2 (u)=jS DPC2 (u0)[P(u-u0)-P(u+u0)du0 Among them, H DPC1 (u) is the phase contrast transfer function in the left and right directions, H DPC2 (u) is the phase contrast transfer function in the vertical direction, and u0 is the spatial frequency coordinate corresponding to the current illumination angle; Let P(u) be the illumination intensity distribution function in the left, top, right, and bottom illumination directions, respectively, and let P(u) be the pupil function of the objective lens. The phase is solved by deconvolution using the differential phase-contrast transfer function and the differential phase-contrast intensity map: Where α is the regularization parameter, F{·}, F -1 {·} represent the Fourier transform and inverse Fourier transform, respectively; using the solved phase image φ 0 (x) Initialize the sample complex amplitude o 0 (x): In the formula, NA obj This is the numerical aperture of the objective lens.

6. The large field-of-view, high-resolution microscopic imaging method based on partially coherent coded illumination according to claim 1, characterized in that, The pupil function of the objective lens is as follows: Where u is the spatial frequency coordinate.

7. The large field-of-view, high-resolution microscopic imaging method based on partially coherent coded illumination according to claim 1, characterized in that, Step 5: Generate an estimated light intensity value based on the current complex amplitude of the sample. The calculation formula is: In the formula, O represents the illumination intensity distribution function of the m-th partially coherent illumination mode in the dark field. n (uu i ) represents the spectrum of the current sample, and P(u) represents the pupil function of the objective lens.

8. The large field-of-view, high-resolution microscopic imaging method based on partially coherent coded illumination according to claim 1, characterized in that, Complex amplitude o after light intensity constraint update The formula for calculating (x) is: In the formula, This is the light intensity distribution in the actual photograph. P(u) is the light intensity estimate generated in the positive direction based on the current complex amplitude of the sample, where P(u) is the pupil function of the objective lens, and O is the light intensity estimate generated in the positive direction based on the current complex amplitude of the sample. n (uu i ) represents the spectrum of the current sample.

9. The large field-of-view, high-resolution microscopic imaging method based on partially coherent coded illumination according to claim 1, characterized in that, Step 7: Update the i-th sub-aperture in the spectrum O(u) using the complex amplitude spectrum after light intensity constraint. The update formula is: Where n is the current iteration number, O(u) is the sample spectrum, is the Fourier transform of the complex amplitude o(x), β is the update step size, and P * (uu i ) is the conjugate of the pupil function, O update (u) represents the complex amplitude spectrum after light intensity constraint.

10. The large field-of-view, high-resolution microscopic imaging method based on partially coherent coded illumination according to claim 1, characterized in that, The method for determining the stable convergence of a high-resolution complex amplitude distribution is as follows: when the following conditions are met: Then stop iterating; at this point, o n+1 (x) represents the reconstructed high-resolution complex amplitude of the sample, o n (x) represents the high-resolution complex amplitude of the current sample at the start of each reconstruction.

Images