Lensless Fourier plyometric tomography based on wavelength scanning

Abstract

The present invention discloses a wavelength scanning-based lensless Fourier stacking diffraction tomography microscopy imaging method, which uses only a wavelength-tunable light source for illumination on a lensless on-chip microscopy experimental system to collect a series of coaxial holograms. Then, the three-dimensional scattering potential spectrum is filled by an iterative Fourier stacking method to directly restore the three-dimensional refractive index distribution of the sample. The present invention does not require complex modifications to conventional lensless on-chip microscopes, and can give lensless on-chip microscopes the ability of pixel super-resolution three-dimensional tomography.

Description

Technical Field

[0001] The invention belongs to three-dimensional refractive index imaging technology, in particular to a lensless Fourier stack diffraction tomography microscopy imaging method based on wavelength scanning. Background Art

[0002] High-throughput microscopy, that is, the ability to record images with a large field of view without compromising spatial and temporal resolution, is crucial to imaging science. For example, in applications such as neuroscience, stem cell biology, developmental biology, early cancer diagnosis, and personalized drug screening, we need to perform high-content quantitative analysis of multiple events in a large number of cell populations. However, the amount of information that traditional microscopy systems can provide is always limited, which is determined by the space-bandwidth product (SBP), usually in the order of 10 million pixels (10Megapixels). More specifically, there is a contradiction between resolution and field of view that is difficult to take into account at the same time in traditional microscopes. The field of view is large under low-power microscopes, but the resolution is low; after switching to high-power microscopes, although the resolution is improved, the field of view is correspondingly reduced by a higher proportion. Recently developed computational optical microscopy techniques, such as lensless on-chip holographic microscopy, Fourier stack microscopy, synthetic aperture / synthetic field of view holographic microscopy, etc. Among these methods, lensless super-resolution holography can be said to be the most promising technology. It achieves a large effective numerical aperture (NA) close to unity over the native field of view of the imaging sensor without the need for any lenses and other intermediate optical elements. This further simplifies the imaging setup while effectively circumventing the optical aberration and chromaticity issues inherent in traditional lens-based imaging systems. In addition, the entire system can be built in a miniaturized and low-cost form, providing a potential solution for point-of-care diagnosis in resource-limited environments to reduce medical costs.

[0003] Although lensless on-chip holographic microscopy seems to have solved the limited spatial bandwidth product in traditional microscopy systems, there are still many problems that limit its practical application. First, because the sample is placed close to the sensor surface, the resolution of the imaging is limited by the pixel size of the imaging device. Due to the limitations of technology and process level, the current sensor resolution is still far below the optical diffraction limit. Therefore, a lot of research work in this field is focused on "pixel super-resolution", and many methods have been invented, such as two-dimensional lateral sub-pixel scanning of sensors, micro-displacement sub-pixel scanning of light sources, axial multi-defocus distance scanning of sensors, and wavelength scanning of illumination light, to achieve detection with more than 2 times the super-pixel resolution. Secondly, when studying the dynamic process of living cells and their various physiological activities, label-free microscopy is an ideal detection method. Many quantitative phase imaging technologies based on lensless on-chip microscopy have emerged. For example, we have achieved long-term living cell observation through active plate scanning and multi-wavelength scanning methods. Finally, the above-mentioned lensless microscopy methods, whether for intensity measurement or phase recovery, are for two-dimensional thin sample imaging and lack three-dimensional tomography capabilities. So far, only a few works have explored lensless imaging of three-dimensional thick samples. Isikman et al. combined the basic concept of lensless holographic microscopy with multi-angle illumination (Isikman SO, Bishara W, Mavandadi S, et al. Lens-free optical tomographic microscope with a large imaging volume on a chip [J]. Proceedings of the National Academy of Sciences, 2011, 108 (18): 7296-7301.) and used the filtered back projection (FBP) algorithm to reconstruct the volume image of the object. They ignored the object diffraction information and thus could not image the phase object. In addition, the use of a robotic arm made the experimental setup complicated and expensive.Zuo et al. used LED array illumination to achieve mechanical displacement-free experimental system (Zuo C, Sun J, Zhang J, et al. Lensless phase microscopy and diffraction tomography with multi-angle and multi-wavelength illuminations using a LED matrix [J]. Optics express, 2015, 23 (11): 14314-14328.), and solved this problem by replacing filtered back projection with optical diffraction tomography in the tomographic reconstruction process and implementing a multi-wavelength light source at each illumination angle to restore the phase of the hologram. Restoring the refractive index distribution of a three-dimensional object from a hologram sequence is essentially an inverse scattering problem. Berdeu et al. used a 360° axially rotatable robotic arm equipped with a light source with a fixed tilt angle of 45° to establish a lensless on-chip diffraction tomography platform (Berdeu A, Momey F, Laperrousaz B, et al. Comparative study of fully three-dimensional reconstruction algorithms for lens-free microscopy [J]. Applied optics, 2017, 56 (13): 3939-3951.). They calculated the complex amplitude of each illumination angle by phase slope or two-dimensional phase retrieval method, taking into account the diffraction effect. Then, the full three-dimensional reconstruction was obtained according to the Fourier diffraction theorem. The above methods all use an experimental platform based on multi-angle illumination. However, in lensless imaging, objects are imaged out of focus. When the illumination angle changes, the imaging position of the sample will change by tens or hundreds of microns. This will cause objects close to the edge of the sensor to run out of the field of view, that is, the effective field of view in the multi-angle illumination method is much smaller than the size of the sensor target surface. Moreover, the movement of the object will also lead to the difficulty of image registration. It is precisely because of the above reasons that the results of lensless 3D imaging are not ideal, and the resolution is still limited by the discrete sampling of the sensor. Summary of the invention

[0004] The object of the present invention is to provide a lensless Fourier stack diffraction tomography microscopy imaging method based on wavelength scanning.

[0005] The technical solution to achieve the purpose of the present invention is: a lensless Fourier stack diffraction tomography microscopy imaging method based on wavelength scanning, the steps are as follows:

[0006] Step 1, collecting the original intensity map;

[0007] Step 2, constructing a three-dimensional refractive index space of the object to be measured;

[0008] Step 3, determining the corresponding position of the hologram collected at the corresponding wavelength on the three-dimensional spectrum, and obtaining a new refractive index distribution of the sample;

[0009] Step 4: Repeat step 3 according to the new refractive index distribution of the sample to complete the three-dimensional spectrum iteration at a single wavelength and obtain the final refractive index distribution of the sample.

[0010] Preferably, the original intensity map is collected using a lensless on-chip microscopy system, which includes a wavelength scanning light source and a sensor, wherein the wavelength scanning light source is a combination of a supercontinuum laser and an acousto-optic tunable filter or a plurality of monochromatic light sources coupled or a wavelength scanning laser. When the wavelength scanning light source is a combination of a supercontinuum laser and an acousto-optic tunable filter, a broadband light beam emitted by the supercontinuum laser is filtered by the acousto-optic tunable filter and then irradiated onto the sample, and the sample is disposed on the sensor.

[0011] Preferably, the pixel size of the three-dimensional refractive index space n(r) of the measured object meets the final imaging resolution, and the number of pixels N in the three-dimensional matrix x ,N y ,N z Meet the minimum number of samples in each direction.

[0012] Preferably, the specific steps of determining the positions corresponding to the holograms collected at different wavelengths on the three-dimensional spectrum are:

[0013] Step 3.1, calculate the sample scattering potential at the corresponding wavelength. The specific formula is:

[0014]

[0015] Where V(r,ω) is the sample scattering potential, r = (r x ,r y ,r z ) represents the three-dimensional space coordinates, ω=2πc / λ is the angular frequency, c is the light beam in vacuum, represents the wave number in vacuum, n(r) is the refractive index distribution of the sample, and n m is the refractive index of the background medium;

[0016] Step 3.2: Perform a three-dimensional Fourier transform on the sample scattering potential V(r,ω) to obtain a three-dimensional Fourier spectrum where u=(u x ,u y ,u z ) is the spatial frequency coordinate;

[0017] For the three-dimensional sub-spectrum along u zDirection projection, get the two-dimensional sub-spectrum The formula is as follows

[0018]

[0019] where u T =(u x ,u y ) represents the two-dimensional spatial frequency coordinate; k m (ω) is the wave vector in the surrounding medium, k m (ω)=|k m (ω)|=k0(ω)n m is the wave number in the surrounding medium, k0(ω)n m is the radius of the three-dimensional sub-spectrum, δ(·) is the Dirac delta function;

[0020] Step 3.3, perform inverse Fourier transform on the two-dimensional sub-spectrum to obtain the normalized first-order scattered field complex amplitude U on the focal plane s1n (r T ,ω); According to the normalized first-order scattered field complex amplitude on the focal plane, combined with the Rytov approximation, the complex amplitude on the focal plane is obtained;

[0021] Step 3.4, use the angular spectrum method to transfer the complex amplitude on the focal plane to the sensor plane to obtain the complex amplitude U(r T ,ω), and use the intensity map I(r T ,ω) to update the amplitude, propagate the update to the focusing surface, and obtain the updated complex amplitude of the scattered field on the focusing surface

[0022] Step 3.5, complex amplitude Take the ln(·) operation to obtain the updated normalized first-order scattered field right Perform Fourier transform to get the updated two-dimensional spectrum Will Remap to Ewald spherical shell and insert into the original 3D spectrum The corresponding position in the updated three-dimensional spectrum After performing a three-dimensional inverse Fourier transform, the new refractive index distribution of the sample is obtained

[0023] Preferably, the complex amplitude on the focal plane is specifically:

[0024] U s1 (r T ,ω)=U in (r T ,ω)exp[U s1n (r T,ω)]

[0025] Where U s1 (r T ,ω) is the complex amplitude of the first-order scattered field, U in (r T ,ω) is the complex amplitude of the incident field.

[0026] Preferably, the intensity map I(r T The specific formula for updating the amplitude by taking the square root of ω is:

[0027]

[0028] where j is the imaginary unit and arg(·) is the function that takes the argument of the complex amplitude.

[0029] Compared with the prior art, the present invention has the following significant advantages: (1) the present invention can achieve pixel super-resolution three-dimensional imaging with uniform resolution over the entire field of view of the sensor; (2) the present invention has only one light source at a fixed position, can maintain relatively high coherence, and does not introduce mechanical displacement, thereby improving stability.

[0030] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of the experimental setup for multi-wavelength tomography based on lensless on-chip microscopy.

[0032] Figure 2 It is a flow chart of a large-field-of-view pixel super-resolution lensless Fourier stack diffraction tomography method based on multi-wavelength scanning.

[0033] Figure 3 It is the corresponding position of the hologram collected at different wavelengths on the three-dimensional spectrum.

[0034] Figure 4 This is a schematic diagram of the spectrum shape corresponding to the final reconstructed sample refractive index distribution.

[0035] Figure 5 The three-dimensional refractive index distribution rendering of a single diatom and the two-dimensional refractive index distribution at different z-axis depths reconstructed using this method. DETAILED DESCRIPTION

[0036] like Figure 2 As shown, a lensless Fourier stack diffraction tomography microscopy imaging method based on wavelength scanning includes the following four steps:

[0037] Step 1: Collect raw intensity images using a lensless on-chip microscopy system.

[0038] like Figure 1As shown, the present invention is based on a conventional lensless on-chip microscopy system, which includes: a wavelength scanning light source, a sample 3 and a sensor 4. The wavelength tunable light source used in the present invention is achieved by filtering a broadband light beam from a supercontinuum laser 1 (YSLSC-Pro7) through an acousto-optic tunable filter 2 (AOTF, YSLAOTF-Pro, bandwidth: 2-11 nanometers, RF1: 430-780 nanometers, RF2: 780-1450 nanometers), which can achieve wavelength scanning within a wavelength range of 430-1450 nanometers, with a minimum interval of 1 nanometer. In addition, multiple monochromatic light sources (lasers, LEDs) can be coupled or wavelength scanning lasers can be used to replace the supercontinuum laser-acoustic-optic filter combination to achieve wavelength scanning. The sample 3 is directly placed on the image sensor for imaging, and the system is captured using a board-level monochrome CMOS sensor 4 (1.67 microns, 3872×2764, The ImagingSourceDMK24UJ003).

[0039] The specific implementation process is: using a coherent light source with a wide range of tunable wavelengths, sequentially tuning the illumination wavelength {λ i , i=1,2,...,N}, illuminate the sample, and synchronously trigger the camera to record the holographic image sequence at different wavelengths {I cap {r T ,ω),ω=ω 1 ,ω 2 ,...,ω N}.

[0040] Step 2: construct a three-dimensional refractive index space with large field of view and high resolution for the object being measured.

[0041] The specific implementation process is: Assume that the refractive index of the background medium is a constant, and its refractive index is n m Construct a three-dimensional refractive index space n(r) with a large field of view and high resolution for the object being measured. The pixel size of the three-dimensional refractive index space must meet the final imaging resolution, and the number of pixels in the three-dimensional matrix N x ,N y ,N z Meet the minimum number of samples in each direction.

[0042] Step 3, determining the corresponding positions of the holograms collected at different wavelengths on the three-dimensional spectrum, the specific implementation process is:

[0043] Step 3.1, at the illumination wavelength λ, by the formula

[0044]

[0045] Calculate the sample scattering potential V(r,ω) at this wavelength. x ,ry ,r z ) represents the three-dimensional space coordinates; ω=2πc / λ is the angular frequency, c is the light beam in vacuum, and ω is used in the formula to represent the variable under the wavelength λ; represents the wave number in vacuum when the incident wavelength is λ; n(r) is the refractive index distribution of the sample, n m is the refractive index of the background medium.

[0046] Step 3.2: Perform a three-dimensional Fourier transform on the sample scattering potential V(r,ω) to obtain its three-dimensional Fourier spectrum. where u=(u x ,u y ,u z ) is the spatial frequency coordinate. Figure 3 As shown, the three-dimensional sub-spectrum of the sample at the illumination wavelength λ is a radius k0(ω)n m , a spherical shell with the vertex at the origin of the frequency domain, and its value is given by Mid-edge spherical shell (uk m The three-dimensional sub-spectrum is determined by the value of (ω)). z After projection in the direction, a two-dimensional sub-spectrum is obtained. The formula is as follows

[0047]

[0048] where u T =(u x ,u y ) represents the two-dimensional spatial frequency coordinate; k m (ω) is the wave vector in the surrounding medium, k m (ω)=|k m (ω)|=k0(ω)n m is the wave number in the surrounding medium and δ(·) is the Dirac delta function.

[0049] Step 3.3, perform inverse Fourier transform on the resulting spectrum to obtain the normalized first-order scattered field complex amplitude U on the focal plane. s1n (r T ,ω), defined as U s1n (r T ,ω)=U s1 (r T ,ω) / U in (r T ,ω), where U s1 (r T ,ω) is the complex amplitude of the first-order scattered field, U in (r T ,ω) is the complex amplitude of the incident field.

[0050] Combining the Rytov approximation, the complex amplitude on the focal plane is written as:

[0051] U s1 (r T ,ω)=U in (r T ,ω)exp[U s1n (r T ,ω)]

[0052] Step 3.4: Use the angular spectrum method to convert U s1 (r T ,ω) is transmitted to the sensor plane, and the complex amplitude U(r T ,ω), and use the intensity graph I(r T ,ω) to update the amplitude

[0053]

[0054] Get the updated complex amplitude Back to the focal plane, where j is the imaginary unit and arg(·) is the function of the argument angle of the complex amplitude. Then Propagate to the focal plane and obtain the updated complex amplitude of the scattered field on the focal plane

[0055] Step 3.5, Take the ln(·) operation to obtain the updated normalized first-order scattered field Again Perform Fourier transform to get the updated two-dimensional spectrum Then Remap to Ewald spherical shell and insert into the original 3D spectrum Finally, the updated three-dimensional spectrum After performing a three-dimensional inverse Fourier transform, the new refractive index distribution of the sample is obtained This completes one sub-iteration of the lensless diffraction tomography algorithm based on multi-wavelength scanning.

[0056] Step 4, perform a complete iteration of the object to be measured to obtain the refractive index distribution n(r) of the sample.

[0057] The new refractive index distribution obtained in step 3.5 Update to step 3.1, calculate the corresponding position of the hologram collected at another wavelength on the three-dimensional spectrum, and obtain the new refractive index distribution of the sample, repeat step 3, and repeat the entire iterative process multiple times to obtain a converged result. Figure 4 Represents the range covered by the Ewald sphere in the three-dimensional frequency domain space under multi-wavelength illumination. Figure 5The three-dimensional refractive index distribution rendering of a single diatom and the two-dimensional refractive index distribution at different z-axis depths reconstructed using this method.

[0058] The present invention only needs to tune the illumination wavelength under vertical illumination of the light source to obtain a series of holograms, and then use a multi-wavelength-based Fourier stack diffraction tomography reconstruction algorithm combined with a propagation model to gradually combine these intensity images into a three-dimensional refractive index distribution of the sample. The present invention has only one light source at a fixed position, can maintain relatively high coherence, and does not introduce mechanical displacement, thereby improving the stability of the system.

Claims

1. A lensless Fourier stack diffraction tomography microscopy imaging method based on wavelength scanning, characterized in that: Here are the steps: Step 1, collecting the original intensity map; Step 2, constructing a three-dimensional refractive index space of the object to be measured; Step 3, determining the corresponding position of the hologram collected at the corresponding wavelength on the three-dimensional spectrum, and obtaining the new refractive index distribution of the sample, the specific steps are: Step 3.1, calculate the sample scattering potential at the corresponding wavelength. The specific formula is: Where V(r,ω) is the sample scattering potential, r = (r x ,r y ,r z ) represents the three-dimensional space coordinates, ω=2πc / λ is the angular frequency, c is the light beam in vacuum, represents the wave number in vacuum, n(r) is the refractive index distribution of the sample, and n m is the refractive index of the background medium; Step 3.2: Perform a three-dimensional Fourier transform on the sample scattering potential V(r,ω) to obtain a three-dimensional Fourier spectrum where u=(u x ,u y ,u z ) is the spatial frequency coordinate; For the three-dimensional sub-spectrum along u z Direction projection, get the two-dimensional sub-spectrum The formula is as follows where u T =(u x ,u y ) represents the two-dimensional spatial frequency coordinate; k m (ω) is the wave vector in the surrounding medium, k m (ω)=|k m (ω)|=k0(ω)n m is the wave number in the surrounding medium, k0(ω)n m is the radius of the three-dimensional sub-spectrum, δ(·) is the Dirac delta function; Step 3.3, perform inverse Fourier transform on the two-dimensional sub-spectrum to obtain the normalized first-order scattered field complex amplitude U on the focal plane s1n (r T ,ω); According to the normalized first-order scattered field complex amplitude on the focal plane, combined with the Rytov approximation, the complex amplitude on the focal plane is obtained; Step 3.4, use the angular spectrum method to transfer the complex amplitude on the focal plane to the sensor plane to obtain the complex amplitude U(r T ,ω0, and use the intensity map I(r T ,ω) to update the amplitude, propagate the update to the focusing surface, and obtain the updated complex amplitude of the scattered field on the focusing surface Step 3.5, complex amplitude Take the ln(·) operation to obtain the updated normalized first-order scattered field right Perform Fourier transform to get the updated two-dimensional spectrum Will Remap to Ewald spherical shell and insert into the original 3D spectrum The corresponding position in the updated three-dimensional spectrum After performing a three-dimensional inverse Fourier transform, the new refractive index distribution of the sample is obtained Step 4: Repeat step 3 according to the new refractive index distribution of the sample to complete the three-dimensional spectrum iteration at a single wavelength and obtain the final refractive index distribution of the sample.

2. The wavelength scanning-based lensless Fourier stack diffraction tomography microscopy imaging method according to claim 1, characterized in that: The original intensity map is collected by using a lensless on-chip microscopy system, which includes a wavelength scanning light source and a sensor. The wavelength scanning light source is a combination of a supercontinuum laser and an acousto-optic tunable filter or a plurality of monochromatic light sources coupled or a wavelength scanning laser. When the wavelength scanning light source is a combination of a supercontinuum laser and an acousto-optic tunable filter, a broadband light beam emitted by the supercontinuum laser is filtered by the acousto-optic tunable filter and then irradiated onto a sample, and the sample is set on the sensor.

3. The wavelength scanning-based lensless Fourier stack diffraction tomography microscopy imaging method according to claim 1, characterized in that: The pixel size of the three-dimensional refractive index space n(r) of the measured object meets the final imaging resolution, and the number of pixels N in the three-dimensional matrix x ,N y ,N z Meet the minimum number of samples in each direction.

4. The wavelength scanning-based lensless Fourier stack diffraction tomography microscopy imaging method according to claim 1, characterized in that: The complex amplitude on the focal plane is specifically: U s1 (r T ,ω)=U in (r T ,ω)exp[U s1n (r T ,ω)] Where U s1 (r T ,ω) is the complex amplitude of the first-order scattered field, U in (r T ,ω) is the complex amplitude of the incident field.

5. The wavelength scanning-based lensless Fourier stack diffraction tomography microscopy imaging method according to claim 1, characterized in that: Using the intensity map I(r T The specific formula for updating the amplitude by taking the square root of ω is: where j is the imaginary unit and arg(·) is the function that takes the argument of the complex amplitude.

Images