Angular spectrum laminated on-chip microscopic imaging system and method

Abstract

The invention provides an angular spectrum laminated on-chip microscopic imaging system and method, and the method comprises the steps: respectively filtering and intercepting diffraction object light in different angular spectrum intervals through an angular spectrum filter, carrying out the optical coding, and collecting an object light diffraction image formed by the coding of the corresponding angular spectrum interval on an image sensor; and the synthetic angular spectrum information of the object is decoded through an angular spectrum lamination iterative algorithm, and finally, large-view-field and high-resolution object amplitude and phase distribution which are not constrained by the aperture of the imaging system are reconstructed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computational optical three-dimensional microscopic imaging, in particular to an angular spectrum stack-on-chip microscopic imaging system and method. BACKGROUND

[0002] The field of optical microscopic imaging has long been faced with the contradiction between field of view and resolution: traditional optical systems are difficult to achieve large field of view and high resolution at the same time due to the limitation of numerical aperture and space-bandwidth product of objective lens. Computational optical imaging technology provides a solution through the way of "optical encoding + computational decoding".

[0003] Lensless encoding stack imaging technology is one of the computational optical imaging technologies, which removes the traditional optical lens, combines front-end optical encoding with back-end computational reconstruction algorithm to realize image acquisition and reproduction, replaces the optical imaging function of the lens through the diffraction characteristics of light wave and computational power, thereby breaking through the physical limitations of traditional imaging systems. It has the imaging advantages of simple system structure, large field of view, high resolution, and no aberration, but its encoding mode is composed of randomly distributed microspheres on the plane, which has the problems of needing accurate displacement calibration, collecting a large number of original images, and poor stability.

[0004] Fourier stack imaging technology is also one of the computational optical imaging technologies, the core of which is to synthesize local spectral information in a stack way by multi-angle illumination and collecting corresponding low-resolution sample information. Compared with traditional optical imaging systems, the use of low-power objective lens greatly alleviates the contradiction between resolution and field of view. However, the uniformity, coherence and illumination intensity of the programmable LED array light source used for illumination are not as good as those of laser light source, which has the problems of long exposure time and wavefront distortion.

[0005] More importantly, the current computational optical imaging systems generally use array image sensors, which have the problems of insufficient dynamic range of image sensors leading to saturation (overexposure) of high brightness areas and low signal-to-noise ratio (underexposure) of low brightness areas of the image, and the problem of small space-bandwidth product caused by large pixel size and small number of pixels, which leads to the problem that high-frequency signals representing sample details cannot be sampled and are overwhelmed by low-frequency signals. SUMMARY

[0006] In order to overcome the problems of complex optical imaging system, large space volume, mutual restriction between field of view and resolution, limited light source equipment, insufficient dynamic range and space-bandwidth product of image sensor, and high sampling redundancy rate in the prior art, the present application provides an angular spectrum stack-on-chip microscopic imaging system and method.

[0007] An angular spectrum stacked on-chip microscopic imaging system, arranged in order of light propagation: a coherent or partially coherent light source for illumination; a sample stage for carrying a sample; an angular spectrum filter for angular spectrum selection; an image sensor for collecting diffraction light intensity.

[0008] Specifically, the angular spectrum filter is an optical element that selectively transmits plane waves incident from different angles, which can pass object light within a certain diffraction angle range and block light outside this diffraction angle range.

[0009] More specifically, the angular spectrum filter is an interference filter or an angle-sensitive super surface or a volume Bragg grating or a volume holographic grating.

[0010] There is no distance restriction between the sample stage, the angular spectrum filter, and the image sensor, which can be monolithically integrated or set a distance according to the use scenario. When the distance between them is zero, an on-chip imaging system can be achieved.

[0011] An angular spectrum stacked on-chip microscopic imaging method, comprising the following steps: illuminating an object using a coherent or partially coherent light source and generating diffracted object light; using an angular spectrum filter to angularly filter the object light, respectively filtering and intercepting object light in different angular spectrum intervals; using an image sensor to collect different angular spectrum filtered object light and an object light diffraction image without angular spectrum filtering; using the object light diffraction intensity without angular spectrum filtering and an angular spectrum propagation algorithm to obtain an initial guess of the angular spectrum on the sample plane for iteration; using a stacked iterative phase recovery algorithm to decode the synthesized angular spectrum information of the object to be measured; and using the synthesized angular spectrum information to reconstruct the object amplitude and phase distribution without the aperture constraint of the imaging system, large field of view, and high resolution.

[0012] Specifically, the angular spectrum filtering is achieved by changing the passband of the angular spectrum filter to form different angular spectrum filtering functions.

[0013] More specifically, the angular spectrum filtering is achieved by replacing the angular spectrum filter with different angular spectrum passbands or changing the azimuth and elevation angle combination of the angular spectrum filter relative to the optical axis.

[0014] Compared with the prior art, the present application has the following advantages: (1) The system and method provided by the present application are not limited by any optical element aperture, and the field of view is determined only by the image sensor target surface, so that three-dimensional imaging with large field of view and high resolution can be achieved.

[0015] (2) The imaging system provided by the present application cancels the 4f system required by the traditional spatial filtering, uses the angular spectrum filtering technology, and has no distance limit among the sample, the filter and the image sensor, so that the system volume is greatly reduced compared with the prior art, on-chip three-dimensional imaging system can be realized, and the imaging system has simple and compact structure, low cost, high stability and can be applied to a wider application scenario.

[0016] (3) The method provided by the present application has a flexible image acquisition strategy, can adjust the angular spectrum filtering interval size according to the pixel size of the image sensor, reduce the interference fringe density, eliminate the problem of undersampling, acquire different angular spectrum components with large intensity difference in batches, overcome the defect that weak high-frequency signals are submerged by strong background light and low-frequency signals, reduce the requirement for the large dynamic range of the image sensor, efficiently and sufficiently acquire high-frequency information, and improve the imaging resolution.

[0017] (4) The method provided by the present application only needs to acquire three original images at least, the measurement process is simple and fast, the sampling redundancy rate is low, the calculation amount is small during reproduction, and the algorithm is efficient.

[0018] The present application will be further described in detail below with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 It is a schematic diagram of the angular spectrum laminated on-chip microscopic imaging system of the present application.

[0020] Figure 2 It is a schematic diagram of another angular spectrum laminated on-chip microscopic imaging system of the present application.

[0021] Figure 3 It is a step flow chart of the angular spectrum laminated on-chip microscopic imaging method of the present application.

[0022] Figure 4 It is a schematic diagram of the angular spectrum filter of the present application.

[0023] Figure 5 It is a schematic diagram of the angular spectrum filtering function formed by replacing different center wavelength filters.

[0024] Figure 6 It is a schematic diagram of the angular spectrum filtering function formed by using different azimuth angles and elevation angles of the same filter.

[0025] Figure 7 It is an object amplitude and phase distribution image recovered when different filters are replaced as angular spectrum filters.

[0026] Figure 8 It is an object amplitude and phase distribution image recovered when different azimuth angles and elevation angles of the same filter are used as angular spectrum filters.

[0027] Among them, the appendix Figure 1 The specific markings are: 1 is a 671nm red solid-state laser; 2 is a concave lens; 3 is a convex lens; 4 is the sample stage (sample); 5 represents an angular spectrum filter; 6 is an image sensor.

[0028] Appendix Figure 2 The specific markings are: 1 is a beam angle scanning device; 2 is an angular spectrum filter; 3 is an image sensor. Detailed Implementation

[0029] The angular spectrum stacked on-sheet microscopy imaging system and method provided by the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0030] First Embodiment The angular spectrum stacked on-sheet microscopy imaging system of this embodiment, such as Figure 1 As shown, the system includes: a red solid-state laser 1 with a wavelength of 671nm, whose emitted laser light is expanded and collimated by a concave lens 2 and a convex lens 3 to form a plane wave for illumination; the plane wave generates diffracted object light after passing through the sample 4 on the sample stage; the object light is selectively filtered by an angular spectrum filter 5 to intercept object light in different angular spectrum intervals; and an image sensor 6 is used to collect the object light intensity in the corresponding angular spectrum interval.

[0031] As a preferred embodiment, the distance d between the sample to be tested 4 and the image sensor 6 is 11 mm; the angular spectral filter 5 is a rotating wheel equipped with a bandpass filter group. Figure 4 As shown in (a), the filter group includes a narrowband filter with a center wavelength of 671nm and a passband width of 10nm and a narrowband filter with a center wavelength of 680nm and a passband width of 10nm. Different angular spectrum filtering in different angular spectrum ranges can be achieved by changing different filters through a rotating wheel.

[0032] As another preferred embodiment, the angular spectral filter 5 is a rotating tilting stage carrying a bandpass filter, such as... Figure 4 As shown in (b), a narrowband filter with a center wavelength of 671nm and a passband width of 10nm is mounted. The rotating tilting stage rotates around the optical axis. By rotating and tilting, different combinations of azimuth and elevation angles are formed to achieve angular spectrum filtering in different angular spectrum ranges.

[0033] As a preferred embodiment, the angular spectrum filter 5 is a transmittance angle-sensitive metasurface prepared using micro-nano fabrication technology. The metasurface transmits incident light within a specific angle range and blocks incident light outside that angle range, thereby achieving angular spectrum filtering.

[0034] As a preferred embodiment, the angular spectrum filter 5 is a volume Bragg grating, which transmits incident light within a specific angular range and blocks incident light outside that angular range, thereby achieving angular spectrum filtering.

[0035] As a preferred embodiment, the angular spectrum filter 5 is a volume holographic grating, which transmits incident light within a specific angular range and blocks incident light outside that angular range, thereby achieving angular spectrum filtering.

[0036] Second Embodiment The angular spectrum stacked on-sheet microscopy imaging system of this embodiment, such as Figure 2 As shown, the system includes: a beam angle scanning device 1, which incident a (quasi-)plane wave (wavelength 671nm) after beam expansion and collimation onto the sample plane at different azimuth and elevation angles; a narrowband filter 2 with a center wavelength of 671nm and a passband width of 10nm, which is integrated above the image sensor, serves as a sample stage to support the sample and intercepts the angular spectrum diffracted from the sample to achieve angular spectrum filtering; and an image sensor 3, which is used to collect the object light intensity in the corresponding angular spectrum range of the beam angle scanning device at different azimuth and elevation angles.

[0037] Third Embodiment This embodiment will further illustrate the angular spectrum stacked on-sheet microscopy imaging method provided by the present invention in conjunction with the accompanying drawings and embodiments.

[0038] Step 1: Illuminate the object using a (quasi)plane wave and generate an object light wave. The (quasi)plane wave is represented as:

[0039] The object wavefront is represented as follows:

[0040] In the formula, The object light emitted from the target is the wavefront. The complex amplitude of the target to be measured is denoted as .

[0041] object light wavefront Perform a Fourier transform to obtain the angular spectrum on the sample plane. ,in, This represents the wave vector of the object light in the x and y directions.

[0042] Step 2: different angular spectrum filtering is performed on the object light using an angular spectrum filter, and the object light in different angular spectrum intervals is intercepted, wherein the mthfilter function of the angular spectrum filtering is expressed as:

[0043] In the formula , the amplitude transfer function of the mthangular spectrum filter function is expressed as , and the phase transfer function is expressed as

[0044] Specifically, the angular spectrum filter in Step 2 is a narrowband filter group with a center wavelength of 671 nm and 680 nm and a passband half-width of 10 nm. The filter group is perpendicular to the optical axis. When the working wavelength is 671 nm, the transmittance functions of the filters change with the incident light angle, as shown in Figure 5 (a) and Figure 5 (b), which form circular and annular bandpass in the angular spectrum domain, as shown in Figure 5 (c) and Figure 5 (d), respectively. By using different filters in rotation, angular spectrum filtering in different angular spectrum intervals is achieved.

[0045] As another preferred solution, the angular spectrum filter in Step 2 is a filter with a center wavelength equal to the working wavelength and a passband half-width of 10 nm. The filter is rotated around the optical axis, and the pitch angle between the optical axis is adjustable. By combining different azimuth angles and pitch angles, different filter functions are formed. At this time, the filter function is equivalent to the translation of the filter function when the optical axis is perpendicular to the filter function in the angular spectrum domain. The specific expression is:

[0046]

[0047]

[0048] In the formula and are the corresponding pitch angle and azimuth angle of the angular spectrum filter, respectively. , are the angular spectrum offsets introduced by the corresponding pitch angle and azimuth angle in the x-axis direction and the y-axis direction, respectively. is the optical transfer function of the filter when it is perpendicular to the optical axis. As shown in Figure 6 , the amplitude transfer functions of the nine different angular spectrum filter functions are shown.

[0049] As another preferred solution, the angular spectrum filter in step 2 is a filter with the same center wavelength as the working wavelength and a passband half-width of 10 nm, which is fixedly integrated on the image sensor, and different angular spectrum filtering is realized by changing the incident light angle (azimuth angle and elevation angle) incident to the sample plane to intercept the diffraction light of different angles of the sample.

[0050] Step 3: Collecting the filtered object light diffraction intensity on the image sensor plane at a distance d from the sample plane At the same time, collecting an object light diffraction intensity without angular spectrum filtering .

[0051] Specifically, the diffraction process with a propagation distance of d in step 3 can be represented as:

[0052] In the formula, is the angular spectrum after the mthangular spectrum filtering and propagating to the image sensor plane, is the optical transfer function of the free space propagation distance . Taking the inverse Fourier transform of to obtain the wavefront on the image sensor plane , the angular spectrum filtered diffraction intensity collected on the image sensor plane is:

[0053] Step 4: Obtaining the initial guess angular spectrum of the sample plane as the initial value of the superposition iteration phase recovery algorithm.

[0054] Specifically, the object light diffraction intensity without angular spectrum filtering in step 3 is subjected to square root, upsampling and Fourier transform operations in sequence to obtain the angular spectrum on the image sensor plane :

[0055] In the formula, represents N times upsampling, represents Fourier transform. Using the angular spectrum propagation model, inversely propagate to the sample plane at a distance d from the image sensor plane to obtain the initial guess angular spectrum of the sample plane :

[0056] In the formula, represents inverse propagation of the angular spectrum.

[0057] Step 5: Using the diffraction intensity collected after filtering by different angular spectrum (including no angular spectrum filtering) to iterate repeatedly the stack iteration on the sample plane and the image sensor plane to obtain the synthesized angular spectrum on the sample plane.

[0058] Specifically, it includes the following steps: using the initial guess angular spectrum in step 4 as the initial value of the angular spectrum on the sample plane Filtering with the mthangular spectrum filtering function in step 2 to obtain the filtered angular spectrum

[0059] Using the angular spectrum propagation to diffract the filtered angular spectrum to the image sensor plane at a distance d to obtain the predicted angular spectrum

[0060] Performing inverse Fourier transform and N times downsampling operation on the predicted angular spectrum to obtain the complex amplitude with the same number of pixels as the image sensor

[0061] Using the amplitude obtained after square root of the mthdiffraction image collected by the image sensor Update the amplitude part of the current predicted complex amplitude , and then perform upsampling to obtain the updated complex amplitude with the same number of pixels as the initial value on the image sensor plane Performing inverse Fourier transform on the updated complex amplitude , and using the angular spectrum propagation to update the filtered angular spectrum on the sample plane

[0062] Using the rPIE algorithm to update the angular spectrum on the sample plane , and using the updated result as the input of the next round of stack; in the next round of stack, using the diffraction images filtered by the remaining angular spectrum on the image sensor plane to update the amplitude, and using the rPIE algorithm on the sample plane to update the angular spectrum, after iterating through all the m angular spectrum filtered diffraction images and the diffraction image without angular spectrum filtering, repeating the steps described in step 5 to iterate until the result converges, and obtaining the synthesized angular spectrum on the sample plane.

[0063] Step 6: Performing inverse Fourier transform on the synthesized angular spectrum in step 5 to obtain the complex amplitude distribution of the sample with large field of view and high resolution .

[0064] Figure 7 ​​​​​The results of angular spectrum filtering using a replacement filter are shown, where Fig. (a), (d) are the amplitude distribution and phase distribution true values of the sample, Fig. (b), (e) are the image amplitude and phase results obtained by N=2 up-sampling reconstruction, and Fig. (c), (f) are the image amplitude and phase results obtained without up-sampling reconstruction.

[0065] Figure 8 The results of angular spectrum filtering using the same filter with different azimuth and elevation angle combinations are shown, where Fig. (a), (d) are the amplitude image and phase image true values of the sample, Fig. (b), (e) are the image amplitude and phase results obtained by N=2 up-sampling reconstruction, and Fig. (c), (f) are the image amplitude and phase results obtained without up-sampling reconstruction.

[0066] The core of the present application is to select the angular spectrum in a certain angle range for optical encoding, and to perform calculation decoding on this basis. Those skilled in the art can understand that any optical element that can selectively transmit light rays within a certain angle range and block light rays outside the angle range can be used as an alternative to the angular spectrum filter described in the present application, and any method that can intercept part of the angular spectrum can be used as an alternative to the angular spectrum filtering method described in the present application, and is not limited to the above several. In order to cover a wider angular spectrum range and improve imaging resolution, the number of filters can be increased or different elevation angles can be increased, and the diffraction patterns in the corresponding angular spectrum range can be collected. According to the above content, according to the ordinary technical knowledge and conventional means in the art, without departing from the above basic technical ideas of the present application, the present application can also be made into other various forms of equivalent modifications, replacements or changes, such as using different wavelengths, different polarization directions of light source combined with angular spectrum filter to realize optical encoding, using single wavelength light source and multiple angular spectrum filter in series combination, using multiple wavelength light source and single angular spectrum filter combination, using a single filter with multiple passbands as an angular spectrum filter, etc. These all fall within the protection scope of the present application.

Claims

1. An angular-spectral stack-on-chip microscopic imaging system, characterized in that, In order of light propagation: a coherent or partially coherent light source for illumination; a sample stage for carrying a sample; an angular spectrum filter for angular spectrum selection; an image sensor for collecting diffracted light intensity.

2. The angular-spectral stack-on-chip microscopic imaging system of claim 1, wherein, Further comprising a beam angle scanning device for carrying the coherent or partially coherent light source, for making the illumination light incident on the sample stage at different angles.

3. The angular-spectral stack-on-chip microimaging system of claim 1 or 2, wherein, The angular spectrum filter is an optical element that selectively transmits plane waves incident from different angles, which can pass the object light within a certain diffraction angle range and block the light outside this diffraction angle range.

4. The angular-spectral stack-on-chip microscopic imaging system of claim 1 or 2, wherein, The angular spectrum filter is an interference filter or an angle-sensitive transmission super surface or a volume Bragg grating or a volume holographic grating.

5. A method of angular-spectral lamination on-chip microscopy, characterized in that The method comprises the following steps: (1) illuminating the object with a coherent or partially coherent light source and generating diffracted object light; (2) using an angular spectrum filter to filter the object light, respectively filtering and intercepting object light in different angular spectrum intervals; (3) using an image sensor to collect diffracted images of object light after different angular spectrum filtering and one without angular spectrum filtering; (4) using the diffracted light intensity of the object light without angular spectrum filtering and the angular spectrum propagation algorithm to obtain the initial guess map of the angular spectrum on the sample plane for iteration; (5) using a stack iterative phase retrieval algorithm to decode the synthesized angular spectrum information of the object to be measured; (6) using the synthesized angular spectrum information to reconstruct the object amplitude and phase distribution without the aperture constraint of the imaging system.

6. The angular-spectral stack-on-chip microscopic imaging method of claim 5, wherein, In the step of illuminating the object with a coherent or partially coherent light source and generating diffracted object light, the beam angle scanning device is used to illuminate the object from different angles and generate diffracted object light.

7. The angular-spectral stack-on-chip microscopic imaging method of claim 5, wherein, The angular spectrum filtering is achieved by changing the angle of the angular spectrum filter or by replacing the angular spectrum filter with different angular spectrum passbands to form different angular spectrum filtering functions.

8. The angular-spectral stack microscopy method of claim 6, wherein, The angular spectrum filtering is achieved by changing the angle of the incident light through the beam angle scanning device.

9. The angular-spectral stack microscopy method of claim 7 or 8, wherein, The overlap between different angular spectrum intervals is zero.

10. The angular-spectral stack microscopy method of claim 7 or 8, wherein, There is a gap between different angular spectrum intervals.

Images