Field-dependent aberration correction in coherent imaging, in particular digital holography

Abstract

Various examples of the disclosure relate to coherent imaging techniques such as digital holography or Fourier ptychography. Aberrations are corrected. The aberrations are corrected according to the field location.

Description

[0001] DESCRIPTION

[0002] FIELD-DEPENDENT ABERRATION CORRECTION IN COHERENT IMAGING, ESPECIALLY DIGITAL HOLOGRAPHY

[0003] TECHNICAL FIELD

[0004] Various examples of the disclosure relate to methods for imaging using an optical imaging system configured for coherent measurement of a sample object using amplitude and phase information. Various examples relate to digital holography. Various examples particularly relate to aberration correction in digital holography.

[0005] BACKGROUND

[0006] Digital holography is a coherent imaging technique that allows the reconstruction of the complex-valued image field of an object. Due to the coherent relationship between the image field and the exit pupil, the complex-valued field in the exit pupil of the imaging system is also known.

[0007] This property allows for a number of interesting manipulations of digital holograms, such as digital refocusing or the compensation of optical aberrations. This is described in: Myung K. Kim, "Principles and techniques of digital holographic microscopy," SPIE Rev. 1 (1 ) 018005 (1 April 2010)

[0008] Image aberrations in holographic imaging not only lead to a distorted visualization of the object but also introduce quantitative errors in the image phase, which, for example, renders surface or dieken metrology based on the measured phase of the sample object inaccurate. This also applies to other imaging techniques that measure a sample object coherently using amplitude and phase information. An example of another imaging technique would be Fourier ptychography. See, for example, Tian, ​​Lei, and Laura Waller. "Quantitative differential phase contrast imaging in an LED array microscope." Optics express 23.9 (2015): 11394–11403. See also Song, Pengming, et al. "Full-field Fourier ptychography (FFP): Spatially varying pupil modeling and its application for rapid field-dependent aberration metrology." APL Photonics 4.5 (2019): 050802.

[0009] For example, Alexander Stadelmaier and Jürgen H. Massig, "Compensation of lens aberrations in digital holography," Opt. Lett. 25, 1630-1632 (2000), describe techniques for reducing aberrations in digital holography. The techniques described there exhibit limited accuracy.

[0010] SHORT DESCRIPTION

[0011] Therefore, there is a need for improved techniques for digitally correcting aberrations in imaging techniques with quantitative phase information.

[0012] This problem is solved by the features of the independent patent claims. The features of the dependent patent claims define embodiments.

[0013] A method is disclosed with which the field-independent and field-dependent image errors (aberrations) can be measured in a coherent imaging process and can then be calculated out of a measured object field.

[0014] The techniques disclosed herein can be used, for example, in digital holography or Fourier ptychography.

[0015] A computer-implemented method for imaging using an optical imaging system is disclosed. The optical imaging system is configured for coherent measurement of a sample object using amplitude information and phase information. The method comprises controlling the optical imaging system to measure the pupil function of an imaging optics of the optical imaging system using a calibration object at a plurality of discrete field positions as part of a calibration process. The method also comprises determining a steady and continuous representation of the pupil function as a function of the field position based on the pupil function measured at the plurality of discrete field positions. The method further comprises controlling the optical imaging system to measure the sample object.Based on the measurement of the sample object, an image of the sample object is reconstructed taking into account the continuous and steady representation of the pupil function.

[0016] A two-step process is disclosed. First, calibration is performed, followed by measurement of the sample object. During calibration, a calibration object is used that allows pupil function to be measured. The calibration object is different from the sample object. The calibration object has properties that allow pupil function to be measured.

[0017] The pupil function describes the change in the complex amplitude of a light source after imaging by an optical system at a specific location. The pupil function is related to the point transfer function or point spread function. The point transfer function (sometimes also called point spread function) is used to describe the intensity distribution of the image generated by an optical system of a point-like object lying on the optical axis. The point transfer function is the absolute square of the normalized Fourier transform of the pupil function of the imaging optical system.

[0018] Therefore, in these examples, one or more point-shaped objects are used as calibration objects. In practice, "point-shaped" means a particularly small object (e.g., with a diameter smaller than the resolution limit of the optical system).

[0019] The multiple discrete field positions correspond to the different positions of the point-like object perpendicular to the optical axis (different lateral positioning). This corresponds to specific sampling points at which the pupil function is measured. The continuous representation of the pupil function corresponds to an interpolation between the different sampling points at the multiple discrete field positions.

[0020] By using a steady and continuous representation of the pupil function, a more accurate and numerically easy-to-handle reconstruction of the image of the sample object can be achieved taking the pupil function into account.

[0021] The features set forth above and features described below may be used not only in the corresponding explicitly set forth combinations, but also in further combinations or in isolation, without departing from the scope of the present invention.

[0022] SHORT DESCRIPTION OF THE CHARACTERS

[0023] FIG. 1 schematically illustrates a system with a data processing device and an optical imaging system designed as a microscope for the coherent measurement of a sample object with amplitude information and phase information according to various examples.

[0024] FIG. 2 is a flowchart of an exemplary method.

[0025] Fig. 3 illustrates a simulated pupil function with aberrations at a specific field position.

[0026] FIG. 4 illustrates basis functions of a principal component analysis of the field-dependent pupil function from FIG. 3 up to the third order.

[0027] FIG. 5 illustrates the magnitude of the coefficients of the basis functions from FIG. 4.

[0028] FIG. 6 illustrates the image field simulated with the pupil function from FIG. 4 for a calibration object with multiple point-shaped apertures according to various examples.

[0029] FIG. 7 illustrates the pupil function measured using the image field from FIG. 6 at a specific field position according to various examples. FIG. 8 illustrates the image field of a sample object when imaged using the pupil function according to FIG. 3.

[0030] FIG. 9 illustrates the corrected image field according to various examples.

[0031] DETAILED DESCRIPTION

[0032] The above-described properties, features and advantages of this invention, as well as the manner in which they are achieved, will become clearer and more clearly understood in connection with the following description of the embodiments, which are explained in more detail in connection with the drawings.

[0033] The present invention is explained in more detail below using preferred embodiments with reference to the drawings. In the figures, identical reference numerals designate identical or similar elements. The figures are schematic representations of various embodiments of the invention. Elements shown in the figures are not necessarily drawn to scale. Rather, the various elements shown in the figures are depicted in such a way that their function and general purpose will be understood by those skilled in the art. Connections and couplings between functional units and elements shown in the figures can also be implemented as an indirect connection or coupling. A connection or coupling can be implemented wired or wirelessly. Functional units can be implemented as hardware, software, or a combination of hardware and software.

[0034] The following describes techniques for coherent imaging with amplitude and phase information for a sample object. For example, digital holography or Fourier ptychography could be used as imaging techniques.

[0035] In particular, techniques are described below for reconstructing an image of a sample based on the measurement of the sample using the corresponding imaging technique. The reconstruction is based on incorporating information about one or more aberrations of the optical imaging system, so that the influence of these one or more aberrations on the reconstructed image can be reduced. The image is obtained with higher quality, and image artifacts due to imperfections in the optical imaging system are reduced.

[0036] The techniques disclosed herein are based on calibration with a corresponding calibration object.

[0037] For example, calibration could be performed immediately before measuring the sample object. It would also be conceivable to perform calibration once for a specific optical setup of the optical imaging system. Calibration could, for example, be performed during an end-of-line test during production.

[0038] As part of the calibration process, the pupil function of the imaging optics of the optical imaging system is measured using the calibration object.

[0039] Let 0(%,y) be the (generally complex-valued) object transmission distribution (or reflection distribution) of the light and P(f, g) be the imaging pupil function of the imaging optics, which includes the numerical aperture (NA) of the exit pupil as well as the image aberrations of the imaging optics. Then, the electric field in the sensor plane / image plane for coherent imaging (e.g., digital holography) is given by x, y and x', y' are coordinates in position space. f, g are coordinates in a pupil plane. T is the Fourier transform; J 7-1 the inverse Fourier transform.

[0040] This electric field (which corresponds to the image of the object in the sensor plane, and is therefore also referred to as the image field) is particularly accessible through digital holography. It is reconstructed from the measured quantities; the specific reconstruction method, based on the measured interferogram or interference pattern between the object and reference waves, can be implemented according to reference techniques, see, for example, Myung K. Kim, "Principles and techniques of digital holographic microscopy," SPIE Rev. 1 (1) 018005 (April 1, 2010). Typically, a CMOS or CCD sensor is used to measure the interference pattern between the object wave (sample beam) and the reference wave (reference beam). The image field can then be reconstructed from the interference pattern; a process known as numerical diffraction. The image field is also accessible using other coherent imaging techniques, such as Fourier ptychography.

[0041] First, the calibration process is described. This process is used to measure the pupil function P(f,g).

[0042] During the calibration process, a calibration object is measured. The calibration object can include at least one point-shaped contrast object, e.g., at least one point-shaped aperture or at least one point-shaped aperture.

[0043] If a small, point-shaped aperture is chosen as the object structure for calibration, its spectrum T[0(x,y)] is a very weakly varying function of f and g, which is generally known. Small here means that the diameter is on the order of the resolution limit of the optical system. For a circular aperture, for example, one has the central part of an Airy function, while for rectangular apertures, one has the central part of a sinc function. See also TABLE 1 below.

[0044] This means that the complex-valued pupil function P f,g) within the aperture of the imaging optics is derived from the reconstructed image field E and the known object spectrum of the calibration object can be obtained as:

[0045] The concrete image errors of the system can then be determined (in a conventional way) from a decomposition of the phase of P f,g) into Zernike basis functions Z k (f,g can be obtained: arg[

[0046] In other words, if the pupil function is known, the field distribution of one or more aberrations in the imaging optics can be determined based on the Zernike basis functions. This can be helpful, for example, to estimate the resolution or inaccuracy in imaging the sample object. This can be useful in a quantitative evaluation for determining uncertainties or error bars.

[0047] If the point aperture of the calibration object is positioned on the optical axis, i.e., generally in the center of the image field (x = 0, y = 0), then the pupil function is measured at this field location, i.e., P f,g; x = 0, y = 0). By choosing an arbitrary location (x c ,y c ) for the point-shaped opening of the calibration object, the pupil function can also be determined for other field locations, and thus the field-dependent pupil function can be determined:

[0048] In other words, this means that during the calibration process, the pupil function is measured at several discrete field positions {x c ,y c} is measured.

[0049] The field-dependent pupil function is connected to the field-dependent complex-valued amplitude point transfer function (sometimes also called amplitude point spread function, APSF) via a Fourier transform: There are various options for selecting the calibration object and implementing the calibration process. Some variants for circular contrast objects are summarized in Table 1.

[0050] TABLE 1: Various examples for the implementation of a calibration object with one or more point-like contrast objects and, associated with this, for measuring the pupil functions at several discrete field positions. Using the preceding calibration process, the pupil function is then known at several discrete field positions; see equation (4).

[0051] The following describes how this expanded knowledge of the pupil function—not just at one field position, but at multiple field positions—can be used to correct aberrations particularly effectively. It should first be noted that the correction of field-independent aberrations (i.e., for a pupil function that was not determined field-dependently) is already known, see Jürgen H. Massig, "Compensation of lens aberrations in digital holography," Opt. Lett. 25, 1630-1632 (2000). The correction according to such reference techniques follows directly from equation (1). The desired, undisturbed object field is given by the reconstructed image field by

[0052] If the pupil function has only a phase effect, but no amplitude effect over the range of the NA, the calculation can also be carried out as follows:

[0053] In the following, the correction of field-dependent aberrations according to the techniques described herein is described, going beyond these conventional techniques. This involves interpolation of the pupil function P(f,g; x c ,y c ). A continuous representation of pupil function is determined based on a principal component analysis. Principal component analysis is also called the Karhunen-Loeve transformation: In the literature, the method is also known as Proper Orthogonal Decomposition (pod). See Kerschen, G., Golinval, Je., VAKAKIS, AF et al. The Method of Proper Orthogonal Decomposition for Dynamical Characterization and Order Reduction of Mechanical Systems: An Overview. Nonlinear Dyn 41 , 147-169 (2005). In practice, the spatial dependence of the pupil function iA is a weak and continuously varying function. Therefore, the spatially dependent coefficients Kj decay rapidly over j, and it suffices to consider a finite and generally small number M of basis functions Bj:

[0054] Typically, M is in the range of 5 to 10. Even such an expansion up to the fifth or tenth order shows a high accuracy in aberration correction.

[0055] Furthermore, the continuous spatial dependence allows an interpolation of the coefficient matrix Kj(x c ,y cto a finer spatial grid, e.g. the target grid (x,y) of the image acquisition:

[0056] Kj x,y) = interpol [ / <7(x c ,y c )]

[0057] (10)

[0058] The correction of field-dependent aberrations is achieved by a backward propagation of the measured complex image field E(x',y') by the complex conjugate field-dependent pupil function (which reverses the phase effect of the Zernikes). From equations (7) and (9) follows for the object field: This result for the corrected field now provides the appropriate signal for a quantitative phase or height evaluation of the given object. For example, a quantitative phase image of the sample object can be output.

[0059] Furthermore, from (1 ) and (9) follows the forward calculation of a holographic image, with field-dependent image errors:

[0060] FIG. 1 schematically illustrates a system 70 comprising a data processing device 90 and a microscope 80.

[0061] Microscope 80 can, for example, be a digital holographic microscope that uses a detector to record an interference pattern between a sample beam and a reference beam from a coherent light source, e.g., a laser. An interferometric image can be generated. A Mach-Zehnder interferometer or a Michelson interferometer can be used.

[0062] It would also be possible for the microscope to be a digital Fourier ptychography microscope, which has an illumination module in the illumination pupil plane with several coherent light sources that can be individually switched on and off. This allows multiple intensity images to be captured with oblique illumination, and based on these multiple intensity images, the phase of the object can then be quantitatively reconstructed.

[0063] Microscope 80 can provide corresponding measurement data to an interface 91 of data processing device 90. A processor 92 can process the corresponding measurement data. For this purpose, processor 92 can load and execute program code from a memory 93.More generally, executing the program code from memory 93 causes processor 92 to perform techniques as described herein, for example: controlling microscope 80 by sending control data via interface 91; performing a calibration process; measuring a pupil function at several discrete field positions; determining a steady and continuous representation of the pupil function; controlling microscope 80 to measure a sample object or a calibration object; reconstructing an image of the sample object using a reconstruction algorithm, for example, a numerical diffraction algorithm associated with digital holography or a Fourier ptychography reconstruction algorithm; etc.

[0064] Some general aspects of digital holography are explained below. In digital holography, the holographic interference pattern is generated optically by the superposition of object and reference beams, which are digitally scanned by a camera (e.g., a CCD camera) and transmitted to a computer as a series of numbers. The interference between the object and reference beams creates an interference pattern, also known as a hologram. The interference pattern contains both the amplitude and phase information of the object beam, allowing the reconstruction of a three-dimensional image of the object. The propagation of (typically fully coherent) optical fields is completely and accurately described by diffraction theory, which allows numerical reconstruction of the image as a series of complex numbers representing the amplitude and phase of the optical field.

[0065] The reference beam can be obtained as an external reference, for example, and can be generated separately from the object beam or split off from the object beam before the object is illuminated. A Michelson interferometer setup or a Mach-Zehnder interferometer setup can be used to generate the interference pattern. In a Michelson interferometer, the incoming light beam is split into two paths at a beam splitter. One of these paths illuminates the object (object beam), while the other serves as the reference beam. After reflection or transmission, the two beams are recombined at the beam splitter, where they interfere. The Mach-Zehnder interferometer works in a similar way, but with a different arrangement of the optical components: here, the object beam and the reference beam are spatially separated and guided along different paths before being recombined to generate the interference pattern.However, it would also be possible to couple the reference beam into the object beam using an optical fiber, thus avoiding the need for beam splitters and beam combiners. This allows for a more compact and robust design.

[0066] Details relating to techniques that may be performed by processor 92 are described below.

[0067] FIG. 2 is a flowchart of an exemplary method. The method of FIG. 2 is computer-implemented, meaning it can be executed, for example, by a data processing device such as the data processing device 90 of FIG. 1. The method of FIG. 2 is used for imaging using an optical imaging system, such as the microscope 80 of FIG.

[0068] 1 . In particular, a coherent measurement of a sample object with amplitude and phase information can be achieved. Aberration correction can also be enabled.

[0069] In box 3005, the optical imaging system is controlled to measure the pupil function of an imaging optics of the optical imaging system using a calibration object at several discrete field positions as part of a calibration process. Examples of calibration objects were discussed above in connection with Table 1. The measurement of the pupil function at the various discrete field positions was described above in connection with equation (4).

[0070] In box 3010, a continuous representation of the pupil function is determined as a function of the field position based on the pupil function measured at several discrete field positions. For example, interpolation can be performed between the measured sampling points. A principal component analysis can be performed; corresponding techniques were described above in connection with equation (9).

[0071] Then, the pupil function of the optical imaging system is known. This pupil function includes the aberrations. For example, it would be conceivable that, based on the continuous and steady representation of the pupil function, a field distribution of at least one of the aberrations of the imaging optics could be quantified. Corresponding techniques were described above in connection with equation (3), where the decomposition of the phase of the pupil function into Zernike basis functions was discussed. The components of the various Zernike basis functions are indicative of the respective strength of the associated aberration.

[0072] The sample object can then be measured in box 3015. This means that the calibration object is removed from the microscope's beam path and the sample object is placed in the beam path instead.

[0073] It would then be conceivable, for example, for the measurement signal for the sample object to be corrected for these aberrations. The image of the sample object can then be reconstructed in box 3020 after the measurement signals have been corrected. Particularly in the context of digital holography, a scenario has been described in which the object field for the sample object is determined based on a backpropagation of the image field (which is conventionally reconstructed) from the detector plane to the object plane using the inverse of the continuous representation of the pupil function. This was described in connection with equation (11).

[0074] An exemplary implementation is described below using a simulation of a digital holographic image. A field-dependent coma profile and a field-dependent field curvature (as an example aberration) are modeled. The pupil function is calculated using the corresponding Zernike description of coma and sphere at eleven field locations (x n ,y = 0),n = 1 ... 11, along the x-axis (lateral, perpendicular to the optical axis, which is oriented in the z-direction). As an example, FIG. 3 shows the phase 62 of the pupil function used for the simulation for a position at the field edge.

[0075] The corresponding Karhunen-Loeve decomposition (principal component analysis) of this field-dependent pupil is shown in FIG. 4 and FIG. 5. In detail, FIG. 4 shows the phase 62 of the first three basis functions ( / =1, 2, 3) of the principal component analysis; and FIG. 5 shows the magnitudes of the coefficients of the principal component analysis ( |Ä | in equation (8)) for different field positions (at a comparatively low lateral resolution; this can be increased by interpolation). In FIG. 5, the legends show the assignment of the values ​​of the plotted magnitudes of the components of the principal component analysis to gray levels. It can be seen that the magnitudes decrease from j=1 to j=3 (from approximately 160 to 45). This corresponds to the finding that only a limited number of principal components (up to approximately M=5 or / W=10) need to be considered for a precise analysis (beyond this, the coefficients are so small that they can be neglected).

[0076] Now, using equation (12), the holographic image field can be simulated, which would have been measured by the holographic reconstruction of an interferogram recorded with an optical system with the given field-dependent aberrations (assuming the aberrations are present). Seven pinholes on the x-axis with a diameter below the resolution limit of the optics used are chosen as the object structure, which sample the field-dependent aberrations on the x-axis. This corresponds to a calibration object with a 1x7 array of point-shaped openings; see TABLE 1: Example 2. The corresponding image field is shown in FIG. 6 – obtained by imaging with field-dependent coma and field-dependent field curvature (specifically, the amplitude λ1 and the phase λ2; as well as a line plot of the amplitude at y=0).

[0077] From the simulated image field shown in FIG. 6, the field-dependent pupil function can be obtained by cropping the point images (inset in FIG. 6 and dashed squares; the cropping is shown in FIG. 6 as an example for a single position at the field edge, but is repeated for all seven point images to obtain multiple support points for the pupil function) and Fourier transforming. The pupil function measured based on the cropped image of the corresponding point image in FIG. 6 is shown in FIG. 7. A comparison of FIG. 3 (“ground truth”) with FIG. 7 (sampling of the pupil function) shows that the pupil function can be measured to a good approximation in this way.

[0078] The pupil function can thus be determined at several support points (at different field positions). A principal component analysis can then be carried out. Calibration is then complete. The “backward path” of the calculation is shown as an example in FIG. 8 and FIG. 9. If the field-dependent pupil function is known through such a calibration step, any sample object can be corrected in a field-dependent manner as described above. This is shown in FIG. 8 and FIG. 9. The image field of a row of points, imaged with image field curvature and coma (FIG. 8), can be corrected by backpropagation according to equation (11) using the conjugate or inverted pupil function. In particular, the deviations in the image field between the image and the actual object at the field edge are highlighted with the arrow in FIG. 9.

[0079] Of course, the features of the previously described embodiments and aspects of the invention can be combined with one another. In particular, the features can be used not only in the described combinations, but also in other combinations or on their own, without departing from the scope of the invention.

[0080] Techniques related to aberration correction, particularly for imaging using digital holography, were described above. However, similar techniques are also conceivable for other coherent imaging techniques, such as ptychography, Fourier ptychography, or iterative methods such as Gerchberg-Saxton.

Claims

PATENT CLAIMS 1 . A computer-implemented method for imaging using an optical imaging system configured for coherent measurement of a sample object with amplitude information and phase information, the method comprising: - Controlling the optical imaging system in order to measure the pupil function of an imaging optics of the optical imaging system using a calibration object at several discrete field positions as part of a calibration process, - based on the pupil function measured at several discrete field positions: determining a steady and continuous representation of the pupil function as a function of the field position, - Controlling the optical imaging system to measure the sample object, and - based on the measurement of the sample object, reconstructing an image of the sample object taking into account the continuous and steady representation of the pupil function.

2. The computer-implemented method of claim 1, wherein the steady and continuous representation of the pupil function is determined based on a principal component analysis of the pupil function measured at the plurality of discrete field positions.

3. A computer-implemented method according to claim 1 or 2, wherein the optical imaging system (80) is a digital holographic microscope configured to record an interference pattern between a sample beam and a reference beam by means of a detector.

4. A computer-implemented method according to any one of the preceding claims, wherein reconstructing the image of the sample object comprises: - Determining an object field for the sample object based on a backpropagation of an image field acquired during the measurement of the sample object from a detector plane to an object plane using an inverse of the continuous representation of the pupil function.

5. A computer-implemented method according to claim 3 and 4, wherein the method further comprises: - Determining the image field based on the interference pattern using digital holography techniques.

6. Computer-implemented method according to one of the preceding claims, wherein the calibration object comprises at least one point-shaped contrast object.

7. The computer-implemented method of claim 6, wherein the at least one point-shaped contrast object comprises at least one of at least one point-shaped aperture or at least one point-shaped aperture.

8. A computer-implemented method according to any one of the preceding claims, wherein the calibration process comprises arranging the calibration object in an object plane and at a plurality of positions perpendicular to the optical axis of the imaging optics in order to obtain measured values ​​for the pupil function at the plurality of field positions.

9. A method according to any one of the preceding claims, wherein the method further comprises: - based on the continuous and steady representation of the pupil function: quantifying a field profile of at least one aberration of the imaging optics.

10. Method according to one of the preceding claims, where the image is a quantitative phase image of the sample object.

11. Apparatus (90) comprising at least one processor (92) and a memory (93), wherein the at least one processor (92) is configured to load and execute program code from the memory (93), wherein the at least one processor is configured to execute the computer-implemented method according to one of the preceding claims based on the execution of the program code.

Images