Hybrid differential defocus phase contrast

Abstract

To provide advanced techniques of digital phase contrast microscopy.SOLUTION: Phase contrast images are calculated by digitally post-processing microscope images acquired at different angled illumination configurations and defocus values.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] Various examples of the present disclosure generally relate to digital phase contrast techniques that determine a phase contrast image by digitally post-processing multiple microscope images. In particular, techniques are disclosed that enhance a cumulative phase transfer function based on the use of multiple angle illumination arrangements and multiple amounts of defocus. [Background technology]

[0002] When light interacts with a specimen of interest, such as biological tissue, two main contrast mechanisms leave a footprint on the incident light. First, the specimen can attenuate the incident light by absorption. Second, the specimen can modify the wavefront of the incident light leaving a phase contrast. For non-thick, unlabeled biological specimens, absorption can generally be neglected. In these situations, phase contrast microscopy is used to generate specimen images.

[0003] Phase-sensitive techniques have a long history, including Zernike phase contrast imaging [1] (Non-Patent Document 1), Gabor in-line homography [2] (Non-Patent Document 2), and Nomarski differential interferometry (DIC) [3] (Non-Patent Document 3), to name a few. While phase-sensitive, the above methods are generally not quantitative, and the final images are prone to artifacts that do not originate from the specimen itself. For example, the halo artifact observed at phase discontinuities in Zernike phase contrast imaging prevents direct phase quantification. Gabor in-line homography suffers from so-called double image artifacts, while Nomarski DIC requires additional hardware elements such as phase-shifting elements and polarizing optics. The latter problem poses hurdles for microscope users in terms of cost, complexity, and calibration difficulties.

[0004] More recently, quantitative phase imaging systems have moved towards a data-driven approach: here, typically, multiple images are recorded and numerically combined into a single phase-contrast image using first principles of physical image formation. This type of technique is therefore sometimes called digital phase contrast: the phase contrast is obtained by digital post-processing of the acquired intensity images. Prominent examples are the transport of intensity equation (TIE) [4] and the differential phase contrast method (DPC) [5]. To acquire a TIE dataset, the specimen is translated in the optical axis (z-direction) and a so-called z-stack, consisting of at least two images, is recorded.

[0005] The data is then converted into a phase contrast image by solving a diffusion-type partial differential equation. For example, US2020 / 209604A1 discloses annular illumination combined with TIE - i.e., image acquisition with a non-zero defocus amount - and accurate phase transfer function (PTF) calculation. In DPC, the specimen is illuminated from at least three different directions (angular illumination) while maintaining a unique z-position. Possible sources of angular illumination include any kind of segmented illumination source, for example segmented diodes [5] (Non-Patent Document 5), light-emitting diode (LED) arrays [6] (Non-Patent Document 6), digital micromirror devices (DMDs), liquid crystal displays (LCDs), or variable condenser irises. The recorded data is then converted into a phase contrast image by solving a deconvolution problem.

[0006] TIE and DPC have fundamental drawbacks: (1) TIE usually employs a point source. To improve the lateral resolution achievable with the optical system, it is desirable to have a radiation source that is extended laterally to the optical axis. (2) The performance of DPC degrades when the numerical apertures (NA) of the illumination and detection are mismatched [6, 7]. In particular, the latter is problematic when the illumination NA is smaller than the detection NA. In such a situation, the phase image lacks contrast.

[0007] A possible solution to problem (1) has been reported recently by Zuo and co-workers [7], where the specimen is illuminated sequentially by individual LEDs, which are simultaneously defocused. Unfortunately, sequentially using each LED of a large LED matrix, combined with defocusing, requires acquiring hundreds of images, making this technique time-consuming and computationally complex in that the data must be recorded, transferred, processed and stored.

[0008] A possible solution to problem (2) is to combine DPC with an improved objective, for example as shown in PCT / EP2020 / 050307. In the latter patent, the authors used an amplitude-modulated pupil ring in combination with a DPC acquisition scheme. Unfortunately, the latter has the drawback that some of the light entering the pupil is absorbed - an undesirable effect on the fluorescence image when combined with DPC without a change in the objective. [Prior art documents] [Non-patent literature]

[0009] [Non-Patent Document 1] Zernike, Frits. "Phase contrast, a new method for the microscopic observation of transparent objects part II." Physica 9.10 (1942): 974-986. [Non-Patent Document 2] Gabor, Dennis. "Microscopy by reconstructed wave-fronts." Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences 197.1051 (1949): 454-487. [Non-Patent Document 3] Nomarski, Georges. "Microinterferometre differentiel a ondes polarisees." J. Phys. Rad. 16 (1955): 9S-13S. [Non-Patent Document 4] Streibl, Norbert. "Phase imaging by the transport equation of intensity." Optics com munications 49.1 (1984): 6-10. [Non-Patent Document 5] Mehta, Shalin B., and Colin JR Sheppard. "Quantitative phase-gradient imaging at high resolution with asymmetric illumination-based differential phase contrast." Optics letters 34.13 (2009): 1924-1926. [Non-Patent Document 6] Tian, ​​Lei, and Laura Waller. "Quantitative differential phase contrast imaging in an LED array microscope." Optics express 23.9 (2015): 11394-11403. [Non-Patent Document 7] Zhou, Shun, et al. "Transport-of-intensity Fourier ptychographic diffraction tomography: defying the matched illumination condition." Optica 9.12 (2022): 1362-1373. [Patent documents]

[0010] [Patent Document 1] International Publication No. 2020 / 144228 (PCT / EP2020 / 050307) Summary of the Invention [Problem to be solved by the invention]

[0011] There is therefore a need for an improved technique for digital phase contrast microscopy. A need exists to alleviate at least the limitations and disadvantages identified above. This need is met by the features of the independent claims. The features of the dependent claims define embodiments. [Means for solving the problem]

[0012] In the following, a digital phase contrast technique is disclosed: Multiple microscope images are acquired at multiple settings, each setting characterized by a different amount of defocus and a different angular illumination arrangement. Various techniques are based on the discovery that both defocusing of the imaging plane and angular illumination of the imaging plane have the potential to obtain images that encode phase information. By combining angular illumination and defocusing, phase information can be reconstructed in a wider spatial frequency range compared to referenced techniques such as TIE and DPC. In this way, good quality phase contrast images can be obtained.

[0013] A method for determining a phase contrast image is disclosed, the phase contrast image being determined based on a plurality of microscopic images acquired using a microscope.

[0014] The microscope images are intensity images, i.e., they are not acquired with phase contrast, nevertheless, phase information is encoded in the microscope images that allows the phase of the imaged specimen to be reconstructed.

[0015] The microscope includes an illumination module configured to provide switchable angular illumination, i.e., drive different angular illumination configurations. The angular illumination provides a range of angles at which light is incident on an imaging plane of the microscope, and the angular spectrum of the angular illumination is fixed across the imaging plane.

[0016] The microscope further includes an optical system, the optical system including one or more optical elements, e.g., a lens. The optical system is configured to illuminate the imaging plane. The optical system is further configured to image the imaging plane onto at least one camera of the microscope.

[0017] The method includes controlling an illumination module to drive a plurality of angle illumination arrangements, in at least some examples, at least one of the plurality of angle illumination arrangements includes a range of illumination directions, i.e., the at least one angle illumination arrangement each comprises an extended angular spectrum (rather than just a single illumination angle).

[0018] Only one of the multiple angle lighting arrangements may include a range of lighting directions. Each of the multiple angle lighting arrangements may include a range of lighting directions. If all of the multiple angle lighting arrangements do not include a range of lighting directions, the remaining angle lighting arrangements include one lighting direction. The multiple angle lighting arrangements may each have an angular spectrum with a different width. For example, a first angle lighting arrangement has a first angular spectrum and a second angle lighting arrangement has a second angular spectrum with a different second width. A wider angular spectrum can be used with a narrower angular spectrum. The narrower angular spectrum can have a minimum width achievable for each lighting module (i.e., a single lighting direction).

[0019] For example, each of a number of angular illumination arrangements (indexed by j) may be denoted by a respective illumination numerical aperture (NA), NA i,j The illumination NA defines the range of angles over which the imaging plane is illuminated. The higher the NA, the greater the inarrows the range of angles at which light can enter, so NA i Increasing generally increases the coherence of the illumination, effectively narrowing the light source and increasing spatial coherence.

[0020] The detection optical system, e.g., the objective lens or objective lens system, has a NA, NA d The NA of a detector is defined by the objective lens or lens system. The NA of a detector affects how fine the details of an image can be resolved. The higher the NA of a detector, the greater the light it can collect from a wider angle, improving the resolution.

[0021] Each illumination numerical aperture (associated with each of the multiple angle illumination arrangements) is smaller than the detection numerical aperture, e.g., the ratio is at least 2, and σ=NA i,j / NA d where σ≦1 / 2 for all j. σ is sometimes called the coherence parameter, i.e., σ can be less than or equal to 1 / 2 for any image acquired and post-processed.

[0022] In the above examples, at least one of the multiple angle lighting arrangements is disclosed as including a range of lighting directions, but this is generally arbitrary. In some examples, each of the multiple angle lighting arrangements includes a single lighting angle. By "single lighting angle" we mean that the width of each of the lighting angle distributions is as small as achievable with the lighting module. Generally, this means that a single light source, e.g., a single LED in each array, is switched on.

[0023] The method further includes controlling at least one camera to capture a plurality of images at a plurality of angular illumination arrangements and at a plurality of amounts of defocus.

[0024] The method further includes combining the images to determine a phase contrast image. In principle, combining two or more images may include calculating pairwise differences. Combining two or more images may also include calculating a complex-valued representation of the phase, i.e., sum, of the two or more image combinations, with each phase shift applied to a separate image.

[0025] Each of the multiple images can be combined with each of the further images. In some scenarios, multiple subsets of images may be formed, each including only some of the images. Each image in a given subset is then combined with any other image in that subset. Such an approach may be applied, especially when multiple phase reconstruction algorithms are used and run on separate subsets.

[0026] A phase contrast image can have a quantitative phase contrast, ie, the contrast can quantitatively encode the phase shift imparted by an object.

[0027] While in the above at least one camera and illumination module are controlled to acquire an image, in other scenarios multiple images can be pre-acquired, with multiple defocus amounts and multiple angle illumination configurations, which are loaded from a database or image archive and then combined to determine a phase contrast image.

[0028] The computer device includes a processor and a memory. The memory stores program code. The processor loads the program code and executes the program code. When the processor executes the program code, it performs the method disclosed above.

[0029] The computer program or computer readable storage medium includes program code that is loaded and executed by a processor, which, when executed, performs the method disclosed above.

[0030] It is to be understood that the features mentioned above and those to be described below can be used not only in the respective combinations shown, but also in other combinations, or alone, without departing from the scope of the invention. [Brief description of the drawings]

[0031] [Figure 1] 1A and 1B are schematic diagrams illustrating a system including a microscope and a computer according to various examples. [Diagram 2] 1 shows a schematic of a DPC digital phase contrast technique according to a prior art implementation; [Diagram 3] 1 shows a schematic diagram of a prior art implementation of a TIE digital phase contrast technique; [Figure 4] 1A-1D are schematic diagrams illustrating various examples of phase contrast techniques; [Diagram 5] 5 is a comparison of the phase transfer functions of the phase contrast techniques according to Figs. 2, 3 and 4. [Figure 6] 1 is a flowchart of a method according to various examples. [Figure 7] 1 shows angular spectra for multiple angle illumination arrangements according to various examples. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] Some embodiments of the present disclosure generally provide a number of circuits or other electrical devices. References to circuits, other electrical devices, and functionality provided by each are not intended to be limiting to include only those shown and described in the specification. Although the various circuits and other electrical devices disclosed may be labeled with specific reference numbers, such reference numbers do not limit the scope of operation of the circuits or other electrical devices. Such circuits and other electrical devices may be combined with each other and / or separated in any manner based on the particular type of electrical implementation desired. It is understood that any circuit or other electrical device disclosed herein may include any number of microcontrollers, graphic processor units (GPUs), integrated circuits, storage devices (e.g., flash memory, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), or other suitable variations thereof), and software that cooperate with others to perform the operations disclosed herein. Additionally, any one or more electrical devices may be configured to execute program code embodied in a non-transitory computer-readable medium to perform any number of functions as disclosed.

[0033] Hereinafter, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be understood that the following description of the embodiments is not to be taken in a limiting sense. The scope of the present invention is not intended to be limited by the embodiments or drawings described below, which are merely illustrative.

[0034] The drawings should be considered as schematic representations, and the components shown in the drawings are not necessarily drawn to scale. Rather, the various components are depicted such that their functions and general purpose are apparent to those skilled in the art. Connections or couplings between functional blocks, devices, components, or other physical or functional units shown in the drawings or described herein may also be implemented by indirect connections or couplings. Couplings between components may also be established via wireless connections. Functional blocks may be implemented in hardware, firmware, software, or a combination thereof.

[0035] The technique disclosed herein can be called "Hybrid Differential Defocus (HDD) Phase Contrast". HDD phase contrast combines the advantages of TIE and DPC while resolving the shortcomings of each technique. In HDD, the specimen is defocused and illuminated by an asymmetric illumination pattern. Unlike reference [7], the disclosed technique does not exclusively employ individual LED illumination, which would result in a large number of images being acquired. HDD instead utilizes, at least to some extent, multiplexing of illumination directions, for example by simultaneously lighting multiple facets (e.g., multiple LEDs) of a programmable light source. For example, half-pupil illumination can be used. That is, a split light source is used, which is switched on exactly half in terms of area (e.g., top half vs. bottom half, or left half vs. right half). The split is symmetrical with respect to the optical axis, and the pupil itself can be square, rectangular, elliptical, circular, or annular. More generally, multiple illumination arrangements are activated, at least one of which includes a range of illumination angles at the imaging plane.

[0036] In addition, HDD uses a variable pattern displayed by a programmable light source instead of TIE, which uses exclusively point sources. Whereas in DPC the specimen is held at a fixed position on the focal plane, HDD uses axial movement of the specimen to achieve improved phase contrast. Since the imaging plane is uniformly illuminated in the spatial domain with an angular spectrum (i.e., the angular spectrum does not depend on the lateral position relative to the optical axis within the associated aperture), the angular illumination arrangement is not to be confused with, for example, light sheet microscopy, where only a sheet of a portion of the specimen is illuminated.

[0037] The techniques are based on the discovery that a multiple angle illumination arrangement combined with defocusing provides a phase contrast modality that improves lateral resolution compared to TIE, while not suffering from the NA mismatch problem that leads to low phase contrast as seen in DPC. Also, by multiplexing multiple LEDs in parallel, HDD data acquisition protocols are orders of magnitude faster than conventional techniques [7].

[0038] 1 is a schematic diagram of a system 70 according to various examples. The system 70 includes a microscope 90 and a computer 80. The microscope 90 includes an illumination module 91, an optical system 92, and a detection module 93.

[0039] The detection module 93 includes detection optics. The numerical aperture NA of the detector d The detection module 93 further includes one or more cameras for acquiring the microscopic images.

[0040] The illumination module 91 is configured to provide switchable / reconfigurable angular illumination of an imaging plane defined along the optical path 94 of the system 70. This means that the illumination angle is controllable. In addition to controlling the main illumination angle, it is also possible to control the angular spectrum, e.g. width or contributions. For example, multiple illumination arrangements with angular spectra of different widths can be driven. Sometimes only a single illumination direction is driven (angular spectrum with smallest width) and sometimes multiple illumination directions are superimposed (angular spectrum with larger width).

[0041] Optical system 92 is configured to illuminate the imaging surface and further image the imaging surface onto at least one camera of detection module 93 .

[0042] The microscope 90 further includes a control module 95 configured to control various components of the microscope. For example, the control module 95 can be implemented using a CPU, a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC). The control module 95 can include a memory. The control module 95 can be configured to control an illumination module that drives a multiple angle illumination arrangement. The control module 95 can also be configured to control the detection module 93, and in particular at least one camera of the detection module 93, to acquire multiple images. Optionally, the control module 95 can be configured to control the optical system 92 to, for example, move a sample holder configured to hold a sample to implement multiple defocus amounts.

[0043] Also shown is a computer 80 that includes an interface 84 configured to communicate with the microscope 90, and specifically with a control module 95. For example, a processor 81, e.g., a CPU, FPGA, ASIC, of ​​the computer 80, provides control data to the control module 95 to perform control functions, e.g., triggering image acquisition, triggering activation of a certain angle illumination arrangement, triggering implementation of a particular amount of defocus, etc. The processor 81 can also acquire image data from the microscope 90 through the interface 84 and post-process the image data. The processor 81 can also load image data from a database or image archive.

[0044] The processor 81 is configured to load and execute program code from the memory 83 to perform such techniques. In particular, the processor 81 may perform digital post-processing to determine phase contrast based on a plurality of intensity images acquired from the microscope 90. The computer 80 may also include a user interface 82, e.g., a GUI, for outputting the phase contrast thus determined.

[0045] 1 illustrates a situation in which the computing structure is split between control module 95 and computer 80, in some situations digital post-processing of image data may be performed by control module 95. Alternatively, or additionally, component-level control of various components of microscope 90 may be at least partially delegated to computer 80.

[0046] 2, 3, and 4 illustrate the differential phase contrast (DPC), transport of intensity equation (TIE), and hybrid differential defocus (HDD) schemes, respectively.

[0047] DPC (FIG. 2) uses a programmable illumination unit (PIU) 201. It requires the acquisition of at least three focal plane images with varying illumination patterns 202 to generate a phase-contrast image of the specimen. In particular, these variable illumination patterns 202 correspond to different angular illumination arrangements that illuminate the imaging plane 203 at different angles when the specimen is held in a fixed position. Each pattern 202 includes LEDs that are driven with different asymmetric distributions with respect to the optical axis 207 (dashed lines and open circles). That is, each illumination arrangement includes multiple illumination directions (defined by the LEDs being driven). An objective lens 204 and a tube lens 205 are used to image the imaging plane 203 onto a pixel detector 206 (camera) (part of the optical system, see FIG. 1: optical system 92). Different LEDs are driven for different variable illumination patterns 202. A subsequent numerical deconvolution routine converts the recorded images into a phase-contrast image of the specimen, see [6]. Details related to such DPC phase reconstruction algorithms are described next.

[0048] A minimum of three images are assumed to be recorded from equiangular directions (azimuth angles) spaced 120° apart (this angle represents the azimuth position of the average angular spectrum, for example, when using a semicircular illumination pattern centered at that azimuth angle). All of these images are acquired with the same amount of defocus, i.e., in this example, the defocus amount is nominally zero defocus so that the sample is in the imaging plane. These three images are included in the DPC subset. The two-dimensional (2D) raw images are referred to here as I 0° , I 120° , I 240° These can be generated, for example, by three equiangularly spaced LEDs, with or without a diffuser to extend the angular range. Alternatively, any angularly separated light source (digital micromirror device, spatial light modulator, etc.) can be used. Details regarding the illumination module 91 are described above in relation to FIG. 1. A complex-valued real spatial phase image is then calculated as the combination of these three images in the DPC subset.

[0049]

number

[0050] The DPC phase contrast image is calculated using the formula of the phase reconstruction algorithm.

number

[0051] H DPC (k)=exp(j arctan(k y / k x )) is the Hilbert transform kernel (see Kieran G. Larkin and Peter A. Fletcher, "Isotropic scalar image visualization of vector differential image data using the inverse Riesz transform," Biomed. Opt. Express 5, 907-920 (2014), and the appendix of this specification), and imag[...] selects the imaginary part of the complex number. Instead of using the Hilbert transform kernel, one can also employ a DPC transform function using the "Weak Object Transfer Function" (WOTF) formula: Equation 6. Since the WOTF itself requires at least two Fourier transform calculations, the Hilbert transform is an approximation but faster. Meanwhile, it has been observed that for relevant spatial frequencies, the loss in accuracy is quite limited.

[0052] DPC phase-contrast images suffer from artifacts induced by shadows cast from scatterers located outside the imaging plane (out-of-focus scatterers). Shadow casting is sensitive to the angular illumination and is preserved in the real space phase and image when making the above calculations. Scatterers outside the imaging plane can be, for example, scratches on optical elements in the light path or dust on the backside of the slide in the sample holder. These shadows cast in DPC phase-contrast images are typically poorly localized, i.e., confined to small spatial frequencies.

[0053] Reference is now made to FIG. 3. TIE phase reconstruction is based on a diffusion equation that relates the axial intensity derivative to the phase of the specimen. The axial intensity derivative is approximated by recording at least two images with axial translation of the imaging plane and specimen closer to the focal plane, which allows multiple amounts of defocus 311 to be implemented. The illumination unit can be fully coherent, such as an on-axis point source 301 or an on-axis collimated laser. Alternatively, partially coherent illumination may be used. After image recording under varying defocus (also known as z-stack), sufficient information is available to solve the underlying diffusion equation to yield the phase information of the specimen [4].

[0054] The TIE subset contains two images, and these two images are I ―ΔZ , I +ΔZ The images are acquired with two different defocus amounts given by ΔZ=c λ / NA 2 is the distance to the focal plane of the specimen. Thus, positive and negative defocus amounts are used, equidistant from the focal plane. λ and NA are the central wavelength of the illumination and the numerical aperture used. The c parameter thus refers to the magnitude of the defocus with respect to the depth of field of the detection optics. In practice, the c parameter (for the images in the TIE subset) is chosen between 1 and 10 to control the defocus distance. The two images for the TIE subset are acquired with the same illumination arrangement, e.g., on-axis illumination. With such on-axis illumination, the angular spectrum is centered at the apex and is partially coherent. This can be achieved by driving only a single light source centered on the optical axis 207 for an array of light sources, as shown in FIG. 3 for light source 301 centered on the optical axis 207. However, a range of illumination directions can also be used. For example, σ≦1 / 3, specifically 1 / 10≦σ≦1 / 3, for images that are post-processed using TIE reconstruction. The TIE phase-contrast image φ of the specimen TIE is then generated by the formula given by the TIE reconstruction algorithm.

number

[0055] H TIE (k)=k x 2 +k Y 2 is a second-order transfer function, and FFT denotes the 2D Fast Fourier Transform. This simple form of the transfer function is preferable to the calculation of the WOTF transfer function, which is numerically less efficient and provides only a small improvement at high spatial frequencies. This optical transfer function H TIE (k) is -H DPC (k) - but is a simplified transfer function that is predefined independent of any microscope properties, i.e., dependent only on the k-space position.

[0056] As previously explained in relation to the DPC phase reconstruction algorithm, TIE phase contrast images, unlike DPC phase contrast images, are less affected by shadowing artifacts caused by out-of-focus scatterers such as impurities, dust, scratches, etc. This is because the TIE phase reconstruction algorithm removes angle-sensitive shadowing from the pairwise difference images according to Equation 3 above, without changing the illumination geometry. On the other hand, in TIE phase contrast images, high image frequencies are not preserved, which results in blurred edges and reduced image resolution.

[0057] According to some examples, two separate phase reconstruction algorithms can be implemented, one based on the DPC subset and Equation 2, and one based on the TIE subset and Equation 3.

[0058] In such an overview, two temporary phase-contrast images are acquired and calculated based on the TIE phase reconstruction algorithm and the DPC phase reconstruction algorithm, respectively. The image information associated with these two temporary phase-contrast images can then be combined to obtain a final phase-contrast image. For example, a low-pass filter can be applied to the temporary phase-contrast image acquired from the TIE phase reconstruction algorithm, and a high-pass filter can be applied to the temporary phase-contrast image acquired from the DPC phase reconstruction algorithm. The filtered representations of each of these temporary phase-contrast images, i.e. the low-pass and high-pass filtered image information, can then be combined pixel-wise to obtain a final phase-contrast image. In this way, the temporary phase-contrast image φ TIE and φ DPC Given according to the computational steps described above, the final HDD phase contrast image is computed, optionally using high-pass-low-pass filtering.

number

[0059] In this way, high-pass filtered image information obtained from the DPC phase-contrast image is combined with low-pass filtered image information obtained from the TIE phase-contrast image. The kernel width of the Gaussian filter is controlled by the parameter σ, and the complementary transfer weight function (1-w(k)) is selected to be at least large enough to filter background artifacts such as dust from the DPC temporal phase-contrast image. The scalar parameter d is selected to ensure continuity of the combined DPC and TIE spectra in the circular spatial frequency annulus given by Equation 5.

number

[0060] The combination of DPC and TIE phase reconstruction algorithms followed by TIE and DPC temporal phase contrast images provides a final phase contrast image with superior properties when compared to the DPC temporal phase contrast image and the TIE temporal phase contrast image. In particular, the angle-sensitive shadowing due to out-of-focus scatter in the imaging path, as seen in the DPC phase contrast image, is not observed in the final phase contrast image because high-pass filtering is applied to the DPC phase contrast image. In this way, blurring artifacts resulting from out-of-focus scatter in the imaging path are removed. Meanwhile, the lower spatial frequencies that are less protected by the TIE phase reconstruction algorithm are filtered from the TIE phase contrast image, so that most of the information of each image is obtained from the DPC phase contrast image. In this way, the resulting HDD phase contrast image does not have the same background artifacts as the DPC phase contrast image, and at the same time, shows good image contrast for all spatial frequencies. Furthermore, it is sufficient to acquire a total of five different images, namely two images in the TIE subset at two different defocus amounts and three images in the DPC subset at three different angle illumination configurations. Image acquisition times are significantly reduced compared to competing approaches such as [7], requiring only five image acquisitions, or less than 10 if multiple acquisitions are used to improve the signal-to-noise ratio. Furthermore, exposure of the specimen to light is reduced, which is favorable for certain light-sensitive analytes, such as cellular structures.

[0061] The images acquired and their association with the two subsets are shown below. [Table 1]

[0062] Table 1: DPC and TIE subsets are based on a total of 5 images. The subsets are non-overlapping, i.e., no image belongs to both subsets. The c-parameter of the images in the TIE subset may range from 1 to 10. For the images used for DPC, semicircular illumination and σ=1 / 2 are used, and for the images used for TIE, σ=1 / 3 is used.

[0063] Above, we disclose a HDD scheme employing two separate phase reconstruction algorithms, one for acquiring a temporary DPC phase contrast image and one for acquiring a temporary TIE phase contrast image. The DPC and TIE phase contrast images are then fused through selective high-pass and low-pass digital signal processing. In another variation, a joint HDD reconstruction algorithm can be employed. This is described in relation to FIG. 4.

[0064] Now, reference is made to Figure 4. Such a joint HDD reconstruction algorithm combines the working principles of DPC (see Figure 2) and TIE (see Figure 3) in accordance with the disclosed technique.

[0065] In the illustrated example, the illumination module 91 is implemented by the PIU 201, which includes an LED array including multiple LEDs located at different angles and distances relative to the optical axis 207. The multiple LEDs are driven separately, for example, by the control module 95 (see FIG. 1). Driving multiple LEDs, for example according to asymmetric patterns, creates different illumination patterns 202, as described above in connection with FIG. 2. Each of the multiple illumination patterns 202 includes multiple driven LEDs, implementing a wide angular spectrum of illumination angles to simultaneously drive multiple illumination directions. The asymmetric arrangement of the illumination patterns 202 results in an average of the angular spectrum tilted relative to the optical axis 207. It should be noted that while FIG. 4 shows a semicircular angular spectrum, other illumination patterns are also contemplated. Furthermore, the HDD is not limited to being implemented by the PIU 201 as shown in FIG. 4. In principle, the illumination module may include at least one of a programmable LED array, a digital micromirror device, a liquid crystal display, a variable condenser aperture, an angular diffuser, and an aperture wheel.

[0066] For example, a digital micromirror device can be illuminated by a light source, and depending on the orientation of each mirror of the digital micromirror device, each illumination spot in the illumination pupil plane can be driven or not driven - equivalent to switching on versus switching off an LED in a programmable LED array of the programmable illumination unit 201. This allows control of both the dominant angle of illumination (i.e., the average of the angular spectrum) as well as further characteristics of the angular spectrum such as width and side peaks.

[0067] A similar effect can be achieved with a liquid crystal display with a suitably illuminated background. Individual pixels of the liquid crystal display are activated and deactivated. A variable capacitor iris can include multiple apertures that can be selectively opened and closed.

[0068] An angular diffuser can be placed between the imaging plane 203 and the illumination pupil plane to help widen the aperture.

[0069] The HDD is not limited to lighting arrangements including multiple lighting angles, such as that shown in lighting pattern 202 of FIG. 4, and one or more angle lighting arrangements including a single lighting direction may be used, in addition to lighting arrangements including multiple lighting directions.

[0070] In accordance with the scenario and HDD technology of Figure 4, a multiple angle illumination arrangement is implemented in combination with multiple defocus amounts 311. In the schematic of Figure 4, two defocus amounts are illustrated (zero defocus and offset by Δz), however, defocus amounts with larger count values ​​are contemplated.

[0071] Furthermore, in the scenario of FIG. 4, multiple defocus amounts are implemented by shifting the sample stage to reposition the specimen along the optical path, specifically along the optical axis 207. For example, this is implemented by motorizing the sample stage of the optical system 92, which is controlled by the control module (see FIG. 1). This is just one option for using multiple defocus amounts. In other scenarios, multiple cameras combined with a beam splitter can be used. Multiple cameras are positioned at different distances relative to the sample surface along the optical axis. The use of multiple cameras has the advantage that the image acquisition at multiple defocus positions can be multiplexed. This can reduce the overall measurement time required to acquire all the images that are to be post-processed. Alternatively or additionally, electrically adjustable lenses that can change the focal length can be used. This allows for a more compact arrangement without the need for multiple cameras. Since electrically adjustable lenses can be focused quickly, the overall measurement time required for image acquisition can be reduced, for example, when compared to the overview of repositioning the sample state.

[0072] Further reductions in measurement time can be achieved by multiplexing multiple angular illumination configurations in polarization space and / or wavelength space. For example, a multi-color illumination module 91 (e.g. red-green-blue) can be used, which can drive different angular illumination configurations at different wavelengths. This can then be combined with a wavelength-selective detection module 93, which can simultaneously acquire images at different wavelengths. A similar scheme can be envisioned for different polarizations.

[0073] Above, various aspects related to the hardware implementation of the microscope 90 as well as its various components and image acquisition measurement methods are disclosed. Next, aspects related to the digital post-processing of the joint phase reconstruction algorithm are discussed. In particular, it is explained how the combination of multiple intensity images associated with different combinations of multiple angular illumination arrangements and multiple defocus amounts can be implemented to obtain a phase contrast image.

[0074] Various aspects are based on the following findings: Several issues have to be addressed in digitally post-processing acquired HDD microscopy images: (1) the typical transfer function deconvolution analysis used for DPC data with the specimen fixed in the focal plane is no longer valid; (2) the diffusion equation underlying TIE is not valid under partially coherent illumination such as that produced by the PIU, which changes illumination patterns, whereas TIE assumes a stable illumination pattern; (3) the transfer function analysis of [7] for individual point sources is not valid, since multiple points in the PIU are switched on simultaneously.

[0075] To solve these problems, multiple images acquired with different combinations of defocus amount and angular illumination arrangement are subtracted from each other. This allows background subtraction. Furthermore, for each of these pairs, the difference of the respective phase transfer functions is calculated (thereby obtaining the so-called differential phase transfer function). Then, based on the information of the difference image and the differential phase transfer function, the phase of the specimen can be restored, for example, using a deconvolution analysis, which restores the phase of the object with the phase given by the optical system. An optimization problem can be formulated to obtain the specimen phase, i.e., a phase contrast image, based on the difference image and the differential phase transfer function. Although a deconvolution analysis is one option for solving such an optimization problem, other scenarios are conceivable, especially iterative approaches. Sometimes such iterative approaches are more robust to noise or converge robustly.

[0076] Furthermore, various options are available for determining the phase transfer function. One particular scheme, detailed below, is based on the weak object transfer function formula. The assumption of a "weak object" is used when dealing with objects that only slightly modulate the phase and amplitude of the light passing through them. WOTF is the concept used under this weak object. WOTF contains the Fourier transform of an object that is weak in phase or amplitude. It describes how the object affects the phase and amplitude of the light in frequency space, assuming that these effects are relatively small. For a weak object, the outgoing wave just beyond the object is approximately the sum of the incoming wave and a small perturbation due to the object itself. The WOTF formula allows to incorporate partially coherent illumination conditions generated by the PIU, as well as defocusing due to axial specimen movement. However, other options different from the WOTF formula are also available. For example, one can use predefined parametrized templates retrieved from a database and instantiated according to a specific combination of angular illumination arrangement and defocus amount. It is also possible to use the Hirbert transform to determine the phase transfer function.

[0077] The WOTF format, see [4], provides the generalized transfer function (complex valued) of an optical system as follows:

number

[0078] k=(k x ,k y ) and k' = (k' x ,k' y ) denotes the spatial frequency, S is the light source 92 (i.e., the illumination pattern at the illumination pupil), and P is the focused pupil of the optical system. Alternative formulations, such as using a two-dimensional integral over the illumination angle present at the focused pupil, are possible. Because HDD phase contrast imaging uses defocus and a variable light source pattern, two metrics are needed to represent the changing illumination and defocus conditions of the system. We define bimodal weak object transfer as follows:

number

[0079] The indices m and n respectively allow to distinguish between different light source patterns and defocus conditions. Here, Sm(k) denotes the mth illumination pattern displayed on the PIU (however, as mentioned above, this can be formulated similarly for many different light modules, e.g. by integrating the illumination angles provided by a particular light module). In contrast, the amount of defocus can be included in this description by adding an appropriate phase to the pupil. Below is an example of a spherical phase correction, but other phase corrections are also possible, e.g., a quadratic equation as an approximation.

number

[0080] Here, Zn denotes the relative movement of the specimen with respect to the focal plane in the nth axial acquisition plane. The modulated amplitude (ATF) and phase (PTF) contrast functions are defined as follows:

number

[0081] For weakly absorbing samples and aberration-free optical systems, the ATF can be neglected (see [6]). However, in some scenarios, aberrations in the optical system can be taken into account, e.g. spherical aberration, coma, astigmatism, etc.

[0082] Under the assumption that the ATF is zero, in a partially coherent microscope, the spatial two-dimensional Fourier transform FT[I] of the observed image can be modeled as Equation 9:

number

[0083] where b is a constant offset and φ denotes the unknown phase of the specimen in the estimated imaging plane. It should be noted that the latter is a set of equations; i.e., a separate equation for each combination of source pattern (m) (or more general angular illumination arrangement) and amount of defocus (n).

[0084] The background terms need to be removed from these equations, so to obtain Equation 10, we calculate the difference between the equations with pairs of indices (m,n) and (m',n').

number

[0085] The image intensity difference (eg, the difference between acquired microscopic images, ie, image combination) is obtained as Equation 11, and the PTF difference (differential phase contrast function; Equation 12).

number

number

[0086] Equation 10 defines the optimization problem, which allows the latter set of equations to be written as a least-squares problem.

number

[0087] An optimization problem can be solved for the phase φ (i.e., to determine the phase contrast image) via numerical iterative techniques or deconvolution. For example, the optimal phase estimate is given by:

number

[0088] iFT[…] denotes the Fourier transform, and for simplicity of notation, dPTF at spatial frequency vector k * m,n,m’,n’ and FT[dI m,n,m’,n’ ] dependency. This corresponds to a Wiener filter.

[0089] Modeling known aberrations in the PTF and / or ATF can increase the robustness of the reconstruction.

[0090] The main challenge in performing the deconvolution of the immediate equation is primarily the denominator term (Equation 16) which may have zero crossings.

number

[0091] Division by values ​​close to or equal to zero leads to amplification of noise and numerical problems, so a regularization term (α>0) is added to the denominator to prevent division by zero.

number

[0092] If the regularization term is too large, the deconvolution process will attenuate low spatial frequencies, resulting in poor phase contrast and lateral resolution. Conversely, if the regularization term is too small, values ​​close to zero in the denominator will amplify noise. To avoid over-regularization and the resulting attenuation of spatial frequencies, the form of the denominator d(k) is ideally adjusted to have as few zero regions as possible.

[0093] 5 shows the PTF and associated achievable numerical aperture (NA) coverage for different phase contrast techniques - DPC, TIE, and HDD. Example phase images 501-503 for each technique are shown.

[0094] 5 shows the adjustment process of the spatial frequency coverage of d(k). Hereinafter, the quantity in Equation 18 is called the cumulative NA coverage.

number

[0095] The first row in Fig. 5 shows the DPC (see Fig. 2) under mismatched illumination conditions (numerical aperture NAi=0.3 of the illumination part of the optical system and numerical aperture NAd=0.6 of the detection part of the optical system). Although the DPC is essentially able to reach a cumulative NA radius of NAi+NAd (radius of the dashed line in the middle of the first row in Fig. 5), a hole of approximately radius NAd-NAi appears in the NA coverage, resulting in a loss of coverage of low frequencies. This leads to a lower contrast of the obtained phase reconstruction (as is evident from the phase contrast image 501).

[0096] TIE (see FIG. 3) employs - as shown in the second row of FIG. 5 - an axial defocus. When evaluating the corresponding phase transfer function (second row of FIG. 5, left column), the NA coverage reaches a smaller value in the k interval compared to DPC (simply put, the black circles around the k-space center are smaller). This leads to an improved contrast in the final phase reconstruction compared to DPC (see phase contrast image 502). However, since only an on-axis point source is used (see FIG. 3), the illumination NA is vanished in standard TIE phase contrast systems. Thus, the NA coverage only reaches the bandwidth of NAd (simply put, the radius of the white donut is limited), which is a disadvantage when compared to DPC in terms of the achievable lateral resolution (given by λ / [NAi+NAd], where λ is the wavelength). By combining both the TIE and DPC phase reconstruction algorithms using appropriate bandpass filters (see Equation 4), a better NA coverage can be reached.

[0097] The HDD (see Fig. 4) is shown in the third row of Fig. 5. By synthesizing the PTF from both the variable illumination pattern and the defocus, a wider range of spatial frequencies can be covered (the donut in the middle column has a smaller inner hole and a wider radius). The resulting phase contrast (right column) features superior phase contrast compared to the DPC and reaches a higher resolution compared to the TIE (see phase contrast image 503). The dark areas in the PTF require regularization.

[0098] 6 is a flow chart of a method according to various examples, which relates to determining a phase contrast image. It is based on the HDD technique disclosed herein, which uses digital post-processing of multiple intensity microscopy images to reconstruct a phase contrast image.

[0099] The method of Figure 6 can be implemented using a processor based on program code loaded from a memory. For example, the method of Figure 6 can be implemented by a processor 81 of a computer 80 by loading and executing program code stored in a memory 83.

[0100] In option box 605, a specimen is loaded onto the microscope. For example, a biological sample including a cell culture may be loaded onto the microscope. Loading the specimen onto the microscope includes holding the sample on a sample stage of the microscope.

[0101] In the option box 610, an angular illumination arrangement is driven. Depending on the specific implementation of the illumination module, the box 610 can take different forms. For example, an aperture wheel can be placed at a rotational position. A certain LED pattern of the LED array of the PIU can also be driven according to, for example, a desired angular spectrum. The digital micromirrors of the DMD (Digital Micromirror Device) can also be reconfigured. The pixels of the liquid crystal display can be switched on and off. The angular illumination arrangement driven in the box 610 provides full illumination of the imaging surface under a range of illumination angles. In this way, the brightness level does not change much across the imaging surface (within the aperture of the system), and the illumination angles observed at different area positions perpendicular to the optical axis do not change.

[0102] Box 610 can drive multiple types of angular illumination arrangements, each characterized by a different width of the angular spectrum. For example, a first type of angular illumination arrangement that can be driven by box 610 includes a range of illumination directions. The angular spectrum is therefore relatively wide. A second type of angular illumination arrangement that can be driven by box 610 includes a single illumination direction. The angular spectrum is therefore relatively narrow. This is illustrated in FIG. 7, which shows multiple angular spectrums 701, 702, 711-714. The angular spectrum indicates the distribution of angles at which light is incident across the imaging plane (in the set in FIG. 7, light 762 is incident on imaging plane 761, with respective angle 763 highlighted).

[0103] 7 shows that the angular spectrums 701, 702 include a range of illumination directions. Here, "range of illumination directions" means that the illumination module used can provide illumination using a smaller angular spectrum, but the illumination module is controlled to illuminate the imaging surface using a wider angular spectrum, i.e. combining multiple illumination directions that can be switched on by the illumination module individually. For example, in an LED array scenario, this means that multiple LEDs (each LED defining a single illumination direction) are switched on simultaneously.

[0104] In contrast, each of the angular spectra 711-714 includes a single illumination direction. Nevertheless, each angular spectrum 711-714 has a certain width 751 that is significantly smaller than the width 752 of the angular spectrum 701, 702. This finite width 751 is due to the stretching of the minimum optical aperture of the light source module in the illumination pupil plane, for example the stretching of the exit area of ​​a light-emitting diode or the pixel size of a liquid crystal display or the micromirror of a micromirror device. The hardware of the optical module does not allow providing an illumination arrangement with a narrower width than the angular spectra 711-714, and therefore they can be said to include a single illumination direction.

[0105] The bottom of FIG. 7 shows the illumination pattern 202 of a programmable light source (LED array) for angular spectra 701, 702, 711-714 (the optical axis 207 is shown as the open circle in each of these plots).

[0106] As can be seen, the illumination arrangements associated with the angular spectra 701, 702 correspond to DPC type illumination, whereas the illumination arrangements associated with the angular spectra 711-714 correspond to TIE type illumination. This type of technique is based on the following discovery: Illumination arrangements providing a range of illumination directions and illumination arrangements providing a single illumination direction (or more generally wide and narrow angular spectra) have different advantages and disadvantages. A wide angular spectrum provides high resolution in the phase contrast image. A narrow angular spectrum provides high contrast in the phase contrast image. Combining a wide and narrow angular spectrum allows a combination of high contrast and high resolution.

[0107] In option box 615, one or more components of the microscope are controlled to drive a certain amount of defocus. For example, box 615 includes repositioning a sample stage for holding a specimen along the optical path or optical axis, i.e., Z repositioning is performed. However, it is not necessary in all scenarios to drive a certain amount of defocus. In some scenarios, multiple cameras can acquire images of the imaging plane at different offsets with respect to the imaging plane, thereby implementing different amounts of defocus. As a further option, an electrically configurable lens, such as an objective lens or a tube lens, can be used in box 615 to implement a certain amount of defocus.

[0108] Then, in box 620, an image is acquired, the image being acquired at some combination of angular illumination arrangement and defocus amount, as per the precedent by execution of boxes 610 and 615. Generally, it should be noted that box 620 is optional, since in some scenarios image reconstruction (discussed below in connection with box 635) may be performed on pre-acquired images. An example sequence of images that may be acquired is disclosed above in connection with Table 1.

[0109] Box 625 checks whether a further defocus amount is needed; if so, a further iteration 626 of boxes 615 and 620 is performed to select another defocus amount. This corresponds to incrementing the index n in the formula given above.

[0110] If all defocus amounts have been obtained for a certain angular illumination configuration, the method starts at box 630, where index n is reinitialized and continues by checking whether more angular illumination configurations are needed in box 630. If so, a further iteration 631 of boxes 610, 615, 620, 625 is performed, and index m in the formula is incremented.

[0111] In the scenario of FIG. 6, loop 626 is the inner loop and loop 631 is the outer loop, and other scenarios are possible that switch between different amounts of defocus and angular illumination arrangements.

[0112] Once all images have been acquired in box 620 (ie, all different combinations of index m and index n), image reconstruction begins in box 635.

[0113] Several options are available for the implementation of box 635. In one example (box 636), the joint phase reconstruction includes images acquired with different illumination configurations and different defocus amounts. This involves solving an optimization problem, for example using deconvolution analysis as explained above. An example optimization problem is described above in relation to Equation 14 and FIG. 4. Optional regularization can be considered. In another example (box 637), separate phase reconstruction algorithms are performed for each of the image subsets, i.e., the TIE subset and the DPC subset (as described above in relation to FIGS. 2 and 3). A downstream combination of the TIE and DPC phase contrast images can then be performed to obtain a final phase contrast image.

[0114] Thereafter, once the phase contrast image has been determined, in box 640 the phase contrast image can be optionally output to a user, for example via a computer screen or by uploading the image to a server.

[0115] In summary, at least the following embodiments are disclosed:

[0116] Example 1 1. A method for determining a phase contrast image based on a plurality of microscopy images acquired using a microscope, comprising: the microscope comprises an illumination module configured to provide switchable angular illumination of an imaging plane of the microscope, the microscope further comprising an optical system for illuminating the imaging plane and imaging the imaging plane onto at least one camera of the microscope; The method comprises: controlling the lighting module to drive a plurality of angular lighting arrangements, at least one of the angular lighting arrangements including a corresponding range of lighting directions; Controlling at least one camera to capture a plurality of images at a plurality of angular illumination arrangements and a plurality of amounts of defocus; and combining the multiple images to determine a phase contrast image.

[0117] Example 2 The method of example 1, further comprising applying multiple defocus amounts by at least one of moving a sample stage along the optical path of the optical system, using multiple cameras positioned along the optical path of the optical system, or controlling an electrically adjustable lens.

[0118] Example 3 3. The method of example 1 or 2, wherein at least some of the multiple angular illumination arrangements are multiplexed in polarization space or wavelength space.

[0119] Example 4 The method of any one of the preceding embodiments, wherein the illumination module comprises at least one of a programmable LED array, a digital micromirror device, a liquid crystal display, a variable condenser aperture, an angular diffuser, and an aperture wheel.

[0120] Example 5 combining the images includes calculating pairwise differences between each of the images in the first subset using a first phase reconstruction algorithm to obtain a first temporary phase contrast image; combining the images further comprises calculating pairwise differences between each of the images in the second subset using a second phase reconstruction algorithm to obtain a second temporary phase contrast image; The method comprises: The method of any one of the preceding examples, further comprising combining first image information associated with the first temporary phase contrast image and second image information associated with the second temporary phase contrast image to obtain the phase contrast image.

[0121] Example 6 applying a low pass filter to the first temporary phase contrast image to obtain the first image information; 6. The method of example 5, further comprising applying a high pass filter to the second temporary phase contrast image to obtain the second image information.

[0122] Example 7 the first subset comprises a first image of the plurality of images and a second image of the plurality of images; the first image included in the first subset has a positive defocus amount; the second image in the second subset is a negative defocus amount; 7. The method of claim 5 or 6, wherein the first and second images included in the first subset are both acquired with the same angular illumination arrangement.

[0123] Example 8 the second subset comprises a first image of the plurality of images, a second image of the plurality of images, and a third image of the plurality of images; The method according to any one of Examples 5 to 7, wherein the first image included in the second subset, the second image included in the second subset, and the third image included in the second subset are all acquired with the same defocus amount and different angle illumination arrangements.

[0124] Example 9 A method according to any one of Examples 5 to 8, wherein at least one of the first phase reconstruction algorithm and the second phase reconstruction algorithm is based on a predefined optical transfer function that is independent of the properties of the microscope.

[0125] Example 10 5. The method according to any one of claims 1 to 4, wherein combining the images comprises solving an optimization problem according to differences between pairs of the images and associated differences of each phase transfer function, the phase transfer function being determined for each combination of the angular illumination arrangements and the defocus values.

[0126] Example 11 11. The method of example 10, further comprising determining a phase transfer function for each combination of a plurality of angular illumination configurations and a plurality of defocus amounts based on a parameterized template stored in a database.

[0127] Example 12 The method of embodiment 11 further comprises calculating a phase transfer function for each combination of the plurality of angular illumination configurations and the plurality of defocus amounts based on a weak object transfer function formula that includes a two-dimensional integral over a light field at a pupil plane and a light source plane.

[0128] Example 13 13. The method of embodiment 12, wherein the weak transfer function formula takes into account one or more aberrations of the optical system.

[0129] Example 14 14. The method according to any one of claims 10 to 13, wherein the optimization problem is solved using deconvolution analysis.

[0130] Example 15 The method of example 14, wherein the convolution analysis takes into account a regularization term.

[0131] Example 16 4. The method according to any one of the preceding examples, wherein at least one of the plurality of angular illumination arrangements includes a single illumination direction.

[0132] Example 17 13. The method of any one of the preceding examples, wherein the multiple angle illumination arrangement has an angular spectrum of multiple widths.

[0133] Example 18 The method according to any one of the preceding embodiments, wherein the magnitude of each of the plurality of defocus amounts is related to the depth of field of the detection optics of the microscope by a ratio in the range of 1-10.

[0134] Example 19 1. A method for determining a phase contrast image based on a plurality of microscopy images acquired using a microscope, the microscope comprising an illumination module configured to provide switchable angular illumination of an imaging plane of the microscope, the microscope further comprising an optical system for illuminating the imaging plane and imaging the imaging plane onto at least one camera of the microscope; The method comprises: controlling the illumination module to drive a plurality of angular illumination configurations having a plurality of angular spectrum widths; controlling the at least one camera to capture a plurality of images at a plurality of angular illumination arrangements and a plurality of amounts of defocus; combining the multiple images to determine a phase contrast image.

[0135] Example 20 at least one processing device comprising a memory storing program code and a processor configured to read and execute the program code, The processing device, wherein the processor is configured to execute the method according to any one of Examples 1 to 19 when the processor executes the program code.

[0136] Example 21 20. Program code which is executed by a processor, the processor being configured to, when executing the program code, perform a method according to any one of claims 1 to 19.

[0137] While the present invention has been shown and described with respect to certain preferred embodiments, equivalents and modifications will occur to those skilled in the art upon reading and understanding the specification. The present invention includes all such equivalents and modifications, and is limited only by the scope of the appended claims.

[0138] For purposes of illustration, the above describes a scenario in which the multiple angle illumination arrangement includes at least one angle illumination arrangement that includes multiple illumination directions. More generally, the multiple angle illumination configurations used in combination with multiple defocus amounts may have angular spectra with different widths, e.g., a wide angular spectrum and a narrow angular spectrum (see FIG. 6).

[0139] Addendum Related to Equation 2, H DPC (k)=exp(j arctan(k Y / k x ) is used as the Hirbert transformation kernel. The equation is derived as follows:

[0140] First, consider 1D DPC. The raw images (e.g., left and right parts of an LED array) look like DIC images. DIC or DPC phase contrast images are related to the gradient of phase. Images from left and right half illumination are subtracted from each other to obtain an approximate gradient dphase / dx. In the Fourier domain, this corresponds to multiplication by jk… i.e.

number

[0141] An approximation is possible, the essential feature is that the premultiplier k changes sign at the origin of the coordinate system. This is the main feature of the transfer function. That is, approximately,

number

[0142] Then, in two dimensions,

number

Claims

1. 1. A method for determining a phase contrast image based on a plurality of microscopy images acquired using a microscope, comprising: the microscope comprises an illumination module configured to provide switchable angular illumination of an imaging plane of the microscope, the microscope further comprising an optical system for illuminating the imaging plane and imaging the imaging plane onto at least one camera of the microscope; The method comprises: controlling the lighting module to drive a plurality of angular lighting arrangements, at least one of the angular lighting arrangements including a corresponding range of lighting directions; controlling at least one camera to capture a plurality of images at a plurality of angular illumination arrangements and a plurality of amounts of defocus; and combining the multiple images to determine a phase contrast image.

2. 10. The method of claim 1, further comprising applying multiple amounts of defocus by at least one of: moving a sample stage along the optical path of the optical system, using multiple cameras positioned along the optical path of the optical system, or controlling an electrically adjustable lens.

3. The method of claim 1 or 2, wherein at least some of the multiple angular illumination arrangements are multiplexed in polarization space or wavelength space.

4. 4. The method of claim 1, wherein the illumination module comprises at least one of a programmable LED array, a digital micromirror device, a liquid crystal display, a variable condenser aperture, an angular diffuser, and an aperture wheel.

5. combining the images includes calculating pairwise differences between each of the images in the first subset using a first phase reconstruction algorithm to obtain a first temporary phase contrast image; combining the images further comprises calculating pairwise differences between each of the images in the second subset using a second phase reconstruction algorithm to obtain a second temporary phase contrast image; The method comprises:

5. The method of claim 1, further comprising combining first image information associated with the first temporary phase contrast image and second image information associated with the second temporary phase contrast image to obtain the phase contrast image.

6. applying a low pass filter to the first temporary phase contrast image to obtain the first image information; The method of claim 5 , further comprising applying a high pass filter to the second temporary phase contrast image to obtain the second image information.

7. the first subset comprises a first image of the plurality of images and a second image of the plurality of images; the first image included in the first subset has a positive defocus amount; the second image in the second subset is a negative defocus amount; The method of claim 5 or 6, wherein the first and second images in the first subset are both acquired with the same angular illumination arrangement.

8. the second subset comprises a first image of the plurality of images, a second image of the plurality of images, and a third image of the plurality of images; 8. The method of claim 5, wherein the first image in the second subset, the second image in the second subset, and the third image in the second subset were all acquired with the same amount of defocus but with different angular illumination arrangements.

9. 9. The method of claim 5, wherein at least one of the first phase reconstruction algorithm and the second phase reconstruction algorithm is based on a predefined optical transfer function that is independent of the properties of the microscope.

10. The method of claim 1 , wherein at least one of the plurality of angular illumination arrangements comprises a single illumination direction.

11. The method of claim 1 , wherein the multiple angle illumination arrangement has an angular spectrum of multiple widths.

12. 12. The method of claim 1, wherein the magnitude of each of the plurality of defocus amounts is related to the depth of field of the detection optics of the microscope by a ratio in the range of 1 to 10.

13. 1. A method for determining a phase contrast image based on a plurality of microscopy images acquired using a microscope, the microscope comprising an illumination module configured to provide switchable angular illumination of an imaging plane of the microscope, the microscope further comprising an optical system for illuminating the imaging plane and imaging the imaging plane onto at least one camera of the microscope; The method comprises: controlling the illumination module to drive a plurality of angular illumination configurations having a plurality of angular spectrum widths; controlling the at least one camera to capture a plurality of images at a plurality of angular illumination arrangements and a plurality of amounts of defocus; combining the multiple images to determine a phase contrast image.

14. At least one processing device comprising a memory storing program code and a processor configured to read and execute the program code, A processing device, the processor being configured to perform the method of any one of claims 1 to 13 when executing the program code.

15. 14. Program code which is executed by a processor, the processor being configured to, when executing the program code, perform a method according to any one of claims 1 to 13.

Images