Hybrid differential focal phase contrast

Abstract

Phase contrast images are calculated by digital post-processing of microscope images acquired at different angular illumination configurations and defocus values.

Description

Technical Field

[0001] Various examples of the present application generally relate to digital phase contrast techniques for determining phase contrast images by digitally post-processing multiple microscope images. In particular, techniques are disclosed for enhancing the cumulative phase transfer function based on the use of multiple angled illumination configurations and multiple defocus values. Background Art

[0002] When light interacts with a target sample, such as biological tissue, two main contrast mechanisms leave traces on the incident radiation. First, the sample attenuates the incident light by absorption; second, the sample deforms the incident light wavefront, leaving behind phase contrast. For thin and unlabeled biological samples, absorption is usually negligible. In this case, phase contrast microscopy is used to generate an image of the sample.

[0003] Phase-sensitive techniques have a long history, and a few examples include Zernike phase contrast microscopy [1], Gabor on-axis holography [2], and Nomarski differential interference contrast (DIC) microscopy [3]. Although phase-sensitive, the above techniques are generally not quantitative and can result in artifacts in the final image that do not originate from the sample itself. For example, in Zernike phase contrast microscopy, halo artifacts observed at phase discontinuities hinder direct phase quantification. Gabor on-axis holography suffers from the so-called double image artifact, while Nomarski DIC requires additional hardware components such as phase shifting elements or polarization optics. The latter issues present obstacles to microscope users in terms of cost, complexity, and calibration difficulties.

[0004] Recently, quantitative phase imaging systems have undergone a shift towards a data-driven approach: typically multiple images are recorded and then numerically combined into a single phase contrast image using the basic principles of physical image formation. This technique can therefore be referred to as digital phase contrast: the phase contrast is obtained by digital post-processing of the acquired intensity images. Prominent examples are the Transport of Intensity Equation (TIE) [4] and Differential Phase Contrast (DPC) [5]. To acquire a TIE dataset, the sample is translated along the optical axis (z-direction) and a so-called z-stack consisting of at least two images is recorded. The data is then converted numerically into a phase contrast image by solving a diffusion-type partial differential equation. For example, US2020 / 209604A1 discloses an annular illumination combined with TIE (i.e., image acquisition at non-zero defocus values) and a rigorous calculation of the Phase Transfer Function (PTF). In DPC, the sample is illuminated from at least three different directions (angular illumination) while the sample is held at a fixed z position. Possible sources of angular illumination are any type of segmented light source; for example, segmented diodes [5], light emitting diode (LED) arrays [6], digital micromirror devices (DMDs), liquid crystal displays (LCDs), or variable focusing aperture diaphragms. The recorded data are then converted into a phase contrast image by solving a deconvolution problem.

[0005] TIE and DPC have fundamental drawbacks: (1) TIE usually uses a point source for illumination. In order to improve the lateral resolution achievable by the optical system, it is desirable to use a radiation source that extends laterally relative to the optical axis; (2) DPC performs poorly when the illumination and detection numerical apertures (NA) are mismatched [6,7]. The latter is particularly problematic when the illumination NA is smaller than the detection NA. In this case, the generated phase image lacks contrast.

[0006] Zuo et al. recently reported a possible solution to problem (1) [7]. In this study, the specimen was illuminated sequentially by individual LEDs, while the specimen was also defocused. Unfortunately, using each LED in a large LED matrix sequentially, combined with defocusing, requires capturing hundreds of images, making the technique slow and computationally complex in terms of the data that needs to be recorded, transmitted, processed, and stored.

[0007] One possible solution to problem (2) is to combine DPC with a modified objective lens, for example as described in PCT / EP2020 / 050307 [8]. In the latter patent, the authors combined an amplitude modulated pupil ring with a DPC acquisition mechanism. Unfortunately, the latter has the disadvantage that part of the light entering the pupil is absorbed - this has an adverse effect on fluorescence imaging when used in combination with DPC without changing the objective lens. Summary of the invention

[0008] Therefore, there is a need for advanced digital phase contrast microscopy techniques. There is a need for techniques that can alleviate at least some of the above limitations or disadvantages.

[0009] This need is met by the features of the independent claim. Embodiments are defined by the features of the dependent claims.

[0010] Digital phase contrast techniques are disclosed below. Multiple microscope images are acquired at multiple setups, each setup being characterized by a corresponding defocus value and a corresponding angular illumination configuration. The various techniques are based on the discovery that both the defocus of the imaging plane and the angular illumination of the imaging plane make it possible to obtain an image that encodes phase information. By combining both angular illumination and defocus, phase information can be reconstructed over a larger range of spatial frequencies than reference techniques such as TIE or DPC. Thus, high quality phase contrast images can be obtained.

[0011] A method for determining a phase contrast image is disclosed. The phase contrast image is determined based on a plurality of microscope images acquired using a microscope.

[0012] The plurality of microscope images are intensity images. That is, they are not acquired by phase contrast. Nevertheless, information enabling reconstruction of the phase of the imaged sample is encoded in the microscope images.

[0013] The microscope includes an illumination module. The illumination module is configured to provide switchable angular illumination. That is, different angular illumination configurations can be enabled. The angular illumination provides a certain range of angles at which light is incident in an imaging plane of the microscope. This angular spectrum of the angular illumination is fixed across the imaging plane.

[0014] The microscope also includes an optical system. The optical system includes one or more optical elements, such as lenses. The optical system is configured to illuminate an imaging plane. In addition, the optical system is configured to image the imaging plane onto at least one camera of the microscope.

[0015] The method includes controlling the lighting module to enable multiple angular lighting configurations. In at least some examples, at least one of the multiple angular lighting configurations includes a range of lighting directions. That is, each of the at least one angular lighting configuration includes a corresponding extended angular spectrum (rather than just a single lighting angle).

[0016] It is possible that only one of the multiple angle lighting configurations includes a series of lighting directions. It is also possible that each of the multiple angle lighting configurations includes a series of lighting directions. When not all of the multiple angle lighting configurations include a series of lighting directions, the remaining angle lighting configurations include a single lighting direction. The multiple angle lighting configurations can each have an angle spectrum with a different width. For example, a first angle lighting configuration has a first angle spectrum with a first width, and a second angle lighting configuration has a second angle spectrum with a different second width. A wider angle spectrum can be used together with a narrower angle spectrum. The narrower angle spectrum can have a minimum achievable width for the corresponding lighting module (i.e., a single lighting direction).

[0017] For example, each of a plurality of angular illumination configurations (indexed as j) may be associated with a corresponding illumination numerical aperture (NA) NA i,j The numerical aperture of illumination defines the range of angles over which the object plane is illuminated. Increasing NA i Generally increases the coherence of the illumination because the higher NA i This narrows the angular range over which light is emitted, effectively shrinking the light source and increasing its spatial coherence.

[0018] The detector optics, such as an objective or objective lens system, defines the detector numerical aperture NA d The detector numerical aperture is defined by the objective or lens system. The detector numerical aperture affects how fine the details in the image can be resolved; a higher detector numerical aperture allows light to be collected from a wider angle, thus improving resolution.

[0019] It is possible that each illumination numerical aperture (associated with each of the multi-angle illumination configurations) is smaller than the detector numerical aperture, for example at least a factor of 2 smaller: σ = NA i，j / NA d , where σ≤1 / 2 for all j. σ is sometimes called the coherence parameter. That is, for any image acquired and post-processed, σ may be 1 / 2 or less.

[0020] Although the above examples have been disclosed where at least one of the multiple angle lighting configurations includes a range of lighting directions, this is generally optional. In some examples, each of the multiple angle lighting configurations includes a single lighting angle, respectively. "Single lighting angle" means that the width of the corresponding distribution of lighting angles that can be achieved using the lighting module is the smallest achievable. Typically, this may mean lighting a single light source, such as a single LED of a corresponding array.

[0021] The method also includes controlling at least one camera to acquire a plurality of images under the multi-angle lighting configuration and at a plurality of defocus values.

[0022] The method also includes combining the multiple images to determine a phase contrast image. As a general rule, combining two or more images may include calculating pairwise differences. Combining two or more images may also include calculating a phasor, i.e., a complex-valued representation of the sum of the two or more images being combined, wherein a corresponding phase shift is applied to each individual image.

[0023] It is often possible to combine each of the multiple images with each of the remaining images. Alternatively, in some scenarios, multiple subsets of the image may be formed, each subset including only portions of the image. Each image in a given subset is then combined with any other image in the subset. This approach may be particularly useful if multiple phase reconstruction algorithms operating on separate subsets are used.

[0024] A phase contrast image may have quantitative phase contrast, ie the contrast may quantitatively encode the phase shift imposed by the object.

[0025] Although a method of controlling at least one camera and an illumination module to acquire an image is disclosed above, in other scenarios, multiple images may be acquired in advance. In this scenario, multiple images previously acquired at multiple defocus values ​​and multiple angular illumination configurations may be loaded from a database or a picture archive and then combined to determine a phase contrast image.

[0026] A computing device includes a processor and a memory. The memory stores program code. The processor is configured to load the program code and execute the program code. When the processor executes the program code, the method described above is performed.

[0027] A computer program or a computer-readable storage medium includes program code. The program code can be loaded and executed by a processor. When the processor executes the program code, it performs the method described above.

[0028] It is to be understood that the features mentioned above and those yet to be explained below can be used not only in the respective combination indicated but also in other combinations or alone, without departing from the scope of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A system including a microscope and a computer according to various examples is schematically illustrated.

[0030] Figure 2The DPC digital phase contrast technology implemented according to the prior art is schematically shown.

[0031] Figure 3 The figure schematically shows the TIE digital phase contrast technology implemented according to the prior art.

[0032] Figure 4 Phase contrast techniques according to various examples are schematically illustrated.

[0033] Figure 5 is based on Figure 2 , Figure 3 and Figure 4 Comparison of the phase transfer functions of the phase contrast techniques.

[0034] Figure 6 is a flow chart of a method according to various examples.

[0035] Figure 7 Angular spectra of multiple angular lighting configurations according to various examples are schematically illustrated. DETAILED DESCRIPTION

[0036] Some examples of the present disclosure generally provide multiple circuits or other electrical devices. All references to circuits and other electrical devices and the functions provided by each circuit and other electrical device are not intended to be limited to the contents shown and described herein. Although specific labels can be assigned to the various circuits or other electrical devices disclosed, these labels are not intended to limit the operating range of circuits and other electrical devices. These circuits and other electrical devices can be combined and / or separated in any way based on the specific type of electrical implementation required. It can be recognized that any circuit or other electrical device disclosed herein may include any number of microcontrollers, graphics processor units (GPUs), integrated circuits, storage devices (e.g., FLASH, random access memory (RAM), read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or other suitable variants thereof) and software that cooperates with each other to perform the operations disclosed herein. In addition, any one or more electrical devices may be configured to execute a program code embodied in a non-transient computer-readable medium that is programmed to perform any number of functions disclosed.

[0037] Below, embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be understood that the following description of the embodiments should not be considered restrictive. The scope of the present invention is not intended to be limited by the embodiments or drawings described below, which are merely illustrative.

[0038] The accompanying drawings should be considered schematic diagrams, and the elements shown in the accompanying drawings are not necessarily shown to scale. Instead, the various elements are represented in such a way that their functions and general uses are apparent to those skilled in the art. Any connection or coupling between the functional blocks, devices, components, or other physical or functional units shown in the accompanying drawings or described herein may also be achieved by indirect connection or coupling. Coupling between components may also be established by wireless connection. Functional blocks may be implemented in hardware, firmware, software, or a combination thereof.

[0039] The technique disclosed herein may be referred to as "Hybrid Differential Defocus (HDD) phase contrast". HDD phase contrast combines the advantages of TIE and DPC while addressing the disadvantages of each technique. In HDD, the sample is defocused and illuminated by an asymmetric illumination pattern. Unlike reference [7], the disclosed technique does not exclusively employ single LED illumination, which results in a large number of image acquisitions. Instead, HDD exploits illumination direction multiplexing, at least to some extent, where, for example, multiple facets (e.g., multiple LEDs) of a programmable light source are illuminated simultaneously. For example, half-pupil illumination may be used. That is, a segmented light source is used that illuminates exactly half in area (e.g., the top half versus the bottom half, or the left half versus the right half). The segments are symmetric with respect to the optical axis. The pupil itself may be square, rectangular, elliptical, circular, or annular. More generally, multiple illumination configurations are enabled, where at least one of the multiple illumination configurations includes a range of illumination angles within the imaging plane.

[0040] Furthermore, HDD exploits different patterns displayed on a programmable light source, as opposed to TIE which uses only point source illumination. In DPC, the sample is held in a fixed position in the focal plane, whereas HDD exploits axial sample translation to achieve improved phase contrast. In the spatial domain, the imaging plane is uniformly illuminated with a specific angular spectrum (i.e., the angular spectrum is independent of the lateral position relative to the optical axis within the associated aperture), so the angular illumination configuration should not be confused with, for example, light sheet microscopy, where only a certain slice of the sample is illuminated.

[0041] Various techniques are based on the finding that combining defocus with multiple angular illumination configurations produces a phase contrast morphology that exhibits improved lateral resolution compared to TIE, while not suffering from the mismatched NA problem that results in lower phase contrast as encountered in DPC. Furthermore, by multiplexing multiple LEDs in parallel, the data acquisition protocol of HDD is orders of magnitude faster than existing techniques [7].

[0042] Figure 1Schematically illustrated is a system 70 according to various examples. The system 70 comprises a microscope 90 and a computer 80. The microscope 90 comprises an illumination module 91 , an optical system 92 and a detector module 93.

[0043] The detector module 93 includes detector optics. The detector optics define the detector numerical aperture NA d The detector module 93 also includes one or more cameras to acquire microscope images.

[0044] The illumination module 91 is configured to provide switchable / reconfigurable angular illumination of an imaging plane defined along an optical path 94 of the system 70. This means that the illumination angle can be controlled. In addition to controlling the main illumination angle, the angular spectrum, such as the width or contribution, can also be selectively controlled. For example, multiple illumination configurations with angular spectrums of different widths can be enabled. Sometimes, only a single illumination direction may be enabled (smallest width of the angular spectrum), and sometimes multiple illumination directions may be superimposed (larger width of the angular spectrum).

[0045] The optical system 92 is configured to illuminate the imaging plane and further image the imaging plane onto at least one camera of the detector module 93 .

[0046] The microscope 90 also includes a control module 95, which is configured to control various components of the microscope. For example, the control module 95 can be implemented using a CPU or a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The control module 95 may include a memory. The control module 95 can be configured to control the lighting module to enable multiple angle lighting configurations. The control module 95 can also be configured to control the detector module 93, and specifically, control at least one camera of the detector module 93 to collect multiple images. Optionally, the control module 95 can be configured to control the optical system 92, for example, to move a sample holder configured to hold a sample, so as to achieve multiple defocus values.

[0047] Also shown is a computer 80, which includes an interface 84 configured to communicate with the microscope 90, in particular the control module 95. For example, a processor 81 (e.g., a CPU, FPGA, or ASIC) of the computer 80 can provide control data to the control module 95 to implement certain control functions, such as triggering image acquisition, triggering the enabling of certain angular illumination configurations, triggering the implementation of certain defocus values, etc. The processor 81 can also retrieve image data from the microscope 90 via the interface 84 and perform post-processing on the image data. The processor 81 can also load image data from a database or picture archive.

[0048] The processor 81 is configured to load program code from the memory 83 and execute the program code to perform such techniques. In particular, the processor 81 can perform digital post-processing to determine a phase contrast image based on a plurality of intensity images retrieved from the microscope 90. The computer 80 can also include a user interface 82 (e.g., a GUI) to output the phase contrast image thus determined.

[0049] Although in Figure 1 , the computer architecture is split between the control module 95 and the computer 80, but in some cases the control module 95 may also perform digital post-processing of the image data. Alternatively or additionally, component-level control of individual components of the microscope 90 may also be a task at least partially delegated to the computer 80.

[0050] Figure 2 , Figure 3 and Figure 4 The experimental hardware used in the differential phase contrast (DPC), transfer intensity equation (TIE) and hybrid differential defocus (HDD) phase contrast imaging setups are shown respectively.

[0051] DPC( Figure 2 ) uses a programmable illumination unit (PIU) 201. It needs to acquire at least three focal plane images under a variable illumination pattern 202 to produce a phase contrast image of the sample. In particular, these variable illumination patterns 202 correspond to different angular illumination configurations for illuminating the imaging plane 203 at different angles while the sample remains in a fixed position. Each pattern 202 includes enabled LEDs with different asymmetric distributions relative to the optical axis 207 (dashed lines and hollow circles). That is, each illumination configuration includes multiple illumination directions (defined by enabled LEDs). Objective lens 204 and tube lens 205 (part of the optical system, see Figure 1 : The optical system 92 is used to image the imaging plane 203 onto a pixelated detector 206 (camera). Different LEDs are enabled for different variable illumination patterns 202. A subsequent numerical deconvolution routine converts the recorded image into a phase contrast image of the sample, see [6]. Details about such a DPC phase reconstruction algorithm will be explained next.

[0052] Assume that a minimum of three images are recorded from equally spaced angular directions (azimuths) 120° apart (the angles represent the azimuth position of the average angular spectrum, e.g., using a semicircular illumination pattern centered at that azimuth). These images are all acquired at the same defocus value, i.e. nominally zero defocus in this example, so that the sample is located in the imaging plane. These three images are contained in the DPC subset. The two-dimensional (2D) raw image is referred to here as I 0° , I 120° , I240° These can be generated by, for example, three angularly equally spaced LEDs with or without a diffuser to extend the angular range. Alternatively, any angularly segmented source can be used (digital micromirror device, spatial light modulator, etc.). Details on illumination module 91 have been described above in conjunction with Figure 1 Then, the complex-valued real-space phasor image is calculated as the combination of these three images in the DPC subset.

[0053]

[0054] The DPC phase contrast image is calculated using the formula of the phase reconstruction algorithm:

[0055]

[0056] Among them, H DPC (k) = exp(j·arctan(k y / k x )) is the Hilbert transform kernel (see Kieran G.Larkinand Peter A.Fletcher, "Isotropic scalar image visualization of vector differential image data using the inverse Riesz transform," Biomed.Opt.Express 5, 907-920 (2014); and the appendix to this document), and imag[…] selects the imaginary part of the complex number. In addition to using the Hilbert transform kernel, the DPC transfer function using the "weak object transfer function" (WOTF) formula (i.e., Formula 6) can also be used. The calculation of the Hilbert transform is an approximation, but it is faster - because the WOTF itself requires the calculation of at least two Fourier transforms. On the other hand, it is observed that for relevant spatial frequencies, the reduction in accuracy is very limited.

[0057] DPC phase contrast images are affected by artifacts caused by shadow casting by scatterers located outside the imaging plane (out-of-focus scatterers). Such shadow casting is sensitive to the direction of the angular illumination and is therefore retained in the real-space phase or image calculated as described above. For example, scatterers outside the imaging plane may be scratches on optical elements in the optical path or dust on the back of a sample holder slide. Typically, these shadows cast into the DPC phase contrast image are not well localized, i.e., blurred and extended. That is, they are limited to small spatial frequencies.

[0058] Reference now Figure 3: TIE phase reconstruction is based on a diffusion equation that relates the axial intensity derivative to the phase of the sample. The axial intensity derivative can be approximated by recording at least two images, where the imaging plane and the sample are axially shifted and moved closer to the focal plane, so that multiple defocus values ​​311 are achieved. The illumination unit is fully coherent, such as an on-axis point source 301 or an on-axis collimated laser. Alternatively, partially coherent illumination can be used. After recording images with varying defocus (also called z-stacks), there is enough information to solve the associated diffusion equation and thus obtain the phase information of the sample [4].

[0059] The TIE subset contains two images; these two images are generated by I -Δz and I +Δz Given two different defocus values, where Δz = c·λ / NA 2 is the distance of the sample from the focal plane. In this way, positive and negative defocus values ​​equidistant from the focal plane are used. λ and "NA" are the central wavelength of the illumination and the numerical aperture of the objective used. Correspondingly, the c-parameter describes the size of the defocus related to the depth of field of the detector optics. In practice, the c-parameter (for images in the TIE subset) is chosen between c=1 and c=10 to control the defocus distance. The two images of the TIE subset are acquired under the same illumination configuration, such as on-axis illumination. In such on-axis illumination, the angular spectrum is centered on the vertex and is partially coherent. This can be achieved by enabling a single light source in the light source array centered on the optical axis, such as Figure 3 centered on the optical axis 201. However, a range of illumination directions may also be used. For example, for an image post-processed using TIE reconstruction, σ≤1 / 3, or, in particular, 1 / 10≤σ≤1 / 3. The TIE phase contrast image φ of the sample is then generated using the formula given by the TIE phase reconstruction algorithm: TIE :

[0060]

[0061] in is the second-order transfer function, and FFT stands for two-dimensional fast Fourier transform. The simple form of the transfer function compares favorably with the WOTF transfer function calculation, which is numerically less efficient and yields only slight improvements for high spatial frequencies. This optical transfer function H TIE (k) - similar to H DPC (k) - does not depend on any properties of the microscope, i.e., it is a predefined simplified transfer function that depends only on the k-space position.

[0062] Unlike DPC phase contrast images, TIE phase contrast images are not significantly affected by shadow casting artifacts (as explained above in conjunction with the DPC phase reconstruction algorithm) caused by out-of-focus scattering (e.g., impurities, dust, scratches, etc.). This is because the TIE phase reconstruction algorithm does not change the illumination configuration, so that such angle-sensitive shadow casting is removed from the paired difference images according to the previous formula 3. On the other hand, high image frequencies are not preserved in TIE phase contrast images, making edges appear blurred and imaging resolution reduced.

[0063] According to some examples, two separate phase reconstruction algorithms may be performed, one based on the DPC subset and Equation 2, and another based on the TIE subset and Equation 3.

[0064] In this case, two temporary phase contrast images calculated based on the TIE phase reconstruction algorithm and the DPC phase reconstruction algorithm, respectively, are obtained. The image information associated with the two temporary phase contrast images can then be combined to obtain a final phase contrast image. For example, a low-pass filter can be applied to the temporary phase contrast image obtained from the TIE phase reconstruction algorithm, while a high-pass filter can be applied to the temporary phase contrast image obtained from the DPC phase reconstruction algorithm. Then, the corresponding filtered representations of these temporary phase contrast images, that is, the corresponding low-pass filtered and high-pass filtered image information, can be combined pixel-wise to obtain the final phase contrast image. In this way, given the temporary phase contrast image φ obtained according to the previous calculation steps TIE and φ DPC , using selective high-pass and low-pass filters to calculate the final HDD phase contrast image

[0065] φ HDD =iFFT(w(k)·FFT(φ TIE )+d·(1-w(k))·FFT(φ DPC )),

[0066] Formula 4 where the weight function is a Gaussian function . Thus, the high-pass filtered image information obtained from the DPC phase contrast image is combined with the low-pass filtered image information obtained from the TIE phase contrast image. The width of the Gaussian filter kernel is controlled by the parameter σ and is chosen to be at least large enough to ensure that the complementary transfer weight function (1-w(k)) filters out background artifacts (e.g., dust) from the DPC temporary phase contrast image. The scalar parameter d is chosen to ensure that the combined DPC and TIE spectra are given by Describes the continuity of circular spatial frequency rings.

[0067] This combination of DPC and TIE phase reconstruction algorithms, together with the subsequent combination of TIE and DPC temporary phase contrast images, enables the provision of a synthetic phase contrast image with superior performance compared to temporary DPC and TIE phase contrast images. In particular, since a high-pass filter is applied to the DPC phase contrast image, angle-sensitive shadow casting due to out-of-focus scattering centers in the imaging path, which is present in the DPC phase contrast image, is not observed in the final phase contrast image. In this way, blurring artifacts caused by out-of-focus scatterers in the imaging path are removed. On the other hand, low spatial frequencies that are not well preserved by the TIE phase reconstruction algorithm are filtered out from the TIE phase contrast image, so that the corresponding image information is mainly obtained from the DPC phase contrast image. In this way, the resulting HDD phase contrast image does not have background artifacts similar to the DPC phase contrast image, and at the same time, exhibits good image contrast at all spatial frequencies. In addition, it is sufficient to acquire a total of 5 different images, namely 2 images in the TIE subset at two different defocus values, and 3 images in the DPC subset at three different angular illumination configurations. Since only 5 images are acquired, or more generally less than 10 images if multiple acquisitions are performed to improve the signal-to-noise ratio, the image acquisition time can be significantly faster compared to competing methods such as [7]. In addition, the illumination of the sample can be reduced, which is beneficial for certain light-sensitive samples (such as cellular structures, etc.).

[0068] The acquired images and their association with the two subsets are shown below.

[0069]

[0070] Table 1: Two subsets of DPC and TIE based on a total of 5 images. The subsets do not overlap, i.e. no image belongs to both subsets. The c-parameter of the images of the TIE subset can be in the range of 1 to 10. For images used for DPC, semicircular illumination and σ = 1 / 2 can be used; for images used for TIE, σ ≤ 1 / 3

[0071] The above discloses an HDD scenario in which two separate phase reconstruction algorithms are used, one for obtaining a temporary DPC phase contrast image and one for obtaining a temporary TIE phase contrast image. The DPC phase contrast image and the TIE phase contrast image are then fused via selective high-pass and low-pass digital signal processing. In another variant, a joint HDD reconstruction algorithm may be used. Figure 4 This is explained.

[0072] Reference now Figure 4 According to the disclosed technology, this joint HDD reconstruction algorithm combines DPC (see Figure 2 ) and TIE (see Figure 3) works.

[0073] In the example shown, the lighting module 91 is implemented by the PIU 201 and includes an LED array, which includes a plurality of LEDs positioned at different angles and distances relative to the optical axis 207. Each of the plurality of LEDs can be individually enabled, for example, by the control module 95 (see Figure 1 ) is enabled. By enabling multiple batches of LEDs (for example, in an asymmetric pattern), different lighting patterns 202 can be formed, as shown in the previous combination Figure 2 Each of the plurality of illumination patterns 202 includes a plurality of enabled LEDs to achieve a wide angular spectrum of illumination angles; multiple illumination directions are enabled simultaneously. The asymmetric arrangement of the illumination patterns 202 results in an average value of the angular spectrum being tilted relative to the optical axis 207. Note that although Figure 4 A semicircular lighting pattern 202 is shown, but other lighting patterns are also contemplated. Figure 4 As shown, the HDD is not limited to implementing the illumination module 91 via the PIU 201. As a general rule, the illumination module may include at least one of a programmable LED array, a digital micromirror device, a liquid crystal display, a variable focusing aperture stop, an angular diffuser, or an aperture wheel.

[0074] For example, the digital micromirror device can be illuminated by one light source, and depending on the orientation of each mirror of the digital micromirror device, the corresponding illumination point in the illumination pupil plane can be enabled or disabled, which corresponds to lighting up or turning off the LEDs of the programmable LED array of the programmable illumination unit 201. In this way, the main angle of illumination (i.e., the average value of the angular spectrum) as well as other characteristics of the angular spectrum, such as width or side peaks, etc., can be controlled.

[0075] A similar effect can be achieved using a liquid crystal display with appropriate backlighting. Individual pixels of the liquid crystal display can be enabled or disabled. A variable focusing aperture stop can include multiple apertures that can be selectively opened or closed.

[0076] An angular diffuser may help broaden the angular spectrum by being positioned between the imaging plane 203 and the illumination pupil plane.

[0077] HDD is not limited to lighting configurations that include multiple lighting directions, such as Figure 4 In particular, in addition to a lighting configuration including multiple lighting directions, one or more angular lighting configurations including a single lighting direction may also be used.

[0078] exist Figure 4 In the scene of , according to the HDD technology, multiple angle lighting configurations are implemented in combination with multiple defocus values ​​311. Figure 4A combination of two defocus values ​​(zero defocus and offset Δz) is shown in the scene of , but a larger number of defocus values ​​can be envisaged.

[0079] In addition, Figure 4 In the scenario of FIG. 20 , multiple defocus values ​​are achieved by moving the sample stage to reposition the sample along the optical path, in particular the optical axis 207. For example, this can be achieved by a motorized sample stage of the optical system 92, which is controlled by the control module 95 (see FIG. 20 ). Figure 1 ) control. This is just one option for applying multiple defocus values. In another scenario, multiple cameras can be used, in combination with a beam splitter. Multiple cameras can be arranged at different distances along the optical path relative to the sample plane. The advantage of using multiple cameras is that multiple image acquisition can be performed at multiple defocus positions. Thereby, the total measurement time required to acquire all images that are subsequently post-processed can be reduced. Alternatively or additionally, an electrically adjustable lens with a reconfigurable focal length can be used. This enables a more compact arrangement without the need to provide multiple cameras. The electrically adjustable lens can be refocused quickly, so that the overall measurement time required to acquire an image can be reduced, for example compared to a scenario where the sample state is repositioned.

[0080] The measurement time can be further reduced by multiplexing multi-angle illumination configurations in polarization space or wavelength space. For example, a multi-color illumination module 91 (e.g., red-green-blue) can be used, which enables different angular illumination configurations for different wavelengths. This can then be combined with a wavelength selective detection module 93 that can simultaneously acquire images of different wavelengths. Similar setups with different polarizations can be envisioned.

[0081] Above, various aspects related to the hardware implementation of the microscope 90 and its various components and image acquisition measurement methods have been disclosed. Next, aspects related to digital post-processing of the joint phase reconstruction algorithm will be discussed. In particular, it will be explained how to achieve the combination of multiple intensity images associated with different combinations of multiple angular illumination configurations and multiple defocus values ​​to produce a phase contrast image.

[0082] The various aspects are based on the finding that in order to digitally post-process acquired HDD microscope images, several challenges must be addressed: (1) The typical transfer function deconvolution analysis for DPC data is no longer valid when the sample is fixed in the focal plane. (2) The diffusion equation underlying TIE is invalid under the partially coherent illumination generated by the PIU. In addition, the illumination pattern changes, while TIE assumes a static illumination pattern. (3) Since multiple points on the PIU are illuminated simultaneously, the transfer function analysis proposed in [7] for individual point sources is invalid.

[0083] In order to solve these problems, multiple images acquired at different combinations of defocus values ​​and angular illumination configurations are subtracted from each other. This will enable background subtraction. Further, for each of these pairs, the difference of the corresponding phase transfer functions is calculated (thereby generating a so-called differential phase transfer function). Then, based on the knowledge of the difference image and the differential phase transfer function, the sample phase can be reconstructed, for example using deconvolution analysis to restore the phase of an object with a phase imposed by the optical system. Based on the difference image and the differential phase transfer function, an optimization problem can be formulated to obtain the sample phase, i.e., a phase contrast image. Deconvolution analysis is an option for solving such optimization problems, but other scenarios, in particular iterative methods, are also conceivable. Sometimes, such iterative methods are more robust to noise or converge more robustly.

[0084] In addition, there are multiple options for determining the phase transfer function. One specific scenario, which will be explained in detail below, is based on the weak object transfer function form. The "weak object" assumption is used to deal with objects that only slightly modulate the phase and amplitude of the light passing through them. WOTF is a concept used under this weak object assumption. WOTF includes the Fourier transform of a weak phase or amplitude object. It describes how the object affects the phase and amplitude of the light in frequency space, assuming that these effects are relatively small. For weak objects, the outgoing wave just beyond the object is approximately the sum of the incident wave plus a small perturbation caused by the object itself. This WOTF form allows the incorporation of partially coherent illumination states generated by the PIU and defocus under axial sample translation. However, there are other options different from the WOTF form. For example, a predetermined parameterized template retrieved from a database and instantiated according to a specific combination of angular illumination configuration and defocus value can be used. In addition, the Hilbert transform can be used to determine the phase transfer function.

[0085] The WOTF formalism (see [4]) provides the following generalized transfer function (complex valued) for an optical system:

[0086] WOTF(k)=∫S(k′)[P * (k′)P(k′+k)-P(k′)P * (k′+k)]dk′,

[0087] Formula 5

[0088] where k = (k x , k y ) and k′=(k′ x , k′ y) represents the spatial frequency, S is the source 92 (i.e., the illumination pattern in the illumination pupil), and P is the in-focus pupil of the optical system. Alternative formulas are conceivable, for example, using the two-dimensional integral of the illumination angle present in the in-focus pupil. Since HDD phase contrast imaging exploits both defocus and a changing source pattern, two metrics are needed to describe the changes in the illumination and defocus states of the system. Dual-mode weak object transfer is defined as follows:

[0089]

[0090] The indices m and n are used to distinguish different source patterns and defocus states, respectively. m (k) describes the mth illumination pattern displayed on the PIU (however, as described above, this can be formulated similarly for other types of light modules, for example by integrating the illumination angle provided by a particular light module), and the defocus value can be included in this description by adding the appropriate phase to the pupil. Spherical phase correction is used below as an example, but other phase corrections, such as quadratic - as an approximation - are also conceivable.

[0091]

[0092] Here, z n Describes the relative motion of the sample with respect to the focal plane at the nth axial acquisition plane.

[0093] The modulated amplitude (ATF) and phase (PTF) contrast functions are defined as follows:

[0094] ATF m,n (k) = Real [WOTF m,n (k)]

[0095] PTF m,n (k) = Imag[WOTF m,n (k)].

[0096] Equation 8 For weakly absorbing samples or aberration-free optical systems, the ATF can be neglected, see reference [6]. However, in some scenarios, aberrations of the optical system can be considered, such as spherical aberration, coma, or astigmatism as a few examples.

[0097] Assuming that the ATF is zero, the spatial two-dimensional Fourier transform FT[I] of the observed image in a partially coherent microscope can be modeled as

[0098] FT[I m,n ](j)=b+PTF m，n (k) · FT[φ](k), for all m = 1, ..., M; n = 1, ..., N

[0099] Equation 9 where b is a constant offset and φ is the unknown phase of the sample in the imaging plane to be estimated. Note that the latter is a set of equations; that is, there is a separate equation for each combination of source pattern (m) (or more generally, angular illumination configuration) and defocus value (n).

[0100] Since background terms need to be eliminated from these formulas, we calculate the difference between the formulas with index pairs (m,n) and (m',n') to obtain

[0101] FT[dI m，n，m′，n′ ](k) = dPTF m，n，m′，n′ (k) · FT[φ](k),

[0102] Formula 10 where the image intensity difference (the difference between the acquired microscope images, i.e. the combination of images) is defined as

[0103] dI m，n，m′，n′ =I m，n -I m′，n′

[0104] Formula 11 and PTF difference (differential phase contrast function)

[0105] dPTF m，n，m′，n′ (k) = PTF m，n (k)-dPTF m′，n′ (k)

[0106] Equation 12 Equation 10 defines an optimization problem. This can be written as the latter set of equations written as a least squares problem:

[0107]

[0108] The optimization problem for phase φ (i.e., determining the phase contrast image) can be solved by numerical iteration techniques or deconvolution. For example, the optimal phase estimate is given by

[0109]

[0111] where iFT[...] denotes the Fourier transform, and we omit it to simplify the notation. and FT[dI m，n，m′，n′ ] on the spatial frequency vector k. This corresponds to the Wiener filter.

[0112] When modeling known aberrations in the PTF and / or ATF, the robustness of the reconstruction can be improved.

[0113] The main challenge in performing deconvolution in the last formula is the term in the denominator,

[0114]

[0115] It may have mainly zero crossings. Division by values ​​close to or equal to zero may lead to noise amplification or numerical problems. To prevent division by zero, a regularization term (α>0) is added to the denominator,

[0116]

[0117] If the regularization term is chosen too large, the deconvolution process will suppress low spatial frequencies, resulting in poor phase contrast and lateral resolution. Conversely, if the regularization term is too low, values ​​close to zero in the denominator can lead to noise amplification. To avoid over-regularization and thus suppression of spatial frequencies, the form of the denominator d(k) is ideally adjusted to have as few regions as possible that are zero.

[0118] Figure 5 The PTFs of different digital phase contrast techniques - DPC, TIE and HDD - and the associated achievable numerical aperture (NA) coverage are shown. One example phase image 501-503 is shown for each technique.

[0119] Figure 5 The process of adjusting the spatial frequency coverage of d(k) is shown in FIG. is called the cumulative NA coverage.

[0120] Figure 5 The first row shows the numerical aperture of the illumination part of the optical system under mismatched illumination conditions (NA i = 0.3, the numerical aperture NA of the detector part of the optical system d = 0.6) DPC (see Figure 2 ). Although DPC can in principle achieve NA i +NA d The cumulative NA radius ( Figure 5 The dashed radius in the middle of the first row), but the center of the NA coverage exhibits a radius of approximately NA d -NA i This in turn causes the resulting phase reconstruction to exhibit low contrast (as is evident from the phase contrast image 501).

[0121] TIE - such as Figure 5 The second line (see Figure 3 ) - axial defocus is used. When evaluating the corresponding phase transfer function ( Figure 5, second row, left column), it can be seen that the NA coverage reaches lower values ​​in k-space compared to DPC (simply, the black circle around the center of k-space is smaller). This results in improved contrast in the final phase reconstruction compared to DPC (see phase contrast image 502). However, since only a point source on a single axis is used (see Figure 3 ), the illumination NA of the standard TIE phase contrast system disappears. Therefore, the NA coverage only reaches NA d bandwidth (simply put, the white ring has a finite radius), which has a significant impact on the achievable lateral resolution (given by λ / [NA i +NA d ], where λ is the wavelength). Better NA coverage can be achieved by combining the TIE phase reconstruction algorithm with the DPC phase reconstruction algorithm using appropriate bandpass filtering (see Equation 4).

[0122] HDD Figure 5 The third line of (see Figure 4 ). By synthesizing the PTF from a variable illumination pattern and defocus, a wider range of spatial frequencies can be covered (the rings in the middle column have a smaller inner hole and a larger radius). The resulting phase reconstruction (right column) has superior phase contrast compared to DPC while achieving higher resolution compared to TIE (see phase contrast image 503). The black regions in the PTF require regularization.

[0123] Figure 6 is a flow chart of a method according to various examples. Figure 6 The method involves determining a phase contrast image. This is based on the HDD technique disclosed in this article. The HDD technique uses digital post-processing of multiple intensity microscopy images to reconstruct phase contrast.

[0124] Figure 6 The method can be implemented using a processor based on program code loaded from a memory. For example, after loading and executing the program code stored in the memory 83, the processor 81 of the computer 80 can execute Figure 6 method.

[0125] At optional block 605, a sample is loaded into a microscope. For example, a biological sample including a cell culture may be loaded into a microscope. Loading a sample into a microscope may include securing a sample holder to a sample stage of the microscope.

[0126] At optional block 610, an angular illumination configuration is enabled. Depending on the specific implementation of the illumination module, block 610 can take different forms. For example, the aperture wheel can be positioned at a certain rotational position. A certain LED pattern of the LED array of the PIU can also be enabled, for example, depending on the desired angular spectrum. The digital micromirrors of the DMD can also be reconfigured. The pixels of the LCD can be turned on or off. The angular illumination configuration enabled at block 610 provides full-area illumination of the imaging plane at a range of illumination angles. In this way, the brightness level does not change significantly on the imaging plane (within the aperture of the system), and the illumination angle observed at different field positions perpendicular to the optical axis does not change.

[0127] At block 610, multiple types of angular lighting configurations may be enabled. The multiple types may be characterized by different widths of the corresponding angular spectrums. For example, a first type of angular lighting configuration that may be enabled at block 610 includes a range of lighting directions. Accordingly, the angular spectrum is relatively wide. A second type of angular lighting configuration that may be enabled at block 610 includes a single lighting direction. Accordingly, the angular spectrum is relatively narrow. This may be achieved by enabling a plurality of angular lighting configurations, such as a plurality of angular lighting configurations, and a plurality of angular lighting configurations. Figure 7 Shown in. Figure 7 A plurality of angle spectra 701, 702, 711-714 are shown. The angle spectrum shows the distribution of the angle of light incidence on the entire image plane (see Figure 7 , where light 762 is incident into image plane 761 and the corresponding angle 763 is highlighted).

[0128] Figure 7 It is shown that the angular spectrum 701, 702 includes a series of lighting directions. Here, "series of lighting directions" means that the lighting module used is capable of providing lighting using a smaller angular spectrum; however, the lighting module is controlled to illuminate the imaging plane using a wider angular spectrum (i.e., by combining multiple lighting directions that can be switched separately by the lighting module). For example, for a scene with an LED array, this means that multiple LEDs (each LED defining a lighting direction) are lit at the same time.

[0129] Differently, each of the angular spectra 711-714 comprises a single illumination direction. However, each angular spectrum 711-714 has a certain width 751 - which is significantly smaller than the width 752 of the angular spectra 701, 702. This limited width 751 is due to the extension of the smallest light aperture of the light module in the illumination pupil plane, for example, the emission area of ​​a light emitting diode or the pixel size of a liquid crystal display or the extension of the micromirrors of a micromirror device. The hardware of the light module does not allow providing illumination configurations with narrower widths than the angular spectra 711-714; therefore, these can be said to comprise a single illumination direction.

[0130] Figure 7The bottom portion shows the illumination pattern 202 of a programmable light source (LED array) for an angular spectrum 701, 702, 711-714, (the optical axis 207 is shown as an open circle in each figure).

[0131] As can be appreciated, the illumination configuration is associated with an angular spectrum 701, 702 corresponding to a DPC type illumination; while the illumination configuration is associated with an angular spectrum 711-714 corresponding to a TIE type illumination. This technique is based on the discovery that illumination configurations that provide a range of illumination directions on the one hand and illumination configurations that provide a single illumination direction (or more generally, wide angular spectrums and narrow angular spectrums) on the other hand have different advantages and disadvantages: a wide angular spectrum provides high resolution in a phase contrast image; a narrow angular spectrum provides high contrast in a phase contrast image. By combining wide and narrow angular spectra, high contrast and high resolution can be combined.

[0132] At optional block 615, one or more elements of the microscope are controlled to enable a certain defocus value. For example, block 615 may include repositioning a sample stage for holding a sample along an optical path or optical axis. That is, z-repositioning may be implemented. However, it is not necessary to enable a certain defocus value in all scenarios. In some cases, multiple cameras may capture images of an imaging plane at different offsets relative to the imaging plane; thereby achieving different defocus values. Another option is to use an electrically configurable lens (e.g., an objective lens or a tube lens) to achieve a certain defocus value at block 615.

[0133] Then, an image is acquired at block 620. The image is acquired with a combination of an angular illumination configuration and a defocus value according to the previous execution instances of blocks 610 and 615. It is generally noted that in some scenarios, image reconstruction may be performed on the pre-acquired image (which will be explained later in conjunction with block 635), so block 620 is also optional. An example sequence of acquired images is disclosed above in conjunction with Table 1.

[0134] At block 625, it is checked whether a further defocus value is required; if so, a further iteration 626 of blocks 615 and 620 is performed, selecting another defocus value. This corresponds to increasing the index n in the above formula.

[0135] If all defocus values ​​have been acquired for a certain angular illumination configuration, the method starts at block 630. Here, the index n is reinitialized and then it is checked at block 630 whether a further angular illumination configuration is needed. If so, a further iteration 631 of blocks 610, 615, 620, 625 is performed. In addition, the index m in the above formula is increased.

[0136] It should be noted that although Figure 6In the scenario of FIG. 6 , loop 626 is the inner loop and loop 631 is the outer loop, and other scenarios of switching between different defocus values ​​and angular lighting configurations are also conceivable.

[0137] Once all images (ie, all different combinations of index m and index n) are acquired at block 620 , image reconstruction begins at block 635 .

[0138] There are a number of options for implementing block 635. In one example (block 636), the joint phase reconstruction includes images acquired under different illumination configurations and different defocus values. This includes solving an optimization problem, such as using a deconvolution analysis as described above. Figure 4 An exemplary optimization problem is disclosed. Optional regularization may be considered. In another example (block 637), separate phase reconstruction algorithms (such as previously combined with Figure 2 and Figure 3 Then, a downstream combination of the TIE phase contrast image and the DPC phase contrast image can be performed to obtain a final phase contrast image.

[0139] Then, once the phase contrast image has been determined, the phase contrast image may optionally be output to a user at block 640, for example, via a computer screen or by uploading the image to a server.

[0140] In summary, at least the following examples are disclosed.

[0141] Example 1. A method of determining a phase contrast image based on a plurality of microscope images acquired using a microscope, the microscope comprising an illumination module configured to provide switchable angle illumination of an imaging plane of the microscope, the microscope further comprising an optical system to illuminate the imaging plane and image the imaging plane onto at least one camera of the microscope,

[0142] Wherein, the method comprises:

[0143] - controlling the lighting module to enable a plurality of angular lighting configurations, at least one of the plurality of angular lighting configurations comprising a corresponding series of lighting directions,

[0144] - controlling the at least one camera to acquire a plurality of images at a plurality of angular lighting configurations and at a plurality of defocus values, and

[0145] -Combining multiple images to determine a phase contrast image.

[0146] Example 2. The method of Example 1 further includes:

[0147] - applying the plurality of defocus values ​​by at least one of: moving the sample stage along the optical path of the optical system, by using a plurality of cameras arranged along the optical path of the optical system, or by controlling an electrically tunable lens.

[0148] Example 3. The method of Example 1 or 2,

[0149] Wherein at least some of the plurality of angular illumination configurations are multiplexed in polarization space or wavelength space.

[0150] Example 4. The method of any of the preceding examples,

[0151] The illumination module includes at least one of a programmable LED array, a digital micromirror device, a liquid crystal display, a variable focusing aperture diaphragm, an angle diffuser, or an aperture wheel.

[0152] Example 5. The method of any of the preceding examples,

[0153] wherein combining the plurality of images comprises calculating a pairwise difference between each of the plurality of images included in the first subset using a first phase reconstruction algorithm, thereby obtaining a first temporary phase contrast image,

[0154] wherein combining the plurality of images further comprises calculating a pairwise difference between each of the plurality of images included in the second subset using a second phase reconstruction algorithm, thereby obtaining a second temporary phase contrast image,

[0155] Wherein the method further comprises:

[0156] - combining first image information associated with the first temporary phase contrast image and second image information associated with the second temporary phase contrast image, thereby obtaining a phase contrast image.

[0157] Example 6. The method of Example 5, further comprising:

[0158] - applying a low pass filter to the first temporary phase contrast image, thereby obtaining first image information, and

[0159] - applying a high pass filter to the second temporary phase contrast image, thereby obtaining second image information.

[0160] Example 7. The method of Example 5 or 6,

[0161] wherein the first subset consists of a first image from the plurality of images and a second image from the plurality of images,

[0162] The first image contained in the first subset is at a positive defocus value,

[0163] The second images contained in the first subset are at negative defocus values,

[0164] The first image and the second image included in the first subset are both acquired under the same angular illumination configuration.

[0165] Example 8. The method of any one of Examples 5 to 7,

[0166] The second subset consists of a first image from the plurality of images, a second image from the plurality of images, and a third image from the plurality of images,

[0167] The first image contained in the second subset, the second image contained in the second subset, and the third image contained in the second subset are all acquired under the same defocus value and different angular illumination configurations.

[0168] Example 9. The method of any one of Examples 5 to 8,

[0169] Therein, at least one of the first phase reconstruction algorithm or the second phase reconstruction algorithm is based on a corresponding predefined optical transfer function, which is independent of the characteristics of the microscope.

[0170] Example 10. The method of any one of Examples 1 to 4,

[0171] The combining of the plurality of images comprises solving an optimization problem that depends on differences between the plurality of image pairs and also on associated differences of respective phase transfer functions, wherein the phase transfer function is determined for each combination of a plurality of angular illumination configurations and a plurality of defocus values.

[0172] Example 11. The method of Example 10, further comprising:

[0173] - determining a phase transfer function for each combination of a plurality of angular illumination configurations and a plurality of defocus values ​​based on a parameterized template stored in a database.

[0174] Example 12. The method of Example 11, further comprising:

[0175] - calculating a phase transfer function for each combination of a plurality of angular illumination configurations and a plurality of defocus values ​​based on a weak object transfer function form comprising a two-dimensional integral over the entire light field at a pupil plane and a source plane.

[0176] Example 13. The method of Example 12,

[0177] The weak transfer function form takes into account one or more aberrations of the optical device.

[0178] Example 14. The method of any one of Examples 10 to 13,

[0179] Deconvolution analysis is used to solve the optimization problem.

[0180] Example 15. The method of Example 14,

[0181] The deconvolution analysis takes the regularization term into account.

[0182] Example 16. The method of any of the preceding examples,

[0183] Wherein at least one of the plurality of angular lighting configurations further comprises a single lighting direction.

[0184] Example 17. The method of any of the preceding examples,

[0185] The multiple angular lighting configurations have angular spectra of multiple widths.

[0186] Example 18. The method of any of the preceding examples,

[0187] The magnitude of each of the plurality of defocus values ​​is related to the depth of field of the detector optics of the microscope by a factor in the range of 1 to 10.

[0188] Example 19. A method of determining a phase contrast image based on a plurality of microscope images acquired using a microscope, the microscope comprising an illumination module configured to provide switchable angle illumination of an imaging plane of the microscope, the microscope further comprising an optical system to illuminate the imaging plane and image the imaging plane onto at least one camera of the microscope,

[0189] Wherein, the method comprises:

[0190] - controlling the lighting module to enable a plurality of angular lighting configurations, the plurality of angular lighting configurations having angular spectra of a plurality of widths,

[0191] - controlling the at least one camera to acquire a plurality of images at a plurality of angular lighting configurations and at a plurality of defocus values, and

[0192] -Combining multiple images to determine a phase contrast image.

[0193] Example 20. At least one processing device, comprising a memory storing a program code and a processor configured to load the program code and execute the program code, the processor being configured to perform the method of any one of Examples 1 to 19 when executing the program code.

[0194] Example 21. Program code executable by a processor, the processor being configured to perform the method of any one of Examples 1 to 19 when executing the program code.

[0195] Although the invention has been shown and described with respect to certain preferred embodiments, equivalents and modifications will occur to those skilled in the art upon the reading and understanding of the specification. The present invention includes all such equivalents and modifications, and is limited only by the scope of the appended claims.

[0196] For illustration, the above describes a scenario in which multiple angular lighting configurations include at least one angular lighting configuration including multiple lighting directions. More generally, a multi-angle lighting configuration used in conjunction with multiple defocus values ​​may have angular spectra of different widths, such as a wide angular spectrum and a narrow angular spectrum (see Figure 6 ).

[0197] appendix

[0198] Regarding Formula 2, H DPC (k) = exp(j·arctan(k y / k x )) has been used as the Hilbert transform kernel. The motivation for this formulation is as follows.

[0199] First, consider a one-dimensional DPC. The original images (e.g., the left portion of the LED array and the right portion of the LED array) have a DIC image-like appearance. DIC or DPC phase contrast images are related to phase gradients. By subtracting the images from the left and right half illuminations, the gradient dphase / dx can be approximated. In the Fourier domain, this is equivalent to multiplying by jk…

[0200] FT(dphase / dx)=j·k·FT(phase)

[0201] An approximation can be made, the essential feature of which is that the premultiplier k changes sign at the origin of the coordinate system. This is the main feature of the transfer function. Therefore, it is roughly as follows:

[0202] FT(dphase / dx)=sign(k)·FT(phase)

[0203] Next, let’s look at it from two dimensions:

[0204]

[0205] where φ is k x , k y The polar angle in the plane. In summary, the arctan operationally takes over the role of the sign change that the strict transfer function does.

[0206] References

[0207] [1]Zernike,Frits."Phase contrast,a new method for the microscopicobservation of transparent objects part II."Physica 9.10(1942):974-986.

[0208] [2]Gabor,Dennis."Microscopy by reconstructed wave-fronts."Proceedingsof the Royal Society of London.Series A.Mathematical and Physical Sciences197.1051(1949):454-487.

[0209] [3]Nomarski,Georges."Microinterféromètre différentielàondes polarisées."J.Phys.Rad.16(1955):9S-13S.

[0210] [4]Streibl,Norbert."Phase imaging by the transport equation ofintensity."Optics communications 49.1(1984):6-10.

[0211] [5]Mehta,Shalin B.,and Colin JR Sheppard."Quantitative phase-gradientimaging at high resolution with asymmetric illumination-based differentialphase contrast."Optics letters 34.13(2009):1924-1926.

[0212] [6]Tian,Lei,and Laura Waller."Quantitative differential phasecontrast imaging in anLEDarray microscope."Optics express 23.9(2015):11394-11403.

[0213] [7] Zhou, Shun, et al. "Transport-of-intensity Fourier ptychographicdiffraction tomography:defying the matched illumination condition." Optica9.12(2022):1362-1373.

[0214] [8] Amplitude filter loop according to PCT / EP2020 / 050307 (10 2019 100 419.5)

Claims

1. A method for determining a phase contrast image based on a plurality of microscope images acquired using a microscope, the microscope comprising an illumination module configured to provide switchable angular illumination of an imaging plane of the microscope, the microscope further comprising an optical system to illuminate the imaging plane and to image the imaging plane onto at least one camera of the microscope, in, The method comprises: - controlling the lighting module to enable a plurality of angular lighting configurations, at least one of the plurality of angular lighting configurations comprising a corresponding series of lighting directions, - controlling said at least one camera to acquire a plurality of images at said plurality of angular lighting configurations and at a plurality of defocus values, and - combining the multiple images to determine a phase contrast image.

2. The method of claim 1, further comprising: - applying the plurality of defocus values ​​by at least one of: moving a sample stage along an optical path of the optical system, by using a plurality of cameras arranged along the optical path of the optical system, or by controlling an electrically tunable lens.

3. The method according to claim 1 or 2, Wherein at least some of the plurality of angular illumination configurations are multiplexed in polarization space or wavelength space.

4. A method as claimed in any one of the preceding claims, in, The illumination module includes at least one of a programmable LED array, a digital micromirror device, a liquid crystal display, a variable focusing aperture stop, an angular diffuser, or an aperture wheel.

5. A method as claimed in any one of the preceding claims, wherein combining the plurality of images comprises calculating a pairwise difference between each of the plurality of images included in the first subset using a first phase reconstruction algorithm, thereby obtaining a first temporary phase contrast image, wherein combining the plurality of images further comprises calculating a pairwise difference between each of the plurality of images included in the second subset using a second phase reconstruction algorithm, thereby obtaining a second temporary phase contrast image, Wherein the method further comprises: - combining first image information associated with the first temporary phase contrast image and second image information associated with the second temporary phase contrast image, thereby obtaining the phase contrast image.

6. The method of claim 5, further comprising: - applying a low pass filter to the first temporary phase contrast image, thereby obtaining the first image information, and - applying a high pass filter to the second temporary phase contrast image, thereby obtaining the second image information.

7. The method according to claim 5 or 6, wherein the first subset consists of a first image in the plurality of images and a second image in the plurality of images, The first images contained in the first subset are at positive defocus values, The second images contained in the second subset are at negative defocus values, The first image and the second image contained in the first subset are both acquired under the same angular illumination configuration.

8. The method according to any one of claims 5 to 7, in, the second subset consists of a first image of the plurality of images, a second image of the plurality of images, and a third image of the plurality of images, The first image contained in the second subset, the second image contained in the second subset, and the third image contained in the second subset are all acquired under the same defocus value and under different angular illumination configurations.

9. The method according to any one of claims 5 to 8, in, At least one of the first phase reconstruction algorithm or the second phase reconstruction algorithm is based on a respective predefined optical transfer function which is independent of properties of the microscope.

10. The method according to any one of the preceding claims, Wherein at least another of the plurality of angular lighting configurations comprises a single lighting direction.

11. A method as claimed in any one of the preceding claims, The multiple angular lighting configurations have angular spectra of multiple widths.

12. A method as claimed in any one of the preceding claims, The magnitude of each of the plurality of defocus values ​​is related to the depth of field of the detector optics of the microscope by a factor in the range of 1 to 10.

13. A method of determining a phase contrast image based on a plurality of microscope images acquired using a microscope, the microscope comprising an illumination module configured to provide switchable angular illumination of an imaging plane of the microscope, the microscope further comprising an optical system to illuminate the imaging plane and image the imaging plane onto at least one camera of the microscope, in, The method comprises: - controlling the lighting module to enable a plurality of angular lighting configurations, the plurality of angular lighting configurations having angular spectra of a plurality of widths, - controlling said at least one camera to acquire a plurality of images at said plurality of angular lighting configurations and at a plurality of defocus values, and - combining the multiple images to determine a phase contrast image.

14. At least one processing device, comprising a memory storing a program code and a processor configured to load the program code and execute the program code, wherein the processor is configured to perform the method of any one of claims 1 to 13 when executing the program code.

15. Program code executable by a processor, the processor being configured to perform the method of any one of claims 1 to 13 when executing the program code.

Images