Observation device and observation method

Abstract

An illumination unit 31 illuminates an object to be observed with first illumination light along a first illumination direction under a dark-field illumination condition and illuminates the object to be observed with second illumination light along a second illumination direction under a bright-field illumination condition in a first illumination mode, and illuminates the object to be observed with the second illumination light along the second illumination direction under the bright-field illumination condition in a second illumination mode. A processing unit 60 generates a first complex amplitude image on the basis of an intensity image at each of a plurality of focal planes acquired in the first illumination mode, generates a second complex amplitude image on the basis of an intensity image at each of a plurality of focal planes acquired in the second illumination mode, and generates a complex amplitude image when the object to be observed is illuminated with light along the first illumination direction under the dark-field illumination condition on the basis of the difference between the first complex amplitude image and the second complex amplitude image. Thus, an observation device capable of easily acquiring a complex amplitude image of an object to be observed in dark-field illumination with a simple configuration is achieved.

Description

Observation Device and Observation Method

[0001] The present disclosure relates to an observation device and an observation method.

[0002] In the observation of an observation object by a microscope, there is generally a trade-off relationship between resolution and field of view. That is, when attempting to observe an observation object with high resolution, the field of view becomes narrow, and when attempting to observe an observation object with a wide field of view, the resolution becomes low. On the other hand, there are applications (such as pathological slide observation) that require the observation of an observation object with a wide field of view and high resolution.

[0003] As a technique for meeting the requirement of wide-field and high-resolution observation, there is a technique in which the stage on which the observation object is placed is moved to acquire images of the observation object at a plurality of positions, and then these plurality of images are stitched together. However, this technique has a problem in that it takes a long time to acquire a plurality of images because it requires mechanical movement of the stage.

[0004] Also, as another technique for meeting the requirement of wide-field and high-resolution observation, there is a synthetic aperture method that can acquire a high-resolution image even when using a low-magnification objective lens (Non-Patent Document 1). The synthetic aperture method does not require mechanical movement of the stage.

[0005] The synthetic aperture method illuminates light along each of a plurality of illumination directions with respect to an observation object to acquire images, and then synthesizes these plurality of images in the frequency space to generate a wide-field and high-resolution image. The synthetic aperture method utilizes the fact that high-frequency components in the frequency space correspond to high resolution in the real space, and that a fine sampling interval in the frequency space corresponds to a wide field of view in the real space.

[0006] Since the synthetic aperture method synthesizes a plurality of images in the frequency space, it is necessary to acquire each of these plurality of images as a complex amplitude image. Also, in order to obtain a higher-resolution image, it is necessary to increase the incident angle of light on the observation object to acquire a complex amplitude image having high-frequency components.

[0007] Therefore, it is desirable to perform bright-field illumination on the object to obtain multiple bright-field images, as well as dark-field illumination on the object to obtain multiple dark-field images, and then use these multiple bright-field and dark-field images for composite aperture processing. When the incident NA is less than or equal to the detected NA, a bright-field image is obtained, and when the incident NA is greater than the detected NA, a dark-field image is obtained.

[0008] In addition to two-beam interferometry, there is a technique for acquiring complex amplitude images of an object being observed that uses the transport of intensity equation (TIE) (Non-Patent Literature 2). The technique using TIE acquires intensity images at multiple z-positions (positions on the z-axis parallel to the optical axis of the objective lens) of the object being observed, and generates a complex amplitude image using TIE based on these multiple intensity images.

[0009] Moonseok Kim et al., "High-speed synthetic aperture microscopy for live cell imaging", OPTICS LETTERS, Vol.36, No.2, pp.148-150, 2011Chao Zuo et al., "Transport of intensity equation: a tutorial", Optics and Lasers in Engineering, Vol.135, 106187, 2020

[0010] Two-beam interferometry can acquire complex amplitude images containing phase information, not only under bright-field illumination but also under dark-field illumination. However, two-beam interferometry has drawbacks: the optical system is not easily adjustable, it has stability issues, and the equipment is complex and expensive.

[0011] Techniques using TIE can overcome the problems of two-beam interferometry described above. However, while TIE techniques can acquire bright-field images relatively accurately under bright-field illumination, when attempting to acquire dark-field images under dark-field illumination, the error in phase calculation by TIE becomes large, making it difficult to accurately obtain complex amplitude images.

[0012] The requirement for acquiring complex amplitude images (dark-field images) of an object under dark-field illumination is not limited to synthetic aperture processing.

[0013] The embodiment aims to provide an observation device and observation method that can easily acquire complex amplitude images of an object under dark-field illumination with a simple configuration.

[0014] The embodiment is an observation device. The observation device comprises (1) a light source that outputs light, (2) an illumination unit that, in a first illumination mode, illuminates an object to be observed with the first illumination light along a first illumination direction under dark-field illumination conditions and the second illumination light along a second illumination direction under bright-field illumination conditions, with the difference in optical path length between the first illumination light and the second illumination light generated based on the light output from the light source being less than or equal to the coherent length, and in a second illumination mode, illuminates the object to be observed with the second illumination light along a second illumination direction under bright-field illumination conditions, and (3) an illumination unit that, in each of the first illumination mode and the second illumination mode, displays intensity images at each of the multiple focus planes of the object to be observed. The system comprises (4) an imaging unit for acquiring images, and (5) a processing unit for processing based on the intensity images acquired by the imaging unit. The processing unit generates a first complex amplitude image using an intensity transport equation based on the intensity images at each of a plurality of focus planes acquired by the imaging unit in the first illumination configuration, and generates a second complex amplitude image using an intensity transport equation based on the intensity images at each of a plurality of focus planes acquired by the imaging unit in the second illumination configuration. The processing unit generates a complex amplitude image when light is shone on an object along a first illumination direction under dark-field illumination conditions, based on the difference between the first complex amplitude image and the second complex amplitude image.

[0015] The embodiment is an observation method. The observation method includes (1) a first intensity image acquisition step in which, in a first illumination configuration, the illumination unit illuminates the object to be observed with the first illumination light along a first illumination direction under dark-field illumination conditions and the second illumination light along a second illumination direction under bright-field illumination conditions, and the imaging unit acquires intensity images at each of the multiple focus planes of the object to be observed; and (2) a second illumination configuration in which the illumination unit illuminates the object to be observed with the second illumination light along a second illumination direction under bright-field illumination conditions and the imaging unit acquires intensity images at each of the multiple focus planes of the object to be observed. The system comprises: (3) a second intensity image acquisition step for acquiring an intensity image; (4) a first complex amplitude image generation step for generating a first complex amplitude image using an intensity transport equation based on the intensity images at each of the multiple focus planes acquired in the first intensity image acquisition step; (5) a second complex amplitude image generation step for generating a second complex amplitude image using an intensity transport equation based on the intensity images at each of the multiple focus planes acquired in the second intensity image acquisition step; and (6) a dark-field image generation step for generating a complex amplitude image when light is shone on an object to be observed along a first illumination direction under dark-field illumination conditions, based on the difference between the first complex amplitude image and the second complex amplitude image.

[0016] According to the observation apparatus and observation method of the embodiment, complex amplitude images of the object to be observed under dark-field illumination can be easily obtained with a simple configuration.

[0017] Figure 1 shows the configuration of the observation device 1. Figure 2 shows the configuration of the illumination unit 32. Figure 3 shows the configuration of the illumination unit 33. Figure 4 shows the configuration of the illumination unit 33A. Figure 5 shows the configuration of the illumination unit 33B. Figure 6 is a flowchart of the observation method. Figure 7 shows the intensity image I(r) acquired in the first illumination mode during the first intensity image acquisition step S1, and consists of (a) the intensity image at z = -50λ and (b) the intensity image at z = +50λ. Figure 8 shows the intensity image I(r) acquired in the second illumination mode during the second intensity image acquisition step S2, and consists of (a) the intensity image at z = -50λ and (b) the intensity image at z = +50λ. Figure 9 shows the images generated in the first complex amplitude image generation step S3 under the first illumination mode, and consists of (a) the image shown on the left side of the TIE, (b) the phase image φ(r) obtained by the TIE calculation, and (c) the real part of the complex amplitude image. Figure 10 shows the images generated in the second complex amplitude image generation step S4 under the second illumination mode, and consists of (a) the image shown on the left side of the TIE, (b) the phase image φ(r) obtained by the TIE calculation, and (c) the real part of the complex amplitude image. Figure 11 shows (a) the real part of the dark-field image generated in the dark-field image generation step S5, (b) the exact solution of the dark-field image, and (c) the difference between the dark-field image (Figure 11(a)) and the exact solution (Figure 11(b)) generated in the dark-field image generation step S5. Figure 12 shows the distribution of illumination directions in wavenumber space for each of the dark-field illumination conditions and bright-field illumination conditions. Figure 13 shows the exact solution obtained by performing a composite aperture processing based on multiple darkfield images and multiple brightfield images, and (a) is a phase image and (b) is an image showing the intensity distribution in wavenumber space. Figure 14 shows the brightfield image obtained when the illumination direction under brightfield illumination conditions is set to the vertical direction, and (a) is a phase image and (b) is an image showing the intensity distribution in wavenumber space. Figure 15 shows the image obtained by performing a composite aperture processing based on multiple brightfield images obtained for each of multiple illumination directions under brightfield illumination conditions in the brightfield image generation step, and (a) is a phase image and (b) is an image showing the intensity distribution in wavenumber space.Figure 16 shows an image obtained by performing synthetic aperture processing based on multiple bright-field images and multiple dark-field images in the synthetic aperture processing step, and (a) is a phase image, and (b) is an image showing the intensity distribution in wavenumber space. Figure 17 is an image showing the difference between the image obtained by performing synthetic aperture processing based on multiple bright-field images and multiple dark-field images in the synthetic aperture processing step (Figure 16) and the exact solution (Figure 13), and (a) is a phase image, and (b) is an image showing the intensity distribution in wavenumber space. Figure 18 shows the intensity image I(r) obtained in the first illumination mode in the first intensity image acquisition step S1, and (a) is an intensity image at z = 950λ, and (b) is an intensity image at z = 1050λ. Figure 19 shows the intensity image I(r) obtained in the second illumination mode in the second intensity image acquisition step S2, and (a) is an intensity image at z = 950λ, and (b) is an intensity image at z = 1050λ. Figure 20 shows the images generated in the first complex amplitude image generation step S3 for the first illumination mode, and consists of (a) the image shown on the left side of the TIE, (b) the phase image φ(r) obtained by the TIE calculation, and (c) the real part image of the complex amplitude image. Figure 21 shows the images generated in the second complex amplitude image generation step S4 for the second illumination mode, and consists of (a) the image shown on the left side of the TIE, (b) the phase image φ(r) obtained by the TIE calculation, and (c) the real part image of the complex amplitude image. Figure 22 shows (a) the real part image of the darkfield image generated in the darkfield image generation step S5, and (b) the exact solution of the darkfield image. Figure 23 shows the exact solution of the darkfield image at z=0, and consists of (a) the phase image and (b) an image showing the intensity distribution in wavenumber space. Figure 24 shows an image at z = 1000λ obtained by performing a composite aperture processing based on multiple bright-field images obtained in each of multiple illumination directions under bright-field illumination conditions during the bright-field image generation step, and (a) is a phase image, and (b) is an image showing the intensity distribution in wavenumber space. Figure 25 shows an image at z = 1000λ obtained by performing a composite aperture processing based on multiple bright-field images and multiple dark-field images during the composite aperture processing step, and (a) is a phase image, and (b) is an image showing the intensity distribution in wavenumber space.Figure 26 shows an image obtained by wavefront propagation in the wavefront step, where multiple bright-field images and multiple dark-field images at z = 1000λ are wavefront propagated to generate a complex amplitude image at z = 0, and then performing a composite aperture processing based on the multiple complex amplitude images at z = 0. The image shows (a) a phase image and (b) an image showing the intensity distribution in wavenumber space.

[0018] Embodiments of the observation apparatus and observation method will be described in detail below with reference to the attached drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant descriptions are omitted. The present invention is not limited to these examples, but is indicated by the claims, and all modifications within the meaning and scope equivalent to the claims are intended to be included.

[0019] First, we will explain an example of the optical system configuration of the observation device using Figures 1 to 5, and then we will explain the detailed contents of the processing in the observation device and observation method.

[0020] Figure 1 shows the configuration of the observation device 1. This observation device 1 includes a light source 10, an illumination unit 31, an imaging unit 50, and a processing unit 60, etc. The light source 10 outputs spatially coherent light. The light source 10 may be a laser light source, or it may be a light source such as an SLD (Super Luminescent Diode), SC (Super Continuum) light source, or optical frequency comb light source. Alternatively, spatially incoherent light output from an LED (Light Emitting Diode) or mercury lamp may be passed through a pinhole or the like to enhance spatial coherence.

[0021] Lens 21 is optically connected to light source 10 and focuses the light output from light source 10 onto the optical incident end 22 of optical fiber 23, causing the light to enter the optical incident end 22. Optical fiber 23 guides the light that entered the optical incident end 22 to the optical exit end 24. The light guided by optical fiber 23 is emitted as divergent light from the optical exit end 24. Lens 25 is optically connected to the optical exit end 24 and receives the light output as divergent light from the optical exit end 24, collimates it, and outputs the collimated light to the illumination unit 31.

[0022] The illumination unit 31 receives light output from the light source 10, which has passed through the lens 21, optical fiber 23, and lens 25. The illumination unit 31 can switch between a first illumination mode and a second illumination mode.

[0023] In the first illumination mode, the illumination unit 31 illuminates the object to be observed S with the first illumination light along the first illumination direction under dark-field illumination conditions, and with the difference in optical path length between the first illumination light and the second illumination light generated based on the light output from the light source 10 being less than or equal to the coherence length. In the second illumination mode, the illumination unit 31 illuminates the object to be observed S with the second illumination light along the second illumination direction under bright-field illumination conditions.

[0024] The illumination unit 31 includes a beam splitter 311, a phase-modulated spatial light modulator 313, a polarizer 314, a half-wave plate 315, a polarizer 316, a lens 318, and an objective lens 319.

[0025] The beam splitter 311 reflects the light that has arrived via the polarizer 314 and half-wave plate 315, which are placed between it and the lens 25, to the spatial light modulator 313. The beam splitter 311 also receives the light that has arrived from the spatial light modulator 313 and outputs this light to the polarizer 316.

[0026] The spatial light modulator 313 selectively phase-modulates the linearly polarized light of the first direction from among the linearly polarized light of the first and second directions, which are mutually orthogonal, that is incident on the modulation plane. The polarizer 314 and half-wave plate 315 set the polarization state of the light so that the light incident from the beam splitter 311 to the modulation plane of the spatial light modulator 313 contains linearly polarized components of the first and second directions to an equal degree.

[0027] The polarizer 316 receives light that has arrived from the spatial light modulator 313 via the beam splitter 311 and makes it possible to interfere with the linearly polarized light of the first and second directions contained in that light. The polarizer 316 has an optical axis that is 45 degrees different in direction from the polarization direction of the light that has arrived from the spatial light modulator 313 via the beam splitter 311 (linearly polarized light of the first and second directions), and selectively transmits the polarization component of the input light in the direction of the optical axis. The lens 318 and the objective lens 319 illuminate the object to be observed S with the light output from the polarizer 316 (first illumination light and second illumination light) as a plane wave.

[0028] The illumination unit 31 having such a configuration can illuminate an object S under dark-field illumination conditions along a first illumination direction using linearly polarized light of a first direction, which has been phase-modulated by the spatial light modulator 313, as the first illumination light. The illumination unit 31 can also illuminate an object S under bright-field illumination conditions along a second illumination direction using linearly polarized light of a second direction, which has not been phase-modulated by the spatial light modulator 313, as the second illumination light.

[0029] The direction of illumination of the object to be observed S with the first illumination light can be set by the period of the phase modulation pattern on the modulation plane of the spatial light modulator 313. Therefore, if the numerical aperture of the illumination-side objective lens 319 is greater than the numerical aperture of the detection-side objective lens 41, in the second illumination configuration, the illumination unit 31 can illuminate the object to be observed S with only the second illumination light without illuminating the object to be observed S with the first illumination light by adjusting the period of the phase modulation pattern.

[0030] The objective lens 41 receives light that has been illuminated by the illumination unit 31 onto the object to be observed S and has reached the object to be observed S, and outputs that light to the mirror 42. The lens 43 receives the light that has been output from the objective lens 41 and reflected by the mirror 42, and causes that light to enter the imaging surface of the imaging unit 50.

[0031] The imaging unit 50 receives light that has reached the imaging surface from the lens 43. In both the first illumination mode and the second illumination mode, the imaging unit 50 acquires intensity images at each of the multiple focal planes of the object to be observed S. In order to acquire intensity images at each of the multiple focal planes of the object to be observed S, the position of the imaging unit 50 in the optical axis direction may be adjusted by the position adjustment unit 51 (e.g., a stage or a piezo actuator), or the position of the object to be observed S, the objective lens 41, or the lens 43 in the optical axis direction may be adjusted.

[0032] The processing unit 60 is electrically connected to the imaging unit 50 and performs processing based on the intensity image acquired by the imaging unit 50. The processing details of the processing unit 60 will be described later.

[0033] The observation device can also have other configurations. In particular, the illumination unit can have various configurations other than the illumination unit 31 described above. Figures 2 to 5 show examples of other configurations of the illumination unit.

[0034] Figure 2 shows the configuration of the illumination unit 32. The illumination unit 32 includes a beam splitter 321, a mirror 322, an intensity-modulated spatial light modulator 323, a lens 328, and an objective lens 329. The spatial light modulator 323 may be a DMD (Digital Micromirror Device).

[0035] The beam splitter 321 splits the light arriving from the lens 25 into a first-branched beam and a second-branched beam. The first-branched beam is output to the spatial light modulator 323, and the second-branched beam is output to the mirror 322. The beam splitter 321 also receives the first-branched beam, which has been intensity-modulated by the spatial light modulator 323, and the second-branched beam, which has been reflected by the mirror 322, and outputs these first-branched and second-branched beams to the lens 328. The lens 328 and the objective lens 329 illuminate the object S to be observed with the first-branched beam and the second-branched beam, respectively, output from the beam splitter 321 as plane waves.

[0036] The illumination unit 32 having such a configuration can illuminate the object to be observed S along a first illumination direction under dark-field illumination conditions using the first branched light, whose intensity has been modulated by the spatial light modulator 323, as the first illumination light. The illumination unit 32 can also illuminate the object to be observed S along a second illumination direction under bright-field illumination conditions using the second branched light, whose intensity has been reflected by the mirror 322, as the second illumination light.

[0037] The direction of illumination of the object to be observed S with the first illumination light can be set by the period of the intensity modulation pattern on the modulation plane of the spatial light modulator 323. Therefore, if the numerical aperture of the illumination-side objective lens 329 is greater than the numerical aperture of the detection-side objective lens 41, in the second illumination configuration, the illumination unit 32 can illuminate the object to be observed S with only the second illumination light without illuminating the object to be observed S with the first illumination light by adjusting the period of the intensity modulation pattern.

[0038] Figure 3 shows the configuration of the illumination unit 33. The illumination unit 33 includes a beam splitter 331, mirrors 332 and 333, a lens 338, and an objective lens 339. The orientation of the reflective surface of mirror 333 is variable, for example, by using a galvanometer mirror, voice coil mirror, or piezo tilt mirror. The orientation of the reflective surface of mirror 332 may also be variable.

[0039] The beam splitter 331 splits the light arriving from lens 25 into a first-branched beam and a second-branched beam, outputs the first-branched beam to mirror 333, and outputs the second-branched beam to mirror 332. The beam splitter 331 also receives the first-branched beam reflected by mirror 333 and the second-branched beam reflected by mirror 332, and outputs these first-branched and second-branched beams to lens 338. Lens 338 and objective lens 339 illuminate the object S to be observed as plane waves, respectively, with the first-branched beam and the second-branched beam output from beam splitter 331.

[0040] The lighting unit 33 having such a configuration can use the first branched light reflected by the mirror 333 as the first illumination light and illuminate the observation object S with this first illumination light along the first illumination direction under the dark-field illumination condition. The lighting unit 33 can use the second branched light reflected by the mirror 332 as the second illumination light and illuminate the observation object S with this second illumination light along the second illumination direction under the bright-field illumination condition.

[0041] The illumination direction of the first illumination light to the observation object S can be set by the orientation of the reflecting surface of the mirror 333. Therefore, if the numerical aperture of the objective lens 339 on the illumination side is larger than the numerical aperture of the objective lens 41 on the detection side, in the second illumination mode, the lighting unit 33 can illuminate only the second illumination light to the observation object S without illuminating the first illumination light to the observation object S by adjusting the orientation of the reflecting surface of the mirror 333.

[0042] The lighting unit 33A shown in FIG. 4 is a modified example of the lighting unit 33 shown in FIG. 3 and includes a shutter 334 provided between the beam splitter 331 and the mirror 332. In this configuration, with the light blocked by the shutter 334, by changing the orientation of the reflecting surface of the mirror 333, light can be illuminated to the observation object along each of a plurality of illumination directions under the bright-field illumination condition.

[0043] The lighting unit 33B shown in FIG. 5 is another modified example of the lighting unit 33 shown in FIG. 3 and includes a shutter 335 provided between the beam splitter 331 and the mirror 333. In this configuration, the lighting unit 33B can illuminate only the second illumination light to the observation object S without illuminating the first illumination light to the observation object S by blocking the light with the shutter 335.

[0044] Next, an observation method using the observation device 1 will be described. FIG. 6 is a flowchart of the observation method. This observation method includes a first intensity image acquisition step S1, a second intensity image acquisition step S2, a first complex amplitude image generation step S3, a second complex amplitude image generation step S4, and a dark-field image generation step S5.

[0045] The first intensity image acquisition step S1 and the second intensity image acquisition step S2 are performed using the illumination unit 31 (or illumination units 32, 33, 33A, 33B) and the imaging unit 50. The first complex amplitude image generation step S3, the second complex amplitude image generation step S4, and the darkfield image generation step S5 are performed by the processing unit 60.

[0046] In the first intensity image acquisition step S1, in the first illumination configuration, the illumination unit illuminates the object under observation with the first illumination light along the first illumination direction under dark-field illumination conditions, and with the difference in optical path length between the first illumination light and the second illumination light generated based on the light output from the light source being less than or equal to the coherence length. Then, in this first illumination configuration, the imaging unit acquires intensity images at each of the multiple focus planes of the object under observation.

[0047] In the second intensity image acquisition step S2, in the second illumination configuration, the illumination unit illuminates the object to be observed with a second illumination light along the second illumination direction under bright-field illumination conditions. At this time, illumination light is not provided under dark-field illumination conditions. Then, in this second illumination configuration, the imaging unit acquires intensity images at each of the multiple focus planes of the object to be observed.

[0048] In the first complex amplitude image generation step S3, a first complex amplitude image is generated using an intensity transport equation based on the intensity images at each of the multiple focus planes acquired in the first intensity image acquisition step S1.

[0049] In the second complex amplitude image generation step S4, a second complex amplitude image is generated using an intensity transport equation based on the intensity images at each of the multiple focus planes acquired in the second intensity image acquisition step S2.

[0050] In the dark-field image generation step S5, a complex amplitude image (dark-field image) is generated based on the difference between the first complex amplitude image and the second complex amplitude image when light is shone onto the object under dark-field illumination conditions along the first illumination direction.

[0051] The order of processing between the first intensity image acquisition step S1 and the second intensity image acquisition step S2 is arbitrary. The processing of the first complex amplitude image generation step S3 may be performed after the processing of the first intensity image acquisition step S1 and before the processing of the dark-field image generation step S5. The processing of the second complex amplitude image generation step S4 may be performed after the processing of the second intensity image acquisition step S2 and before the processing of the dark-field image generation step S5.

[0052] The intensity transport equations (TIEs) used in the first complex amplitude image generation step S3 and the second complex amplitude image generation step S4 are as follows. The TIEs are expressed by equations (1) and (2) below.

[0053] Here, equation (1) is the TIE when illumination light is obliquely incident on the object being observed, and k in the equation in This represents the wave vector of the illumination light incident on the object being observed. Equation (2) is the TIE when the illumination light is perpendicularly incident on the object being observed, and k in the equation ０ is the wavenumber of the illumination light incident on the object being observed. r is a variable representing the two-dimensional position on the xy-plane perpendicular to the z-axis (the axis parallel to the optical axis of the objective lens). φ(r) represents the phase image. I(r) is the intensity image at each of the multiple focus planes acquired in the first intensity image acquisition step S1 or the second intensity image acquisition step S2. The number of focus planes from which intensity images are acquired can be two or more, for example, five or less.

[0054] Further explanation of equation (1) above: Assuming I(r) is a constant, we obtain equation (3) below, and further, equation (4) below. This equation is the Poisson equation. Therefore, by solving this Poisson equation, the phase image φ(r) can be calculated. In addition, the amplitude image can be obtained from the intensity image. And from these phase image and amplitude image, a complex amplitude image can be generated.

[0055] The observation method of this embodiment may include synthetic aperture processing (see Simulation B described later). In this case, in the dark-field image generation step S5, a complex amplitude image (dark-field image) is generated based on the difference between the first complex amplitude image and the second complex amplitude image when the object to be observed is illuminated with light along each of the multiple first illumination directions under dark-field illumination conditions.

[0056] Furthermore, the observation method of this embodiment further comprises a bright-field image generation step and a synthetic aperture processing step. In the bright-field image generation step, a complex amplitude image (bright-field image) is generated by the illumination unit when the object to be observed is illuminated with light along each of a plurality of illumination directions under bright-field illumination conditions, using an intensity transport equation based on the intensity images at each of a plurality of focus planes acquired by the imaging unit.

[0057] Then, in the synthetic aperture processing step, synthetic aperture processing is performed based on the complex amplitude images (darkfield images) for each of the multiple first illumination directions under darkfield illumination conditions generated in the darkfield image generation step, and the complex amplitude images (brightfield images) for each of the multiple illumination directions under brightfield illumination conditions generated in the brightfield image generation step.

[0058] Furthermore, the observation method of this embodiment may further include a wavefront propagation step in which the complex amplitude image generated by TIE is wavefront propagated (digital refocused) to generate a complex amplitude image at another location (see Simulation C described later).

[0059] Next, we will explain the results of simulations A to C performed on the observation method of this embodiment.

[0060] Figures 7 to 11 show the results of simulation A. Here, the first illumination direction (θ) under dark-field illumination conditions is shown. ｘ , θ ｙThe angles were set to (0.22°, 1.16°), and the second illumination direction under bright-field illumination conditions was set to the vertical direction (parallel to the z-axis). In the first intensity image acquisition step S1 and the second intensity image acquisition step S2, the imaging unit acquired an intensity image I(r) at two positions, z = -50λ and z = +50λ, with wavelength λ, and a dark-field image was generated in the dark-field image generation step S5.

[0061] Figure 7 shows the intensity image I(r) acquired in the first intensity image acquisition step S1 under the first illumination mode. Figure 8 shows the intensity image I(r) acquired in the second intensity image acquisition step S2 under the second illumination mode. In Figures 7 and 8, (a) is the intensity image at z = -50λ and (b) is the intensity image at z = +50λ.

[0062] Figure 9 shows the images generated in the first complex amplitude image generation step S3 when the first illumination mode is used. Figure 10 shows the images generated in the second complex amplitude image generation step S4 when the second illumination mode is used. In Figures 9 and 10, (a) is the image shown on the left side of the TIE, (b) is the phase image φ(r) obtained by calculating the TIE, and (c) is the real part of the complex amplitude image. Figure 9(c) is the first complex amplitude image, and Figure 10(c) is the second complex amplitude image.

[0063] Figure 11(a) is the real part of the dark-field image generated in dark-field image generation step S5. Figure 11(b) is the exact solution of the dark-field image. Figure 11(c) is the difference between the dark-field image generated in dark-field image generation step S5 (Figure 11(a)) and the exact solution (Figure 11(b)). As shown in this figure, the dark-field image generated in dark-field image generation step S5 (Figure 11(a)) was in good agreement with the exact solution (Figure 11(b)).

[0064] Figures 12 to 17 show the results of Simulation B. Here, in the dark-field image generation step, complex amplitude images (dark-field images) are acquired when light is shone on the object under multiple illumination directions under dark-field illumination conditions. In this case, the second illumination direction under bright-field illumination conditions was set to the vertical direction. In the bright-field image generation step, complex amplitude images (bright-field images) are acquired when light is shone on the object under multiple illumination directions under bright-field illumination conditions.

[0065] The imaging unit was used to acquire intensity images I(r) at two positions, z = -50λ and z = +50λ, with wavelength λ. In the synthetic aperture processing step, synthetic aperture processing was performed based on multiple complex amplitude images (darkfield images) generated in the darkfield image generation step and multiple complex amplitude images (brightfield images) generated in the brightfield image generation step.

[0066] Figure 12 shows the distribution of illumination directions in wavenumber space for dark-field and bright-field illumination conditions. This figure shows one element k of the wavenumber vector of the illumination light incident on the object being observed. ｘ With the horizontal axis being the other element k ｙ In wavenumber space with the vertical axis, each position (illumination direction) is discretely arranged in a square grid, and different symbols indicate whether the conditions are dark-field illumination or bright-field illumination.

[0067] Figure 13 shows the exact solution obtained by performing a composite aperture process based on multiple dark-field images and multiple bright-field images. Figure 14 shows the bright-field image obtained when the illumination direction is set to the vertical under bright-field illumination conditions. Figure 15 shows the image obtained by performing a composite aperture process based on multiple bright-field images obtained for each of multiple illumination directions under bright-field illumination conditions in the bright-field image generation step. Figure 16 shows the image obtained by performing a composite aperture process based on multiple bright-field images and multiple dark-field images in the composite aperture process step. Figure 17 is an image showing the difference between the image obtained by performing a composite aperture process based on multiple bright-field images and multiple dark-field images in the composite aperture process step (Figure 16) and the exact solution (Figure 13).

[0068] In Figures 13 to 17, (a) is a phase image, and (b) is an image showing the intensity distribution in wavenumber space. The intensity distribution in wavenumber space was obtained by performing a Fourier transform on the complex amplitude image.

[0069] As shown in these figures, the resolution of the image obtained by performing a composite aperture processing on multiple bright-field images obtained in multiple illumination directions under bright-field illumination conditions (Figure 15) is higher than the resolution of the bright-field image obtained in one illumination direction under bright-field illumination conditions (Figure 14). Furthermore, the resolution of the image obtained by performing a composite aperture processing on multiple bright-field images and multiple dark-field images in the composite aperture processing step (Figure 16) is even higher. In addition, the image obtained by performing a composite aperture processing in the composite aperture processing step (Figure 16) showed good agreement with the exact solution (Figure 13).

[0070] Figures 18 to 26 show the results of simulation C. Here, the first illumination direction (θ) under dark-field illumination conditions is shown. ｘ , θ ｙThe angles were set to (0.22°, 1.16°), and the second illumination direction under brightfield illumination conditions was set to the vertical direction. Based on the intensity images I(r) acquired by the imaging unit at two positions z=950λ and z=1050λ, with wavelength λ, a complex amplitude image at z=1000λ was generated by TIE, and in the wavefront propagation step, the wavefront propagation was performed to generate a complex amplitude image at z=0. Synthetic aperture processing was also performed.

[0071] Figure 18 shows the intensity image I(r) acquired in the first intensity image acquisition step S1 under the first illumination mode. Figure 19 shows the intensity image I(r) acquired in the second intensity image acquisition step S2 under the second illumination mode. In Figures 18 and 19, (a) is the intensity image at z = 950λ and (b) is the intensity image at z = 1050λ.

[0072] Figure 20 shows the images generated in the first complex amplitude image generation step S3 when the first illumination mode is used. Figure 21 shows the images generated in the second complex amplitude image generation step S4 when the second illumination mode is used. In Figures 20 and 21, (a) is the image shown on the left side of the TIE, (b) is the phase image φ(r) obtained by the TIE calculation, and (c) is the real part of the complex amplitude image. Figure 20(c) is the first complex amplitude image at z = 1000λ, and Figure 21(c) is the second complex amplitude image at z = 1000λ.

[0073] Figure 22(a) is the real part of the dark-field image generated in dark-field image generation step S5. Figure 22(b) is the exact solution of the dark-field image. These images are dark-field images at z = 1000λ. As shown in this figure, the dark-field image generated in dark-field image generation step S5 (Figure 22(a)) showed good agreement with the exact solution (Figure 22(b)).

[0074] Figure 23 shows the exact solution of the dark-field image at z=0. Figure 24 shows the image at z=1000λ obtained by performing a composite aperture process based on multiple bright-field images obtained in each of multiple illumination directions under bright-field illumination conditions in the bright-field image generation step. Figure 25 shows the image at z=1000λ obtained by performing a composite aperture process based on multiple bright-field images and multiple dark-field images in the composite aperture process step. Figure 26 shows the image obtained by performing a composite aperture process based on multiple complex amplitude images at z=0, after generating a complex amplitude image at z=0 by wavefront propagation of multiple bright-field images and multiple dark-field images at z=1000λ in the wavefront propagation step.

[0075] In Figures 23 to 26, (a) is a phase image, and (b) is an image showing the intensity distribution in wavenumber space. As shown in these figures, the image obtained by performing the wavefront propagation step and the synthetic aperture processing step (Figure 26) was in good agreement with the exact solution (Figure 23).

[0076] As described above, in this embodiment, the illumination unit 31 (or illumination units 32, 33, 33A, 33B) can be integrated, and compared to the case where a two-beam interferometer is used, the optical system is easier to adjust, stability is superior, and the device can be made simpler in configuration.

[0077] Furthermore, in this embodiment, since the complex amplitude image of the object being observed under dark-field illumination is obtained by utilizing both the illumination light from dark-field illumination and the illumination light from bright-field illumination, the dark-field image can be accurately obtained. In addition, by performing composite aperture processing based on multiple dark-field images and multiple bright-field images, a wide-field and high-resolution image can be generated.

[0078] The observation apparatus and observation method are not limited to the embodiments and configuration examples described above, and various modifications are possible.

[0079] The observation apparatus of the first embodiment according to the above embodiment includes (1) a light source that outputs light, (2) an illumination unit that, in the first illumination mode, illuminates the object to be observed with the first illumination light along the first illumination direction under dark-field illumination conditions and the second illumination light along the second illumination direction under bright-field illumination conditions, with the difference in optical path length between the first illumination light and the second illumination light generated based on the light output from the light source being less than or equal to the coherent length, and in the second illumination mode, illuminates the object to be observed with the second illumination light along the second illumination direction under bright-field illumination conditions, and (3) in each of the first illumination mode and the second illumination mode, for each of the multiple focus planes of the object to be observed. The system comprises (4) an imaging unit that acquires intensity images, and (5) a processing unit that processes based on the intensity images acquired by the imaging unit, wherein the processing unit generates a first complex amplitude image using an intensity transport equation based on the intensity images at each of a plurality of focus planes acquired by the imaging unit in the first illumination configuration, generates a second complex amplitude image using an intensity transport equation based on the intensity images at each of a plurality of focus planes acquired by the imaging unit in the second illumination configuration, and generates a complex amplitude image when light is shone on an object along a first illumination direction under dark-field illumination conditions, based on the difference between the first complex amplitude image and the second complex amplitude image.

[0080] In the observation apparatus of the second embodiment, in the configuration of the first embodiment, the processing unit generates a complex amplitude image when the object to be observed is illuminated with light along each of the multiple first illumination directions under dark-field illumination conditions, based on the difference between the first complex amplitude image and the second complex amplitude image. The illumination unit generates a complex amplitude image when the object to be observed is illuminated with light along each of the multiple illumination directions under bright-field illumination conditions, based on the intensity images at each of the multiple focus planes acquired by the imaging unit using an intensity transport equation. The composite aperture processing may be performed based on the complex amplitude images for each of the multiple first illumination directions under dark-field illumination conditions and the complex amplitude images for each of the multiple illumination directions under bright-field illumination conditions.

[0081] In the observation apparatus of the third embodiment, in the configuration of the first or second embodiment, the processing unit may be configured to generate a complex amplitude image at another location by wavefront propagation of the complex amplitude image generated by the intensity transport equation.

[0082] In the observation apparatus of the fourth embodiment, in any configuration of the first to third embodiments, the illumination unit includes a beam splitter that splits the light output from the light source into a first branched beam and a second branched beam, a first reflector that reflects the first branched beam output from the beam splitter, and a second reflector that reflects the second branched beam output from the beam splitter. In the first illumination mode, one of the first branched beam reflected by the first reflector and the second branched beam reflected by the second reflector may be used to illuminate the object under dark-field illumination conditions, while the other illuminates the object under bright-field illumination conditions.

[0083] In the observation device of the fifth embodiment, in the configuration of the fourth embodiment, the orientation of the reflective surface of the first reflector and the orientation of the reflective surface of the second reflector, or both, may be variable.

[0084] In the observation apparatus of the sixth embodiment, in the configuration of the fourth or fifth embodiment, both or either of the first reflector and the second reflector may be configured as a spatial light modulator.

[0085] In the observation apparatus of the seventh embodiment, in any configuration of the fourth to sixth embodiments, the illumination unit may further include a shutter provided on both or either the optical path of the first branched light from the beam splitter to the object to be observed, and the optical path of the second branched light from the beam splitter to the object to be observed.

[0086] In the observation apparatus of the eighth embodiment, in any configuration of the first to third embodiments, the illumination unit includes a spatial light modulator that selectively phase-modulates light of either a first polarization component or a second polarization component, which are mutually orthogonal, to the light output from the light source. In the first illumination embodiment, one of the light components may be used to illuminate the object under dark-field illumination conditions, while the other is used to illuminate the object under bright-field illumination conditions.

[0087] The observation method of the first embodiment according to the above embodiment includes: (1) In a first illumination configuration, the illumination unit illuminates the object to be observed with the first illumination light along a first illumination direction under dark-field illumination conditions and the second illumination light along a second illumination direction under bright-field illumination conditions, and the imaging unit acquires an intensity image of each of the multiple focus planes of the object to be observed; and (2) In a second illumination configuration, the illumination unit illuminates the object to be observed with the second illumination light along a second illumination direction under bright-field illumination conditions and the imaging unit acquires an intensity image of each of the multiple focus planes of the object to be observed. The system comprises: (3) a second intensity image acquisition step of acquiring intensity images at each of the multiple focus planes acquired in the first intensity image acquisition step, a first complex amplitude image generation step of generating a first complex amplitude image using an intensity transport equation based on the intensity images at each of the multiple focus planes acquired in the first intensity image acquisition step, a second complex amplitude image generation step of generating a second complex amplitude image using an intensity transport equation based on the intensity images at each of the multiple focus planes acquired in the second intensity image acquisition step, and a dark-field image generation step of generating a complex amplitude image when light is shone on an object to be observed along a first illumination direction under dark-field illumination conditions, based on the difference between the first complex amplitude image and the second complex amplitude image.

[0088] In the observation method of the second embodiment, the configuration of the first embodiment may further include: a dark-field image generation step in which a complex amplitude image is generated when light is shone on the object to be observed along each of a plurality of first illumination directions under dark-field illumination conditions, based on the difference between the first complex amplitude image and the second complex amplitude image; a bright-field image generation step in which a complex amplitude image is generated when light is shone on the object to be observed along each of a plurality of illumination directions under bright-field illumination conditions by the illumination unit, based on the intensity image at each of a plurality of focus planes acquired by the imaging unit using an intensity transport equation; and a synthetic aperture processing step in which a synthetic aperture processing is performed based on the complex amplitude images for each of the plurality of first illumination directions under dark-field illumination conditions and the complex amplitude images for each of the plurality of illumination directions under bright-field illumination conditions.

[0089] In the observation method of the third embodiment, the configuration of the first or second embodiment may further include a wavefront propagation step in which a complex amplitude image generated by the intensity transport equation is wavefront propagated to generate a complex amplitude image at another location.

[0090] In the observation method of the fourth embodiment, in any configuration of the first to third embodiments, the illumination unit includes a beam splitter that splits the light output from the light source into a first branched beam and a second branched beam, a first reflector that reflects the first branched beam output from the beam splitter, and a second reflector that reflects the second branched beam output from the beam splitter. In the first illumination embodiment, one of the first branched beam reflected by the first reflector and the second branched beam reflected by the second reflector may be used to illuminate the object under dark-field illumination conditions, while the other illuminates the object under bright-field illumination conditions.

[0091] In the observation method of the fifth embodiment, in the configuration of the fourth embodiment, the orientation of the reflective surface of the first reflector and the orientation of the reflective surface of the second reflector, or both, may be variable.

[0092] In the observation method of the sixth embodiment, in the configuration of the fourth or fifth embodiment, both or either of the first reflector and the second reflector may be configured as a spatial light modulator.

[0093] In the observation method of the seventh embodiment, in any configuration of the fourth to sixth embodiments, the illumination unit may further include a shutter provided on both or either the optical path of the first branched light from the beam splitter to the object to be observed, and the optical path of the second branched light from the beam splitter to the object to be observed.

[0094] In the observation method of the eighth embodiment, in any of the configurations of the first to third embodiments, the illumination unit includes a spatial light modulator that selectively phase-modulates light of either a first polarization component or a second polarization component, which are mutually orthogonal, to the light output from the light source. In the first illumination embodiment, one of the light components of the first and second polarizations may be used to illuminate the object under dark-field illumination conditions, while the other is used to illuminate the object under bright-field illumination conditions.

[0095] The embodiment can be used as an observation device and observation method that can easily acquire complex amplitude images of an object under dark-field illumination with a simple configuration.

[0096] 1... Observation device, 10... Light source, 21... Lens, 22... Light incident end, 23... Optical fiber, 24... Light exit end, 25... Lens, 31-33, 33A, 33B... Illumination unit, 41... Objective lens, 42... Mirror, 43... Lens, 50... Imaging unit, 60... Processing unit, 311... Beam splitter, 313... Spatial light modulator, 314... Polarizer, 315... Half-wave plate, 316... Polarizer, 318... Lens, 319... Objective lens, 321... Beam splitter, 322... Mirror, 323... Spatial light modulator, 328... Lens, 329... Objective lens, 331... Beam splitter, 332, 333... Mirror, 334, 335... Shutter, 338... Lens, 339... Objective lens, 51... Position adjustment unit, S... Object to be observed.

Claims

1. A light source that outputs light; an illumination unit that, in a first illumination mode, illuminates an object to be observed with the first illumination light along a first illumination direction under dark-field illumination conditions and illuminates the object to be observed with the second illumination light along a second illumination direction under bright-field illumination conditions, with the difference in optical path length between a first illumination light and a second illumination light generated based on the light output from the light source being less than or equal to the coherent length; and in a second illumination mode, illuminates the object to be observed with the second illumination light along the second illumination direction under bright-field illumination conditions; an imaging unit that acquires intensity images at each of a plurality of focus planes of the object to be observed in each of the first illumination mode and the second illumination mode; and a processing unit that processes based on the intensity images acquired by the imaging unit, wherein the processing unit generates a first complex amplitude image using an intensity transport equation based on the intensity images at each of the plurality of focus planes acquired by the imaging unit in the first illumination mode, and generates a second complex amplitude image using an intensity transport equation based on the intensity images at each of the plurality of focus planes acquired by the imaging unit in the second illumination mode. An observation device that generates a complex amplitude image when light is shone onto the object to be observed along the first illumination direction under dark-field illumination conditions, based on the difference between the first complex amplitude image and the second complex amplitude image.

2. The observation apparatus according to claim 1, wherein the processing unit generates a complex amplitude image when the object to be observed is illuminated with light along each of the plurality of first illumination directions under dark-field illumination conditions, based on the difference between the first complex amplitude image and the second complex amplitude image; generates a complex amplitude image when the object to be observed is illuminated with light along each of the plurality of illumination directions under bright-field illumination conditions by the illumination unit, based on the intensity transport equation using intensity images at each of the plurality of focus planes acquired by the imaging unit; and performs composite aperture processing based on the complex amplitude images for each of the plurality of first illumination directions under dark-field illumination conditions and the complex amplitude images for each of the plurality of illumination directions under bright-field illumination conditions.

3. The observation apparatus according to claim 1 or 2, wherein the processing unit generates a complex amplitude image at another location by wavefront propagation of the complex amplitude image generated by the intensity transport equation.

4. The observation apparatus according to any one of claims 1 to 3, wherein the illumination unit includes a beam splitter that splits the light output from the light source into a first branched beam and a second branched beam, a first reflecting unit that reflects the first branched beam output from the beam splitter, and a second reflecting unit that reflects the second branched beam output from the beam splitter, wherein in the first illumination mode, one of the first branched beam reflected by the first reflecting unit and the second branched beam reflected by the second reflecting unit is used to illuminate the object to be observed under dark-field illumination conditions, and the other is used to illuminate the object to be observed under bright-field illumination conditions.

5. The observation apparatus according to claim 4, wherein the orientation of the reflective surface of the first reflecting part and the orientation of the reflective surface of the second reflecting part, or either one thereof, are variable.

6. The observation apparatus according to claim 4 or 5, wherein both or either of the first reflector and the second reflector are spatial light modulators.

7. The observation apparatus according to any one of claims 4 to 6, wherein the illumination unit further includes a shutter provided on both or either the optical path of the first branched light from the beam splitter to the object to be observed, and the optical path of the second branched light from the beam splitter to the object to be observed.

8. The observation apparatus according to any one of claims 1 to 3, wherein the illumination unit includes a spatial light modulator that selectively phase modulates the light of either a first polarization component or a second polarization component of the light output from the light source, and in the first illumination mode, one of the light of the first polarization component and the light of the second polarization component illuminates the object to be observed under dark-field illumination conditions, and the other illuminates the object to be observed under bright-field illumination conditions.

9. In a first illumination configuration, the illumination unit illuminates the object to be observed with the first illumination light along a first illumination direction under dark-field illumination conditions and with the second illumination light along a second illumination direction under bright-field illumination conditions, and the imaging unit acquires intensity images at each of the multiple focus planes of the object to be observed in a first intensity image acquisition step; In a second illumination configuration, the illumination unit illuminates the object to be observed with the second illumination light along a second illumination direction under bright-field illumination conditions, and the imaging unit acquires intensity images at each of the multiple focus planes of the object to be observed in a second intensity image acquisition step; and based on the intensity images at each of the multiple focus planes acquired in the first intensity image acquisition step, the imaging unit generates a first complex amplitude image using an intensity transport equation. An observation method comprising: a second complex amplitude image generation step of generating a second complex amplitude image using an intensity transport equation based on intensity images at each of the plurality of focus planes acquired in the second intensity image acquisition step; and a dark-field image generation step of generating a complex amplitude image when light is shone on the object to be observed along the first illumination direction under dark-field illumination conditions, based on the difference between the first complex amplitude image and the second complex amplitude image.

10. The observation method according to claim 9, further comprising: a dark-field image generation step, in which a complex amplitude image is generated when light is shone on the object to be observed along each of the plurality of first illumination directions under dark-field illumination conditions, based on the difference between the first complex amplitude image and the second complex amplitude image; a bright-field image generation step, in which a complex amplitude image is generated when light is shone on the object to be observed along each of the plurality of illumination directions under bright-field illumination conditions by the illumination unit, based on intensity images at each of the plurality of focus planes acquired by the imaging unit using an intensity transport equation; and a synthetic aperture processing step, in which a synthetic aperture processing is performed based on the complex amplitude images for each of the plurality of first illumination directions under dark-field illumination conditions and the complex amplitude images for each of the plurality of illumination directions under bright-field illumination conditions.

11. The observation method according to claim 9 or 10, further comprising a wavefront propagation step of generating a complex amplitude image at another location by wavefront propagation of a complex amplitude image generated by the intensity transport equation.

12. The observation method according to any one of claims 9 to 11, wherein the illumination unit includes a beam splitter that splits the light output from the light source into a first branched beam and a second branched beam, a first reflecting unit that reflects the first branched beam output from the beam splitter, and a second reflecting unit that reflects the second branched beam output from the beam splitter, wherein in the first illumination mode, one of the first branched beam reflected by the first reflecting unit and the second branched beam reflected by the second reflecting unit is used to illuminate the object to be observed under dark-field illumination conditions, and the other is used to illuminate the object to be observed under bright-field illumination conditions.

13. The observation method according to claim 12, wherein the orientation of the reflective surface of the first reflecting part and the orientation of the reflective surface of the second reflecting part, or either one thereof, are variable.

14. The observation method according to claim 12 or 13, wherein both or either of the first reflector and the second reflector are spatial light modulators.

15. The observation method according to any one of claims 12 to 14, wherein the illumination unit further includes a shutter provided on both or either the optical path of the first branched light from the beam splitter to the object to be observed, and the optical path of the second branched light from the beam splitter to the object to be observed.

16. The observation method according to any one of claims 9 to 11, wherein the illumination unit includes a spatial light modulator that selectively phase modulates the light of either a first polarization component or a second polarization component of the light output from the light source, and in the first illumination mode, one of the light of the first polarization component and the light of the second polarization component is used to illuminate the object to be observed under dark-field illumination conditions, and the other is used to illuminate the object to be observed under bright-field illumination conditions.

Images