Observation apparatus and observation method

Abstract

This invention provides an observation device that can easily acquire complex amplitude images of an object under dark-field illumination using a simple configuration. [Solution] In the first illumination configuration, the illumination unit 31 illuminates with a first illumination light along a first illumination direction under dark-field illumination conditions, and illuminates with a second illumination light along a second illumination direction under bright-field illumination conditions. In the second illumination configuration, the illumination unit illuminates with a second illumination light along a second illumination direction under bright-field illumination conditions. The processing unit 60 generates a first complex amplitude image based on the intensity images at each of the multiple focus planes acquired by the imaging unit 50 in the first illumination configuration, generates a second complex amplitude image based on the intensity images at each of the multiple focus planes acquired by the imaging unit in the second illumination configuration, and generates a complex amplitude image when light is illuminated along the first illumination direction under dark-field illumination conditions based on the difference between the first complex amplitude image and the second complex amplitude image.

Description

【Technical Field】 【0001】 The present invention relates to an observation device and an observation method. 【Background Art】 【0002】 In the observation of an object to be observed by a microscope, there is generally a trade-off relationship between resolution and field of view. That is, when trying to observe an object to be observed with high resolution, the field of view becomes narrow, and when trying to observe an object to be observed with a wide field of view, the resolution becomes low. On the other hand, there are applications (for example, pathological slide observation) that require the observation of an object to be observed with a wide field of view and high resolution. 【0003】 As a technique for meeting the requirement of wide-field and high-resolution observation, there is a technique in which a stage on which an object to be observed is placed is moved to acquire images of the object to be observed at a plurality of positions, and then these plurality of images are stitched together. However, since this technique requires mechanical movement of the stage, it has a problem that it takes a long time to acquire a plurality of images. 【0004】 Further, as another technique for meeting the requirement of wide-field and high-resolution observation, there is a synthetic aperture method (Non-Patent Document 1) that can acquire a high-resolution image even when using a low-magnification objective lens. The synthetic aperture method does not require mechanical movement of the stage. The synthetic aperture method illuminates an object to be observed with light along a plurality of illumination directions respectively to acquire images, and then synthesizes these plurality of images in the frequency space to generate a wide-field and high-resolution image. The synthetic aperture method utilizes the fact that high-frequency components in the frequency space correspond to high resolution in the real space, and that a fine sampling interval in the frequency space corresponds to a wide field of view in the real space. 【0005】 Since the synthetic aperture method synthesizes multiple images in wavenumber space, it is necessary to acquire each of these multiple images as a complex amplitude image. Furthermore, to obtain higher resolution images, it is necessary to increase the angle of incidence of light on the object being observed to acquire a complex amplitude image with a high wavenumber component. Therefore, it is desirable to obtain multiple bright-field images by illuminating the object being observed with bright-field light, and also obtain multiple dark-field images by illuminating the object with dark-field light, and then perform synthetic aperture processing using not only these multiple bright-field images but also multiple dark-field images. When the incident NA is less than or equal to the detected NA, a bright-field image is obtained, and when the incident NA is greater than the detected NA, a dark-field image is obtained. 【0006】 In addition to two-beam interferometry, there is a technique for acquiring complex amplitude images of an object being observed that uses the transport of intensity equation (TIE) (Non-Patent Literature 2). The technique using TIE acquires intensity images at multiple z-positions (positions on the z-axis parallel to the optical axis of the objective lens) of the object being observed, and generates a complex amplitude image using TIE based on these multiple intensity images. [Prior art documents] [Non-patent literature] 【0007】 [Non-Patent Document 1] Moonseok Kim, et al,"High-speed synthetic aperture microscopy for live cell imaging,"OPTICS LETTERS, Vol.36, No.2, January 15, 2011 [Non-Patent Document 2] Chao Zuo, et al, "Transportof intensity equation: a tutorial," Optics and Lasers in Engineering 135(2020) 106187 [Overview of the Initiative] [Problems that the invention aims to solve] 【0008】 Two-beam interferometry can acquire complex amplitude images containing phase information, not only under bright-field illumination but also under dark-field illumination. However, two-beam interferometry has drawbacks: the optical system is not easily adjustable, it has stability issues, and the equipment is complex and expensive. 【0009】 Techniques using TIE can overcome the problems of two-beam interferometry described above. However, while TIE techniques can acquire bright-field images relatively accurately under bright-field illumination, when attempting to acquire dark-field images under dark-field illumination, the error in phase calculation by TIE becomes large, making it difficult to accurately obtain complex amplitude images. 【0010】 The requirement for acquiring complex amplitude images (dark-field images) of an object under dark-field illumination is not limited to synthetic aperture processing. 【0011】 The present invention was made to solve the above-mentioned problems, and aims to provide an observation device and observation method that can easily acquire complex amplitude images of an object to be observed under dark-field illumination with a simple configuration. [Means for solving the problem] 【0012】 A first aspect of the observation apparatus of the present invention comprises: (1) a light source that outputs light; (2) an illumination unit that, in a first illumination mode, illuminates an object to be observed with first illumination light along a first illumination direction under dark-field illumination conditions and second illumination light along a second illumination direction under bright-field illumination conditions, with the difference in optical path length between first illumination light and second illumination light generated based on the light output from the light source being less than or equal to the coherent length; and in a second illumination mode, illuminates an object to be observed with second illumination light along a second illumination direction under bright-field illumination conditions; (3) an imaging unit that acquires intensity images at each of a plurality of focus planes of an object to be observed in both the first and second illumination modes; and (4) a processing unit that performs processing based on the intensity images acquired by the imaging unit. The processing unit generates a first complex amplitude image using an intensity transport equation based on the intensity images at each of the multiple focus planes acquired by the imaging unit in the first illumination configuration, generates a second complex amplitude image using an intensity transport equation based on the intensity images at each of the multiple focus planes acquired by the imaging unit in the second illumination configuration, and generates a complex amplitude image when light is shone on the object being observed along the first illumination direction under dark-field illumination conditions, based on the difference between the first complex amplitude image and the second complex amplitude image. 【0013】 In a second aspect of the observation apparatus of the present invention, in addition to the first aspect, the processing unit generates a complex amplitude image when the object to be observed is illuminated with light along each of a plurality of first illumination directions under dark-field illumination conditions, based on the difference between the first complex amplitude image and the second complex amplitude image. The illumination unit generates a complex amplitude image when the object to be observed is illuminated with light along each of a plurality of illumination directions under bright-field illumination conditions, based on the intensity image at each of a plurality of focus planes acquired by the imaging unit using an intensity transport equation. A composite aperture processing is performed based on the complex amplitude images for each of the plurality of first illumination directions under dark-field illumination conditions and the complex amplitude images for each of the plurality of illumination directions under bright-field illumination conditions. 【0014】 In a third aspect of the observation apparatus of the present invention, in addition to the first or second aspect, the processing unit generates a complex amplitude image at another location by wavefront propagation of the complex amplitude image generated by the intensity transport equation. 【0015】 In a fourth aspect of the observation apparatus of the present invention, in addition to any of the first to third aspects, the illumination unit includes a beam splitter that splits light output from a light source into a first branched beam and a second branched beam, a first reflector that reflects the first branched beam output from the beam splitter, and a second reflector that reflects the second branched beam output from the beam splitter. In the first illumination mode, one of the first branched beam reflected by the first reflector and the second branched beam reflected by the second reflector is used to illuminate the object under dark-field illumination conditions, and the other is used to illuminate the object under bright-field illumination conditions. 【0016】 In a fifth aspect of the observation apparatus of the present invention, in addition to the fourth aspect, the orientation of the reflective surface of the first reflector and the orientation of the reflective surface of the second reflector, or either one thereof, are variable. 【0017】 In the sixth aspect of the observation apparatus of the present invention, in addition to the fourth or fifth aspect, both or either of the first reflector and the second reflector are spatial light modulators. 【0018】 In the seventh aspect of the observation apparatus of the present invention, in addition to any of the fourth to sixth aspects, the illumination unit further includes a shutter provided on both or either the optical path of the first branched light from the beam splitter to the object to be observed, and the optical path of the second branched light from the beam splitter to the object to be observed. 【0019】 In the eighth aspect of the observation apparatus of the present invention, in addition to any of the first to third aspects, the illumination unit includes a spatial light modulator that selectively phase modulates the light of either a first polarization component or a second polarization component, which are mutually orthogonal, to the light output from the light source, wherein in the first illumination aspect, one of the light of the first polarization component and the light of the second polarization component is used to illuminate the object under dark-field illumination conditions, and the other is used to illuminate the object under bright-field illumination conditions. 【0020】 A first aspect of the observation method of the present invention is: (1) In a first illumination configuration, the illumination unit illuminates the object to be observed with the first illumination light along a first illumination direction under dark-field illumination conditions and the second illumination light along a second illumination direction under bright-field illumination conditions, and the imaging unit acquires an intensity image at each of the multiple focus planes of the object to be observed (first intensity image acquisition step); (2) In a second illumination configuration, the illumination unit illuminates the object to be observed with the second illumination light along a second illumination direction under bright-field illumination conditions, and the imaging unit acquires an intensity image at each of the multiple focus planes of the object to be observed (second intensity image acquisition step); (3) Based on the intensity images at each of the multiple focus planes acquired in the first intensity image acquisition step, the imaging unit generates a first complex amplitude image using an intensity transport equation (4) The system includes (5) a second complex amplitude image generation step which generates a second complex amplitude image using an intensity transport equation based on the intensity images at each of the multiple focus planes acquired in the second intensity image acquisition step, and (6) a dark-field image generation step which generates a complex amplitude image when light is shone on the object to be observed along a first illumination direction under dark-field illumination conditions, based on the difference between the first complex amplitude image and the second complex amplitude image. 【0021】 In a second aspect of the observation method of the present invention, in addition to the first aspect, in the dark-field image generation step, a complex amplitude image is generated based on the difference between a first complex amplitude image and a second complex amplitude image when the object to be observed is illuminated with light along each of a plurality of first illumination directions under dark-field illumination conditions. Furthermore, the second aspect further comprises a bright-field image generation step in which a complex amplitude image is generated by an illumination unit when the object to be observed is illuminated with light along each of a plurality of illumination directions under bright-field illumination conditions, based on intensity images at each of a plurality of focus planes acquired by an imaging unit using an intensity transport equation, and a synthetic aperture processing step in which a synthetic aperture processing is performed based on the complex amplitude images for each of the plurality of first illumination directions under dark-field illumination conditions and the complex amplitude images for each of the plurality of illumination directions under bright-field illumination conditions. 【0022】 In addition to the first aspect or the second aspect, the third aspect of the observation method of the present invention further includes a wavefront propagation step of wavefront propagating a complex amplitude image generated by the intensity transport equation to generate a complex amplitude image at another position. 【0023】 In the fourth aspect of the observation method of the present invention, in addition to any one of the first to third aspects, the illumination unit includes a beam splitter that splits the light output from the light source into first branched light and second branched light, a first reflection unit that reflects the first branched light output from the beam splitter, and a second reflection unit that reflects the second branched light output from the beam splitter. In the first illumination mode, one of the first branched light reflected by the first reflection unit and the second branched light reflected by the second reflection unit illuminates the observation object under dark field illumination conditions, and the other illuminates the observation object under bright field illumination conditions. 【0024】 In the fifth aspect of the observation method of the present invention, in addition to the fourth aspect, both or either one of the orientation of the reflection surface of the first reflection unit and the orientation of the reflection surface of the second reflection unit is variable. 【0025】 In the sixth aspect of the observation method of the present invention, in addition to the fourth aspect or the fifth aspect, both or either one of the first reflection unit and the second reflection unit is a spatial light modulator. 【0026】 In the seventh aspect of the observation method of the present invention, in addition to any one of the fourth to sixth aspects, the illumination unit further includes shutters provided on both or either one of the optical paths of the first branched light from the beam splitter to the observation object and the optical paths of the second branched light from the beam splitter to the observation object. 【0027】 In the eighth aspect of the observation method of the present invention, in addition to any of the first to third aspects, the illumination unit includes a spatial light modulator that selectively phase modulates the light of either a first polarization component or a second polarization component, which are mutually orthogonal, of the light output from the light source, and in the first illumination aspect, one of the light of the first polarization component and the light of the second polarization component is used to illuminate the object under dark-field illumination conditions, and the other is used to illuminate the object under bright-field illumination conditions. [Effects of the Invention] 【0028】 According to the present invention, complex amplitude images of objects under dark-field illumination can be easily acquired with a simple configuration. [Brief explanation of the drawing] 【0029】 [Figure 1] Figure 1 shows the configuration of the observation device 1. [Figure 2] Figure 2 shows the configuration of the lighting unit 32. [Figure 3] Figure 3 shows the configuration of the lighting unit 33. [Figure 4] Figure 4 shows the configuration of the lighting unit 33A. [Figure 5] Figure 5 shows the configuration of the lighting unit 33B. [Figure 6] Figure 6 is a flowchart of the observation method. [Figure 7] Figure 7 shows the intensity image I(r) acquired in the first intensity image acquisition step S1 under the first illumination mode. Figure 7(a) is the intensity image at z = -50λ, and Figure 7(b) is the intensity image at z = +50λ. [Figure 8] Figure 8 shows the intensity image I(r) acquired in the second intensity image acquisition step S2 when the second illumination mode is used. Figure 8(a) is the intensity image at z = -50λ, and Figure 8(b) is the intensity image at z = +50λ. [Figure 9]Figure 9 shows the images produced in the first complex amplitude image generation step S3 when the first illumination mode is used. Figure 9(a) is the image shown on the left side of the TIE, Figure 9(b) is the phase image φ(r) obtained by calculating the TIE, and Figure 9(c) is the real part of the complex amplitude image. [Figure 10] Figure 10 shows the images generated in the second complex amplitude image generation step S4 when the second illumination mode is in use. Figure 10(a) is the image shown on the left side of the TIE, Figure 10(b) is the phase image φ(r) obtained by calculating the TIE, and Figure 10(c) is the real part of the complex amplitude image. [Figure 11] Figure 11(a) is the real part of the dark-field image generated in dark-field image generation step S5. Figure 11(b) is the exact solution of the dark-field image. Figure 11(c) is the difference between the dark-field image (Figure 11(a)) generated in dark-field image generation step S5 and the exact solution (Figure 11(b)). [Figure 12] Figure 12 shows the distribution of illumination directions in wavenumber space for dark-field and bright-field illumination conditions. Figure 12(a) is a phase image, and Figure 12(b) is an image showing the intensity distribution in wavenumber space. [Figure 13] Figure 13 shows the exact solution obtained by performing a composite aperture process based on multiple dark-field and multiple bright-field images. Figure 13(a) is the phase image, and Figure 13(b) is an image showing the intensity distribution in wavenumber space. [Figure 14] Figure 14 shows a bright-field image obtained under bright-field illumination conditions with the illumination direction set to the vertical. Figure 14(a) is a phase image, and Figure 14(b) is an image showing the intensity distribution in wavenumber space. [Figure 15] Figure 15 shows an image obtained by performing a composite aperture processing based on multiple brightfield images obtained under multiple illumination directions under brightfield illumination conditions during the brightfield image generation step. Figure 15(a) is a phase image, and Figure 15(b) is an image showing the intensity distribution in wavenumber space. [Figure 16]Figure 16 shows an image obtained by performing synthetic aperture processing based on multiple bright-field images and multiple dark-field images in the synthetic aperture processing step. Figure 16(a) is a phase image, and Figure 16(b) is an image showing the intensity distribution in wavenumber space. [Figure 17] Figure 17 shows the difference between the image obtained by performing synthetic aperture processing based on multiple bright-field and multiple dark-field images in the synthetic aperture processing step (Figure 16) and the exact solution (Figure 13). Figure 17(a) is the phase image, and Figure 17(b) is the image showing the intensity distribution in wavenumber space. [Figure 18] Figure 18 shows the intensity image I(r) acquired in the first intensity image acquisition step S1 under the first illumination mode. Figure 18(a) is the intensity image at z=950λ, and Figure 18(b) is the intensity image at z=1050λ. [Figure 19] Figure 19 shows the intensity image I(r) acquired in the second intensity image acquisition step S2 when the second illumination mode is used. Figure 19(a) is the intensity image at z=950λ, and Figure 19(b) is the intensity image at z=1050λ. [Figure 20] Figure 20 shows the images generated in the first complex amplitude image generation step S3 when the first illumination mode is in operation. Figure 20(a) is the image shown on the left side of the TIE, Figure 20(b) is the phase image φ(r) obtained by calculating the TIE, and Figure 20(c) is the real part of the complex amplitude image. [Figure 21] Figure 21 shows the images generated in the second complex amplitude image generation step S4 when the second illumination mode is in use. Figure 21(a) is the image shown on the left side of the TIE, Figure 21(b) is the phase image φ(r) obtained by calculating the TIE, and Figure 21(c) is the real part of the complex amplitude image. [Figure 22] Figure 22(a) shows the real part of the dark-field image generated in dark-field image generation step S5. Figure 22(b) shows the exact solution of the dark-field image. [Figure 23]Figure 23 shows the exact solution of the dark-field image at z=0. Figure 23(a) is the phase image, and Figure 23(b) is the image showing the intensity distribution in wavenumber space. [Figure 24] Figure 24 shows the image at z=1000λ obtained by performing a composite aperture processing based on multiple brightfield images obtained under multiple illumination directions under brightfield illumination conditions during the brightfield image generation step. Figure 24(a) is the phase image, and Figure 24(b) is an image showing the intensity distribution in wavenumber space. [Figure 25] Figure 25 shows the image at z=1000λ obtained by performing synthetic aperture processing based on multiple bright-field images and multiple dark-field images in the synthetic aperture processing step. Figure 25(a) is the phase image, and Figure 25(b) is an image showing the intensity distribution in wavenumber space. [Figure 26] Figure 26 shows an image obtained by wavefront propagation in the wavefront step, where multiple bright-field images and multiple dark-field images at z=1000λ are wavefront propagated to generate a complex amplitude image at z=0, and then performing a composite aperture processing based on the multiple complex amplitude images at z=0. Figure 26(a) is a phase image, and Figure 26(b) is an image showing the intensity distribution in wavenumber space. [Modes for carrying out the invention] 【0030】 Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the attached drawings. In the description of the drawings, the same elements will be denoted by the same reference numerals, and redundant descriptions will be omitted. The present invention is not limited to these examples, but is indicated by the claims, and all modifications within the meaning and scope equivalent to the claims are intended to be included. 【0031】 First, we will explain an example of the optical system configuration of the observation device using Figures 1 to 5, and then we will explain the detailed contents of the processing in the observation device and observation method. Figure 1 shows the configuration of the observation device 1. This observation device 1 includes a light source 10, an illumination unit 31, an imaging unit 50, and a processing unit 60, etc. The light source 10 outputs spatially coherent light. The light source 10 may be a laser light source, or it may be a light source such as an SLD (Super Luminescent Diode), SC (Super Continuum) light source, or optical frequency comb light source. Alternatively, spatially incoherent light output from an LED (Light Emitting Diode) or mercury lamp may be passed through a pinhole or the like to enhance spatial coherence. 【0032】 Lens 21 is optically connected to the light source 10 and focuses the light output from the light source 10 onto the optical incident end 22 of the optical fiber 23, causing the light to enter the optical incident end 22. The optical fiber 23 guides the light that entered the optical incident end 22 to the optical output end 24. The light guided by the optical fiber 23 is emitted as divergent light from the optical output end 24. Lens 25 is optically connected to the optical output end 24 and receives the light output as divergent light from the optical output end 24, collimates it, and outputs the collimated light to the illumination unit 31. 【0033】 The illumination unit 31 receives light output from the light source 10 that has passed through lens 21, optical fiber 23, and lens 25. The illumination unit 31 can switch between a first illumination mode and a second illumination mode. In the first illumination mode, the illumination unit 31 sets the difference in optical path length between the first illumination light and the second illumination light generated based on the light output from the light source 10 to be less than or equal to the coherence length, and illuminates the object to be observed S with the first illumination light along the first illumination direction under dark-field illumination conditions, and with the second illumination light along the second illumination direction under bright-field illumination conditions. In the second illumination mode, the illumination unit 31 illuminates the object to be observed S with the second illumination light along the second illumination direction under bright-field illumination conditions. 【0034】 The illumination unit 31 includes a beam splitter 311, a phase-modulated spatial light modulator 313, a polarizer 314, a half-wave plate 315, a polarizer 316, a lens 318, and an objective lens 319. 【0035】 The beam splitter 311 reflects the light that has passed through the polarizer 314 and half-wave plate 315, which are placed between it and the lens 25, to the spatial light modulator 313. The beam splitter 311 also receives the light that has come from the spatial light modulator 313 and outputs this light to the polarizer 316. 【0036】 The spatial light modulator 313 selectively phase-modulates the linearly polarized light of the first direction from among the linearly polarized light of the first and second directions, which are mutually orthogonal, that are incident on the modulation plane. The polarizer 314 and half-wave plate 315 set the polarization state of the light so that the light incident from the beam splitter 311 to the modulation plane of the spatial light modulator 313 contains linearly polarized components of the first and second directions to an equal degree. 【0037】 The polarizer 316 receives light that has arrived from the spatial light modulator 313 via the beam splitter 311 and makes it possible to interfere with the linearly polarized light of the first and second directions contained in that light. The polarizer 316 has an optical axis that is 45 degrees different in direction from the polarization direction of the light that has arrived from the spatial light modulator 313 via the beam splitter 311 (linearly polarized light of the first and second directions), and selectively transmits the polarization component of the input light in the direction of the optical axis. The lens 318 and the objective lens 319 illuminate the object of observation S with the light output from the polarizer 316 (first illumination light and second illumination light) as a plane wave. 【0038】 The illumination unit 31 having such a configuration can illuminate an object S under dark-field illumination conditions along a first illumination direction using linearly polarized light of a first direction, which has been phase-modulated by the spatial light modulator 313, as the first illumination light. The illumination unit 31 can also illuminate an object S under bright-field illumination conditions along a second illumination direction using linearly polarized light of a second direction, which has not been phase-modulated by the spatial light modulator 313, as the second illumination light. 【0039】 The illumination direction of the first illumination light onto the object to be observed S can be set by the period of the phase modulation pattern on the modulation plane of the spatial light modulator 313. Therefore, if the numerical aperture of the illumination-side objective lens 319 is greater than the numerical aperture of the detection-side objective lens 41, in the second illumination embodiment, the illumination unit 31 can illuminate the object to be observed S with only the second illumination light without illuminating the object to be observed S with the first illumination light by adjusting the period of the phase modulation pattern. 【0040】 The objective lens 41 receives light that has been illuminated by the illumination unit 31 onto the object to be observed S and has reached the object to be observed S, and outputs that light to the mirror 42. The lens 43 receives the light that has been output from the objective lens 41 and reflected by the mirror 42, and directs that light onto the imaging surface of the imaging unit 50. 【0041】 The imaging unit 50 receives light that has reached the imaging surface from the lens 43. In both the first illumination mode and the second illumination mode, the imaging unit 50 acquires intensity images at each of the multiple focal planes of the object to be observed S. To acquire intensity images at each of the multiple focal planes of the object to be observed S, the position of the imaging unit 50 in the optical axis direction may be adjusted by the position adjustment unit 51 (e.g., a stage or a piezo actuator), or the position of the object to be observed S, the objective lens 41, or the lens 43 in the optical axis direction may be adjusted. 【0042】 The processing unit 60 is electrically connected to the imaging unit 50 and performs processing based on the intensity image acquired by the imaging unit 50. The processing details of the processing unit 60 will be described later. 【0043】 The observation device can also be configured in other ways. In particular, the illumination unit can be configured in various ways other than the illumination unit 31 described above. Figures 2 to 5 show examples of other configurations for the illumination unit. 【0044】 Figure 2 shows the configuration of the illumination unit 32. The illumination unit 32 includes a beam splitter 321, a mirror 322, an intensity-modulated spatial light modulator 323, a lens 328, and an objective lens 329. The spatial light modulator 323 may be a DMD (Digital Micromirror Device). 【0045】 The beam splitter 321 splits the light arriving from lens 25 into a first-branched beam and a second-branched beam. It outputs the first-branched beam to the spatial light modulator 323 and the second-branched beam to mirror 322. The beam splitter 321 also receives the first-branched beam, which has been intensity-modulated by the spatial light modulator 323, and the second-branched beam, which has been reflected by mirror 322, and outputs these first-branched and second-branched beams to lens 328. Lens 328 and objective lens 329 illuminate the object S as a plane wave using the first-branched and second-branched beams output from beam splitter 321, respectively. 【0046】 The illumination unit 32 having such a configuration can illuminate the object to be observed S along a first illumination direction under dark-field illumination conditions using the first branched light, whose intensity has been modulated by the spatial light modulator 323, as the first illumination light. The illumination unit 32 can also illuminate the object to be observed S along a second illumination direction under bright-field illumination conditions using the second branched light, whose intensity has been reflected by the mirror 322, as the second illumination light. 【0047】 The direction of illumination of the object to be observed S with the first illumination light can be set by the period of the intensity modulation pattern on the modulation plane of the spatial light modulator 323. Therefore, if the numerical aperture of the illumination-side objective lens 329 is greater than the numerical aperture of the detection-side objective lens 41, in the second illumination configuration, the illumination unit 32 can illuminate the object to be observed S with only the second illumination light without illuminating the object to be observed S with the first illumination light by adjusting the period of the intensity modulation pattern. 【0048】 Figure 3 shows the configuration of the illumination unit 33. The illumination unit 33 includes a beam splitter 331, mirrors 332 and 333, a lens 338, and an objective lens 339. The orientation of the reflective surface of mirror 333 is variable, for example, by using a galvanometer mirror, voice coil mirror, or piezo tilt mirror. The orientation of the reflective surface of mirror 332 may also be variable. 【0049】 The beam splitter 331 splits the light arriving from lens 25 into a first-branched beam and a second-branched beam, outputting the first-branched beam to mirror 333 and the second-branched beam to mirror 332. The beam splitter 331 also receives the first-branched beam reflected by mirror 333 and the second-branched beam reflected by mirror 332, and outputs these first-branched and second-branched beams to lens 338. Lens 338 and objective lens 339 illuminate the object S as a plane wave with the first-branched beam and the second-branched beam, respectively, output from the beam splitter 331. 【0050】 The illumination unit 33 having this configuration can use the first branched light reflected by the mirror 333 as the first illumination light to illuminate the object to be observed S along the first illumination direction under dark-field illumination conditions. The illumination unit 33 can use the second branched light reflected by the mirror 332 as the second illumination light to illuminate the object to be observed S along the second illumination direction under bright-field illumination conditions. 【0051】 The direction of illumination of the first illumination light onto the object to be observed S can be set by the orientation of the reflective surface of the mirror 333. Therefore, if the numerical aperture of the illumination-side objective lens 339 is greater than that of the detection-side objective lens 41, in the second illumination configuration, the illumination unit 33 can illuminate the object to be observed S with only the second illumination light without illuminating the object to be observed S with the first illumination light by adjusting the orientation of the reflective surface of the mirror 333. 【0052】 The illumination unit 33A shown in Figure 4 is a modified version of the illumination unit 33 shown in Figure 3, and includes a shutter 334 provided between the beam splitter 331 and the mirror 332. In this configuration, by changing the orientation of the reflective surface of the mirror 333 while the light is blocked by the shutter 334, it is possible to illuminate the object under observation along each of multiple illumination directions under bright-field illumination conditions. 【0053】 The illumination unit 33B shown in Figure 5 is another modification of the illumination unit 33 shown in Figure 3, and includes a shutter 335 provided between the beam splitter 331 and the mirror 333. In this configuration, the illumination unit 33B can illuminate the object to be observed S with only the second illumination light without illuminating the object to be observed S with the first illumination light, by blocking the light with the shutter 335. 【0054】 Next, an observation method using the observation device 1 will be described. Figure 6 is a flowchart of the observation method. This observation method comprises a first intensity image acquisition step S1, a second intensity image acquisition step S2, a first complex amplitude image generation step S3, a second complex amplitude image generation step S4, and a dark-field image generation step S5. The first intensity image acquisition step S1 and the second intensity image acquisition step S2 are performed using the illumination unit 31 (or illumination units 33, 33A, 33B) and the imaging unit 50. The first complex amplitude image generation step S3, the second complex amplitude image generation step S4, and the dark-field image generation step S5 are performed by the processing unit 60. 【0055】 In the first intensity image acquisition step S1, in the first illumination configuration, the illumination unit illuminates the object under observation with the first illumination light along the first illumination direction under dark-field illumination conditions, and with the difference in optical path length between the first illumination light and the second illumination light, generated based on the light output from the light source, being less than or equal to the coherence length. Then, in this first illumination configuration, the imaging unit acquires intensity images at each of the multiple focus planes of the object under observation. 【0056】 In the second intensity image acquisition step S2, in the second illumination configuration, the illumination unit illuminates the object under observation with a second illumination light along the second illumination direction under bright-field illumination conditions. At this time, the illumination light is not emitted under dark-field illumination conditions. Then, in this second illumination configuration, the imaging unit acquires intensity images at each of the multiple focus planes of the object under observation. 【0057】 In the first complex amplitude image generation step S3, a first complex amplitude image is generated using an intensity transport equation based on the intensity images at each of the multiple focus planes acquired in the first intensity image acquisition step S1. 【0058】 In the second complex amplitude image generation step S4, a second complex amplitude image is generated using an intensity transport equation based on the intensity images at each of the multiple focus planes acquired in the second intensity image acquisition step S2. 【0059】 In the dark-field image generation step S5, a complex amplitude image (dark-field image) is generated based on the difference between the first complex amplitude image and the second complex amplitude image when light is shone onto the object under dark-field illumination conditions along the first illumination direction. 【0060】 The order of processing between the first intensity image acquisition step S1 and the second intensity image acquisition step S2 is arbitrary. The processing of the first complex amplitude image generation step S3 may be performed after the processing of the first intensity image acquisition step S1 and before the processing of the dark-field image generation step S5. The processing of the second complex amplitude image generation step S4 may be performed after the processing of the second intensity image acquisition step S2 and before the processing of the dark-field image generation step S5. 【0061】 The intensity transport equations (TIE) used in the first complex amplitude image generation step S3 and the second complex amplitude image generation step S4 are as follows. The TIE is expressed by the following equations (1) and (2). Equation (1) is the TIE when illumination light is obliquely incident on the object being observed, and in the equation k inrepresents the wave vector of the illumination light incident on the object being observed. Equation (2) is the TIE when the illumination light is perpendicularly incident on the object being observed, and k0 in the equation is the wave number of the illumination light incident on the object being observed. r is a variable that represents the two-dimensional position on the xy plane perpendicular to the z axis (the axis parallel to the optical axis of the objective lens). φ(r) represents the phase image. I(r) is the intensity image at each of the multiple focus planes acquired in the first intensity image acquisition step S1 or the second intensity image acquisition step S2. The number of focus planes from which intensity images are acquired can be two or more, for example, five or less. 【0062】 【number】 【0063】 【number】 【0064】 Further explanation of equation (1) above: Assuming I(r) is a constant, we obtain equation (3) below, and further, equation (4) below. This equation is the Poisson equation. Therefore, by solving this Poisson equation, we can calculate the phase image φ(r). We can also obtain the amplitude image from the intensity image. And from these phase and amplitude images, we can generate a complex amplitude image. 【0065】 【number】 【0066】 【number】 【0067】 The observation method of this embodiment can include synthetic aperture processing (see Simulation B described later). In this case, in the dark-field image generation step S5, a complex amplitude image (dark-field image) is generated based on the difference between the first complex amplitude image and the second complex amplitude image when the object to be observed is illuminated with light along each of the multiple first illumination directions under dark-field illumination conditions. Furthermore, the observation method of this embodiment further comprises a bright-field image generation step and a synthetic aperture processing step. In the bright-field image generation step, a complex amplitude image (bright-field image) is generated by the illumination unit when the object to be observed is illuminated with light along each of the multiple illumination directions under bright-field illumination conditions using an intensity transport equation based on the intensity images at each of the multiple focus planes acquired by the imaging unit. Then, in the synthetic aperture processing step, synthetic aperture processing is performed based on the complex amplitude images (dark-field images) for each of the multiple first illumination directions under dark-field illumination conditions generated in the dark-field image generation step, and the complex amplitude images (bright-field images) for each of the multiple illumination directions under bright-field illumination conditions generated in the bright-field image generation step. 【0068】 Furthermore, the observation method of this embodiment may further include a wavefront propagation step in which the complex amplitude image generated by TIE is wavefront propagated (digital refocused) to generate a complex amplitude image at another location (see Simulation C described later). 【0069】 Next, we will explain the results of simulations A to C performed on the observation method of this embodiment. 【0070】 Figures 7 to 11 show the results of Simulation A. Here, the first illumination direction (θ) under dark-field illumination conditions is shown. x ,θ yThe 0.22° and 1.16° angles were set, and the second illumination direction under bright-field illumination conditions was set to the vertical direction (parallel to the z-axis). In the first intensity image acquisition step S1 and the second intensity image acquisition step S2, intensity images I(r) were acquired by the imaging unit at two positions, z=-50λ and z=+50λ, with wavelength λ, and a dark-field image was generated in the dark-field image generation step S5. 【0071】 Figure 7 shows the intensity image I(r) acquired in the first intensity image acquisition step S1 under the first illumination mode. Figure 8 shows the intensity image I(r) acquired in the second intensity image acquisition step S2 under the second illumination mode. In Figures 7 and 8, (a) is the intensity image at z = -50λ and (b) is the intensity image at z = +50λ. 【0072】 Figure 9 shows the images generated in the first complex amplitude image generation step S3 when the first illumination mode is used. Figure 10 shows the images generated in the second complex amplitude image generation step S4 when the second illumination mode is used. In Figures 9 and 10, (a) is the image shown on the left side of the TIE, (b) is the phase image φ(r) obtained by calculating the TIE, and (c) is the real part of the complex amplitude image. Figure 9(c) is the first complex amplitude image, and Figure 10(c) is the second complex amplitude image. 【0073】 Figure 11(a) is the real part of the dark-field image generated in dark-field image generation step S5. Figure 11(b) is the exact solution of the dark-field image. Figure 11(c) is the difference between the dark-field image generated in dark-field image generation step S5 (Figure 11(a)) and the exact solution (Figure 11(b)). As shown in this figure, the dark-field image generated in dark-field image generation step S5 (Figure 11(a)) showed good agreement with the exact solution (Figure 11(b)). 【0074】 Figures 12 to 17 show the results of Simulation B. Here, in the dark-field image generation step, complex amplitude images (dark-field images) are acquired when light is shone on the object under multiple illumination directions under dark-field illumination conditions. In this case, the second illumination direction under bright-field illumination conditions was set to the vertical direction. In the bright-field image generation step, complex amplitude images (bright-field images) are acquired when light is shone on the object under multiple illumination directions under bright-field illumination conditions. With wavelength λ, intensity images I(r) are acquired by the imaging unit at two positions z=-50λ and z=+50λ. In the composite aperture processing step, composite aperture processing is performed based on the multiple complex amplitude images (dark-field images) generated in the dark-field image generation step and the multiple complex amplitude images (bright-field images) generated in the bright-field image generation step. 【0075】 Figure 12 shows the distribution of illumination directions in wavenumber space for dark-field and bright-field illumination conditions. This figure shows one element k of the wavenumber vector of the illumination light incident on the object being observed. x With the horizontal axis being k, the other element is y In wavenumber space with the vertical axis, each position (illumination direction) is discretely arranged in a square grid, and different symbols indicate whether the conditions are dark-field illumination or bright-field illumination. 【0076】 Figure 13 shows the exact solution obtained by performing a composite aperture process based on multiple dark-field images and multiple bright-field images. Figure 14 shows the bright-field image obtained when the illumination direction is set to the vertical under bright-field illumination conditions. Figure 15 shows the image obtained by performing a composite aperture process based on multiple bright-field images obtained for each of multiple illumination directions under bright-field illumination conditions in the bright-field image generation step. Figure 16 shows the image obtained by performing a composite aperture process based on multiple bright-field images and multiple dark-field images in the composite aperture process step. Figure 17 is the image of the difference between the image obtained by performing a composite aperture process based on multiple bright-field images and multiple dark-field images in the composite aperture process step (Figure 16) and the exact solution (Figure 13). In Figures 13 to 17, (a) is the phase image and (b) is the image showing the intensity distribution in wavenumber space. The intensity distribution in wavenumber space was obtained by performing a Fourier transform on the complex amplitude image. 【0077】 As shown in these figures, the resolution of the image obtained by performing a composite aperture processing on multiple bright-field images obtained under multiple illumination directions under bright-field illumination conditions (Figure 15) is higher than the resolution of the bright-field image obtained under one illumination direction under bright-field illumination conditions (Figure 14). Furthermore, the resolution of the image obtained by performing a composite aperture processing on multiple bright-field images and multiple dark-field images in the composite aperture processing step (Figure 16) is even higher. In addition, the image obtained by performing a composite aperture processing in the composite aperture processing step (Figure 16) showed good agreement with the exact solution (Figure 13). 【0078】 Figures 18 to 26 show the results of simulation C. Here, the first illumination direction (θ) under dark-field illumination conditions is shown. x ,θ yThe 0.22°, 1.16° angles were set, and the second illumination direction under brightfield illumination conditions was set to the vertical direction. Based on the intensity images I(r) acquired by the imaging unit at two positions, z=950λ and z=1050λ, with wavelength λ, a complex amplitude image at z=1000λ was generated by TIE, and this complex amplitude image at z=0 was generated by wavefront propagation in the wavefront propagation step. Synthetic aperture processing was also performed. 【0079】 Figure 18 shows the intensity image I(r) acquired in the first intensity image acquisition step S1 under the first illumination mode. Figure 19 shows the intensity image I(r) acquired in the second intensity image acquisition step S2 under the second illumination mode. In Figures 18 and 19, (a) is the intensity image at z=950λ and (b) is the intensity image at z=1050λ. 【0080】 Figure 20 shows the images generated in the first complex amplitude image generation step S3 when the first illumination mode is used. Figure 21 shows the images generated in the second complex amplitude image generation step S4 when the second illumination mode is used. In Figures 20 and 21, (a) is the image shown on the left side of the TIE, (b) is the phase image φ(r) obtained by calculating the TIE, and (c) is the real part of the complex amplitude image. Figure 20(c) is the first complex amplitude image at z=1000λ, and Figure 21(c) is the second complex amplitude image at z=1000λ. 【0081】 Figure 22(a) shows the real part of the dark-field image generated in dark-field image generation step S5. Figure 22(b) shows the exact solution of the dark-field image. These images are dark-field images at z=1000λ. As shown in this figure, the dark-field image generated in dark-field image generation step S5 (Figure 22(a)) showed good agreement with the exact solution (Figure 22(b)). 【0082】 Figure 23 shows the exact solution for the dark-field image at z=0. Figure 24 shows the image at z=1000λ obtained by performing a composite aperture process based on multiple bright-field images obtained under multiple illumination directions under bright-field illumination conditions in the bright-field image generation step. Figure 25 shows the image at z=1000λ obtained by performing a composite aperture process based on multiple bright-field images and multiple dark-field images in the composite aperture process step. Figure 26 shows the image obtained by generating a complex amplitude image at z=0 by wavefront propagation of multiple bright-field images and multiple dark-field images at z=1000λ in the wavefront propagation step, and then performing a composite aperture process based on the multiple complex amplitude images at z=0. In Figures 23 to 26, (a) is the phase image, and (b) is the image showing the intensity distribution in wavenumber space. As shown in these figures, the images obtained by performing each process in the wavefront propagation step and the composite aperture process step (Figure 26) were in good agreement with the exact solution (Figure 23). 【0083】 As described above, in this embodiment, the illumination unit 31 (or illumination units 33, 33A, 33B) can be integrated, making optical system adjustment easier, providing superior stability, and allowing for a simpler device configuration compared to using a two-beam interferometer. Furthermore, in this embodiment, since the complex amplitude image of the object observed during dark-field illumination is obtained by utilizing both the illumination light of dark-field illumination and the illumination light of bright-field illumination, the dark-field image can be accurately obtained. In addition, by performing composite aperture processing based on multiple dark-field images and multiple bright-field images, a wide-field and high-resolution image can be generated. [Explanation of symbols] 【0084】 1... Observation device, 10... Light source, 21... Lens, 22... Light incident end, 23... Optical fiber, 24... Light exit end, 25... Lens, 31~33,33A,33B... Illumination unit, 41... Objective lens, 42... Mirror, 43... Lens, 50... Imaging unit, 60... Processing unit, 311... Beam splitter, 313... Spatial light modulator, 314... Polarizer, 315... Half-wave plate, 316... Polarizer, 318... Lens, 319... Objective lens, 321... Beam splitter, 322... Mirror, 323... Spatial light modulator, 328... Lens, 329... Objective lens, 331... Beam splitter, 332,333... Mirror, 334,335... Shutter, 338... Lens, 339... Objective lens.

Claims

[Claim 1] A light source that emits light, In the first illumination configuration, the difference in optical path length between the first illumination light and the second illumination light generated based on the light output from the light source is set to be less than or equal to the coherent length, and the first illumination light illuminates the object to be observed along the first illumination direction under dark-field illumination conditions, and the second illumination light illuminates the object to be observed along the second illumination direction under bright-field illumination conditions. In the second illumination configuration, the illumination unit illuminates the object to be observed along the second illumination direction under bright-field illumination conditions. In each of the first and second illumination modes, an imaging unit acquires intensity images at each of the multiple focus planes of the object to be observed, A processing unit that performs processing based on the intensity image acquired by the imaging unit, Equipped with, The aforementioned processing unit, In the first illumination configuration, a first complex amplitude image is generated using an intensity transport equation based on the intensity images at each of the plurality of focus planes acquired by the imaging unit. In the second illumination configuration, a second complex amplitude image is generated using an intensity transport equation based on the intensity images at each of the multiple focus planes acquired by the imaging unit. A complex amplitude image is generated based on the difference between the first complex amplitude image and the second complex amplitude image when light is shone onto the object to be observed along the first illumination direction under dark-field illumination conditions. Observation device. [Claim 2] The aforementioned processing unit, A complex amplitude image is generated based on the difference between the first complex amplitude image and the second complex amplitude image when the object being observed is illuminated with light along each of the multiple first illumination directions under dark-field illumination conditions. The illumination unit generates a complex amplitude image of the object being observed when it is illuminated by light along each of the multiple illumination directions under bright-field illumination conditions, based on the intensity images at each of the multiple focus planes acquired by the imaging unit, using an intensity transport equation. A composite aperture process is performed based on the complex amplitude images of each of the multiple first illumination directions under dark-field illumination conditions and the complex amplitude images of each of the multiple illumination directions under bright-field illumination conditions. The observation apparatus according to claim 1. [Claim 3] The processing unit generates a complex amplitude image at another location by wavefront propagation of the complex amplitude image generated by the intensity transport equation. The observation apparatus according to claim 1. [Claim 4] The aforementioned lighting unit is The system includes a beam splitter that splits the light output from the light source into a first branched beam and a second branched beam, a first reflector that reflects the first branched beam output from the beam splitter, and a second reflector that reflects the second branched beam output from the beam splitter. In the first illumination mode, one of the first branched light reflected by the first reflector and the second branched light reflected by the second reflector is used to illuminate the object under dark-field illumination conditions, and the other is used to illuminate the object under bright-field illumination conditions. The observation apparatus according to claim 1. [Claim 5] The orientation of the reflective surface of the first reflecting part and the orientation of the reflective surface of the second reflecting part, or both or either of them, are variable. The observation apparatus according to claim 4. [Claim 6] Both or either of the first and second reflectors are spatial light modulators. The observation apparatus according to claim 4. [Claim 7] The illumination unit further includes a shutter provided on either or both of the optical paths of the first branched light from the beam splitter to the object to be observed, and the optical path of the second branched light from the beam splitter to the object to be observed. The observation apparatus according to claim 4. [Claim 8] The aforementioned lighting unit is The system includes a spatial light modulator that selectively phase-modulates light with respect to either one of the mutually orthogonal first and second polarization components of the light output from the light source, In the first illumination mode, one of the light of the first polarization component and the light of the second polarization component is used to illuminate the object under dark-field illumination conditions, and the other is used to illuminate the object under bright-field illumination conditions. The observation apparatus according to claim 1. [Claim 9] In the first illumination configuration, the illumination unit illuminates the object to be observed with the first illumination light along a first illumination direction under dark-field illumination conditions and the second illumination light along a second illumination direction under bright-field illumination conditions, while the imaging unit acquires an intensity image of each of the multiple focus planes of the object to be observed, by setting the difference in optical path length between the first illumination light and the second illumination light generated based on the light output from the light source to be less than or equal to the coherence length. In a second illumination embodiment, the illumination unit illuminates the object to be observed with the second illumination light along the second illumination direction under bright-field illumination conditions, and the imaging unit acquires an intensity image of each of the multiple focus planes of the object to be observed in a second intensity image acquisition step, A first complex amplitude image generation step generates a first complex amplitude image using an intensity transport equation based on the intensity images at each of the plurality of focus planes acquired in the first intensity image acquisition step, A second complex amplitude image generation step is performed in which a second complex amplitude image is generated using an intensity transport equation based on the intensity images at each of the plurality of focus planes acquired in the second intensity image acquisition step, A dark-field image generation step in which a complex amplitude image is generated based on the difference between a first complex amplitude image and a second complex amplitude image when light is shone onto the object to be observed along the first illumination direction under dark-field illumination conditions, An observation method that includes the following features. [Claim 10] In the dark-field image generation step, a complex amplitude image is generated based on the difference between the first complex amplitude image and the second complex amplitude image when light is shone onto the object under each of the multiple first illumination directions under dark-field illumination conditions. A bright-field image generation step in which a complex amplitude image is generated by the illumination unit when the object to be observed is illuminated with light along each of a plurality of illumination directions under bright-field illumination conditions, based on the intensity images at each of a plurality of focus planes acquired by the imaging unit, using an intensity transport equation; A composite aperture processing step is performed based on the complex amplitude images of each of the multiple first illumination directions under dark-field illumination conditions and the complex amplitude images of each of the multiple illumination directions under bright-field illumination conditions. It also has, The observation method according to claim 9. [Claim 11] The system further includes a wavefront propagation step in which a complex amplitude image generated by the intensity transport equation is propagated as a wavefront to generate a complex amplitude image at another location. The observation method according to claim 9. [Claim 12] The aforementioned lighting unit is The system includes a beam splitter that splits the light output from the light source into a first branched beam and a second branched beam, a first reflector that reflects the first branched beam output from the beam splitter, and a second reflector that reflects the second branched beam output from the beam splitter. In the first illumination mode, one of the first branched light reflected by the first reflector and the second branched light reflected by the second reflector is used to illuminate the object under dark-field illumination conditions, and the other is used to illuminate the object under bright-field illumination conditions. The observation method according to claim 9. [Claim 13] The orientation of the reflective surface of the first reflecting part and the orientation of the reflective surface of the second reflecting part, or both or either of them, are variable. The observation method according to claim 12. [Claim 14] Both or either of the first and second reflectors are spatial light modulators. The observation method according to claim 12. [Claim 15] The illumination unit further includes a shutter provided on either or both of the optical paths of the first branched light from the beam splitter to the object to be observed, and the optical path of the second branched light from the beam splitter to the object to be observed. The observation method according to claim 12. [Claim 16] The aforementioned lighting unit is The system includes a spatial light modulator that selectively phase-modulates light with respect to either one of the mutually orthogonal first and second polarization components of the light output from the light source, In the first illumination mode, one of the light of the first polarization component and the light of the second polarization component is used to illuminate the object under dark-field illumination conditions, and the other is used to illuminate the object under bright-field illumination conditions. The observation method according to claim 9.

Images