Observation device and observation method

Abstract

This observation device 1 comprises a light source 11A, a polarization-dependent spatial modulation element 26, a polarization camera 29, and a processing unit 40. The light source 11A outputs polarized spatially coherent light. The polarization-dependent spatial modulation element 26 is provided on a Fourier plane for an imaging surface 29a of the polarization camera 29, receives light that passes through an observation target S, spatially modulates the polarization state of the light, and outputs the spatially modulated light. The polarization camera 29 acquires an intensity image of each of a plurality of polarization components on the imaging surface 29a. The processing unit 40 obtains the complex amplitude distribution of the light at the time of input to the polarization-dependent spatial modulation element 26 on the basis of: the intensity image of each of the plurality of polarization components; and the relationship between the complex amplitude distribution of the light at the time of input to the polarization-dependent spatial modulation element 26 and the complex amplitude distribution of the light on the imaging surface 29a. Thus, an observation device and an observation method are achieved whereby a complex amplitude image of an observation target can be easily acquired with a simple configuration even when light scattering in the observation target is intense.

Description

Observation apparatus and observation method

[0001] This disclosure relates to an observation apparatus and an observation method.

[0002] The amount of phase change of light is proportional to the optical path length, so it can be used to measure the thickness and refractive index of the material through which the light has propagated. In some cases, it is necessary to acquire a phase image as an image of the object being observed. For example, when observing cells, phase images make it easier to observe cells than intensity images. Techniques that apply phase to imaging include quantitative phase microscopy and optical diffraction tomography (ODT).

[0003] Intensity images can be easily obtained by measuring the intensity distribution of light using a standard camera. In contrast, obtaining phase images is not easy. This is because, due to the extremely high frequency of light, standard photodetectors cannot track the vibrations of light and therefore cannot measure the phase.

[0004] Methods for acquiring complex amplitude images including phase include interferometric complex amplitude measurement and intensity measurement complex amplitude measurement. In interferometric complex amplitude measurement, interference images are acquired using a Michelson interferometer or Mach-Zehnder interferometer, and phase images are generated based on these interference images using the phase shift method or fringe interferometry (Non-Patent Literature 1). Intensity measurement complex amplitude measurement techniques include techniques using the transport of intensity equation (TIE) and the spatial Kramers-Kronig method.

[0005] The TIE technique acquires intensity images at multiple z-positions (positions on the z-axis parallel to the optical axis) of the object being observed, and generates a complex amplitude image using TIE based on these multiple intensity images (Non-Patent Literature 2). The spatial Kramers-Kronig method acquires intensity images by irradiating the object being observed with light at a numerical aperture matching the numerical aperture of the detector, and generates a complex amplitude image using the Kramers-Kronig relation based on these intensity images (Non-Patent Literature 3).

[0006] Mitsuo Takeda et al., "Fourier-transform method of fringe-pattern analysis for computer-based topography and interferometry", J. Opt. Soc. Am., Vol.72, No.1, pp.156-160, 1982Chao Zuo et al., "Transport of intensity equation: a tutorial", Optics and Lasers in Engineering, Vol.135, 106187, 2020YoonSeok Baek et al., "Intensity-based holographic imaging via space-domain Kramers-Kronig relations", Nature Photonics, Vol.15, pp.354-360, 2021

[0007] Interferometric complex amplitude measurement can generate accurate phase images even when light scattering is strong in the object being observed. However, because interferometric complex amplitude measurement uses a two-beam interferometer, the optical system is not easy to adjust, there are stability issues, and the equipment is complex and expensive.

[0008] The intensity-based complex amplitude measurement method can overcome the problems of the interferometric complex amplitude measurement method described above. However, because the intensity-based complex amplitude measurement method uses approximations in phase calculations, it is difficult to apply to the wavefront of light strongly scattered by the object being observed.

[0009] The embodiment aims to provide an observation device and observation method that can easily acquire a complex amplitude image of an object with a simple configuration, even when light scattering is strong in the object being observed.

[0010] An embodiment of the first aspect is an observation device. The observation device comprises (1) an illumination unit that irradiates an object to be observed with polarized light, (2) an imaging unit that acquires intensity images of each of a plurality of polarization components on the imaging surface, (3) a polarization-dependent spatial modulation element provided on the Fourier plane with respect to the imaging surface, which receives light that has been irradiated by the illumination unit and passed through the object to be observed, spatially modulates the polarization state of the light, and outputs the modulated light to the imaging unit, and (4) a processing unit that obtains a complex amplitude image of the object to be observed, and (5) the processing unit obtains the complex amplitude distribution of the light at the time of input to the polarization-dependent spatial modulation element and the intensity image I of the j-th polarization component among the intensity images of each of the plurality of polarization components obtained by the imaging unit. ｊ The pupil function P relates the relationship between and ｊ After performing an initialization step to initialize the complex amplitude distribution of light on the Fourier plane, the pupil function P is used to determine the complex amplitude distribution of light on the Fourier plane for each of the multiple polarization components. ｊ A modulation processing step that modulates based on, an inverse Fourier transform processing step that performs an inverse Fourier transform on the complex amplitude distribution of light after this modulation processing to obtain the complex amplitude distribution of light on the imaging plane, and an intensity image I which is the amplitude of the complex amplitude distribution of light after this inverse Fourier transform processing. ｊ An amplitude constraint processing step that constrains based on the amplitude of the light; a Fourier transform processing step that performs a Fourier transform on the complex amplitude distribution of the light after this amplitude constraint processing to obtain the complex amplitude distribution of the light on the Fourier plane; and the complex amplitude distribution of the light and the pupil function P after this Fourier transform processing. ｊ A series of processes, including an update step that updates the complex amplitude distribution of light in the Fourier plane based on the above, is repeated multiple times to obtain the complex amplitude distribution of light at the time of input to the polarization-dependent spatial modulation element, and a complex amplitude image of the object to be observed is obtained based on this complex amplitude distribution.

[0011] An embodiment of the second aspect is an observation device. The observation device includes: (1) an irradiation unit that irradiates an observation object with unpolarized light; (2) an imaging unit that acquires intensity images of a plurality of polarization components on an imaging surface; (3) a polarization-dependent spatial modulation element provided on a Fourier surface with respect to the imaging surface, which inputs the light irradiated by the irradiation unit and passing through the observation object, spatially modulates the polarization state of the light, and outputs the modulated light to the imaging unit; (4) a polarizer provided on the optical path between the observation object and the polarization-dependent spatial modulation element; and (5) a processing unit that obtains a complex amplitude image of the observation object. (6) The processing unit uses a pupil function P that relates the complex amplitude distribution of the light at the input to the polarization-dependent spatial modulation element and the intensity image I of the j-th polarization component among the intensity images of the plurality of polarization components acquired by the imaging unit. ｊ After performing an initialization step of initializing the complex amplitude distribution of the light on the Fourier surface, for each of the plurality of polarization components, a modulation processing step of modulating the complex amplitude distribution of the light on the Fourier surface based on the pupil function P, an inverse Fourier transform processing step of performing an inverse Fourier transform on the complex amplitude distribution of the light after this modulation processing to obtain the complex amplitude distribution of the light on the imaging surface, an amplitude constraint processing step of constraining the amplitude of the complex amplitude distribution of the light after this inverse Fourier transform processing based on the amplitude of the intensity image I, a Fourier transform processing step of performing a Fourier transform on the complex amplitude distribution of the light after this amplitude constraint processing to obtain the complex amplitude distribution of the light on the Fourier surface, and an update processing step of updating the complex amplitude distribution of the light on the Fourier surface based on the complex amplitude distribution of the light after this Fourier transform processing and the pupil function P. A series of these processes are repeatedly performed a plurality of times to obtain the complex amplitude distribution of the light at the input to the polarization-dependent spatial modulation element, and based on this complex amplitude distribution, a complex amplitude image of the observation object is obtained. ｊ as, and ｊ ｊ ｊ

[0012] ​​​An embodiment of the first aspect is an observation method. The observation method is a method for obtaining a complex amplitude image of an object to be observed using (1) an illumination unit that irradiates an object to be observed with polarized light, (2) an imaging unit that acquires intensity images of each of a plurality of polarization components on the imaging surface, and (3) a polarization-dependent spatial modulation element provided on the Fourier plane with respect to the imaging surface, which receives light that has been irradiated by the illumination unit and passed through the object to be observed, spatially modulates the polarization state of the light, and outputs the modulated light to the imaging unit, wherein (4) the complex amplitude distribution of the light at the time of input to the polarization-dependent spatial modulation element and the intensity image I of the j-th polarization component among the intensity images of each of the plurality of polarization components acquired by the imaging unit. ｊ The pupil function P relates the relationship between and ｊ After performing an initialization step to initialize the complex amplitude distribution of light on the Fourier plane, the pupil function P is used to determine the complex amplitude distribution of light on the Fourier plane for each of the multiple polarization components. ｊ A modulation processing step that modulates based on, an inverse Fourier transform processing step that performs an inverse Fourier transform on the complex amplitude distribution of light after this modulation processing to obtain the complex amplitude distribution of light on the imaging plane, and an intensity image I which is the amplitude of the complex amplitude distribution of light after this inverse Fourier transform processing. ｊ An amplitude constraint processing step that constrains based on the amplitude of the light; a Fourier transform processing step that performs a Fourier transform on the complex amplitude distribution of the light after this amplitude constraint processing to obtain the complex amplitude distribution of the light on the Fourier plane; and the complex amplitude distribution of the light and the pupil function P after this Fourier transform processing. ｊ A series of processes, including an update step that updates the complex amplitude distribution of light in the Fourier plane based on the above, is repeated multiple times to obtain the complex amplitude distribution of light at the time of input to the polarization-dependent spatial modulation element, and a complex amplitude image of the object to be observed is obtained based on this complex amplitude distribution.

[0013] The second embodiment is an observation method. The observation method is a method for obtaining a complex amplitude image of an object using (1) an illumination unit that irradiates an object to be observed with unpolarized light, (2) an imaging unit that acquires intensity images of each of a plurality of polarization components on the imaging surface, (3) a polarization-dependent spatial modulation element provided on the Fourier plane with respect to the imaging surface, which receives light that has been irradiated by the illumination unit and passed through the object to be observed, spatially modulates the polarization state of the light, and outputs the modulated light to the imaging unit, and (4) a polarizer provided in the optical path between the object to be observed and the polarization-dependent spatial modulation element, wherein (5) the complex amplitude distribution of the light at the time of input to the polarization-dependent spatial modulation element and the intensity image I of the j-th polarization component among the intensity images of each of the plurality of polarization components acquired by the imaging unit. ｊ The pupil function P relates the relationship between and ｊ After performing an initialization step to initialize the complex amplitude distribution of light on the Fourier plane, the pupil function P is used to determine the complex amplitude distribution of light on the Fourier plane for each of the multiple polarization components. ｊ A modulation processing step that modulates based on, an inverse Fourier transform processing step that performs an inverse Fourier transform on the complex amplitude distribution of light after this modulation processing to obtain the complex amplitude distribution of light on the imaging plane, and an intensity image I which is the amplitude of the complex amplitude distribution of light after this inverse Fourier transform processing. ｊ An amplitude constraint processing step that constrains based on the amplitude of the light; a Fourier transform processing step that performs a Fourier transform on the complex amplitude distribution of the light after this amplitude constraint processing to obtain the complex amplitude distribution of the light on the Fourier plane; and the complex amplitude distribution of the light and the pupil function P after this Fourier transform processing. ｊ A series of processes, including an update step that updates the complex amplitude distribution of light in the Fourier plane based on the above, is repeated multiple times to obtain the complex amplitude distribution of light at the time of input to the polarization-dependent spatial modulation element, and a complex amplitude image of the object to be observed is obtained based on this complex amplitude distribution.

[0014] According to the observation apparatus and observation method of the embodiment, even when light scattering is strong in the object being observed, a complex amplitude image of the object can be easily obtained with a simple configuration.

[0015] Figure 1 shows the configuration of observation device 1. Figure 2 shows the configuration of observation device 2. Figure 3 shows the configuration of observation device 3. Figure 4 shows the configuration of observation device 4. Figure 5 shows another example of the optical system configuration from the objective lens 21 to the polarizing camera 29 among the configurations of observation devices 1 to 3. Figure 6 shows an example of the image sensor configuration of the polarizing camera 29. Figure 7 shows another example of the optical system configuration from the objective lens 21 onwards. Figure 8 shows an example of the velocity axis orientation distribution pattern in the polarization-dependent spatial modulation element 26. Figure 9 shows another example of the velocity axis orientation distribution pattern in the polarization-dependent spatial modulation element 26. Figure 10 is an example of a flowchart of the observation method. Figure 11 shows (a) and (b) an example of the complex amplitude distribution U(k) before processing in the modulation processing step S5. Figure 12 shows (a) and (b) an example of the complex amplitude distribution U'(k) after processing in the modulation processing step S5. Figure 13 shows (a) and (b) an example of the complex amplitude distribution u(r) after processing in the inverse Fourier transform processing step S6. Figure 14 shows (a) and (b) examples of the complex amplitude distribution u(r) after processing in the amplitude constraint processing step S7. Figure 15 shows (a) and (b) examples of the complex amplitude distribution U'(k) after processing in the Fourier transform processing step S8. Figure 16 shows (a) and (b) examples of the complex amplitude distribution U(k) after processing in the update processing step S9. Figure 17 is a diagram explaining the observed object S assumed in simulations A and B, respectively. Figure 18 shows (a) to (c) the pupil function P for the horizontal polarization component. Ｈ (k) is shown in this figure. Figure 19 shows (a) to (c) the pupil function P for the vertical polarization component. Ｖ Figure (k) is shown. Figure 20 shows the pupil function P for the right circular polarization component (a) to (c). Ｒ (k) is shown in this figure. Figure 21 shows the pupil function P for the left circularly polarized component (a) to (c). Ｌ (k) is shown in the figure. Figure 22 shows (a) intensity image I of the horizontal polarization component. Ｈ (r) is shown in the figure, and (b) is the intensity image I of the vertical polarization component. Ｖ (r) is shown in the figure. Figure 23 is (a) intensity image I of the right circularly polarized component. Ｒ (r) is shown in the figure, and (b) is the intensity image of the left circularly polarized component I.Ｌ Figure 24 shows (a) and (b) the exact solutions. Figure 25 shows (a) and (b) the complex amplitude images obtained when the number of iterations N is 1 in the flowchart shown in Figure 10. Figure 26 shows (a) and (b) the complex amplitude images obtained when the number of iterations N is 11 in the flowchart shown in Figure 10. Figure 27 shows (a) and (b) the complex amplitude images obtained when the number of iterations N is 100 in the flowchart shown in Figure 10. Figure 28 shows (a) to (c) the pupil function P for the horizontal polarization component. Ｈ (k) is shown in this figure. Figure 29 shows (a) to (c) the pupil function P for the vertical polarization component. Ｖ Figure (k) is shown. Figure 30 shows the pupil function P for the right circular polarization component (a) to (c). Ｒ (k) is shown in this figure. Figure 31 shows the pupil function P for the left circularly polarized component (a) to (c). Ｌ (k) is shown in the figure. Figure 32 shows (a) intensity image I of the horizontal polarization component. Ｈ (r) is shown in the figure, and (b) is the intensity image I of the vertical polarization component. Ｖ (r) is shown in the figure. Figure 33 is (a) intensity image I of the right circular polarization component. Ｒ (r) is shown in the figure, and (b) is the intensity image of the left circularly polarized component I. Ｌ Figure 34 shows (r) and (a) the exact solutions. Figure 35 shows (a) and (b) the complex amplitude images obtained when the number of iterations N is 1 in the flowchart shown in Figure 10. Figure 36 shows (a) and (b) the complex amplitude images obtained when the number of iterations N is 11 in the flowchart shown in Figure 10. Figure 37 shows (a) and (b) the complex amplitude images obtained when the number of iterations N is 100 in the flowchart shown in Figure 10. Figure 38 shows (a) the intensity image I of the horizontal polarization component. Ｈ (r) is shown in the figure, and (b) is the intensity image I of the vertical polarization component. Ｖ (r) is shown in the figure. Figure 39 is (a) intensity image I of the right circular polarization component. Ｒ (r) is shown in the figure, and (b) is the intensity image of the left circularly polarized component I. ＬFigure (r) is shown. Figure 40 shows (a) and (b) the exact solution. Figure 41 shows (a) and (b) the complex amplitude images obtained when the number of iterations N is 1 in the flowchart shown in Figure 10. Figure 42 shows (a) and (b) the complex amplitude images obtained when the number of iterations N is 11 in the flowchart shown in Figure 10. Figure 43 shows (a) and (b) the complex amplitude images obtained when the number of iterations N is 100 in the flowchart shown in Figure 10.

[0016] Embodiments of the observation apparatus and observation method will be described in detail below with reference to the attached drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant descriptions are omitted. The present invention is not limited to these examples, but is indicated by the claims, and all modifications within the meaning and scope equivalent to the claims are intended to be included.

[0017] First, we will explain an example of the optical system configuration of the observation device using Figures 1 to 9, and then we will explain the detailed contents of the processing in the observation device and observation method.

[0018] Figure 1 shows the configuration of the observation device 1. This observation device 1 includes a light source 11A, a polarization-dependent spatial modulation element 26, a polarization camera 29, and a processing unit 40, etc. The light source 11A outputs polarized, spatially coherent light. The light source 11A may be a laser light source, or it may be a light source such as an SLD (Super Luminescent Diode), SC (Super Continuum) light source, or optical frequency comb light source. Alternatively, spatially incoherent light output from an LED (Light Emitting Diode) or mercury lamp may be passed through a pinhole or the like to enhance spatial coherence.

[0019] Light emitted from the light source 11A is collimated by lenses 13 and 14, reflected by mirror 15, and then irradiated onto the object to be observed S via lens 16 and objective lens 17. The optical system from the light source 11A to the objective lens 17 constitutes an illumination section that irradiates the object to be observed S with polarized light.

[0020] Light that is irradiated onto the object to be observed S and passes through the object to be observed S is reflected by the mirror 22 after passing through the objective lens 21, and then passes through lens 23, lens 24, polarization-dependent spatial modulation element 26 and lens 28 in order before being input to the imaging surface 29a of the polarization camera 29.

[0021] The polarization-dependent spatial modulation element 26 is located on the Fourier plane of the imaging surface 29a of the polarization camera 29. If the imaging surface 29a of the polarization camera 29 is on a plane in real space, then the polarization-dependent spatial modulation element 26 is on a plane in wavenumber space. The polarization-dependent spatial modulation element 26 receives light that has been irradiated by the illumination unit and passed through the object to be observed S, spatially modulates the polarization state of that light, and outputs the modulated light to the polarization camera 29. The polarization camera 29 is an imaging unit that acquires intensity images of each of multiple polarization components on the imaging surface 29a.

[0022] The processing unit 40 determines the complex amplitude distribution of light at the time of input to the polarization-dependent spatial modulation element 26 based on the intensity images of each of the multiple polarization components acquired by the polarization camera 29, and the relationship between the complex amplitude distribution of light at the time of input to the polarization-dependent spatial modulation element 26 and the complex amplitude distribution of light on the imaging surface 29a.

[0023] In the object S to be observed, the cross-section (observation surface) and the polarization-dependent spatial modulation element 26 may be in an optically conjugate positional relationship with each other. In this case, the complex amplitude distribution of the light input to the polarization-dependent spatial modulation element 26 represents the complex amplitude image of the observation surface.

[0024] The observation surface and the imaging surface 29a of the polarization camera 29 may be in an optically conjugate positional relationship with each other. In this case, the processing unit 40 performs an inverse Fourier transform on the complex amplitude distribution of the light input to the polarization-dependent spatial modulation element 26, and uses the resulting complex amplitude distribution as the complex amplitude image of the observation surface.

[0025] Furthermore, the observation surface does not have to be in an optically conjugate positional relationship with either the polarization-dependent spatial modulation element 26 or the imaging surface 29a. The processing unit 40 can obtain a complex amplitude image of the observation surface at any position by calculating wavefront propagation based on the complex amplitude distribution of light input to the polarization-dependent spatial modulation element 26.

[0026] The mirror 15 may be a galvanometer mirror with a variable orientation of its reflective surface, or it may be a spatial light modulator that can change the direction of light reflection. The spatial light modulator can change the reflection direction by an intensity modulation pattern or phase modulation pattern presented on the reflective surface (modulation surface). The spatial light modulator may be a DMD (Digital Micromirror Device). By setting the direction of light reflection in various ways using the mirror 15 and acquiring a complex amplitude image, ODT becomes possible.

[0027] Figure 2 shows the configuration of observation device 2. This observation device 2 further includes a quarter-wave plate 27 in addition to the configuration of observation device 1 (Figure 1). The quarter-wave plate 27 is provided in the optical path between the polarization-dependent spatial modulation element 26 and the polarization camera 29. In this figure, the quarter-wave plate 27 is provided between the polarization-dependent spatial modulation element 26 and the lens 28. The quarter-wave plate 27 can convert linearly polarized light into circularly polarized light.

[0028] Figure 3 shows the configuration of observation device 3. Compared to the configuration of observation device 2 (Figure 2), observation device 3 is equipped with a light source 11B instead of light source 11A, and further includes a polarization modulation element 12 and a polarizer 25. The polarization modulation element 12 is provided in the optical path of the optical system from light source 11B to objective lens 17. In this figure, the polarization modulation element 12 is provided in the optical path between light source 11B and lens 13.

[0029] In this configuration, the optical system from the light source 11B to the objective lens 17 constitutes an illumination unit that irradiates the object to be observed S with unpolarized light (or light that can be considered unpolarized). Even if the light source 11B is a light source that outputs polarized light, the polarization state of the light output from the light source 11B can be temporally modulated by the polarization modulation element 12 to irradiate the object to be observed S with unpolarized light. The polarization modulation element 12 may be, for example, an electro-optic crystal or an electro-optic polymer. If the light source 11B is a light source that outputs randomly polarized light (for example, a HeNe laser light source), the polarization modulation element 12 is not necessary.

[0030] The polarizer 25 is placed in the optical path between the object to be observed S and the polarization-dependent spatial modulation element 26. In this figure, the polarizer 25 is placed between the lens 24 and the polarization-dependent spatial modulation element 26. The polarizer 25 selectively transmits linearly polarized light of a specific direction from the input light and outputs it to the polarization-dependent spatial modulation element 26.

[0031] In this configuration, for example, even if the object to be observed S is placed in a birefringent plastic petri dish, the object to be observed S can be observed without being affected by its birefringence.

[0032] Figure 4 shows the configuration of the observation device 4. This observation device 4 comprises a light source 11A, a lens 24, a polarization-dependent spatial modulation element 26, a lens 28, a polarization camera 29, and a processing unit 40. In this configuration, the polarization-dependent spatial modulation element 26 and the polarization camera 29 are integrated. In addition to these, the lenses 24 and 28 are also integrated. A quarter-wave plate may be provided in the optical path between the polarization-dependent spatial modulation element 26 and the polarization camera 29, and this quarter-wave plate may also be integrated. Alternatively, a polarizer 25 may be provided in the optical path prior to the polarization-dependent spatial modulation element 26, and this polarizer 25 may also be integrated.

[0033] The light source 11A, which is the illumination unit, outputs polarized, spatially coherent light and irradiates the object to be observed S with this light. The light that is irradiated onto the object to be observed S and passes through the object to be observed S (reflected or scattered light) passes sequentially through lens 24, polarization-dependent spatial modulation element 26 and lens 28, and is input to the imaging surface 29a of the polarization camera 29. By adopting this integrated configuration, the adjustment and handling of the optical system are made easier, and complex amplitude images of the object to be observed S can be acquired even more easily.

[0034] Figure 5 shows another example of the optical system configuration from the objective lens 21 to the polarizing camera 29 among the components of observation devices 1 to 3. The polarization-dependent spatial modulation element 26 only needs to be provided on the Fourier plane with respect to the imaging surface 29a of the polarizing camera 29.

[0035] If the cross-section (observation surface) to be observed in the object S and the imaging surface 29a of the polarizing camera 29 are in an optically conjugate positional relationship, then, for example, as shown in this figure, the polarization-dependent spatial modulation element 26 may be provided at the pupil position of the objective lens 21. In this case, the objective lens 21 and the polarization-dependent spatial modulation element 26 can be integrated, making it easier to adjust and handle the optical system, and further easier to acquire a complex amplitude image of the object S.

[0036] Figure 6 shows an example of the configuration of the image sensor of the polarizing camera 29. The image sensor shown in this figure comprises a photodiode array 291 formed on a semiconductor substrate, a polarizer array 292 provided on the photodiode array, and a lens array 293 provided on the polarizer array, and these are stacked together.

[0037] In the photodiode array 291, multiple photodiodes are arranged in a two-dimensional array. For each photodiode in the photodiode array 291, one polarizer from the polarizer array 292 is provided in correspondence, and one lens from the lens array 293 is provided in correspondence.

[0038] Each polarizer in the polarizer array 292 has a velocity axis in one of four directions (0°, 45°, 90°, 135°). In this figure, the hatching direction of each polarizer in the polarizer array 292 indicates the direction of the velocity axis of that polarizer.

[0039] This image sensor allows for the simultaneous acquisition of two-dimensional images of linearly polarized light in four directions. The image sensor commercialized by Sony Corporation (Polarsens®) has the configuration shown in this figure. The image sensor commercialized by Teledyne DALSA (Area Scan Polarization Sensor) also has the configuration shown in this figure.

[0040] A polarization camera 29 having such an image sensor can simultaneously acquire intensity images of each of the four polarization components. A polarization camera having the configuration shown in Figure 6 can simultaneously acquire intensity images of each component: horizontal polarization, vertical polarization, 45-degree polarization, and -45-degree polarization. If a quarter-wave plate 27 having a speed axis in the horizontal direction is provided, the polarization camera 29 acquires intensity images of each component: horizontal polarization, vertical polarization, right-circular polarization, and left-circular polarization.

[0041] The imaging unit that acquires intensity images of multiple polarization components may have the configuration shown in Figure 7. Figure 7 shows another example of the optical system after the objective lens 21. In this configuration, a polarization beam splitter 31, camera 32 and camera 33 are provided instead of the polarization camera 29. Cameras 32 and 33 may be ordinary cameras instead of polarization cameras. The polarization beam splitter 31 separates the light arriving from the lens 28 into p-polarization components and s-polarization components, outputs the p-polarization component to camera 32 and the s-polarization component to camera 33.

[0042] Cameras 32 and 33 individually acquire intensity images of each polarization component separated by the polarization beam splitter 31. Specifically, camera 32 receives the p-polarization component output from the polarization beam splitter 31 and acquires an intensity image. Camera 33 receives the s-polarization component output from the polarization beam splitter 31 and acquires an intensity image. The imaging unit having the configuration shown in Figure 7 can acquire intensity images of two polarization components.

[0043] Next, the polarization-dependent spatial modulation element 26 will be described. The polarization-dependent spatial modulation element 26 spatially modulates the polarization state of the input light and outputs the modulated light. Examples of the polarization-dependent spatial modulation element 26 include a birefringent pattern retarder and a spatial light modulator.

[0044] A birefringent pattern retarder is constructed by individually placing a birefringent liquid crystal polymer in each of multiple pixels arranged in a two-dimensional array, and setting the orientation of the velocity axis of the liquid crystal polymer for each pixel. A birefringent pattern retarder is equivalent to having half-wave plates arranged in various directions for each pixel. Such birefringent pattern retarders are sold by Thorlabs. Birefringent pattern retarders are sometimes also called phase difference patterning elements or liquid crystal polymer polarizing elements.

[0045] A spatial light modulator selectively spatially modulates the phase of linearly polarized light components in a specific direction from the input light, and the modulation pattern can be set by an external electrical signal.

[0046] Figure 8 shows an example of a velocity axis orientation distribution pattern in a polarization-dependent spatial modulation element 26. This shows an optical vortex pattern in which the velocity axis orientation of each pixel rotates in proportion to the deflection angle θ in polar coordinates (r, θ) on the modulation surface of the polarization-dependent spatial modulation element 26. In the optical vortex pattern, the velocity axis orientation α(r, θ) of the pixel at polar coordinate position (r, θ) rotates in proportion to the deflection angle θ and can be expressed, for example, by equation (1) below. This is a pattern that converts horizontally polarized light into an optical vortex beam with angular momentum m (where m is an integer). Such a pattern generally corresponds to an element called a q-plate.

[0047] Figure 9 shows another example of a velocity axis orientation distribution pattern in the polarization-dependent spatial modulation element 26. This shows a random pattern where the velocity axis orientation of each pixel is random. This pattern is designed so that the result of its inverse Fourier transform fits within the imaging surface 29a of the polarization camera 29.

[0048] An example of a random pattern design method is as follows: In step 1, P(k) is prepared based on a randomly initialized phase pattern φ(k) in the Fourier plane (plane of wave number space) where the polarization-dependent spatial modulation element 26 is located (equations (2) and (3) below). k represents the position in wave number space. The mask function A(k) has a value of 1 when the absolute value of the wave number is 2NA / λ or less, and a value of 0 otherwise. NA is the numerical aperture, and λ is the wavelength.

[0049] In step 2, P(k) is subjected to an inverse Fourier transform to obtain p(r) on the imaging plane 29a (a plane in real space) (equation (4) below). r represents the position in real space.

[0050] In step 3, the p(r) obtained in step 2 is multiplied by the mask function m(r) to obtain a p(r) with a restricted diameter (equations (5) and (6) below). The mask function m(r) is defined as the absolute value of r being r ｄ The value is 1 when the following conditions are met, and 0 otherwise. ｄ This value is less than or equal to the dimension of the imaging surface 29a of the polarizing camera 29.

[0051] In step 4, the p(r) obtained in step 3 is Fourier transformed to obtain P(k) in the Fourier plane (plane of wavenumber space) where the polarization-dependent spatial modulation element 26 is located (equation (7) below).

[0052] In step 5, the amplitude of P(k) obtained in step 4 is restricted to 1, and the diameter is restricted by multiplying it by the mask function A(k) (equation (8) below).

[0053] Steps 2 to 5 are repeated multiple times to update P(k). Based on the P(k) obtained by repeating steps 2 to 5 multiple times, the distribution α(k) of the direction of the velocity axis in the polarization-dependent spatial modulation element 26 is determined (equation (9) below).

[0054] In the following explanation, we will assume that a quarter-wave plate 27 is provided and that the polarization camera 29 described in Figure 6 is used.

[0055] If U(k) is the complex amplitude distribution of light when input to the polarization-dependent spatial modulation element 26, then the complex amplitude distribution u(r) of light on the imaging surface 29a of the polarization camera 29 is expressed by equation (10) below. In addition, the intensity image I of each polarization component acquired by the polarization camera 29 is ｊ (r) is expressed by equation (11) below. The subscript j is H, V, R, and L respectively.

[0056] I Ｈ (r) is the intensity image of the horizontal polarization component, I Ｖ (r) is the intensity image of the vertical polarization component, I Ｒ (r) is the intensity image of the right circularly polarized component, I Ｌ (r) is the intensity image of the left circularly polarized component. P ｊ (k) represents the complex amplitude distribution U(k) of light and the intensity image I of the j-th polarization component when input to the polarization-dependent spatial modulation element 26. ｊ This is the pupil function that relates to (r). Pupillary function P ｊ (k) includes, in addition to the contribution of spatial modulation of the polarization component by the polarization-dependent spatial modulation element 26, the contribution of the conversion of the polarization state by the quarter-wave plate 27 if the quarter-wave plate 27 is provided.

[0057] When a birefringence pattern retarder is used as the polarization-dependent spatial modulation element 26, the pupil function P ｊ (k) is as follows: Assume that the distribution of the direction of the velocity axis is α(k), and that φ(k) = 2α(k). If the light input to the polarization-dependent spatial modulation element 26 is left-circularly polarized with amplitude U(k), then the light output from the polarization-dependent spatial modulation element 26 will be right-circularly polarized with amplitude U(k)exp(iφ(k)). Also, if the light input to the polarization-dependent spatial modulation element 26 is right-circularly polarized with amplitude U(k), then the light output from the polarization-dependent spatial modulation element 26 will be left-circularly polarized with amplitude U(k)exp(-iφ(k)).

[0058] When horizontally polarized light with amplitude U(k) (equation (12) below) is input to the polarization-dependent spatial modulation element 26, the polarization state of the light output from the polarization-dependent spatial modulation element 26 is as shown in equation (13) below. Ｈ is a vector representing horizontal polarization, and e Ｖ is a vector representing perpendicular polarization, and e Ｄ This is a vector representing 45-degree polarization, and e Ａ is a vector representing -45 degree polarization, and e Ｒ is a vector representing right-circular polarization, and e Ｌ This is a vector representing left-circular polarization.

[0059] Pupillary function P ｊ (k) is expressed by equations (14) to (17) below.

[0060] When a spatial light modulator is used as the polarization-dependent spatial modulation element 26, the pupil function P ｊ (k) is as follows. Here again, assume that the distribution of the direction of the velocity axis is α(k), and that φ(k) = 2α(k). When the polarization-dependent spatial modulation element 26 selectively modulates horizontally polarized light vertically, if the light input to the polarization-dependent spatial modulation element 26 is horizontally polarized with amplitude U(k), then the light output from the polarization-dependent spatial modulation element 26 remains horizontally polarized with amplitude U(k). Also, if the light input to the polarization-dependent spatial modulation element 26 is vertically polarized with amplitude U(k), then the light output from the polarization-dependent spatial modulation element 26 becomes vertically polarized with amplitude U(k)exp(iφ(k)).

[0061] When light with amplitude U(k) and 45-degree polarization (equation (18) below) is input to the polarization-dependent spatial modulation element 26, the polarization state of the light output from the polarization-dependent spatial modulation element 26 is as shown in equation (19) below.

[0062] Pupillary function P ｊ (k) is expressed by equations (20) to (23) below.

[0063] Next, the processing performed by the processing unit 40 will be described, as well as the observation method. Figure 10 is an example of a flowchart of the observation method. The observation method of this embodiment includes an intensity image acquisition step S1, an initialization step S2, a modulation processing step S5, an inverse Fourier transform processing step S6, an amplitude constraint processing step S7, a Fourier transform processing step S8, and an update processing step S9, etc.

[0064] In intensity image acquisition step S1, the polarization camera 29 acquires intensity images I for each of the multiple polarization components. ｊ (r) is obtained. Processing from initialization step S2 onward is performed by the processing unit 40, which controls the pupil function P ｊ This is done using (k). Here, the intensity image I of J polarization components is obtained by the polarization camera 29. １ (r) ~ I Ｊ (r) is obtained, and correspondingly J pupil functions P １ (k) ~ P Ｊ (k) shall be used.

[0065] In initialization step S2, the complex amplitude distribution U(k) of light on the Fourier plane (the plane on which the polarization-dependent spatial modulation element 26 is placed) with respect to the imaging surface 29a of the polarization camera 29 is initialized. U(k) can have any initial distribution, but for example, U(k) = 1. Alternatively, U(k) may have a random distribution.

[0066] In step S3, the value of parameter n is initialized to 1. In step S4, the value of parameter j is initialized to 1.

[0067] In the modulation processing step S5, the complex amplitude distribution U(k) of light in the Fourier plane is determined by the pupil function P ｊ The complex amplitude distribution U'(k) of the modulated light is obtained by modulating based on (k) (see equation (24) below).

[0068] In the inverse Fourier transform processing step S6, the complex amplitude distribution U'(k) of the light after modulation processing in the modulation processing step S5 is subjected to an inverse Fourier transform to obtain the complex amplitude distribution u(r) of the light on the imaging surface 29a (see equation (25) below).

[0069] In the amplitude constraint processing step S7, the amplitude of the complex amplitude distribution u(r) of light after the inverse Fourier transform processing in the inverse Fourier transform processing step S6 is used to control the intensity image I ｊ The constraint is based on the amplitude of (r). Specifically, the amplitude of the complex amplitude distribution u(r) is constrained to the intensity image I ｊ Replace (r) with the amplitude (see equation (26) below). Leave the phase of the complex amplitude distribution u(r) (∠u(r)) unchanged.

[0070] In the Fourier transform processing step S8, the complex amplitude distribution u(r) of light after amplitude constraint processing in the amplitude constraint processing step S7 is Fourier transformed to obtain the complex amplitude distribution U'(k) of light on the Fourier plane (equation (27) below).

[0071] In the update process step S9, the complex amplitude distribution U'(k) and pupil function P of the light after the Fourier transform process in the Fourier transform process step S8 are updated. ｊ Based on (k), the complex amplitude distribution U(k) of light in the Fourier plane is updated. The update equation used in this case is, for example, expressed by equation (28) below. ε １ and ε ２ Each value is between 0 and 1. ｊ ＊ (k) is P ｊ It is the complex conjugate of (k).

[0072] In step S10, it is determined whether the value of parameter j has reached J. If the value of parameter j has reached J, the process proceeds to step S11. If the value of parameter j has not reached J, the value of parameter j is incremented by 1 in step S12, and then the process returns to modulation step S5. In other words, a series of processes including steps S5 to S9 are performed for each value of j from 1 to J.

[0073] In step S11, it is determined whether the value of parameter n has reached N. If the value of parameter n has reached N, the process ends. If the value of parameter n has not reached N, the value of parameter n is incremented by 1 in step S13, and then the process returns to step S4. N is an integer greater than or equal to 2. That is, a series of processes including steps S5 to S9 for each value of j = 1 to J is repeated N times.

[0074] The processing unit 40 can determine the complex amplitude distribution U(k) of light at the time of input to the polarization-dependent spatial modulation element 26 by repeating a series of processes including steps S5 to S9 for each value of j=1 to J N times.

[0075] Figure 11 shows an example of the complex amplitude distribution U(k) before processing in the modulation processing step S5. Figure 12 shows an example of the complex amplitude distribution U'(k) after processing in the modulation processing step S5. Figure 13 shows an example of the complex amplitude distribution u(r) after processing in the inverse Fourier transform processing step S6. Figure 14 shows an example of the complex amplitude distribution u(r) after processing in the amplitude constraint processing step S7. Figure 15 shows an example of the complex amplitude distribution U'(k) after processing in the Fourier transform processing step S8. Figure 16 shows an example of the complex amplitude distribution U(k) after processing in the update processing step S9. In these figures, (a) shows the amplitude distribution and (b) shows the phase distribution.

[0076] Next, the results of simulations A and B performed on the observation method of this embodiment will be described. In both simulations A and B, an optical system equipped with a quarter-wave plate 27 shown in Figure 2 was used, and the polarization camera 29 shown in Figure 6 captured intensity images I of the four polarization components. Ｈ (r), I Ｖ (r), I Ｒ (r), I Ｌ We assumed the case where (r) is obtained. Using the update formula of equation (28) above, the parameter ε in this update formula １ , ε ２ Each value was set to 1.

[0077] FIG. 17 is a diagram for explaining an observation object S assumed in each of simulations A and B. As the observation object S, three images arranged at an interval of 4λ were assumed. λ is the wavelength of light. The light is strongly scattered by each image. It was assumed that the image in the center of the three images is at a position optically conjugate to the imaging surface 29a of the polarization camera 29.

[0078] FIGS. 18 to 27 are diagrams showing the conditions and results of simulation A. In simulation A, the pattern of the fast-axis azimuth distribution in the polarization-dependent spatial modulation element 26 was set as the optical vortex pattern shown in FIG. 8.

[0079] FIGS. 18 to 21 are diagrams showing the pupil function P ｊ (k) for each of the four polarization components. FIG. 18 is a diagram showing the pupil function P Ｈ (k) for the horizontal polarization component. FIG. 19 is a diagram showing the pupil function P Ｖ (k) for the vertical polarization component. FIG. 20 is a diagram showing the pupil function P Ｒ (k) for the right circular polarization component. FIG. 21 is a diagram showing the pupil function P Ｌ (k) for the left circular polarization component. In these figures, (a) shows the amplitude distribution of P ｊ (k), (b) shows the phase distribution of P ｊ (k), and (c) shows the amplitude distribution of the inverse Fourier transform of P ｊ (k).

[0080] FIGS. 22 to 27 are diagrams showing the simulation results when light is perpendicularly incident on the observation object S. FIG. 22(a) is a diagram showing the intensity image I Ｈ (r) of the horizontal polarization component. FIG. 22(b) is a diagram showing the intensity image I Ｖ (r) of the vertical polarization component. FIG. 23(a) is a diagram showing the intensity image I Ｒ (r) of the right circular polarization component. FIG. 23(b) is a diagram showing the intensity image I Ｌ (r) of the left circular polarization component.

[0081] FIG. 24 is a diagram showing an exact solution. FIGS. 25 to 27 are complex amplitude images obtained when the number of repetitions N is set to each value in the flowchart shown in FIG. 10. The complex amplitude image shown in FIG. 25 is obtained when N = 1. The complex amplitude image shown in FIG. 26 is obtained when N = 11. The complex amplitude image shown in FIG. 27 is obtained when N = 10% In FIGS. 24 to 27, (a) shows an amplitude distribution, and (b) shows a phase distribution.

[0082] FIGS. 28 to 43 are diagrams showing the conditions and results of Simulation B. In Simulation B, the pattern of the fast axis azimuth distribution in the polarization-dependent spatial modulation element 26 is set to the random pattern shown in FIG. 9.

[0083] FIGS. 28 to 31 are diagrams showing the pupil function P ｊ (k) for each of the four polarization components. FIG. 28 is a diagram showing the pupil function P Ｈ (k) for the horizontal polarization component. FIG. 29 is a diagram showing the pupil function P Ｖ (k) for the vertical polarization component. FIG. 30 is a diagram showing the pupil function P Ｒ (k) for the right circular polarization component. FIG. 31 is a diagram showing the pupil function P Ｌ (k) for the left circular polarization component. In these diagrams, (a) shows the amplitude distribution of P ｊ (k), (b) shows the phase distribution of P ｊ (k), and (c) shows the amplitude distribution of the inverse Fourier transform of P ｊ (k).

[0084] FIGS. 32 to 37 are diagrams showing the simulation results when light is perpendicularly incident on the observation object S. FIG. 32(a) is a diagram showing the intensity image I Ｈ (r) of the horizontal polarization component. FIG. 32(b) is a diagram showing the intensity image I Ｖ (r) of the vertical polarization component. FIG. ) is a diagram showing the intensity image I Ｒ (r) of the right circular polarization component. FIG. 33(b) is a diagram showing the intensity image I Ｌ (r) of the left circular polarization component.

[0085] Figure 34 shows the exact solution. Figures 35 to 37 are complex amplitude images obtained when the number of iterations N is set to the flowchart shown in Figure 10. The complex amplitude image shown in Figure 35 was obtained when N=1. The complex amplitude image shown in Figure 36 was obtained when N=11. The complex amplitude image shown in Figure 37 was obtained when N=100. In Figures 34 to 37, (a) shows the amplitude distribution and (b) shows the phase distribution.

[0086] Figures 38 to 43 show the simulation results when light is incident at an oblique angle on the object S being observed. Incidence direction (θ) ｘ , θ ｙ ) was set to (37.0°, 0°). Figure 38(a) shows the intensity image I of the horizontal polarization component. Ｈ Figure (r) is shown. Figure 38(b) is the intensity image I of the vertical polarization component. Ｖ Figure (r) is shown. Figure 39(a) is the intensity image I of the right circularly polarized component. Ｒ Figure (r) is shown. Figure 39(b) is the intensity image I of the left circularly polarized component. Ｌ This is a figure of (r).

[0087] Figure 40 shows the exact solution. Figures 41 to 43 are complex amplitude images obtained when the number of iterations N is set to the flowchart shown in Figure 10. The complex amplitude image shown in Figure 41 was obtained when N=1. The complex amplitude image shown in Figure 42 was obtained when N=11. The complex amplitude image shown in Figure 43 was obtained when N=100. In Figures 40 to 43, (a) shows the amplitude distribution and (b) shows the phase distribution.

[0088] As described above, in both simulations A and B, the complex amplitude images obtained by repeatedly performing a series of processes including steps S5 to S9 for each of the multiple polarization components showed good agreement with the exact solution. In this embodiment, even when light scattering is strong at the object being observed, a complex amplitude image of the object can be easily obtained with a simple configuration. Furthermore, since a complex amplitude image of the object can be obtained even when light is incident on the object at an oblique angle, phase imaging by ODT is possible.

[0089] The observation apparatus and observation method are not limited to the embodiments and configuration examples described above, and various modifications are possible.

[0090] The observation apparatus according to the first embodiment described above comprises: (1) an illumination unit that irradiates an object to be observed with polarized light; (2) an imaging unit that acquires intensity images of each of a plurality of polarization components on the imaging surface; (3) a polarization-dependent spatial modulation element provided on the Fourier plane with respect to the imaging surface, which receives light that has been irradiated by the illumination unit and passed through the object to be observed, spatially modulates the polarization state of the light, and outputs the modulated light to the imaging unit; and (4) a processing unit that determines a complex amplitude image of the object to be observed, wherein the processing unit determines the complex amplitude distribution of the light at the time of input to the polarization-dependent spatial modulation element and the intensity image I of the jth polarization component among the intensity images of each of the plurality of polarization components acquired by the imaging unit. ｊ The pupil function P relates the relationship between and ｊ After performing an initialization step to initialize the complex amplitude distribution of light on the Fourier plane, the pupil function P is used to determine the complex amplitude distribution of light on the Fourier plane for each of the multiple polarization components. ｊ A modulation processing step that modulates based on, an inverse Fourier transform processing step that performs an inverse Fourier transform on the complex amplitude distribution of light after this modulation processing to obtain the complex amplitude distribution of light on the imaging plane, and an intensity image I which is the amplitude of the complex amplitude distribution of light after this inverse Fourier transform processing. ｊ An amplitude constraint processing step that constrains based on the amplitude of the light; a Fourier transform processing step that performs a Fourier transform on the complex amplitude distribution of the light after this amplitude constraint processing to obtain the complex amplitude distribution of the light on the Fourier plane; and the complex amplitude distribution of the light and the pupil function P after this Fourier transform processing.ｊ A series of processes, including an update step that updates the complex amplitude distribution of light in the Fourier plane based on the above, is repeated multiple times to obtain the complex amplitude distribution of light at the time of input to the polarization-dependent spatial modulation element, and a complex amplitude image of the object to be observed is obtained based on this complex amplitude distribution.

[0091] The observation apparatus of the second embodiment according to the above embodiment comprises: (1) an illumination unit that irradiates an object to be observed with unpolarized light; (2) an imaging unit that acquires intensity images of each of a plurality of polarization components on the imaging surface; (3) a polarization-dependent spatial modulation element provided on the Fourier plane with respect to the imaging surface, which receives light that has been irradiated by the illumination unit and passed through the object to be observed, spatially modulates the polarization state of the light, and outputs the modulated light to the imaging unit; (4) a polarizer provided in the optical path between the object to be observed and the polarization-dependent spatial modulation element; and (5) a processing unit that determines the complex amplitude image of the object to be observed, wherein (6) the processing unit determines the complex amplitude distribution of the light at the time of input to the polarization-dependent spatial modulation element and the intensity image I of the jth polarization component among the intensity images of each of the plurality of polarization components acquired by the imaging unit. ｊ The pupil function P relates the relationship between and ｊ After performing an initialization step to initialize the complex amplitude distribution of light on the Fourier plane, the pupil function P is used to determine the complex amplitude distribution of light on the Fourier plane for each of the multiple polarization components. ｊ A modulation processing step that modulates based on, an inverse Fourier transform processing step that performs an inverse Fourier transform on the complex amplitude distribution of light after this modulation processing to obtain the complex amplitude distribution of light on the imaging plane, and an intensity image I which is the amplitude of the complex amplitude distribution of light after this inverse Fourier transform processing. ｊ An amplitude constraint processing step that constrains based on the amplitude of the light; a Fourier transform processing step that performs a Fourier transform on the complex amplitude distribution of the light after this amplitude constraint processing to obtain the complex amplitude distribution of the light on the Fourier plane; and the complex amplitude distribution of the light and the pupil function P after this Fourier transform processing. ｊ A series of processes, including an update step that updates the complex amplitude distribution of light in the Fourier plane based on the above, is repeated multiple times to obtain the complex amplitude distribution of light at the time of input to the polarization-dependent spatial modulation element, and a complex amplitude image of the object to be observed is obtained based on this complex amplitude distribution.

[0092] In the observation apparatus of the third embodiment, the configuration of the first or second embodiment may further include a quarter-wave plate provided in the optical path between the polarization-dependent spatial modulation element and the imaging unit.

[0093] In the observation apparatus of the fourth embodiment, in any of the configurations of the first to third embodiments, the polarization-dependent spatial modulation element may be provided integrated with the imaging unit.

[0094] In the observation apparatus of the fifth embodiment, in any of the configurations of the first to third embodiments, the polarization-dependent spatial modulation element may be provided at the pupil position of the objective lens located on the optical path between the object to be observed and the imaging unit, and may be integrated with the objective lens.

[0095] In the observation apparatus of the sixth embodiment, in any configuration of the first to fifth embodiments, the polarization-dependent spatial modulation element may be configured to spatially modulate the polarization state based on an optical vortex pattern in which the direction of the velocity axis of each pixel rotates in proportion to the deflection angle θ in polar coordinates (r, θ), or a random pattern in which the direction of the velocity axis of each pixel is random.

[0096] In the observation apparatus of the seventh embodiment, in any of the configurations of the first to fifth embodiments, the polarization-dependent spatial modulation element may be configured as a birefringent pattern retarder or a spatial light modulator.

[0097] In the observation apparatus of the eighth embodiment, in any configuration of the first to seventh embodiments, the imaging unit may include a polarization camera that simultaneously acquires intensity images of multiple polarization components.

[0098] In the observation apparatus of the ninth embodiment, in any configuration of the first to seventh embodiments, the imaging unit may include a polarization beam splitter that separates each of a plurality of polarization components from one another, and a plurality of cameras that individually acquire intensity images of each polarization component separated from the polarization beam splitter.

[0099] The first observation method according to the above embodiment is a method for obtaining a complex amplitude image of an object to be observed using (1) an illumination unit that irradiates an object to be observed with polarized light, (2) an imaging unit that acquires intensity images of each of a plurality of polarization components on the imaging surface, and (3) a polarization-dependent spatial modulation element provided on the Fourier plane with respect to the imaging surface, which receives light that has been irradiated by the illumination unit and passed through the object to be observed, spatially modulates the polarization state of the light, and outputs the modulated light to the imaging unit, wherein (4) the complex amplitude distribution of the light at the time of input to the polarization-dependent spatial modulation element and the intensity image I of the j-th polarization component among the intensity images of each of the plurality of polarization components acquired by the imaging unit. ｊ The pupil function P relates the relationship between and ｊ After performing an initialization step to initialize the complex amplitude distribution of light on the Fourier plane, the pupil function P is used to determine the complex amplitude distribution of light on the Fourier plane for each of the multiple polarization components. ｊ A modulation processing step that modulates based on, an inverse Fourier transform processing step that performs an inverse Fourier transform on the complex amplitude distribution of light after this modulation processing to obtain the complex amplitude distribution of light on the imaging plane, and an intensity image I which is the amplitude of the complex amplitude distribution of light after this inverse Fourier transform processing. ｊ An amplitude constraint processing step that constrains based on the amplitude of the light; a Fourier transform processing step that performs a Fourier transform on the complex amplitude distribution of the light after this amplitude constraint processing to obtain the complex amplitude distribution of the light on the Fourier plane; and the complex amplitude distribution of the light and the pupil function P after this Fourier transform processing. ｊ A series of processes, including an update step that updates the complex amplitude distribution of light in the Fourier plane based on the above, is repeated multiple times to obtain the complex amplitude distribution of light at the time of input to the polarization-dependent spatial modulation element, and a complex amplitude image of the object to be observed is obtained based on this complex amplitude distribution.

[0100] The observation method of the second embodiment according to the above embodiment is a method for obtaining a complex amplitude image of an object using (1) an illumination unit that irradiates an object to be observed with unpolarized light, (2) an imaging unit that acquires intensity images of each of a plurality of polarization components on the imaging surface, (3) a polarization-dependent spatial modulation element provided on the Fourier plane with respect to the imaging surface, which receives light that has been irradiated by the illumination unit and passed through the object to be observed, spatially modulates the polarization state of the light, and outputs the modulated light to the imaging unit, and (4) a polarizer provided in the optical path between the object to be observed and the polarization-dependent spatial modulation element, wherein (5) the complex amplitude distribution of the light at the time of input to the polarization-dependent spatial modulation element and the intensity image I of the j-th polarization component among the intensity images of each of the plurality of polarization components acquired by the imaging unit. ｊ The pupil function P relates the relationship between and ｊ After performing an initialization step to initialize the complex amplitude distribution of light on the Fourier plane, the pupil function P is used to determine the complex amplitude distribution of light on the Fourier plane for each of the multiple polarization components. ｊ A modulation processing step that modulates based on, an inverse Fourier transform processing step that performs an inverse Fourier transform on the complex amplitude distribution of light after this modulation processing to obtain the complex amplitude distribution of light on the imaging plane, and an intensity image I which is the amplitude of the complex amplitude distribution of light after this inverse Fourier transform processing. ｊ An amplitude constraint processing step that constrains based on the amplitude of the light; a Fourier transform processing step that performs a Fourier transform on the complex amplitude distribution of the light after this amplitude constraint processing to obtain the complex amplitude distribution of the light on the Fourier plane; and the complex amplitude distribution of the light and the pupil function P after this Fourier transform processing. ｊ A series of processes, including an update step that updates the complex amplitude distribution of light in the Fourier plane based on the above, is repeated multiple times to obtain the complex amplitude distribution of light at the time of input to the polarization-dependent spatial modulation element, and a complex amplitude image of the object to be observed is obtained based on this complex amplitude distribution.

[0101] In the observation method of the third embodiment, a configuration may be used in which a quarter-wave plate provided in the optical path between the polarization-dependent spatial modulation element and the imaging unit is further used, as in the configuration of the first or second embodiment.

[0102] In the observation method of the fourth embodiment, in any of the configurations of the first to third embodiments, the polarization-dependent spatial modulation element may be provided integrated with the imaging unit.

[0103] In the observation method of the fifth embodiment, in any of the configurations of the first to third embodiments, the polarization-dependent spatial modulation element may be provided at the pupil position of the objective lens located on the optical path between the object to be observed and the imaging unit, and may be integrated with the objective lens.

[0104] In the sixth aspect of the observation method, in any of the configurations of the first to fifth aspects, the polarization-dependent spatial modulation element may be configured to spatially modulate the polarization state based on an optical vortex pattern in which the direction of the velocity axis of each pixel rotates in proportion to the deflection angle θ in polar coordinates (r, θ), or a random pattern in which the direction of the velocity axis of each pixel is random.

[0105] In the observation method of the seventh embodiment, in any of the configurations of the first to fifth embodiments, the polarization-dependent spatial modulation element may be configured as a birefringent pattern retarder or a spatial light modulator.

[0106] In the observation method of the eighth embodiment, in any of the configurations of the first to seventh embodiments, the imaging unit may be configured to include a polarization camera that simultaneously acquires intensity images of multiple polarization components.

[0107] In the observation method of the ninth embodiment, in any of the configurations of the first to seventh embodiments, the imaging unit may include a polarization beam splitter that separates each of a plurality of polarization components from one another, and a plurality of cameras that individually acquire intensity images of each polarization component separated from the polarization beam splitter.

[0108] The embodiment can be used as an observation device and observation method that can easily acquire a complex amplitude image of an object with a simple configuration, even when light scattering is strong in the object being observed.

[0109] 1-4... Observation device, 11A, 11B... Light source, 12... Polarization modulation element, 13, 14... Lens, 15... Mirror, 16... Lens, 17... Objective lens, 21... Objective lens, 22... Mirror, 23, 24... Lens, 25... Polarizer, 26... Polarization-dependent spatial modulation element, 27... Quarter wave plate, 28... Lens, 29... Polarization camera, 29a... Imaging surface, 31... Polarization beam splitter, 32, 33... Camera, 40... Processing unit, S... Object to be observed.

Claims

1. The system comprises: an illumination unit that irradiates an object to be observed with polarized light; an imaging unit that acquires intensity images of each of a plurality of polarization components on an imaging surface; a polarization-dependent spatial modulation element provided on the Fourier plane of the imaging surface, which receives light irradiated by the illumination unit and passed through the object to be observed, spatially modulates the polarization state of the light, and outputs the modulated light to the imaging unit; and a processing unit that determines a complex amplitude image of the object to be observed, wherein the processing unit determines the complex amplitude distribution of the light at the time of input to the polarization-dependent spatial modulation element and the intensity image I of the j-th polarization component among the intensity images of each of the plurality of polarization components acquired by the imaging unit. ｊ The pupil function P relates the relationship between and ｊ After performing an initialization step to initialize the complex amplitude distribution of light on the Fourier plane, the pupil function P is used for each of the multiple polarization components. ｊ A modulation processing step that modulates based on, an inverse Fourier transform processing step that performs an inverse Fourier transform on the complex amplitude distribution of light after this modulation processing to obtain the complex amplitude distribution of light on the imaging surface, and an intensity image I that measures the amplitude of the complex amplitude distribution of light after this inverse Fourier transform processing. ｊ An amplitude constraint processing step that constrains based on the amplitude of the light; a Fourier transform processing step that performs a Fourier transform on the complex amplitude distribution of the light after this amplitude constraint processing to obtain the complex amplitude distribution of the light on the Fourier plane; and the complex amplitude distribution of the light after this Fourier transform processing and the pupil function P. ｊ An observation device that repeatedly performs a series of processes, including an update process step of updating the complex amplitude distribution of light in the Fourier plane based on the above, to obtain the complex amplitude distribution of light at the time of input to the polarization-dependent spatial modulation element, and obtains a complex amplitude image of the object to be observed based on this complex amplitude distribution.

2. An irradiation unit that irradiates an observation object with non-polarized light, an imaging unit that acquires intensity images of a plurality of polarization components on an imaging surface, a polarization-dependent spatial modulation element provided on a Fourier surface with respect to the imaging surface, which inputs light irradiated by the irradiation unit and passed through the observation object, spatially modulates the polarization state of the light, and outputs the modulated light to the imaging unit, a polarizer provided on an optical path between the observation object and the polarization-dependent spatial modulation element, and a processing unit that obtains a complex amplitude image of the observation object. The processing unit relates the complex amplitude distribution of the light at the time of input to the polarization-dependent spatial modulation element to the intensity image I of the j-th polarization component among the intensity images of the plurality of polarization components acquired by the imaging unit. ｊ and P as a pupil function that relates them. ｊ After performing an initialization step of initializing the complex amplitude distribution of the light on the Fourier surface, for each of the plurality of polarization components, a modulation processing step of modulating the complex amplitude distribution of the light on the Fourier surface based on the pupil function P, an inverse Fourier transform processing step of performing an inverse Fourier transform on the complex amplitude distribution of the light after this modulation processing to obtain the complex amplitude distribution of the light on the imaging surface, an amplitude constraint processing step of constraining the amplitude of the complex amplitude distribution of the light after this inverse Fourier transform processing based on the amplitude of the intensity image I, a Fourier transform processing step of performing a Fourier transform on the complex amplitude distribution of the light after this amplitude constraint processing to obtain the complex amplitude distribution of the light on the Fourier surface, and an update processing step of updating the complex amplitude distribution of the light on the Fourier surface based on the complex amplitude distribution of the light after this Fourier transform processing and the pupil function P. A series of these processes are repeatedly performed a plurality of times to obtain the complex amplitude distribution of the light at the time of input to the polarization-dependent spatial modulation element, and based on this complex amplitude distribution, an observation apparatus that obtains a complex amplitude image of the observation object. ｊ Based on the intensity image I of the j-th polarization component among the intensity images of the plurality of polarization components acquired by the imaging unit. ｊ And an amplitude constraint processing step of constraining the amplitude of the complex amplitude distribution of the light after this inverse Fourier transform processing based on the amplitude of the intensity image I. ｊ And an update processing step of updating the complex amplitude distribution of the light on the Fourier surface based on the complex amplitude distribution of the light after this Fourier transform processing and the pupil function P.

3. The observation apparatus according to claim 1 or 2, further comprising a quarter-wave plate provided in the optical path between the polarization-dependent spatial modulation element and the imaging unit.

4. The observation apparatus according to any one of claims 1 to 3, wherein the polarization-dependent spatial modulation element is provided integrated with the imaging unit.

5. The observation apparatus according to any one of claims 1 to 3, wherein the polarization-dependent spatial modulation element is provided at the pupil position of an objective lens located on the optical path between the object to be observed and the imaging unit, and is integrated with the objective lens.

6. The observation apparatus according to any one of claims 1 to 5, wherein the polarization-dependent spatial modulation element spatially modulates the polarization state based on an optical vortex pattern in which the orientation of the velocity axis of each pixel rotates in proportion to the deflection angle θ in polar coordinates (r, θ), or a random pattern in which the orientation of the velocity axis of each pixel is random.

7. The observation apparatus according to any one of claims 1 to 5, wherein the polarization-dependent spatial modulation element is a birefringent pattern retarder or a spatial light modulator.

8. The observation apparatus according to any one of claims 1 to 7, wherein the imaging unit includes a polarization camera that simultaneously acquires intensity images of each of the plurality of polarization components.

9. The observation apparatus according to any one of claims 1 to 7, wherein the imaging unit includes a polarization beam splitter that separates each of the plurality of polarization components from each other, and a plurality of cameras that individually acquire intensity images of each polarization component separated from the polarization beam splitter.

10. A method for obtaining a complex amplitude image of an object to be observed, using: an illumination unit that irradiates an object to be observed with polarized light; an imaging unit that acquires intensity images of each of a plurality of polarization components on an imaging surface; and a polarization-dependent spatial modulation element provided on the Fourier plane of the imaging surface, which receives light irradiated by the illumination unit and passed through the object to be observed, spatially modulates the polarization state of the light, and outputs the modulated light to the imaging unit, wherein the complex amplitude distribution of the light at the time of input to the polarization-dependent spatial modulation element and the intensity image I of the j-th polarization component among the intensity images of each of the plurality of polarization components acquired by the imaging unit. ｊ The pupil function P relates the relationship between and ｊ After performing an initialization step to initialize the complex amplitude distribution of light on the Fourier plane, the pupil function P is used for each of the multiple polarization components. ｊ A modulation processing step that modulates based on, an inverse Fourier transform processing step that performs an inverse Fourier transform on the complex amplitude distribution of light after this modulation processing to obtain the complex amplitude distribution of light on the imaging surface, and an intensity image I that measures the amplitude of the complex amplitude distribution of light after this inverse Fourier transform processing. ｊ An amplitude constraint processing step that constrains based on the amplitude of the light; a Fourier transform processing step that performs a Fourier transform on the complex amplitude distribution of the light after this amplitude constraint processing to obtain the complex amplitude distribution of the light on the Fourier plane; and the complex amplitude distribution of the light after this Fourier transform processing and the pupil function P. ｊ An observation method comprising: performing a series of processes multiple times, including an update process step of updating the complex amplitude distribution of light in the Fourier plane based on the above, to obtain the complex amplitude distribution of light at the time of input to the polarization-dependent spatial modulation element; and obtaining a complex amplitude image of the object to be observed based on this complex amplitude distribution.

11. A method for obtaining a complex amplitude image of an object, using: an illumination unit that irradiates an object with unpolarized light; an imaging unit that acquires intensity images of each of a plurality of polarization components on an imaging surface; a polarization-dependent spatial modulation element provided on the Fourier plane of the imaging surface, which receives light irradiated by the illumination unit and passed through the object, spatially modulates the polarization state of the light, and outputs the modulated light to the imaging unit; and a polarizer provided in the optical path between the object and the polarization-dependent spatial modulation element, wherein the complex amplitude distribution of the light at the time of input to the polarization-dependent spatial modulation element and the intensity image I of the j-th polarization component among the intensity images of each of the plurality of polarization components acquired by the imaging unit. ｊ The pupil function P relates the relationship between and ｊ After performing an initialization step to initialize the complex amplitude distribution of light on the Fourier plane, the pupil function P is used for each of the multiple polarization components. ｊ A modulation processing step that modulates based on, an inverse Fourier transform processing step that performs an inverse Fourier transform on the complex amplitude distribution of light after this modulation processing to obtain the complex amplitude distribution of light on the imaging surface, and an intensity image I that measures the amplitude of the complex amplitude distribution of light after this inverse Fourier transform processing. ｊ An amplitude constraint processing step that constrains based on the amplitude of the light; a Fourier transform processing step that performs a Fourier transform on the complex amplitude distribution of the light after this amplitude constraint processing to obtain the complex amplitude distribution of the light on the Fourier plane; and the complex amplitude distribution of the light after this Fourier transform processing and the pupil function P. ｊ An observation method comprising: performing a series of processes multiple times, including an update process step of updating the complex amplitude distribution of light in the Fourier plane based on the above, to obtain the complex amplitude distribution of light at the time of input to the polarization-dependent spatial modulation element; and obtaining a complex amplitude image of the object to be observed based on this complex amplitude distribution.

12. The observation method according to claim 10 or 11, further comprising using a quarter-wave plate provided in the optical path between the polarization-dependent spatial modulation element and the imaging unit.

13. The observation method according to any one of claims 10 to 12, wherein the polarization-dependent spatial modulation element is provided integrated with the imaging unit.

14. The observation method according to any one of claims 10 to 12, wherein the polarization-dependent spatial modulation element is provided at the pupil position of an objective lens provided on the optical path between the object to be observed and the imaging unit, and is integrated with the objective lens.

15. The observation method according to any one of claims 10 to 14, wherein the polarization-dependent spatial modulation element spatially modulates the polarization state based on an optical vortex pattern in which the direction of the velocity axis of each pixel rotates in proportion to the deflection angle θ in polar coordinates (r, θ), or a random pattern in which the direction of the velocity axis of each pixel is random.

16. The observation method according to any one of claims 10 to 14, wherein the polarization-dependent spatial modulation element is a birefringent pattern retarder or a spatial light modulator.

17. The observation method according to any one of claims 10 to 16, wherein the imaging unit includes a polarization camera that simultaneously acquires intensity images of each of the plurality of polarization components.

18. The observation method according to any one of claims 10 to 16, wherein the imaging unit includes a polarization beam splitter that separates each of the plurality of polarization components from each other, and a plurality of cameras that individually acquire intensity images of each polarization component separated from the polarization beam splitter.

Images