Observation device and observation method
The observation device and method enhance ODT by controlling light irradiation directions to suppress multiple scattered light, enabling accurate three-dimensional refractive index tomography of complex structures like three-dimensional cell tissues.
Patent Information
- Application Number
- JP2022547382
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-09-11
- Filing Date
- 2021-01-21
- Publication Date
- 2024-11-21
- Estimated Expiration
- 2041-01-21
AI Technical Summary
Conventional optical diffraction tomography (ODT) struggles to effectively image multiple scatterers like three-dimensional cell tissues due to the overwhelming influence of multiple scattered light, which deteriorates the Single-scattering to Multi-scattering Ratio (SMR) and causes speckle noise, making it difficult to extract structural information.
An observation device and method that generates interference intensity images, complex amplitude images, and two-dimensional phase images by controlling light irradiation directions, utilizing spatial and temporal gating techniques to suppress multiple scattered light and enhance the Single-scattering to Multi-scattering Ratio (SMR), enabling three-dimensional refractive index tomography.
The method reduces the influence of multiple scattered light, suppresses speckles, and improves the SMR, allowing for accurate three-dimensional refractive index tomography of multiple scatterers such as three-dimensional cell tissues.
Smart Images

Figure 0007588150000007 
Figure 0007588150000008 
Figure 0007588150000009
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an observation device and an observation method. [Background technology]
[0002] In recent years, there has been progress in the technology to create three-dimensional cell tissues called spheroids and organoids. In addition, research is progressing on the application of these three-dimensional cell tissues to drug discovery, regenerative medicine, and other fields. These three-dimensional cell tissues are optically transparent multiple scattering bodies. A wide variety of techniques have been proposed to image such optically transparent scattering bodies. Among these, imaging techniques that use fluorescent probes include confocal microscopes, multiphoton microscopes, and light sheet microscopes. On the other hand, optical coherence tomography (OCT) is known as a non-staining, non-invasive imaging technique that does not use fluorescent probes.
[0003] Although non-stained and non-invasive imaging is often desired for observational objects such as spheroids and organoids, there are few reports of OCT being used to image these objects. This is probably due to the low resolution of OCT imaging and the difficulty of interpreting the signals obtained by OCT imaging. Therefore, it can be said that at present, there is no established technology for imaging non-stained 3D cell tissues that could become the gold standard.
[0004] Quantitative phase imaging (QPI) is also known as a technique that can image the optical path length of an object of observation non-staining and non-invasively. QPI is being applied to the biological field because it can obtain physical information, such as the optical path length of the object of observation (e.g., a cell). Images obtained by QPI can be used to generate other types of images, such as differential interference contrast images and phase contrast microscope images. QPI is a technique that can obtain images with a relatively large amount of information, and is expected to be applicable to higher content analysis than analysis using conventional bright-field images. In addition, high content analysis using non-staining imaging techniques has been actively researched due to the recent improvement in image recognition accuracy through machine learning, and non-staining imaging of multiple scattering objects is expected to play an important role in the future. However, QPI cannot grasp the true three-dimensional structure because the images obtained are merely two-dimensional projections of the optical path length.
[0005] Optical Diffraction Tomography (ODT), described in Patent Document 1, is also known as a technique capable of imaging the optical path length of an object to be observed non-stained and non-invasively. ODT is an extension of QPI into a technique capable of three-dimensional imaging, and can realize three-dimensional refractive index tomography of the object to be observed. By observing cells using ODT, it becomes possible to identify organelles such as cell nuclei and mitochondria, and it is also possible to track three-dimensional morphological changes, and it is expected that even higher content analysis can be performed than with QPI. [Prior art documents] [Patent documents]
[0006] [Patent Document 1] JP 2017-219826 A [Non-patent literature]
[0007] [Non-Patent Document 1] Sungsam Kang, et al, "Imaging deep within a scattering medium using collective accumulation of single-scattered waves," NATURE PHOTONICS, Vol.9, pp.253-258 (2015). Summary of the Invention [Problem to be solved by the invention]
[0008] However, although conventional ODT can be applied to the observation of cells consisting of a few cells, it is difficult to apply it to the observation of multiple scattering bodies such as the above-mentioned three-dimensional cell tissues, because with conventional ODT, when there is a lot of multiple scattered light generated in the observation target, the influence of multiple scattered light appears significantly in the acquired image.
[0009] Light scattering is a phenomenon in which the direction of light travel is changed by interacting with an object. In particular, when the spatial non-uniformity of the refractive index of the object increases, the light interacts with the object multiple times while passing through the object. Light that has interacted with the object multiple times in this way is called multiple scattered light. In contrast, light that has interacted with the object only once is called single scattered light. Multiple scattered light is known to be an obstacle to measurement, causing increased speckle and a deterioration in the single-scattering to multi-scattering ratio (SMR).
[0010] Speckle occurs when light is temporally and spatially coherent, but the interference of multiple scattered light causes large spatial changes in intensity or phase. To suppress speckle, a light source that outputs light that is temporally or spatially incoherent can be used. For example, normal bright-field microscopes such as phase-contrast microscopes use spatially and temporally incoherent light sources such as halogen lamps and light-emitting diodes to obtain images without speckle.
[0011] The deterioration of SMR occurs when multiple scattered light becomes dominant over single scattered light, and the single scattered light is buried in the multiple scattered light. As the observed object becomes larger and the observation depth becomes deeper, the single scattered light component exponentially decreases, while in contrast, the multiple scattered light component increases. Single scattered light is easy to use for measuring the structure of an object because its scattering direction has a direct correspondence with the structure of the object. On the other hand, multiple scattered light has a complex relationship with the structure of the object, making it difficult to extract information about the structure of the object. Therefore, it is known that imaging techniques using single scattered light fail to measure when single scattered light is buried in multiple scattered light (i.e., when SMR deteriorates).
[0012] Suppression of SMR deterioration is possible by a technique called gating, which selectively detects single scattered light among single scattered light and multiple scattered light. Gating suppresses multiple scattered light, so it is possible to suppress speckle while suppressing SMR deterioration. Gating is realized using degrees of freedom such as space, time, and polarization. Confocal microscopy is an example of spatial gating. OCT is an example of temporal and spatial gating.
[0013] Conventional ODT does not eliminate the effect of multiple scattered light, so when there is a lot of multiple scattered light generated in the object being observed, the speckle increases in the acquired image and the SMR deteriorates. Therefore, although conventional ODT can be applied to the observation of cells consisting of a few cells that generate little multiple scattered light, it is difficult to apply it to the observation of multiple scattering bodies such as three-dimensional cell tissues that generate a lot of multiple scattered light.
[0014] The present invention has been made to solve the above problems, and aims to provide an observation device and an observation method that can realize three-dimensional refractive index tomography with reduced effects of multiple scattered light, even when the object being observed is a multiple scatterer. [Means for solving the problem]
[0015] A first aspect of the present invention is an observation device comprising: (1) an interference intensity image acquisition unit that acquires, for each of a plurality of light irradiation directions, an interference intensity image of a reference position resulting from interference between light that is irradiated onto an observation object along each of a plurality of light irradiation directions and passes through the observation object and a reference light, (2) a first complex amplitude image generation unit that generates, for each of the plurality of light irradiation directions, a complex amplitude image of the reference position based on the interference intensity image of the reference position, and (3) a two-dimensional phase image generation unit that generates a two-dimensional phase image based on the complex amplitude image for each of the plurality of light irradiation directions.
[0016] A second aspect of the present invention is an observation method, which includes: (1) an interference intensity image acquisition step of acquiring, for each of a plurality of light irradiation directions, an interference intensity image of a reference position caused by interference between light irradiated onto an observation object along each of a plurality of light irradiation directions and passing through the observation object and a reference light, (2) a first complex amplitude image generation step of generating, for each of the plurality of light irradiation directions, a complex amplitude image of the reference position based on the interference intensity image of the reference position, and (3) a two-dimensional phase image generation step of generating a two-dimensional phase image based on the complex amplitude image for each of the plurality of light irradiation directions.
[0017] A third aspect of the present invention is a program for causing a computer to execute each step of the observation method described above. A fourth aspect of the present invention is a recording medium, which is computer-readable and has the program recorded therein. Effect of the Invention
[0018] According to each aspect of the present invention, even when the object to be observed is a multiple scattering medium, it is possible to realize three-dimensional refractive index tomography in which the influence of multiple scattered light is reduced. [Brief description of the drawings]
[0019] [Figure 1] FIG. 1 is a diagram showing the configuration of an observation apparatus 1A. [Diagram 2] FIG. 2 is a diagram showing the configuration of the observation device 1B. [Diagram 3] FIG. 3 is a flowchart of the observation method. [Figure 4] 4(a) to (c) are diagrams showing an example of scanning the light irradiation direction onto the observation object S in the interference intensity image acquisition step S1. [Diagram 5] FIG. 5 is a diagram for explaining the kernel function g. [Figure 6] FIG. 6 is a flow chart of a first aspect of the two-dimensional phase image generating step S4. [Figure 7] FIG. 7 is an interference intensity image (under perpendicular irradiation) acquired in the interference intensity image acquisition step S1. [Figure 8] FIG. 8 is a complex amplitude image (real part, z=0) generated based on the interference intensity image (FIG. 7) in the first complex amplitude image generating step S2. [Figure 9] FIG. 9 is a complex amplitude image (real part, z=10 μm) generated based on the complex amplitude image (FIG. 8) in the second complex amplitude image generating step S3. [Figure 10] FIG. 10 shows a complex amplitude sum image (real part) generated based on the complex amplitude image (FIG. 9) in step S11 of the two-dimensional phase image generating step S4. [Figure 11] FIG. 11 shows a two-dimensional phase image generated based on the complex amplitude sum image (FIG. 10) in step S12 of the two-dimensional phase image generating step S4. [Figure 12] FIG. 12 is an image at a certain z position of the three-dimensional refractive index distribution obtained in the refractive index distribution calculation step S6. [Figure 13] FIG. 13 is a flowchart of a second aspect of the two-dimensional phase image generating step S4. [Figure 14] FIG. 14 is an interference intensity image (under perpendicular irradiation) acquired in the interference intensity image acquisition step S1. [Figure 15] FIG. 15 is a complex amplitude image (real part, z=0) generated based on the interference intensity image (FIG. 14) in the first complex amplitude image generating step S2. [Figure 16]FIG. 16 is a complex amplitude image (real part, z=20 μm) generated based on the complex amplitude image (FIG. 15) in the second complex amplitude image generating step S3. [Figure 17] FIG. 17 shows complex differential interference contrast images (real and imaginary parts for vertical shear and horizontal shear) generated based on the complex amplitude image (FIG. 16) in step S21 of the two-dimensional phase image generation step S4. [Figure 18] FIG. 18 is an image (real and imaginary parts for each of the vertical shear and horizontal shear) representing the sum of the complex differential interference images (FIG. 17) generated in step S21 of the two-dimensional phase image generating step S4. [Figure 19] FIG. 19 shows differential phase images (vertical shear and horizontal shear) generated based on the image (FIG. 18) representing the sum of complex differential interference images in step S22 of the two-dimensional phase image generating step S4. [Figure 20] FIG. 20 shows a two-dimensional phase image generated based on the differential phase image (FIG. 19) in step S23 of the two-dimensional phase image generating step S4. [Figure 21] FIG. 21 is an image at a certain z position of the three-dimensional refractive index distribution obtained in the refractive index distribution calculation step S6. [Figure 22] FIG. 22 is a flowchart of a third aspect of the two-dimensional phase image generating step S4. [Figure 23] FIG. 23 is an interference intensity image (under perpendicular irradiation) acquired in the interference intensity image acquisition step S1. [Figure 24] FIG. 24 is a complex amplitude image (real part, z=0) generated based on the interference intensity image (FIG. 23) in the first complex amplitude image generating step S2. [Diagram 25] FIG. 25 is a complex amplitude image (real part, z=20 μm) generated based on the complex amplitude image (FIG. 24) in second complex amplitude image generating step S3. [Figure 26] FIG. 26 is a complex amplitude sum image generated based on the complex amplitude image (FIG. 25) in step S31 of the two-dimensional phase image generating step S4. [Figure 27]FIG. 27 is a two-dimensional phase image generated in step S34 in the two-dimensional phase image generating step S4. [Figure 28] FIG. 28 is an image at a certain z position of the three-dimensional refractive index distribution obtained in the refractive index distribution calculation step S6. [Figure 29] FIG. 29 is a diagram for explaining the kernel function. [Diagram 30] FIG. 30 is a diagram showing the configuration of an observation apparatus 1C. [Diagram 31] 31(a) and (b) are diagrams showing an example of scanning the light irradiation direction onto the observation object S in the interference intensity image acquisition step S1. [Diagram 32] 32(a) to (c) are diagrams showing an example of scanning the light irradiation direction onto the observation object S in the interference intensity image acquisition step S1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0020] Hereinafter, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicated descriptions are omitted. The present invention is not limited to these examples, but is indicated by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.
[0021] 1 is a diagram showing the configuration of an observation device 1A. This observation device 1A includes a light source 11, a lens 12, a lens 21, a mirror 22, a lens 23, a condenser lens 24, an objective lens 25, a beam splitter 41, a lens 42, an imaging unit 43, an analyzing unit 50, and the like.
[0022] The light source 11 outputs spatially and temporally coherent light, and is preferably a laser light source. The lens 12 is optically connected to the light source 11, and focuses the light output from the light source 11 onto a light input end 13 of an optical fiber 14, and causes the light to be input to the light input end 13. The optical fiber 14 guides the light input to the light input end 13 by the lens 12 to a fiber coupler 15. The fiber coupler 15 couples light between the optical fiber 14 and optical fibers 16 and 17, and splits the light that is guided by the optical fiber 14 and reaches the optical fiber 14 into two, and guides one branched light by the optical fiber 16 and the other branched light by the optical fiber 17. The light guided by the optical fiber 16 is output as divergent light from a light output end 18. The light guided by the optical fiber 17 is output as divergent light from a light output end 19.
[0023] The lens 21 is optically connected to the light output end 18 and collimates the light output from the light output end 18 as divergent light. The mirror 22 is optically connected to the lens 21 and reflects the light arriving from the lens 21 to the lens 23. The orientation of the reflecting surface of the mirror 22 is variable. The lens 23 is optically connected to the mirror 22. The condenser lens 24 is optically connected to the lens 23. The lens 23 and the condenser lens 24 preferably constitute a 4f optical system. The lens 23 and the condenser lens 24 irradiate light onto the observation object S from a light irradiation direction according to the orientation of the reflecting surface of the mirror 22. The objective lens 25 is optically connected to the condenser lens 24. The observation object S is disposed between the objective lens 25 and the condenser lens 24. The objective lens 25 inputs the light (object light) output from the condenser lens 24 and passing through the observation object S, and outputs the light to the beam splitter 41.
[0024] The beam splitter 41 is optically connected to the objective lens 25 and also to the light output end 19. The beam splitter 41 combines the light (object light) output from the objective lens 25 and arriving therewith and the light (reference light) output from the light output end 19 and arriving therewith, and outputs both lights to the lens 42. The lens 42 is optically connected to the beam splitter 41, and collimates the object light and the reference light arriving therefrom and outputs them to the imaging unit 43. The imaging unit 43 is optically connected to the lens 42, and captures an interference fringe image (interference intensity image) due to interference between the object light and the reference light arriving therefrom. The incident direction of the reference light is inclined with respect to the incident direction of the object light on the imaging surface of the imaging unit 43. The position where the object light and the reference light are combined by the beam splitter 41 may be located after the imaging lens, but considering the effects of aberration, it is preferable that it be located between the objective lens 25 and lens 42 as shown in the figure.
[0025] The analysis unit 50 is electrically connected to the imaging unit 43, and receives an interference intensity image captured by the imaging unit 43. The analysis unit 50 processes the received interference intensity image to calculate a three-dimensional refractive index distribution of the observation object S. The analysis unit 50 may be a computer. The analysis unit 50 includes an interference intensity image acquisition unit 51, a first complex amplitude image generation unit 52, a second complex amplitude image generation unit 53, a two-dimensional phase image generation unit 54, a three-dimensional phase image generation unit 55, a refractive index distribution calculation unit 56, a display unit 57, and a storage unit 58.
[0026] The interference intensity image acquiring unit 51 irradiates the observation object S with light along each of a plurality of light irradiation directions by changing the orientation of the reflecting surface of the mirror 22. The interference intensity image acquiring unit 51 also acquires an interference intensity image at a reference position for each of the plurality of light irradiation directions from the imaging unit 43. The interference intensity image acquiring unit 51 includes a CPU, and has an output port that outputs a control signal for changing the orientation of the reflecting surface of the mirror 22, and also has an input port that inputs the interference intensity image from the imaging unit 43. There is no need to move the objective lens 25 in the optical axis direction. The reference position is an image plane position that is conjugate with the imaging plane of the imaging unit 43.
[0027] The first complex amplitude image generating unit 52, the second complex amplitude image generating unit 53, the two-dimensional phase image generating unit 54, the three-dimensional phase image generating unit 55, and the refractive index distribution calculating unit 56 perform processing based on the interference intensity image, and include processing devices such as a CPU, a DSP, or an FPGA. The display unit 57 displays an image to be processed, an image in the middle of processing, and an image after processing, and includes, for example, a liquid crystal display. The storage unit 58 stores data of various images, and includes a hard disk drive, a flash memory, a RAM, a ROM, and the like. The first complex amplitude image generating unit 52, the second complex amplitude image generating unit 53, the two-dimensional phase image generating unit 54, the three-dimensional phase image generating unit 55, the refractive index distribution calculating unit 56, and the storage unit 58 may be configured by cloud computing.
[0028] The storage unit 58 also stores a program for causing the interference intensity image acquisition unit 51, the first complex amplitude image generation unit 52, the second complex amplitude image generation unit 53, the two-dimensional phase image generation unit 54, the three-dimensional phase image generation unit 55, and the refractive index distribution calculation unit 56 to execute each process. This program may be stored in the storage unit 58 at the time of manufacturing or shipping the observation device 1A, or may be acquired via a communication line after shipping and stored in the storage unit 58, or may be recorded in a computer-readable recording medium 2 and stored in the storage unit 58. The recording medium 2 may be any type of medium, such as a flexible disk, a CD-ROM, a DVD-ROM, a BD-ROM, or a USB memory.
[0029] The processing of the interference intensity image acquisition unit 51, the first complex amplitude image generation unit 52, the second complex amplitude image generation unit 53, the two-dimensional phase image generation unit 54, the three-dimensional phase image generation unit 55 and the refractive index distribution calculation unit 56 will be described in detail later.
[0030] Fig. 2 is a diagram showing the configuration of the observation apparatus 1B, which includes a lens 31, a mirror 32, a lens 34, and the like in addition to the components of the observation apparatus 1A shown in Fig. 1.
[0031] The lens 31 is optically connected to the light output end 19 and collimates the light (reference light) output from the light output end 19 as divergent light. The mirror 32 is optically connected to the lens 31 and reflects the light arriving from the lens 31 to the lens 34. The lens 34 is optically connected to the mirror 32 and outputs the light arriving from the mirror 32 to the beam splitter 41. The light output from the lens 34 is once condensed in front of the beam splitter 41 and then input to the beam splitter 41 as divergent light. The beam splitter 41 combines the light (object light) output from the objective lens 25 and arriving there, and the light (reference light) output from the lens 34 and arriving there, and outputs both lights to the lens 42 with the two lights being coaxial. The imaging unit 43 captures an interference fringe image (interference intensity image) due to interference between the object light and the reference light arriving from the lens 42. The direction of incidence of the reference light on the imaging surface of the imaging section 43 is parallel to the direction of incidence of the object light.
[0032] The driver 33 moves the mirror 32 in a direction perpendicular to the reflecting surface of the mirror 32. The driver 33 is, for example, a piezoelectric actuator. This movement of the mirror 32 changes the difference (phase difference) in the optical path lengths of the object light and the reference light from the optical branching in the fiber coupler 15 to the multiplexing in the beam splitter 41. If this optical path length difference differs, the interference intensity image captured by the imaging unit 43 also differs.
[0033] Fig. 3 is a flowchart of the observation method. This observation method can be performed using either the observation device 1A (Fig. 1) or the observation device 1B (Fig. 2). This observation method includes an interference intensity image acquisition step S1, a first complex amplitude image generation step S2, a second complex amplitude image generation step S3, a two-dimensional phase image generation step S4, a three-dimensional phase image generation step S5, and a refractive index distribution calculation step S6.
[0034] The processing of interference intensity image acquisition step S1 is performed by an interference intensity image acquisition unit 51. The processing of first complex amplitude image generation step S2 is performed by a first complex amplitude image generation unit 52. The processing of second complex amplitude image generation step S3 is performed by a second complex amplitude image generation unit 53. The processing of two-dimensional phase image generation step S4 is performed by a two-dimensional phase image generation unit 54. The processing of three-dimensional phase image generation step S5 is performed by a three-dimensional phase image generation unit 55. The processing of refractive index distribution calculation step S6 is performed by a refractive index distribution calculation unit 56.
[0035] In the interference intensity image acquiring step S1, the interference intensity image acquiring section 51 irradiates the observation object S with light along each of a plurality of light irradiation directions by changing the orientation of the reflecting surface of the mirror 22. Then, the interference intensity image acquiring section 51 acquires, from the imaging section 43, an interference intensity image at a reference position for each of the plurality of light irradiation directions.
[0036] For convenience of explanation, an xyz Cartesian coordinate system is shown in Fig. 1 and Fig. 2. The z axis is parallel to the optical axis of the objective lens 25. The reference position is the image plane position that is in a conjugate relationship with the imaging plane of the imaging unit 43. This position is z=0. The light irradiation direction to the observation object S is determined by the wave vector (k x ,k y ,k z ) k x and k y It can be expressed as follows:
[0037] 4(a) to 4(c) are diagrams showing an example of scanning the light irradiation direction of the observation object S in the interference intensity image acquisition step S1. In these diagrams, the horizontal axis is k x Let the vertical axis be k y k x k y The position of each circle on the plane represents the light irradiation direction. The light irradiation direction is scanned in the direction of k as shown in Fig. 4(a). x k y The electrodes may be arranged in a rectangular grid on a plane, or may be arranged in a rectangular grid on a plane as shown in FIG. x k yThe sensors may be arranged on the circumference of a plurality of concentric circles in a plane, or may be arranged in a k x k y The condenser lens 24 may be arranged in a spiral shape on a plane. In either case, scanning in the light irradiation direction is possible as long as the NA of the condenser lens 24 allows. Either raster scanning or random scanning is possible. In the case of raster scanning, return scanning may or may not be performed.
[0038] In the first complex amplitude image generating step S2, the first complex amplitude image generating unit 52 generates a complex amplitude image of a reference position based on the interference intensity image of the reference position acquired by the interference intensity image acquiring unit 51 for each of the multiple light irradiation directions. In the case of the observation device 1A (FIG. 1), the first complex amplitude image generating unit 52 can generate a complex amplitude image based on one interference intensity image by the Fourier fringe analysis method. In the case of the observation device 1B (FIG. 2), the first complex amplitude image generating unit 52 can generate a complex amplitude image based on three or more interference intensity images in which the optical path length difference (phase difference) between the object light and the reference light is different from each other by the phase shift method.
[0039] In the second complex amplitude image generating step S3, the second complex amplitude image generating unit 53 generates a complex amplitude image for each of the multiple z-direction positions based on the complex amplitude image for the reference position (z=0) generated by the first complex amplitude image generating unit 52 for each of the multiple light irradiation directions. The complex amplitude image u(x, y, 0) for the reference position is transformed into a complex amplitude image U(k x ,k y ,0), the complex amplitude image u(x,y,d) at the position z=d and the two-dimensional Fourier transform U(k x ,k y , d) are expressed by the following formula, where i is the imaginary unit and k 0 is the wave number of the light in the object being observed.
[0040]
number
[0041]
number
[0042] In the two-dimensional phase image generating step S4, the two-dimensional phase image generating unit 54 generates a two-dimensional phase image for each of the multiple positions based on the complex amplitude images in each of the multiple light irradiation directions generated by the second complex amplitude image generating unit 53. The two-dimensional phase image generated here corresponds to a phase image centered on the focused z-direction position. The two-dimensional phase image generating step S4 will be described later in detail.
[0043] Note that the processes in and after the two-dimensional phase image generating step S4 may be performed after all of the complex amplitude images at the multiple positions for each of the multiple light irradiation directions are generated in the second complex amplitude image generating step S3. Alternatively, a complex amplitude image at one z-direction position may be generated for each of the multiple light irradiation directions in the second complex amplitude image generating step S3, and a two-dimensional phase image at that position may be generated in the two-dimensional phase image generating step S4. The unit process may be repeated while scanning the z-direction position. The latter case is preferable in that the amount of image data to be stored in the storage unit 58 can be reduced.
[0044] In a three-dimensional phase image generating step S5, the three-dimensional phase image generating section 55 generates a three-dimensional phase image based on the two-dimensional phase images at each of the multiple positions generated by the two-dimensional phase image generating section 54. The three-dimensional phase image generated here is an image whose variables are positions x, y in the two-dimensional phase image and position z of the two-dimensional phase image.
[0045] In the refractive index distribution calculation step S6, the refractive index distribution calculation unit 56 obtains the three-dimensional refractive index distribution of the observation object by deconvolution based on the three-dimensional phase image generated by the three-dimensional phase image generation unit 55. The refractive index distribution of the observation object is n(x,y,z), the electrical susceptibility distribution is f(x,y,z), and the refractive index of the background medium is n m Then, there is a relationship between the two as shown in the following formula (3). The three-dimensional phase image Φ(x,y,z) generated by the three-dimensional phase image generating unit 55 is expressed by the convolution of the kernel function g(x,y,z) and the electric susceptibility distribution f(x,y,z) as shown in the following formula (4). Therefore, the three-dimensional refractive index distribution n(x,y,z) of the observation object can be obtained by deconvolution based on the three-dimensional phase image Φ(x,y,z).
[0046]
number
[0047]
number
[0048] The kernel function g is based on the Green's function corresponding to the solution of the wave equation. Figure 5 is a diagram explaining the kernel function g. In this diagram, the central position where the value of the kernel function g is the largest is the origin, the vertical direction is the z-axis, and the horizontal direction is the direction perpendicular to the z-axis.
[0049] Next, the two-dimensional phase image generating step S4 will be described in detail. In the two-dimensional phase image generating step S4, the two-dimensional phase image generating section 54 generates a two-dimensional phase image for each of the multiple positions based on the complex amplitude images in each of the multiple light irradiation directions generated by the second complex amplitude image generating section 53. The two-dimensional phase image generating step S4 can have three aspects described below.
[0050] 6 is a flow chart of a first aspect of the two-dimensional phase image generating step S4. In this first aspect, the two-dimensional phase image generating step S4 corrects the phases of the complex amplitude images in each of the multiple light irradiation directions based on the light irradiation direction for each of the multiple positions in step S11, generates a complex amplitude sum image representing the sum of these corrected complex amplitude images, and generates a two-dimensional phase image based on the complex amplitude sum image in step S12.
[0051] The process of step S11 is based on CASS (Collective Accumulation of Single Scattering) technology described in Non-Patent Document 1. Among light that is irradiated to an object along a certain light irradiation direction and passes through the object, the spatial frequency distribution of single scattered light that has interacted with the object only once shifts depending on the light irradiation direction, whereas the spatial frequency distribution of multiple scattered light that has interacted with the object multiple times changes randomly depending on the light irradiation direction. The CASS technology utilizes the difference in the light irradiation direction dependency of the spatial frequency distribution of such single scattered light and multiple scattered light.
[0052] That is, in step S11, the phase of the complex amplitude image for each of the multiple light irradiation directions is corrected based on the light irradiation direction (i.e., the spatial frequency distribution of the complex amplitude image in the spatial frequency domain is translated according to the light irradiation direction), so that the spatial frequency distribution of the single scattered light component in the complex amplitude image has a shape and arrangement that is independent of the light irradiation direction, while the spatial frequency distribution of the multiple scattered light component in the complex amplitude image has a random shape and arrangement. Then, in step S11, a complex amplitude sum image that represents the sum of the multiple complex amplitude images after these corrections is generated (i.e., synthetic aperture processing is performed), so that the single scattered light components in the complex amplitude images are coherently added together, while the multiple scattered light components in the complex amplitude images are offset by each other.
[0053] Therefore, the complex amplitude sum image generated in step S11 has a reduced effect of multiple scattered light, and the three-dimensional refractive index distribution finally obtained in the refractive index distribution calculation step S6 also has a reduced effect of multiple scattered light, suppressing speckles and improving SMR.
[0054] 7 to 12 are diagrams showing examples of images obtained at each step in the first embodiment of the two-dimensional phase image generating step S4. These images are based on the Fourier fringe analysis method using the observation device 1A (FIG. 1). FIG. 7 is an interference intensity image (at normal irradiation) acquired at the interference intensity image acquiring step S1. FIG. 8 is a complex amplitude image (real part, z=0) generated based on the interference intensity image (FIG. 7) at the first complex amplitude image generating step S2. FIG. 9 is a complex amplitude image (real part, z=10 μm) generated based on the complex amplitude image (FIG. 8) at the second complex amplitude image generating step S3.
[0055] Fig. 10 is a complex amplitude sum image (real part) generated based on the complex amplitude image (Fig. 9) in step S11 of the two-dimensional phase image generation step S4. Fig. 11 is a two-dimensional phase image generated based on the complex amplitude sum image (Fig. 10) in step S12 of the two-dimensional phase image generation step S4. Fig. 12 is an image at a certain z position of the three-dimensional refractive index distribution obtained in the refractive index distribution calculation step S6. In this way, the obtained three-dimensional refractive index distribution has a reduced effect of multiple scattered light, suppressed speckle, and improved SMR.
[0056] 13 is a flow chart of a second aspect of the two-dimensional phase image generating step S4. In this second aspect, the two-dimensional phase image generating step S4 generates, for each of the multiple positions, a complex differential interference image for each of the multiple light irradiation directions based on a complex amplitude image for each of the multiple light irradiation directions in step S21, generates a phase differential image based on the sum of the complex differential interference images for each of the multiple light irradiation directions in step S22, and generates a two-dimensional phase image based on the phase differential image in step S23.
[0057] If the complex amplitude image at the position z=d is u(x, y, d), the complex differential interference image q(x, y, d) generated in step S21 is expressed by the following equation (5). At least one of δx and δy is non-zero. If δx ≠ 0, δy = 0, a complex differential interference image q with the x direction as the shear direction is obtained. If δx = 0, δy ≠ 0, a complex differential interference image q with the y direction as the shear direction is obtained. If δx ≠ 0, δy ≠ 0, a complex differential interference image q with a shear direction different from both the x and y directions is obtained.
[0058]
number
[0059] The sum of the complex differential interference images q for each of the multiple light irradiation directions is q sum (x, y, d), the phase differential image φ(x, y, z) generated in step S22 is q sum The phase of (x, y, d) is expressed by the following equation (6): In step S23, a two-dimensional phase image can be generated by integrating or deconvolving this phase differential image φ(x, y, z).
[0060]
number
[0061] In step S21, a complex differential interference image may be generated for each of a plurality of different shear directions on the complex amplitude image. In this case, the two-dimensional phase image generating step S4 generates a complex differential interference image for each of a plurality of different shear directions on the image based on the complex amplitude images for each of the plurality of light irradiation directions for each of the plurality of positions in step S21, generates a phase differential image for each of the plurality of shear directions based on the sum of the complex differential interference images for each of the plurality of light irradiation directions in step S22, and generates a two-dimensional phase image based on the phase differential images for each of the plurality of shear directions in step S23.
[0062] In step S22, the differential phase image generated based on the sum of the complex differential interference images for each of the multiple light irradiation directions has a reduced effect of multiple scattered light. The three-dimensional refractive index distribution finally obtained in step S6 of calculating the refractive index distribution also has a reduced effect of multiple scattered light and suppressed speckle. In addition, when a complex differential interference image is generated for each of the multiple shear directions different from each other on the complex amplitude image in step S21, it is possible to suppress the appearance of line-shaped noise in the two-dimensional phase image obtained in step S23.
[0063] 14 to 21 are diagrams showing examples of images obtained at each step in the case of the second aspect of the two-dimensional phase image generating step S4. These images are based on the Fourier fringe analysis method using the observation device 1A (FIG. 1). In addition, in step S21, complex differential interference images were generated for each of two different shear directions (vertical shear and horizontal shear) on the complex amplitude image. FIG. 14 is an interference intensity image (at vertical irradiation) acquired in the interference intensity image acquiring step S1. FIG. 15 is a complex amplitude image (real part, z=0) generated based on the interference intensity image (FIG. 14) in the first complex amplitude image generating step S2. FIG. 16 is a complex amplitude image (real part, z=20 μm) generated based on the complex amplitude image (FIG. 15) in the second complex amplitude image generating step S3.
[0064] FIG. 17 is a complex differential interference image (real and imaginary parts for vertical shear and horizontal shear) generated based on the complex amplitude image (FIG. 16) in step S21 of the two-dimensional phase image generation step S4. FIG. 18 is an image (real and imaginary parts for vertical shear and horizontal shear) representing the sum of the complex differential interference images (FIG. 17) generated in step S21 of the two-dimensional phase image generation step S4. FIG. 19 is a phase differential image (vertical shear and horizontal shear) generated based on the image (FIG. 18) representing the sum of the complex differential interference images in step S22 of the two-dimensional phase image generation step S4. FIG. 20 is a two-dimensional phase image generated based on the phase differential image (FIG. 19) in step S23 of the two-dimensional phase image generation step S4. FIG. 21 is an image at a certain z position of the three-dimensional refractive index distribution obtained in the refractive index distribution calculation step S6. In this way, the effect of multiple scattered light is reduced in the obtained three-dimensional refractive index distribution, and speckle is suppressed.
[0065] Here, a case has been described in which a two-dimensional phase image is generated by integrating or deconvolving the phase differential image in step S23. However, the phase differential image can also be treated as a two-dimensional phase image. In this case, without performing step S23, a three-dimensional refractive index distribution of the observation object can be obtained from the phase differential image (two-dimensional phase image) generated in step S22 by using a kernel (FIG. 29) including the kernel used in the deconvolution in step S23 in the deconvolution in the refractive index distribution calculation step S6. The kernel shown in FIG. 29 is obtained by convolution integration of the kernel shown in FIG. 5 and the kernel used in the deconvolution in step S23.
[0066] 22 is a flow chart of a third aspect of the two-dimensional phase image generating step S4. In this third aspect, the two-dimensional phase image generating step S4 divides the complex amplitude images of each of the multiple light irradiation directions into multiple batches for each of the multiple positions in step S31, corrects the phase of the complex amplitude image included in each of the multiple batches based on the light irradiation direction, and generates a complex amplitude sum image representing the sum of these corrected complex amplitude images, generates a complex differential interference image for each of the multiple batches based on the complex amplitude sum image for each of the multiple batches in step S32, generates a phase differential image for each of the multiple batches based on the sum of the complex differential interference images for each of the multiple batches in step S33, and generates a two-dimensional phase image based on the phase differential image in step S34.
[0067] The process of step S31 in the third aspect corresponds to dividing the complex amplitude images in each of the multiple light irradiation directions into multiple batches, and then performing the process of step S11 in the first aspect for each of the multiple batches. The processes of steps S32 and S33 in the third aspect correspond to performing the processes of steps S21 and S22 in the second aspect for each of the multiple batches. The process of step S34 in the third aspect corresponds to performing the process of step S23 in the second aspect.
[0068] In step S32, a complex differential interference image may be generated for each of a plurality of different shear directions on the complex amplitude image. In this case, the two-dimensional phase image generating step S4 generates a complex differential interference image for each of a plurality of batches for each of a plurality of different shear directions on the image based on a complex amplitude sum image of each of the plurality of batches in step S32, generates a phase differential image for each of the plurality of shear directions based on the sum of the complex differential interference images of each of the plurality of batches in step S33, and generates a two-dimensional phase image based on the phase differential image for each of the plurality of shear directions in step S34.
[0069] The suppression of speckle in the third embodiment is comparable to that in the first and second embodiments. The improvement of SMR in the third embodiment is intermediate between that in the first and second embodiments.
[0070] 23 to 28 are diagrams showing examples of images obtained at each step in the third embodiment of the two-dimensional phase image generating step S4. These images are based on the Fourier fringe analysis method using the observation device 1A (FIG. 1). In addition, in step S32, complex differential interference contrast images were generated for each of two different shear directions (vertical shear and horizontal shear) on the complex amplitude image. FIG. 23 is an interference intensity image (at normal irradiation) acquired in the interference intensity image acquiring step S1. FIG. 24 is a complex amplitude image (real part, z=0) generated based on the interference intensity image (FIG. 23) in the first complex amplitude image generating step S2.
[0071] Fig. 25 is a complex amplitude image (real part, z=20 μm) generated based on the complex amplitude image (Fig. 24) in the second complex amplitude image generating step S3. Fig. 26 is a complex amplitude sum image generated based on the complex amplitude image (Fig. 25) in step S31 in the two-dimensional phase image generating step S4. Fig. 27 is a two-dimensional phase image generated in step S34 in the two-dimensional phase image generating step S4. Fig. 28 is an image at a certain z position of the three-dimensional refractive index distribution obtained in the refractive index distribution calculating step S6. In this way, the obtained three-dimensional refractive index distribution has a reduced effect of multiple scattered light, suppressed speckle, and improved SMR.
[0072] Here, a case has been described in which a two-dimensional phase image is generated by integrating or deconvolving the phase differential image in step S34. However, the phase differential image can also be treated as a two-dimensional phase image. In this case, the three-dimensional refractive index distribution of the observation object can be obtained from the phase differential image (two-dimensional phase image) generated in step S33 by using a kernel including the kernel used in the deconvolution in step S34 in the deconvolution in the refractive index distribution calculation step S6 without performing step S34.
[0073] When comparing an image of the three-dimensional refractive index distribution obtained in the first embodiment at a certain z position (FIG. 12), an image of the three-dimensional refractive index distribution obtained in the second embodiment at a certain z position (FIG. 21), and an image of the three-dimensional refractive index distribution obtained in the third embodiment at a certain z position (FIG. 28), it is found that, although the speckle suppression effect is about the same, the third embodiment has a greater SMR improvement effect than the second embodiment, and the first embodiment has a greater SMR improvement effect than the third embodiment.
[0074] In this embodiment, when calculating the phase, it is preferable to perform phase unwrapping as necessary.
[0075] The observation device is not limited to the above-mentioned embodiment and configuration example, and various modifications are possible. In the configurations of the observation device 1A (FIG. 1) and the observation device 1B (FIG. 2), the light transmitted through the observation target S is used as the object light, but the light reflected by the observation target S may be used as the object light as in the configuration of the observation device 1C (FIG. 30) described below.
[0076] 30 is a diagram showing the configuration of an observation device 1C. The observation device 1C includes a light source 11, a lens 12, a lens 21, a mirror 22, a lens 23, an objective lens 25, a beam splitter 41, a lens 42, an imaging unit 43, and an analyzing unit 50. The following mainly describes the differences from the observation device 1A (FIG. 1).
[0077] The lens 21 is optically connected to the light output end 18 of the optical fiber 16, and collimates the light output from the light output end 18 as divergent light. The mirror 22 is optically connected to the lens 21, and reflects the light arriving from the lens 21 to the lens 23. The orientation of the reflecting surface of the mirror 22 is variable. The lens 23 is optically connected to the mirror 22. The objective lens 25 is optically connected to the lens 23. A beam splitter 41 is disposed between the lens 23 and the objective lens 25. The lens 23 and the objective lens 25 preferably constitute a 4f optical system. The lens 23 and the objective lens 25 irradiate light onto the observation object S from a light irradiation direction according to the orientation of the reflecting surface of the mirror 22. The objective lens 25 inputs the light (object light) reflected by the observation object S, and outputs the light to the beam splitter 41.
[0078] The beam splitter 41 is optically connected to the objective lens 25 and also optically connected to the light output end 19 of the optical fiber 17. The beam splitter 41 combines the light (object light) output from the objective lens 25 and arriving therewith and the light (reference light) output from the light output end 19 and arriving therewith, and outputs both lights to the lens 42. The lens 42 is optically connected to the beam splitter 41, and collimates the object light and the reference light arriving therefrom and outputs them to the imaging unit 43. The imaging unit 43 is optically connected to the lens 42, and captures an interference fringe image (interference intensity image) due to interference between the object light and the reference light arriving therefrom. The incident direction of the reference light is inclined with respect to the incident direction of the object light on the imaging surface of the imaging unit 43. The position where the object light and the reference light are combined by the beam splitter 41 may be located after the imaging lens, but considering the effects of aberration, it is preferable that it be located between the objective lens 25 and lens 42 as shown in the figure.
[0079] In the configuration of the observation device 1C (Fig. 30), a mechanism for changing the optical path length of the reference light (lens 31, mirror 32, drive unit 33, and lens 34 in Fig. 2) may be provided as in the observation device 1B (Fig. 2) to change the difference in the optical path lengths (phase difference) of the object light and the reference light from the optical branching in the fiber coupler 15 to the multiplexing in the beam splitter 41. In this case, the incident direction of the reference light may be parallel to the incident direction of the object light on the imaging surface of the imaging unit 43.
[0080] Furthermore, each of the processes of the first complex amplitude image generation step S2, the second complex amplitude image generation step S3, the two-dimensional phase image generation step S4, the three-dimensional phase image generation step S5 and the refractive index distribution calculation step S6 may be performed each time an interference intensity image for each of a predetermined number of light irradiation directions is acquired in the interference intensity image acquisition step S1 (FIG. 31), or each time an interference intensity image for one light irradiation direction is acquired in the interference intensity image acquisition step S1 (FIG. 32).
[0081] 31 and 32 are diagrams showing examples of scanning the light irradiation direction of the observation target S in the interference intensity image acquisition step S1. In these diagrams, the horizontal axis is k x Let the vertical axis be k y k x k y The position of each circle on the plane represents the light irradiation direction. In the examples of scanning the light irradiation direction shown in these figures, the light irradiation direction is changed sequentially so that the light irradiation direction when the (N+n)th interference intensity image is acquired coincides with the light irradiation direction when the nth interference intensity image is acquired. n is a positive integer, and N is an integer of 2 or more.
[0082] In the example shown in FIG. 31, when the first to Nth interference intensity images are acquired in the interference intensity image acquisition step S1, each process of steps S2 to S6 is performed based on these first to Nth interference intensity images (FIG. 31(a)). Next, when the (N+1)th to 2Nth interference intensity images are acquired in the interference intensity image acquisition step S1, each process of steps S2 to S6 is performed based on these (N+1)th to 2Nth interference intensity images (FIG. 31(b)). Next, when the (2N+1)th to 3Nth interference intensity images are acquired in the interference intensity image acquisition step S1, each process of steps S2 to S6 is performed based on these (2N+1)th to 3Nth interference intensity images. The same applies thereafter.
[0083] In the example shown in FIG. 32, when the first to Nth interference intensity images are acquired in the interference intensity image acquisition step S1, each process of steps S2 to S6 is performed based on these first to Nth interference intensity images (FIG. 32(a)). Next, when the (N+1)th interference intensity image is acquired in the interference intensity image acquisition step S1, each process of steps S2 to S6 is performed based on the nearest N interference intensity images (the second to (N+1)th interference intensity images) including this (N+1)th interference intensity image (FIG. 32(b)). Next, when the (N+2)th interference intensity image is acquired in the interference intensity image acquisition step S1, each process of steps S2 to S6 is performed based on the nearest N interference intensity images (the third to (N+2)th interference intensity images) including this (N+2)th interference intensity image (FIG. 32(c)). Similarly, when the (N+n)th interference intensity image is acquired in the interference intensity image acquisition step S1, the processes of steps S2 to S6 are performed based on the most recent N interference intensity images (the (1+n)th to (N+n)th interference intensity images) including this (N+n)th interference intensity image.
[0084] Compared with the example shown in FIG. 31, in the example shown in FIG. 32, each time an interference intensity image for one light irradiation direction is acquired in interference intensity image acquisition step S1, the processes of steps S2 to S6 are performed based on the most recent interference intensity images including that interference intensity image, and therefore the number of images obtained per unit time by the processes of steps S2 to S6 is large.
[0085] The observation device of the above embodiment includes: (1) an interference intensity image acquisition unit that acquires, for each of a plurality of light irradiation directions, an interference intensity image of a reference position resulting from interference between light that is irradiated onto an observation object along each of a plurality of light irradiation directions and passes through the observation object and a reference light; (2) a first complex amplitude image generation unit that generates, for each of the plurality of light irradiation directions, a complex amplitude image of the reference position based on the interference intensity image of the reference position; and (3) a two-dimensional phase image generation unit that generates a two-dimensional phase image based on the complex amplitude image for each of the plurality of light irradiation directions.
[0086] In one aspect, the two-dimensional phase image generation unit (a) corrects the phase of a complex amplitude image for each of a plurality of light irradiation directions based on the light irradiation direction, and then generates a complex amplitude sum image representing the sum of these corrected complex amplitude images, and (b) generates a two-dimensional phase image based on the complex amplitude sum image.
[0087] In another aspect, the two-dimensional phase image generating unit (a) generates a complex differential interference image for each of the plurality of light irradiation directions based on the complex amplitude images for each of the plurality of light irradiation directions, and (b) generates a two-dimensional phase image based on the complex differential interference image for each of the plurality of light irradiation directions. It is also preferable that the two-dimensional phase image generating unit (a) generates a complex differential interference image for each of the plurality of light irradiation directions for each of a plurality of different shear directions on the image based on the complex amplitude images for each of the plurality of light irradiation directions, and (b) generates a two-dimensional phase image based on the complex differential interference images for each of the plurality of shear directions and the plurality of light irradiation directions.
[0088] In yet another aspect, the two-dimensional phase image generation unit (a) divides complex amplitude images for each of a plurality of light irradiation directions into a plurality of batches, corrects the phase of the complex amplitude images included in each of the plurality of batches based on the light irradiation direction, and then generates a complex amplitude sum image representing the sum of these corrected complex amplitude images, (b) generates a complex differential interference contrast image for each of the plurality of batches based on the complex amplitude sum image for each of the plurality of batches, and (c) generates a two-dimensional phase image based on the complex differential interference contrast image for each of the plurality of batches. It is also preferable that the two-dimensional phase image generating unit (a) divides complex amplitude images for each of the multiple light irradiation directions into multiple batches, corrects the phase of the complex amplitude images included in each of the multiple batches based on the light irradiation direction, and then generates a complex amplitude sum image representing the sum of these corrected complex amplitude images, (b) generates a complex differential interference contrast image for each of the multiple batches for each of multiple different shear directions on the image based on the complex amplitude sum image for each of the multiple batches, and (c) generates a two-dimensional phase image based on the complex differential interference contrast images for the multiple shear directions and each of the multiple batches.
[0089] In one aspect, it is preferable that the observation device further includes (4) a second complex amplitude image generation unit that generates a complex amplitude image for each of the multiple positions based on a complex amplitude image of a reference position generated by the first complex amplitude image generation unit for each of the multiple light irradiation directions, (5) a three-dimensional phase image generation unit that generates a three-dimensional phase image based on the two-dimensional phase image for each of the multiple positions generated by the two-dimensional phase image generation unit, and (6) a refractive index distribution calculation unit that calculates a three-dimensional refractive index distribution of the observed object based on the three-dimensional phase image.
[0090] The observation method of the above embodiment includes: (1) an interference intensity image acquisition step of acquiring, for each of a plurality of light irradiation directions, an interference intensity image of a reference position resulting from interference between light that is irradiated onto an object to be observed along each of a plurality of light irradiation directions and passes through the object to be observed and a reference light; (2) a first complex amplitude image generation step of generating, for each of the plurality of light irradiation directions, a complex amplitude image of the reference position based on the interference intensity image of the reference position; and (3) a two-dimensional phase image generation step of generating a two-dimensional phase image based on the complex amplitude image for each of the plurality of light irradiation directions.
[0091] In one aspect, the two-dimensional phase image generating step includes: (a) correcting the phase of a complex amplitude image for each of a plurality of light irradiation directions based on the light irradiation direction, and then generating a complex amplitude sum image representing the sum of these corrected complex amplitude images; and (b) generating a two-dimensional phase image based on the complex amplitude sum image.
[0092] In another aspect, the two-dimensional phase image generating step (a) generates a complex differential interference image for each of the plurality of light irradiation directions based on the complex amplitude images for each of the plurality of light irradiation directions, and (b) generates a two-dimensional phase image based on the complex differential interference image for each of the plurality of light irradiation directions. Also, it is preferable that the two-dimensional phase image generating step (a) generates a complex differential interference image for each of the plurality of light irradiation directions for each of a plurality of different shear directions on the image based on the complex amplitude images for each of the plurality of light irradiation directions, and (b) generates a two-dimensional phase image based on the complex differential interference images for each of the plurality of shear directions and the plurality of light irradiation directions.
[0093] In yet another aspect, the two-dimensional phase image generating step includes: (a) dividing complex amplitude images for each of a plurality of light irradiation directions into a plurality of batches; correcting the phase of the complex amplitude images included in each of the plurality of batches based on the light irradiation direction; and generating a complex amplitude sum image representing the sum of these corrected complex amplitude images; (b) generating a complex differential interference contrast image for each of the plurality of batches based on the complex amplitude sum image for each of the plurality of batches; and (c) generating a two-dimensional phase image based on the complex differential interference contrast image for each of the plurality of batches. In addition, in the two-dimensional phase image generating step, it is preferable to (a) divide complex amplitude images for each of the multiple light irradiation directions into multiple batches, correct the phase of the complex amplitude images contained in each of the multiple batches based on the light irradiation direction, and then generate a complex amplitude sum image representing the sum of these corrected complex amplitude images, (b) generate a complex differential interference contrast image for each of the multiple batches for each of multiple different shear directions on the image based on the complex amplitude sum image for each of the multiple batches, and (c) generate a two-dimensional phase image based on the complex differential interference contrast images for the multiple shear directions and each of the multiple batches.
[0094] In one aspect, the observation method preferably further includes: (4) a second complex amplitude image generation step of generating a complex amplitude image for each of the plurality of positions based on the complex amplitude image of the reference position generated in the first complex amplitude image generation step for each of the plurality of light irradiation directions; (5) a three-dimensional phase image generation step of generating a three-dimensional phase image based on the two-dimensional phase image for each of the plurality of positions generated in the two-dimensional phase image generation step; and (6) a refractive index distribution calculation step of determining a three-dimensional refractive index distribution of the object to be observed based on the three-dimensional phase image.
[0095] The program of the above embodiment is for causing a computer to execute each step of the above observation method.The recording medium of the above embodiment is a computer-readable medium having the above program recorded thereon. [Industrial Applicability]
[0096] INDUSTRIAL APPLICABILITY The present invention can be used as an apparatus and method capable of achieving three-dimensional refractive index tomography with reduced influence of multiple scattered light, even when the object to be observed is a multiple scattering medium. [Explanation of symbols]
[0097] Reference Signs List 1A, 1B...observation device, 2...recording medium, 11...light source, 12...lens, 13...light input end, 14...optical fiber, 15...fiber coupler, 16, 17...optical fiber, 18, 19...light output end, 21...lens, 22...mirror, 23...lens, 24...condenser lens, 25...objective lens, 31...lens, 32...mirror, 33...drive unit, 34...lens, 41...beam splitter, 42...lens, 43...imaging unit, 50...analysis unit, 51...interference intensity image acquisition unit, 52...first complex amplitude image generation unit, 53...second complex amplitude image generation unit, 54...two-dimensional phase image generation unit, 55...three-dimensional phase image generation unit, 56...refractive index distribution calculation unit, 57...display unit, 58...memory unit.
Claims
1. an interference intensity image acquisition unit that acquires, for each of a plurality of light irradiation directions, an interference intensity image of a reference position caused by interference between light that is irradiated onto an observation object along each of a plurality of light irradiation directions and passes through the observation object and a reference light; a first complex amplitude image generating unit configured to generate a complex amplitude image of the reference position based on the interference intensity image of the reference position for each of the plurality of light irradiation directions; a two-dimensional phase image generating unit that generates a two-dimensional phase image based on the complex amplitude images in each of the plurality of light irradiation directions; Equipped with The two-dimensional phase image generating unit correcting the phases of the complex amplitude images for each of the plurality of light irradiation directions based on the light irradiation direction, and then generating a complex amplitude sum image representing a sum of the corrected complex amplitude images; generating the two-dimensional phase image based on the complex amplitude sum image; Observation equipment.
2. an interference intensity image acquisition unit that acquires, for each of a plurality of light irradiation directions, an interference intensity image of a reference position caused by interference between light that is irradiated onto an observation object along each of a plurality of light irradiation directions and passes through the observation object and a reference light; a first complex amplitude image generating unit configured to generate a complex amplitude image of the reference position based on the interference intensity image of the reference position for each of the plurality of light irradiation directions; a two-dimensional phase image generating unit that generates a two-dimensional phase image based on the complex amplitude images in each of the plurality of light irradiation directions; Equipped with The two-dimensional phase image generating unit generating a complex differential interference image for each of the plurality of light irradiation directions based on the complex amplitude images for each of the plurality of light irradiation directions; generating the two-dimensional phase image based on the complex differential interference images for each of the plurality of light irradiation directions; Observation equipment.
3. The two-dimensional phase image generating unit generating a complex differential interference contrast image for each of the plurality of light irradiation directions for each of a plurality of shear directions different from each other on the image based on the complex amplitude image for each of the plurality of light irradiation directions; generating the two-dimensional phase image based on the complex differential interference images in each of the plurality of shear directions and the plurality of light irradiation directions; The observation device according to claim 2.
4. an interference intensity image acquisition unit that acquires, for each of a plurality of light irradiation directions, an interference intensity image of a reference position caused by interference between light that is irradiated onto an observation object along each of a plurality of light irradiation directions and passes through the observation object and a reference light; a first complex amplitude image generating unit configured to generate a complex amplitude image of the reference position based on the interference intensity image of the reference position for each of the plurality of light irradiation directions; a two-dimensional phase image generating unit that generates a two-dimensional phase image based on the complex amplitude images in each of the plurality of light irradiation directions; Equipped with The two-dimensional phase image generating unit dividing the complex amplitude images for each of the plurality of light irradiation directions into a plurality of batches, correcting the phase of the complex amplitude image included in each of the plurality of batches based on the light irradiation direction, and then generating a complex amplitude sum image representing a sum of the complex amplitude images after the correction; generating a complex differential interference contrast image for each of the plurality of batches based on the complex amplitude sum image for each of the plurality of batches; generating the two-dimensional phase image based on the complex differential interference images of each of the plurality of batches; Observation equipment.
5. The two-dimensional phase image generating unit dividing the complex amplitude images for each of the plurality of light irradiation directions into a plurality of batches, correcting the phase of the complex amplitude image included in each of the plurality of batches based on the light irradiation direction, and then generating a complex amplitude sum image representing a sum of the complex amplitude images after the correction; generating a complex differential interference contrast image of each of the plurality of batches for each of a plurality of shear directions different from each other on the image based on the complex amplitude sum image of each of the plurality of batches; generating the two-dimensional phase image based on the complex differential interference images for the plurality of shear directions and the plurality of batches; The observation device according to claim 4.
6. a second complex amplitude image generating unit configured to generate a complex amplitude image for each of a plurality of positions based on the complex amplitude image for the reference position generated by the first complex amplitude image generating unit for each of the plurality of light irradiation directions; a three-dimensional phase image generating unit that generates a three-dimensional phase image based on the two-dimensional phase images for each of the plurality of positions generated by the two-dimensional phase image generating unit; a refractive index distribution calculation unit that calculates a three-dimensional refractive index distribution of the observation object based on the three-dimensional phase image; The observation device according to any one of claims 1 to 5, further comprising:
7. an interference intensity image acquisition step of acquiring, for each of a plurality of light irradiation directions, an interference intensity image of a reference position caused by interference between light irradiated onto an observation object along each of a plurality of light irradiation directions and passing through the observation object and a reference light; a first complex amplitude image generating step of generating a complex amplitude image of the reference position based on the interference intensity image of the reference position for each of the plurality of light irradiation directions; a two-dimensional phase image generating step of generating a two-dimensional phase image based on the complex amplitude images in each of the plurality of light irradiation directions; Equipped with In the two-dimensional phase image generating step, correcting the phases of the complex amplitude images for each of the plurality of light irradiation directions based on the light irradiation direction, and then generating a complex amplitude sum image representing a sum of the corrected complex amplitude images; generating the two-dimensional phase image based on the complex amplitude sum image; Observation method.
8. an interference intensity image acquisition step of acquiring, for each of a plurality of light irradiation directions, an interference intensity image of a reference position caused by interference between light irradiated onto an observation object along each of a plurality of light irradiation directions and passing through the observation object and a reference light; a first complex amplitude image generating step of generating a complex amplitude image of the reference position based on the interference intensity image of the reference position for each of the plurality of light irradiation directions; a two-dimensional phase image generating step of generating a two-dimensional phase image based on the complex amplitude images in each of the plurality of light irradiation directions; Equipped with In the two-dimensional phase image generating step, generating a complex differential interference image for each of the plurality of light irradiation directions based on the complex amplitude images for each of the plurality of light irradiation directions; generating the two-dimensional phase image based on the complex differential interference images for each of the plurality of light irradiation directions; Observation method.
9. In the two-dimensional phase image generating step, generating a complex differential interference contrast image for each of the plurality of light irradiation directions for each of a plurality of shear directions different from each other on the image based on the complex amplitude image for each of the plurality of light irradiation directions; generating the two-dimensional phase image based on the complex differential interference images in each of the plurality of shear directions and the plurality of light irradiation directions; The observation method according to claim 8.
10. an interference intensity image acquisition step of acquiring, for each of a plurality of light irradiation directions, an interference intensity image of a reference position caused by interference between light irradiated onto an observation object along each of a plurality of light irradiation directions and passing through the observation object and a reference light; a first complex amplitude image generating step of generating a complex amplitude image of the reference position based on the interference intensity image of the reference position for each of the plurality of light irradiation directions; a two-dimensional phase image generating step of generating a two-dimensional phase image based on the complex amplitude images in each of the plurality of light irradiation directions; Equipped with In the two-dimensional phase image generating step, dividing the complex amplitude images for each of the plurality of light irradiation directions into a plurality of batches, correcting the phase of the complex amplitude image included in each of the plurality of batches based on the light irradiation direction, and then generating a complex amplitude sum image representing a sum of the complex amplitude images after the correction; generating a complex differential interference contrast image for each of the plurality of batches based on the complex amplitude sum image for each of the plurality of batches; generating the two-dimensional phase image based on the complex differential interference images of each of the plurality of batches; Observation method.
11. In the two-dimensional phase image generating step, dividing the complex amplitude images for each of the plurality of light irradiation directions into a plurality of batches, correcting the phase of the complex amplitude image included in each of the plurality of batches based on the light irradiation direction, and then generating a complex amplitude sum image representing a sum of the complex amplitude images after the correction; generating a complex differential interference contrast image of each of the plurality of batches for each of a plurality of shear directions different from each other on the image based on the complex amplitude sum image of each of the plurality of batches; generating the two-dimensional phase image based on the complex differential interference images for the plurality of shear directions and the plurality of batches; The observation method according to claim 10.
12. a second complex amplitude image generating step of generating a complex amplitude image for each of a plurality of positions based on the complex amplitude image for the reference position generated in the first complex amplitude image generating step for each of the plurality of light irradiation directions; a three-dimensional phase image generating step of generating a three-dimensional phase image based on the two-dimensional phase images for each of the plurality of positions generated in the two-dimensional phase image generating step; a refractive index distribution calculation step of calculating a three-dimensional refractive index distribution of the object to be observed based on the three-dimensional phase image; The observation method according to any one of claims 7 to 11, further comprising:
13. A program for causing a computer to execute each step of the observation method according to any one of claims 7 to 12.
14. A computer-readable recording medium having the program according to claim 13 recorded thereon.
Citation Information
Patent Citations
Three-dimensional refractive index tomography and fluorescence structured illumination microscopy system using wavefront shaper, and method of using the same
JP2017219826A
Phase image acquisition device and phase image acquisition method
JP2019124679A
Holographic imaging device and data processing method therefor
WO2019044336A1
Image generation device and image generation method
WO2020013325A1
