Observation device and observation method

Abstract

To provide an observation device that allows observation of an object to be observed while reducing the influence of multiple scattered light.SOLUTION: An observation device 1A comprises a light source 11, a mirror 22, a condenser lens 24, an objective lens 25, a beam splitter 41, an imaging unit 43, and an analysis unit 60. The analysis unit 60 includes an interference intensity image acquisition unit 61, a first complex amplitude image creation unit 62, a second complex amplitude image creation unit 63, a phase conjugation operation unit 64, a two-dimensional phase image creation unit 65, a three-dimensional phase image creation unit 66, and a refractive index distribution calculation unit 67. The analysis unit 60 changes the direction of a reflection surface of the mirror 22 to irradiate an object to be observed S with light along a plurality of light irradiation directions, acquires, from the imaging unit 43, an interference intensity image at a reference position for each of the plurality of light irradiation directions, and performs predetermined processing based on these interference intensity images to determine the three-dimensional refractive index distribution of the object to be observed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an observation device and an observation method. [Background technology]

[0002] In recent years, advances have been made in the technology for creating three-dimensional cellular tissues known as spheroids and organoids. Research is also progressing on the application of these three-dimensional cellular tissues to drug discovery, regenerative medicine, and other fields. These three-dimensional cellular tissues are optically transparent, multiple-scattering bodies. A wide variety of imaging techniques have been proposed for imaging such optically transparent scattering bodies. Among these, imaging techniques that use fluorescent probes include confocal microscopy, multiphoton microscopy, and light-sheet microscopy. Meanwhile, optical coherence tomography (OCT) is a well-known non-staining, non-invasive imaging technique that does not use fluorescent probes.

[0003] Although non-staining and non-invasive imaging is often desirable for observational objects such as spheroids and organoids, there have been few reports of OCT being used to image these objects. This is likely due to the low resolution of OCT imaging and the difficulty in interpreting the signals obtained by OCT imaging. Therefore, it can be said that at present, there is no established gold-standard imaging technology for non-staining 3D cell tissues.

[0004] Quantitative phase imaging (QPI) is a technique that can image the optical path length of an object non-invasively and without staining. QPI is being increasingly applied in the biological field because it can obtain physical information, such as the optical path length of an object (e.g., a cell). Images acquired by QPI can be used to generate other types of images, such as differential interference contrast images and phase-contrast microscopy images. QPI is a technique that can obtain images with relatively high information content, and is expected to be applicable to higher-content analyses than those using conventional bright-field images. Furthermore, with the recent improvement in image recognition accuracy through machine learning, high-content analyses using non-staining imaging techniques have been actively researched, and non-staining imaging of multiple scattering objects is expected to play an important role in the future. However, since the images acquired by QPI are merely two-dimensional projections of the optical path length, they cannot capture true three-dimensional structures.

[0005] Optical Diffraction Tomography (ODT), described in Patent Document 1, is also known as a technology capable of imaging the optical path length of an object of observation non-staining and non-invasively. ODT is an extension of QPI into a technology capable of 3D imaging, and is capable of realizing 3D refractive index tomography of the object of observation. Cell observation using ODT makes it possible to identify organelles such as cell nuclei and mitochondria, and also enables tracking of 3D morphological changes, and is expected to enable even higher-level analysis than QPI. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2017-219826 Summary of the Invention [Problem to be solved by the invention]

[0007] However, while conventional ODT can be applied to the observation of cells consisting of a few cells, it is difficult to apply it to the observation of multiple scattering objects such as the above-mentioned three-dimensional cell tissues, because with conventional ODT, when there is a lot of multiple scattered light generated in the object to be observed, the influence of the multiple scattered light appears significantly in the acquired image.

[0008] Light scattering is a phenomenon in which the direction of light is changed by interacting with an object. In particular, when the spatial non-uniformity of the object's refractive index increases, light interacts with the object multiple times while passing through it. Light that has interacted with the object multiple times in this way is called multiple scattered light. In contrast, light that has interacted with the object only once is called single scattered light. Multiple scattered light is known to cause increased speckle and a worsening of the single-scattering to multi-scattering ratio (SMR), making it an obstacle to measurement.

[0009] Speckle occurs when light that is temporally and spatially coherent is subject to interference from multiple scattered light, resulting in large spatial variations in intensity or phase. Speckle can be suppressed by using a light source that outputs temporally or spatially incoherent light. For example, conventional bright-field microscopes, such as phase-contrast microscopes, obtain speckle-free images by using spatially and temporally incoherent light sources, such as halogen lamps or light-emitting diodes.

[0010] The deterioration of SMR occurs when multiple scattered light becomes dominant over single scattered light, and the single scattered light is buried in the multiple scattered light. As the size of the observed object increases and the observation depth increases, the single scattered light component exponentially decreases, while in contrast, the multiple scattered light component increases. Single scattered light is easy to use for measuring the structure of an object because its scattering direction has a direct correspondence with the structure of the object. On the other hand, the relationship between multiple scattered light and the structure of the object is complex, making it difficult to extract information about the object's structure. Therefore, imaging techniques using single scattered light are known to fail when single scattered light is buried in the multiple scattered light (i.e., SMR deteriorates).

[0011] Suppression of SMR degradation can be achieved by a technique called gating, which selectively detects single-scattered light among single-scattered and multiply-scattered light. Gating suppresses multiply-scattered light, so it is possible to suppress speckle while suppressing SMR degradation. Gating is achieved using degrees of freedom such as space, time, and polarization. Confocal microscopy is an example of spatial gating. OCT is an example of both temporal and spatial gating.

[0012] Because conventional ODT does not eliminate the effects of multiple scattered light, when the object being observed generates a large amount of multiple scattered light, the speckle in the acquired image increases and the SMR deteriorates. Therefore, although conventional ODT can be applied to the observation of cells consisting of a few cells, which generate little multiple scattered light, it is difficult to apply it to the observation of multiple scattering objects such as three-dimensional cellular tissues, which generate a large amount of multiple scattered light.

[0013] The present invention has been made to solve the above problems, and aims to provide an observation device and an observation method that can observe an object by reducing the influence of multiple scattered light, even when the object is a multiple scattering object. [Means for solving the problem]

[0014] The observation device of the present invention includes: (1) an interference intensity image acquisition unit that acquires an interference intensity image of a reference position for each of a plurality of light irradiation directions from an imaging unit that captures an interference intensity image of a reference position resulting from interference between light that has been irradiated onto an observation object along each of a plurality of light irradiation directions and passed through the observation object and a reference light; (2) a first complex amplitude image generation unit that generates a complex amplitude image of the reference position for each of the plurality of light irradiation directions based on the interference intensity image of the reference position; (3) a second complex amplitude image generation unit that generates a complex amplitude image of each of the plurality of positions based on the complex amplitude image of the reference position for each of the plurality of light irradiation directions; (4) a phase conjugate calculation unit that performs a phase conjugate calculation on the complex amplitude image for each of the plurality of light irradiation directions before, during, or after processing by the second complex amplitude image generation unit, and generates a complex amplitude image for each of the plurality of light irradiation directions when the relationship between light irradiation and imaging on the observation object is reversed; and (5) The imaging system includes (6) a two-dimensional phase image generation unit that generates, for each of the plurality of positions, a complex differential interference image for each of the plurality of light irradiation directions based on the complex amplitude image for each of the plurality of light irradiation directions generated by the second complex amplitude image generation unit or the phase conjugate calculation unit, and generates a two-dimensional phase image based on the complex differential interference image for each of the plurality of light irradiation directions, and (7) a three-dimensional phase image generation unit that generates a three-dimensional phase image based on the two-dimensional phase image for each of the plurality of positions. When a phase image generated based on the complex amplitude image before calculation by the phase conjugate calculation unit is defined as a first phase image, and a phase image generated based on the complex amplitude image obtained by calculation by the phase conjugate calculation unit is defined as a second phase image, the two-dimensional phase image generation unit generates a two-dimensional phase image for positions relatively close to the imaging unit based mainly on the first phase image, and generates a two-dimensional phase image for positions relatively far from the imaging unit based mainly on the second phase image.

[0015] In one aspect of the present invention, the two-dimensional phase image generating section generates a two-dimensional phase image based on the sum of complex differential interference images in each of a plurality of light irradiation directions.

[0016] In one aspect of the present invention, the phase conjugate calculation unit divides the complex amplitude image into a plurality of partial images, each having the same number of pixels as the dimension of a matrix in the light irradiation sideband space for the object to be observed, performs a phase conjugate calculation on each of the plurality of partial images, and then combines the plurality of partial images.

[0017] In one aspect of the present invention, the two-dimensional phase image generator uses a weighting function α whose derivative with respect to a variable z, which represents the distance from the imaging unit along the light propagation path, is equal to or less than 0, to generate a two-dimensional phase image as the sum of α times the first phase image and (1-α) times the second phase image. The two-dimensional phase image generator uses, as the weighting function α, a function that is positive when the value of the variable z is equal to or less than a threshold and has a value of 0 otherwise. Alternatively, the two-dimensional phase image generator uses, as the weighting function α, a function whose value depends on the position on a plane perpendicular to the optical axis of the imaging unit.

[0018] In one aspect of the present invention, it is preferable that the two-dimensional phase image generating unit generates a complex differential interference image for each of a plurality of light irradiation directions for each of a plurality of different shear directions on the image based on the complex amplitude image for each of the plurality of light irradiation directions, and generates a two-dimensional phase image based on the complex differential interference images for each of the plurality of shear directions and the plurality of light irradiation directions.

[0019] In one aspect of the present invention, it is preferable that the observation device further comprises a refractive index distribution calculation section that determines the three-dimensional refractive index distribution of the observation object based on the three-dimensional phase image.

[0020] The observation method of the present invention includes: (1) an interference intensity image acquisition step of acquiring an interference intensity image of a reference position for each of a plurality of light irradiation directions from an imaging unit that has captured an interference intensity image of a reference position resulting from interference between light irradiated onto an observation object along each of a plurality of light irradiation directions and passing through the observation object and a reference light; (2) a first complex amplitude image generation step of generating a complex amplitude image of the reference position for each of the plurality of light irradiation directions based on the interference intensity image of the reference position; (3) a second complex amplitude image generation step of generating a complex amplitude image of each of the plurality of positions for each of the plurality of light irradiation directions based on the complex amplitude image of the reference position; (4) a phase conjugate calculation step of performing a phase conjugate calculation on the complex amplitude image for each of the plurality of light irradiation directions before, during, or after processing in the second complex amplitude image generation step, to generate a complex amplitude image for each of the plurality of light irradiation directions when the relationship between light irradiation and imaging on the observation object is reversed; and (5) and (6) a three-dimensional phase image generating step of generating a three-dimensional phase image based on the two-dimensional phase images for each of the plurality of positions, where, in the two-dimensional phase image generating step, a phase image generated based on the complex amplitude image before the phase conjugate calculation step is defined as a first phase image, and a phase image generated based on the complex amplitude image obtained by the phase conjugate calculation step is defined as a second phase image. For positions relatively close to the imaging unit among the plurality of positions, a two-dimensional phase image is generated mainly based on the first phase image, and for positions relatively far from the imaging unit, a two-dimensional phase image is generated mainly based on the second phase image.

[0021] In one aspect of the present invention, the two-dimensional phase image generating step generates the two-dimensional phase image based on a sum of complex differential interference images in each of a plurality of light irradiation directions.

[0022] In one aspect of the present invention, in the phase conjugate calculation step, the complex amplitude image is divided into a plurality of partial images, each having the same number of pixels as the dimension of a matrix in the light irradiation sideband space for the object to be observed, and a phase conjugate calculation is performed on each of these plurality of partial images, and then the plurality of partial images are combined.

[0023] In one aspect of the present invention, the two-dimensional phase image generating step uses a weighting function α whose differential coefficient with respect to a variable z representing the distance from the imaging unit along the light propagation path is equal to or less than 0, and generates a two-dimensional phase image by multiplying the first phase image by α and the second phase image by (1-α). The two-dimensional phase image generating step uses, as the weighting function α, a function whose value is positive when the value of the variable z is equal to or less than a threshold and whose value is 0 otherwise. Alternatively, the two-dimensional phase image generating step uses, as the weighting function α, a function whose value depends on the position on a plane perpendicular to the optical axis of the imaging unit.

[0024] In one aspect of the present invention, it is preferable that in the two-dimensional phase image generating step, a complex differential interference contrast image for each of a plurality of light irradiation directions is generated for each of a plurality of different shear directions on the image based on a complex amplitude image for each of a plurality of light irradiation directions, and a two-dimensional phase image is generated based on the complex differential interference contrast images for each of the plurality of shear directions and the plurality of light irradiation directions.

[0025] In one aspect of the present invention, it is preferable that the observation method further comprises a refractive index distribution calculation step of determining a three-dimensional refractive index distribution of the observation object based on the three-dimensional phase image.

[0026] The program of the present invention causes a computer to execute each step of the observation method of the present invention. The recording medium of the present invention is a computer-readable medium having the program of the present invention recorded thereon. [Effects of the Invention]

[0027] According to the present invention, even when the object to be observed is a multiple scattering medium, the object to be observed can be observed with the influence of multiple scattered light reduced. [Brief explanation of the drawings]

[0028] [Figure 1] FIG. 1 is a diagram showing the configuration of an observation device 1A. [Figure 2] FIG. 2 is a diagram showing the configuration of the observation device 1B. [Figure 3] FIG. 3 is a diagram showing the configuration of the observation device 1C. [Figure 4] FIG. 4 is a flowchart of the observation method. [Figure 5] 5(a) to 5(c) are diagrams showing an example of scanning the light irradiation direction onto the observation object S in the interference intensity image acquisition step S61. [Figure 6] FIG. 6 is a diagram illustrating the kernel function g. [Figure 7] 7(a) and (b) are diagrams showing an example of scanning the light irradiation direction onto the observation object S in the interference intensity image acquisition step S61. [Figure 8] 8(a) to 8(c) are diagrams showing an example of scanning the light irradiation direction onto the observation object S in the interference intensity image acquisition step S61. [Figure 9] FIG. 9 is a flowchart of the two-dimensional phase image generating step S65. [Figure 10] FIG. 10 is a diagram illustrating the kernel function. [Figure 11] FIG. 11 is a diagram illustrating the order of the processes and images in the second complex amplitude image generating step S63 and the two-dimensional phase image generating step S65. [Figure 12] FIG. 12 is a diagram for explaining the order of the processes and images in the second complex amplitude image generating step S63, the phase conjugate calculation step S64, and the two-dimensional phase image generating step S65. [Figure 13] FIG. 13 is a diagram for explaining the order of the processes and images in the second complex amplitude image generating step S63, the phase conjugate calculation step S64, and the two-dimensional phase image generating step S65. [Figure 14]FIG. 14 is a diagram for explaining the order of the processes and images in the second complex amplitude image generating step S63, the phase conjugate calculation step S64, and the two-dimensional phase image generating step S65. [Figure 15] FIG. 15 is a diagram illustrating the order of the processes and images in the three-dimensional phase image generating step S66 and the refractive index distribution calculating step S67. [Figure 16] FIG. 16 is a diagram for explaining an outline of the phase conjugate calculation, showing input light and output light when an interference intensity image is captured by the imaging section. [Figure 17] FIG. 17 is a diagram for explaining an outline of the phase conjugate calculation, showing input light and output light when the relationship between light irradiation and imaging is reversed. [Figure 18] FIG. 18 is a diagram for explaining image division, phase conjugate calculation, and image combination in the phase conjugate calculation step S64. [Figure 19] FIG. 19 is a diagram for explaining the layout during simulation A. [Figure 20] Figure 20(a) shows a phase image obtained by performing a phase conjugate operation when the dimension of the matrix in the light irradiation side wavenumber space is 108 × 108. Figure 20(b) shows a phase image obtained by performing a phase conjugate operation when the dimension of the matrix in the light irradiation side wavenumber space is 54 × 54. [Figure 21] FIG. 21 is a diagram for explaining the layout during simulation B. In FIG. [Figure 22] Figure 22(a) shows a phase image obtained by performing a phase conjugate operation when the dimension of the matrix in the light irradiation side wavenumber space is 108 × 108. Figure 22(b) shows a phase image obtained by performing a phase conjugate operation when the dimension of the matrix in the light irradiation side wavenumber space is 54 × 54. [Figure 23]Fig. 23(a) is a phase image obtained by performing a phase conjugate operation when the imaging unit is focused on the object to be observed and the dimension of the matrix in the light irradiation side wavenumber space is set to 108 × 108. Fig. 23(b) is a phase image obtained by performing a phase conjugate operation when the imaging unit is focused on the object to be observed and the dimension of the matrix in the light irradiation side wavenumber space is set to 36 × 36. [Figure 24] Fig. 24(a) is a phase image obtained by performing a phase conjugate operation when the imaging unit is not focused on the object to be observed and the dimension of the matrix in the light irradiation side wavenumber space is set to 108 × 108. Fig. 24(b) is a phase image obtained by performing a phase conjugate operation when the imaging unit is not focused on the object to be observed and the dimension of the matrix in the light irradiation side wavenumber space is set to 36 × 36. [Figure 25] Fig. 25(a) is a differential phase image obtained by performing a phase conjugate operation when the dimension of the matrix in the light irradiation side wavenumber space is 108 × 108. Fig. 25(b) is a differential phase image obtained by performing a phase conjugate operation when the dimension of the matrix in the light irradiation side wavenumber space is 36 × 36. [Figure 26] FIG. 26 is a diagram for explaining the layout during simulation D. [Figure 27] Figure 27(a) is a phase differential image of the exact solution, Figure 27(b) is a phase differential image obtained without performing phase conjugate calculation, and Figure 27(c) is a phase differential image obtained with performing phase conjugate calculation. [Figure 28] FIG. 28 is an interference intensity image (at normal irradiation) acquired in interference intensity image acquisition step S61. [Figure 29] FIG. 29 is a complex amplitude image (real part, z=0) generated based on the interference intensity image (FIG. 28) in the first complex amplitude image generating step S62. [Figure 30] FIG. 30 shows a complex amplitude image (real part, z=zn) generated based on the complex amplitude image (FIG. 29) in second complex amplitude image generating step S63. [Figure 31]FIG. 31 shows a plurality of partial images (real part, z=zn) obtained by dividing the complex amplitude image (FIG. 30) in the phase conjugate calculation step S64. [Figure 32] FIG. 32 shows partial images (real parts, z=zn) obtained by performing phase conjugate calculation on each of the plurality of partial images (FIG. 31) in the phase conjugate calculation step S64. [Figure 33] FIG. 33 shows a complex amplitude image (real part, z=zn) obtained by combining a plurality of partial images (FIG. 32) after the phase conjugate calculation in the phase conjugate calculation step S64. [Figure 34] FIG. 34 shows a complex differential interference image (imaginary parts for both the x-direction shear and the y-direction shear) generated based on the complex amplitude image (FIG. 33) in step S21 of the two-dimensional phase image generation step S65. [Figure 35] FIG. 35 shows differential phase images (x-direction shear and y-direction shear) generated based on the complex differential interference image (FIG. 34) in step S22 of the two-dimensional phase image generating step S65. [Figure 36] FIG. 36 shows differential phase images (x-direction shear and y-direction shear) generated in step S22 of the two-dimensional phase image generation step S65 based on the complex amplitude image (FIG. 30) without undergoing the phase conjugate calculation step S64. [Figure 37] Figure 37 shows phase differential images (x-direction shear and y-direction shear) obtained by combining the phase differential image (Figure 35) obtained when the phase conjugate calculation step S64 was performed and the phase differential image (Figure 36) obtained when the phase conjugate calculation step S64 was not performed. [Figure 38] FIG. 38 shows the refractive index distribution generated based on the phase differential image in the refractive index distribution calculation step S67. [Figure 39] Figure 39 shows differential phase images at the position z=10.4 μm. Figure 39(a) shows a differential phase image obtained without performing phase conjugate calculation. Figure 39(b) shows a differential phase image obtained with performing phase conjugate calculation. [Figure 40]Figure 40 shows differential phase images at the position z=32.4 μm. Figure 40(a) shows a differential phase image obtained without performing phase conjugate calculation. Figure 40(b) shows a differential phase image obtained with performing phase conjugate calculation. [Figure 41] Figure 41 shows differential phase images at the position z=54.0 μm. Figure 41(a) shows a differential phase image obtained without performing phase conjugate calculation. Figure 41(b) shows a differential phase image obtained with performing phase conjugate calculation. [Figure 42] Figure 42 shows the refractive index distribution at the position z=10.4 μm. Figure 42(a) shows the refractive index distribution obtained without performing phase conjugate calculation. Figure 42(b) shows the refractive index distribution obtained by reconstructing the image by combining the differential phase images obtained with and without performing phase conjugate calculation. [Figure 43] Figure 43 shows the refractive index distribution at the position z=54.0 μm. Figure 43(a) shows the refractive index distribution obtained without performing phase conjugate calculation. Figure 43(b) shows the refractive index distribution obtained by reconstructing the image by combining the differential phase images obtained with and without performing phase conjugate calculation. [Figure 44] Fig. 44(a) shows differential phase images at each z position obtained when no phase conjugate calculation was performed. Fig. 44(b) shows differential phase images at each z position obtained when the processing of phase conjugate calculation step S64 was performed after the processing of second complex amplitude image generation step S63 according to the procedure shown in Fig. 12. Fig. 44(c) shows differential phase images at each z position obtained when the processing of phase conjugate calculation step S64 was performed before the processing of second complex amplitude image generation step S63 according to the procedure shown in Fig. 13. DETAILED DESCRIPTION OF THE INVENTION

[0029] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are designated by the same reference numerals, and duplicate explanations will be omitted. The present invention is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

[0030] 1 is a diagram showing the configuration of an observation device 1 A. This observation device 1 A includes a light source 11, a lens 12, a lens 21, a mirror 22, a lens 23, a condenser lens 24, an objective lens 25, a beam splitter 41, a lens 42, an imaging unit 43, an analyzing unit 60, and the like.

[0031] The light source 11 outputs spatially and temporally coherent light, and is preferably a laser light source. The lens 12 is optically connected to the light source 11 and focuses the light output from the light source 11 onto a light input end 13 of an optical fiber 14, causing the light to be incident on the light input end 13. The optical fiber 14 guides the light incident on the light input end 13 by the lens 12 to a fiber coupler 15. The fiber coupler 15 couples light between the optical fiber 14 and optical fibers 16 and 17. The light guided by the optical fiber 14 branches into two, one branched light being guided by the optical fiber 16 and the other branched light being guided by the optical fiber 17. The light guided by the optical fiber 16 is emitted as diverging light from a light output end 18. The light guided by the optical fiber 17 is emitted as diverging light from a light output end 19.

[0032] The lens 21 is optically connected to the light emitting end 18 and collimates the light output from the light emitting end 18 as divergent light. The mirror 22 is optically connected to the lens 21 and reflects the light reaching it from the lens 21 to the lens 23. The orientation of the reflective surface of the mirror 22 is variable. The lens 23 is optically connected to the mirror 22. The condenser lens 24 is optically connected to the lens 23. The lens 23 and the condenser lens 24 preferably constitute a 4f optical system. The lens 23 and the condenser lens 24 irradiate the observation object S with light from a light irradiation direction that corresponds to the orientation of the reflective surface of the mirror 22. The objective lens 25 is optically connected to the condenser lens 24. The observation object S is disposed between the objective lens 25 and the condenser lens 24. The objective lens 25 receives the light (object light) output from the condenser lens 24 and passing through the observation object S, and outputs the light to the beam splitter 41.

[0033] The beam splitter 41 is optically connected to the objective lens 25 and also to the light output end 19. The beam splitter 41 combines light (object light) output from the objective lens 25 and reaching it with light (reference light) output from the light output end 19 and reaching it, and outputs both lights to the lens 42. The lens 42 is optically connected to the beam splitter 41, and collimates the object light and reference light arriving from the beam splitter 41, and outputs them to the imaging unit 43. The imaging unit 43 is optically connected to the lens 42, and captures an interference fringe image (interference intensity image) resulting from interference between the object light and reference light arriving from the lens 42. The incident direction of the reference light on the imaging surface of the imaging unit 43 is tilted relative to the incident direction of the object light. The position where the object light and reference light are combined by the beam splitter 41 may be after the imaging lens, but considering the effects of aberration, it is preferable to position it between the objective lens 25 and lens 42 as shown in the figure.

[0034] The analysis unit 60 is electrically connected to the imaging unit 43 and receives as input an interference intensity image captured by the imaging unit 43. The analysis unit 60 processes the received interference intensity image to calculate a three-dimensional refractive index distribution of the observation object S. The analysis unit 60 may be a computer. The analysis unit 60 includes an interference intensity image acquisition unit 61, a first complex amplitude image generation unit 62, a second complex amplitude image generation unit 63, a phase conjugate calculation unit 64, a two-dimensional phase image generation unit 65, a three-dimensional phase image generation unit 66, a refractive index distribution calculation unit 67, a display unit 68, and a storage unit 69.

[0035] The interference intensity image acquisition unit 61 irradiates the observation object S with light along each of a plurality of light irradiation directions by changing the orientation of the reflecting surface of the mirror 22. The interference intensity image acquisition unit 61 also acquires an interference intensity image at a reference position for each of the plurality of light irradiation directions from the imaging unit 43. The interference intensity image acquisition unit 61 includes a CPU, and has an output port that outputs a control signal for changing the orientation of the reflecting surface of the mirror 22, and also has an input port that inputs the interference intensity image from the imaging unit 43. There is no need to move the objective lens 25 in the optical axis direction. The reference position is an image plane position that is conjugate with the imaging plane of the imaging unit 43.

[0036] The first complex amplitude image generator 62, the second complex amplitude image generator 63, the phase conjugate calculator 64, the two-dimensional phase image generator 65, the three-dimensional phase image generator 66, and the refractive index distribution calculator 67 perform processing based on the interference intensity image and include processing devices such as a CPU, GPU, DSP, or FPGA. The display 68 displays the image to be processed, the image being processed, and the image after processing, and includes, for example, an LCD display. The memory 69 stores various image data and includes a hard disk drive, flash memory, RAM, ROM, etc. The first complex amplitude image generator 62, the second complex amplitude image generator 63, the phase conjugate calculator 64, the two-dimensional phase image generator 65, the three-dimensional phase image generator 66, the refractive index distribution calculator 67, and the memory 69 may be configured using cloud computing.

[0037] The storage unit 69 also stores programs for causing the interference intensity image acquisition unit 61, the first complex amplitude image generation unit 62, the second complex amplitude image generation unit 63, the phase conjugate calculation unit 64, the two-dimensional phase image generation unit 65, the three-dimensional phase image generation unit 66, and the refractive index distribution calculation unit 67 to execute each process. These programs may be stored in the storage unit 69 at the time of manufacture or shipment of the observation device 1A, or may be acquired via a communication line after shipment and stored in the storage unit 69, or may be stored in the storage unit 69 after being recorded on a computer-readable recording medium 2. The recording medium 2 may be any medium such as a flexible disk, CD-ROM, DVD-ROM, BD-ROM, or USB memory.

[0038] The details of the processing of each of the interference intensity image acquisition unit 61, first complex amplitude image generation unit 62, second complex amplitude image generation unit 63, phase conjugate calculation unit 64, two-dimensional phase image generation unit 65, three-dimensional phase image generation unit 66 and refractive index distribution calculation unit 67 will be described later.

[0039] Fig. 2 is a diagram showing the configuration of observation apparatus 1B. Observation apparatus 1B shown in Fig. 2 includes a lens 31, a mirror 32, a lens 34, and the like in addition to the configuration of observation apparatus 1A shown in Fig. 1.

[0040] The lens 31 is optically connected to the light emitting end 19 and collimates the light (reference light) output as divergent light from the light emitting end 19. The mirror 32 is optically connected to the lens 31 and reflects the light arriving from the lens 31 to the lens 34. The lens 34 is optically connected to the mirror 32 and outputs the light arriving from the mirror 32 to the beam splitter 41. The light output from the lens 34 is temporarily condensed before reaching the beam splitter 41 and then input to the beam splitter 41 as divergent light. The beam splitter 41 combines the light (object light) output from the objective lens 25 and arriving thereat and the light (reference light) output from the lens 34 and arriving thereat, and outputs both light beams coaxially to the lens 42. The imaging unit 43 captures an interference fringe image (interference intensity image) resulting from interference between the object light and the reference light arriving from the lens 42. The direction of incidence of the reference light on the imaging surface of the imaging unit 43 is parallel to the direction of incidence of the object light.

[0041] The driver 33 moves the mirror 32 in a direction perpendicular to the reflecting surface of the mirror 32. The driver 33 is, for example, a piezoelectric actuator. This movement of the mirror 32 changes the difference in optical path length (phase difference) between the object light and the reference light from the optical branching in the fiber coupler 15 to the combination in the beam splitter 41. If this optical path length difference differs, the interference intensity image captured by the imaging unit 43 also differs.

[0042] The observation device is not limited to the configuration examples of Figures 1 and 2, and various modifications are possible. In the configurations of observation device 1A (Figure 1) and observation device 1B (Figure 2), the object light is light that has passed through the observation object S, but light reflected by the observation object S may also be the object light as in the configuration of observation device 1C (Figure 3) described below.

[0043] 3 is a diagram showing the configuration of observation device 1C. Observation device 1C includes a light source 11, a lens 12, a lens 21, a mirror 22, a lens 23, an objective lens 25, a beam splitter 41, a lens 42, an imaging unit 43, and an analyzing unit 60. The following mainly describes the differences from observation device 1A (FIG. 1).

[0044] The lens 21 is optically connected to the light emitting end 18 of the optical fiber 16 and collimates the light output from the light emitting end 18 as divergent light. The mirror 22 is optically connected to the lens 21 and reflects the light reaching it from the lens 21 to the lens 23. The orientation of the reflective surface of the mirror 22 is variable. The lens 23 is optically connected to the mirror 22. The objective lens 25 is optically connected to the lens 23. A beam splitter 41 is disposed between the lens 23 and the objective lens 25. The lens 23 and the objective lens 25 preferably constitute a 4f optical system. The lens 23 and the objective lens 25 irradiate light onto the observation object S from a light irradiation direction according to the orientation of the reflective surface of the mirror 22. The objective lens 25 receives light reflected by the observation object S (object light) and outputs the light to the beam splitter 41.

[0045] The beam splitter 41 is optically connected to the objective lens 25 and also to the light emitting end 19 of the optical fiber 17. The beam splitter 41 combines light (object light) output from the objective lens 25 and reaching it with light (reference light) output from the light emitting end 19 and reaching it, and outputs both lights to the lens 42. The lens 42 is optically connected to the beam splitter 41, and collimates the object light and reference light arriving from the beam splitter 41, and outputs them to the imaging unit 43. The imaging unit 43 is optically connected to the lens 42, and captures an interference fringe image (interference intensity image) resulting from interference between the object light and reference light arriving from the lens 42. The incident direction of the reference light on the imaging surface of the imaging unit 43 is tilted relative to the incident direction of the object light. The position where the object light and reference light are combined by the beam splitter 41 may be after the imaging lens, but considering the effects of aberration, it is preferable to position it between the objective lens 25 and lens 42 as shown in the figure.

[0046] In the configuration of the observation device 1C (FIG. 3), a mechanism for changing the optical path length of the reference light (lens 31, mirror 32, drive unit 33, and lens 34 in FIG. 2) may be provided, similarly to the observation device 1B (FIG. 2), to change the difference in the optical path lengths (phase difference) of the object light and the reference light from the light branching in the fiber coupler 15 to the combination in the beam splitter 41. In this case, the incident direction of the reference light may be parallel to the incident direction of the object light on the imaging surface of the imaging unit 43.

[0047] 4 is a flowchart of the observation method. This observation method can be performed using any of the observation apparatuses 1A (FIG. 1), 1B (FIG. 2), and 1C (FIG. 3). This observation method includes an interference intensity image acquisition step S61, a first complex amplitude image generation step S62, a second complex amplitude image generation step S63, a phase conjugate calculation step S64, a two-dimensional phase image generation step S65, a three-dimensional phase image generation step S66, and a refractive index distribution calculation step S67.

[0048] The processing of interference intensity image acquisition step S61 is performed by the interference intensity image acquisition unit 61. The processing of first complex amplitude image generation step S62 is performed by the first complex amplitude image generation unit 62. The processing of second complex amplitude image generation step S63 is performed by the second complex amplitude image generation unit 63. The processing of phase conjugate calculation step S64 is performed by the phase conjugate calculation unit 64. The processing of two-dimensional phase image generation step S65 is performed by the two-dimensional phase image generation unit 65. The processing of three-dimensional phase image generation step S66 is performed by the three-dimensional phase image generation unit 66. The processing of refractive index distribution calculation step S67 is performed by the refractive index distribution calculation unit 67.

[0049] In the interference intensity image acquisition step S61, the interference intensity image acquisition unit 61 irradiates the observation object S with light along each of a plurality of light irradiation directions by changing the orientation of the reflecting surface of the mirror 22. Then, the interference intensity image acquisition unit 61 acquires, from the imaging unit 43, an interference intensity image at a reference position for each of the plurality of light irradiation directions.

[0050] For convenience of explanation, an xyz Cartesian coordinate system is shown in each of FIGS. 1, 2, and 3. The z axis is parallel to the optical axis of the objective lens 25. The reference position is the image plane position that is conjugate with the imaging plane of the imaging unit 43. This position is defined as z=0. The direction of light irradiation onto the observation object S is determined by the wave vector (k x ,k y ,k z ) k x and k y It can be expressed as:

[0051] 5(a) to 5(c) are diagrams showing an example of scanning the light irradiation direction onto the observation object S in the interference intensity image acquisition step S61. In these diagrams, the horizontal axis represents k x The vertical axis is k y And k x k y The position of each circle on the plane represents the light irradiation direction. The light irradiation direction is scanned by k x k y It may be arranged in a rectangular grid on a plane, or k x k y The sensors may be arranged on the circumference of a plurality of concentric circles in a plane, or may be arranged in a k x k y The scanning direction of the light may be a spirally arranged plane. In either case, scanning in the light irradiation direction is possible as long as the NA of the condenser lens 24 in the configurations of Figures 1 and 2 or the objective lens 25 in the configuration of Figure 3 allows. Either raster scanning or random scanning is possible. In the case of raster scanning, a return scan may or may not be performed.

[0052] In a first complex amplitude image generating step S62, the first complex amplitude image generating unit 62 generates a complex amplitude image of the reference position for each of the plurality of light irradiation directions based on the interference intensity image of the reference position acquired by the interference intensity image acquiring unit 61. In the case of the observation device 1A (FIG. 1) and the observation device 1C (FIG. 3), the first complex amplitude image generating unit 62 can generate a complex amplitude image based on one interference intensity image by the Fourier fringe analysis method. In the case of the observation device 1B (FIG. 2), the first complex amplitude image generating unit 62 can generate a complex amplitude image based on three or more interference intensity images in which the optical path length difference (phase difference) between the object light and the reference light is different from one another by the phase shift method.

[0053] In the second complex amplitude image generating step S63, the second complex amplitude image generating unit 63 generates a complex amplitude image for each of the plurality of z-direction positions for each of the plurality of light irradiation directions based on the complex amplitude image for the reference position (z=0) generated by the first complex amplitude image generating unit 62. The two-dimensional Fourier transform of the complex amplitude image u(x, y, 0) for the reference position is expressed as U(k x ,k y , 0), the complex amplitude image u(x, y, d) at the position z=d and the two-dimensional Fourier transform U(k x ,k y , d) is expressed by the following equation for free propagation, where i is the imaginary unit and k0 is the wave number of light in the object being observed.

[0054]

number

[0055]

number

[0056] The phase conjugate calculation step S64 is performed after the processing of the second complex amplitude image generating step S63. The phase conjugate calculation step S64 may be performed before the processing of the second complex amplitude image generating step S63 (described later). Furthermore, when the second complex amplitude image generating step S63 generates a complex amplitude image at a certain z position from a complex amplitude image at a reference position through multiple stages, the phase conjugate calculation step S64 may be performed between a certain stage and the next stage among the multiple stages (described later). In the phase conjugate calculation step S64, the phase conjugate calculation unit 64 performs a phase conjugate calculation on the complex amplitude images for each of the multiple light irradiation directions to generate complex amplitude images for each of the multiple light irradiation directions when the relationship between light irradiation and imaging of the observation object is reversed.

[0057] The phase conjugate calculation is an operation on a complex amplitude image based on the phase conjugate method, which calculates a transmission matrix that represents the relationship between light irradiation and light output in an object, and includes the inverse of the matrix and coordinate transformation. The phase conjugate method is also called phase conjugation, time reversal method, digital phase conjugation, digital phase conjugate method, etc. Details will be described later.

[0058] In a two-dimensional phase image generating step S65, the two-dimensional phase image generating unit 65 generates a two-dimensional phase image for each of the multiple positions based on the complex amplitude images for each of the multiple light irradiation directions generated by the second complex amplitude image generating unit 63 or the phase conjugate calculating unit 64. The two-dimensional phase image generated here corresponds to a phase image centered on the focused z-direction position.

[0059] In the two-dimensional phase image generation step S65, when the phase image generated based on the complex amplitude image before the processing of the phase conjugate calculation step S64 is defined as the first phase image, and the phase image generated based on the complex amplitude image obtained by the processing of the phase conjugate calculation step S64 is defined as the second phase image, among the multiple positions, for positions relatively close to the imaging unit, two-dimensional phase images are generated mainly based on the first phase image, and for positions relatively far from the imaging unit, two-dimensional phase images are generated mainly based on the second phase image.

[0060] Note that the processes subsequent to the phase conjugate calculation unit 64 may be performed after all complex amplitude images at multiple positions for each of multiple light irradiation directions have been generated in the second complex amplitude image generation step S63. Alternatively, a complex amplitude image at one z-direction position may be generated for each of multiple light irradiation directions in the second complex amplitude image generation step S63, and a two-dimensional phase image at that position may be generated in the two-dimensional phase image generation step S65. This unit process may be repeated while scanning the z-direction position. The latter case is preferable because it reduces the amount of image data to be stored in the storage unit 69.

[0061] In a three-dimensional phase image generating step S66, the three-dimensional phase image generating unit 66 generates a three-dimensional phase image based on the two-dimensional phase images at each of the multiple positions generated by the two-dimensional phase image generating unit 65. The three-dimensional phase image generated here is an image in which the positions x and y in the two-dimensional phase image and the position z of the two-dimensional phase image are variables.

[0062] In the refractive index distribution calculation step S67, the refractive index distribution calculation unit 67 calculates the three-dimensional refractive index distribution of the object to be observed by deconvolution based on the three-dimensional phase image generated by the three-dimensional phase image generation unit 66. The refractive index distribution of the object to be observed is n(x, y, z), the electric susceptibility distribution is f(x, y, z), and the refractive index of the background medium is n mThen, there is a relationship between the two as shown in the following equation (3). The three-dimensional phase image Φ(x, y, z) generated by the three-dimensional phase image generating unit 66 is expressed by the convolution of the kernel function g(x, y, z) and the electric susceptibility distribution f(x, y, z), as shown in the following equation (4). Therefore, the three-dimensional refractive index distribution n(x, y, z) of the observation object can be obtained by deconvolution based on the three-dimensional phase image Φ(x, y, z).

[0063]

number

[0064]

number

[0065] The kernel function g is based on the Green's function corresponding to the solution of the wave equation. Figure 6 is a diagram explaining the kernel function g. In this diagram, the central position where the value of the kernel function g is the largest is the origin, the vertical direction is the z-axis, and the horizontal direction is the direction perpendicular to the z-axis.

[0066] The processes of the first complex amplitude image generating step S62, the second complex amplitude image generating step S63, the phase conjugate calculation step S64, the two-dimensional phase image generating step S65, the three-dimensional phase image generating step S66, and the refractive index distribution calculation step S67 may be performed each time an interference intensity image for each of a predetermined number of light irradiation directions is acquired in the interference intensity image acquiring step S61 (FIG. 7), or each time an interference intensity image for one light irradiation direction is acquired in the interference intensity image acquiring step S61 (FIG. 8).

[0067] 7 and 8 are diagrams showing examples of scanning of the light irradiation direction onto the observation object S in the interference intensity image acquisition step S61. In these diagrams, the horizontal axis is k x The vertical axis is k y And k x k yThe position of each circle on the plane represents the light irradiation direction. In the examples of scanning the light irradiation direction shown in these figures, the light irradiation direction is changed sequentially so that the light irradiation direction when acquiring the (N+n)th interference intensity image coincides with the light irradiation direction when acquiring the nth interference intensity image. n is a positive integer, and N is an integer greater than or equal to 2.

[0068] In the example shown in Fig. 7, when the first to Nth interference intensity images are acquired in interference intensity image acquisition step S61, the processes of steps S62 to S67 are performed based on these first to Nth interference intensity images (Fig. 7(a)). Next, when the (N+1)th to 2Nth interference intensity images are acquired in interference intensity image acquisition step S61, the processes of steps S62 to S67 are performed based on these (N+1)th to 2Nth interference intensity images (Fig. 7(b)). Next, when the (2N+1)th to 3Nth interference intensity images are acquired in interference intensity image acquisition step S61, the processes of steps S62 to S67 are performed based on these (2N+1)th to 3Nth interference intensity images. The same applies thereafter.

[0069] 8, when the first to Nth interference intensity images are acquired in interference intensity image acquisition step S61, the processes of steps S62 to S67 are performed based on these first to Nth interference intensity images (FIG. 8(a)). Next, when the (N+1)th interference intensity image is acquired in interference intensity image acquisition step S61, the processes of steps S62 to S67 are performed based on the N most recent interference intensity images (the second to the (N+1)th interference intensity images) including this (N+1)th interference intensity image (FIG. 8(b)). Next, when the (N+2)th interference intensity image is acquired in interference intensity image acquisition step S61, the processes of steps S62 to S67 are performed based on the N most recent interference intensity images (the third to the (N+2)th interference intensity images) including this (N+2)th interference intensity image (FIG. 8(c)). Similarly, when the (N+n)th interference intensity image is acquired in interference intensity image acquisition step S61, the processes of steps S62 to S67 are performed based on the most recent N interference intensity images (the (1+n)th to (N+n)th interference intensity images) including this (N+n)th interference intensity image.

[0070] Compared with the example shown in FIG. 7, in the example shown in FIG. 8, each time an interference intensity image for one light irradiation direction is acquired in interference intensity image acquisition step S61, the processes of steps S62 to S67 are performed based on the most recent interference intensity images including that interference intensity image, and therefore the number of images obtained per unit time by the processes of steps S62 to S67 is large.

[0071] Next, the two-dimensional phase image generating step S65 will be described in detail. In the two-dimensional phase image generating step S65, the two-dimensional phase image generating unit 65 generates a two-dimensional phase image for each of the multiple positions based on complex amplitude images for each of the multiple light irradiation directions.

[0072] 9 is a flowchart of the two-dimensional phase image generating step S65. In the two-dimensional phase image generating step S65, for each of the multiple positions, in step S21, a complex differential interference image for each of the multiple light irradiation directions is generated based on the complex amplitude image for each of the multiple light irradiation directions, in step S22, a differential phase image is generated based on the sum of the complex differential interference images for each of the multiple light irradiation directions, and in step S23, a two-dimensional phase image is generated based on the differential phase image.

[0073] If the complex amplitude image at the position z=d is u(x, y, d), the complex differential interference image q(x, y, d) generated in step S21 is expressed by the following equation (5). At least one of δx and δy is non-zero. If δx ≠ 0 and δy = 0, a complex differential interference image q with the x direction as the shear direction is obtained. If δx = 0 and δy ≠ 0, a complex differential interference image q with the y direction as the shear direction is obtained. If δx ≠ 0 and δy ≠ 0, a complex differential interference image q with a shear direction different from both the x and y directions is obtained. Note that the complex differential interference image q(x, y, d) may be calculated using equation (5) after converting the complex amplitude image u(x, y, d) as shown in equation (6) below.

[0074]

number

[0075]

number

[0076] The sum of the complex differential interference images q for each of the multiple light irradiation directions is q sum (x, y, d), the phase differential image φ(x, y, z) generated in step S22 is q sum The phase of (x, y, d) is expressed by the following equation (7): In step S23, a two-dimensional phase image can be generated by integrating or deconvolving this differential phase image φ(x, y, z).

[0077]

number

[0078] In step S21, a complex differential interference image may be generated for each of a plurality of different shear directions on the complex amplitude image. In this case, the two-dimensional phase image generating step S65 generates, for each of a plurality of positions, a complex differential interference image for each of a plurality of different shear directions on the image based on the complex amplitude images for each of the plurality of light irradiation directions in step S21, generates a differential phase image for each of the plurality of shear directions based on the sum of the complex differential interference images for each of the plurality of light irradiation directions in step S22, and generates a two-dimensional phase image based on the differential phase images for each of the plurality of shear directions in step S23.

[0079] The differential phase image generated in step S22 based on the sum of the complex differential interference images for each of the multiple light irradiation directions has a reduced effect of multiple scattered light. Finally, the three-dimensional refractive index distribution finally obtained in step S67 of refractive index distribution calculation also has a reduced effect of multiple scattered light, suppressing speckle. Furthermore, when complex differential interference images are generated for each of multiple different shear directions on the complex amplitude image in step S21, it is possible to suppress the appearance of line-shaped noise in the two-dimensional phase image obtained in step S23.

[0080] Here, the case where a two-dimensional phase image is generated by integrating or deconvolving the phase differential image in step S23 has been described. However, the phase differential image can also be treated as a two-dimensional phase image. In this case, without performing step S23, the three-dimensional refractive index distribution of the object to be observed can be obtained from the phase differential image (two-dimensional phase image) generated in step S22 by using a kernel (FIG. 10) including the kernel used in the deconvolution in step S23 in the deconvolution in refractive index distribution calculation step S67. The kernel shown in FIG. 10 is obtained by convolution integration of the kernel shown in FIG. 6 and the kernel used in the deconvolution in step S23.

[0081] 11 is a diagram illustrating the order of processing and images in the second complex amplitude image generating step S63 and the two-dimensional phase image generating step S65. This diagram illustrates a mode in which the processing in the phase conjugate calculation step S64 is not performed. In this mode, in the second complex amplitude image generating step S63, for each of the multiple light irradiation directions, complex amplitude images for each of multiple z-direction positions (z=z1, z2, z3 in this diagram) are generated based on the complex amplitude image for the reference position (z=0) generated in the first complex amplitude image generating step S62 using the free propagation equations (1) and (2) above. Then, in the two-dimensional phase image generating step S65, for each of the multiple positions, a complex differential interference image is generated based on the complex amplitude images for each of the multiple light irradiation directions generated in the second complex amplitude image generating step S63, and further a differential phase image is generated.

[0082] 12 to 14 are diagrams illustrating the order and images of the processes in the second complex amplitude image generating step S63, the phase conjugate calculation step S64, and the two-dimensional phase image generating step S65. These diagrams show how the process in the phase conjugate calculation step S64 is performed before, during, or after the process in the second complex amplitude image generating step S63.

[0083] The first mode shown in Fig. 12 corresponds to the flowchart of Fig. 4. In this first mode, phase conjugate calculation step S64 is performed after the processing of second complex amplitude image generation step S63. In second complex amplitude image generation step S63, for each of a plurality of light irradiation directions, complex amplitude images are generated at each of a plurality of z-direction positions (z = z1, z2, z3 in this figure) based on the complex amplitude image at the reference position (z = 0) generated in first complex amplitude image generation step S62 using the free propagation equations (1) and (2) above.

[0084] In the first mode, subsequently, in a phase conjugate calculation step S64, a phase conjugate calculation is performed on the complex amplitude images in each of the plurality of light irradiation directions for each of the plurality of positions, thereby generating complex amplitude images in each of the plurality of light irradiation directions when the relationship between light irradiation and imaging of the observation object is reversed. Then, in a two-dimensional phase image generation step S65, a complex differential interference image is generated for each of the plurality of positions based on the complex amplitude images in each of the plurality of light irradiation directions generated in the phase conjugate calculation step S64, and further a phase differential image is generated.

[0085] 13, a phase conjugate calculation step S64 is performed before the processing of the second complex amplitude image generating step S63. In the phase conjugate calculation step S64, for each of the plurality of light irradiation directions, a phase conjugate calculation is performed on the complex amplitude image of the reference position (z=0) generated in the first complex amplitude image generating step S62, and a complex amplitude image for each of the plurality of light irradiation directions when the relationship between light irradiation and imaging of the observation object is reversed is generated.

[0086] In the second mode, subsequently, in a second complex amplitude image generating step S63, for each of a plurality of light irradiation directions, a complex amplitude image for each of a plurality of z-direction positions (z=z1, z2, z3 in this figure) is generated based on the complex amplitude image for the reference position (z=0) generated in the phase conjugate calculation step S64 using the free propagation equations (1) and (2) above. Then, in a two-dimensional phase image generating step S65, for each of the plurality of positions, a complex differential interference image is generated based on the complex amplitude images for each of the plurality of light irradiation directions generated in the second complex amplitude image generating step S63, and further a differential phase image is generated.

[0087] In the third mode shown in FIG. 14, when the second complex amplitude image generation step S63 generates complex amplitude images for each of a plurality of positions from a complex amplitude image at a reference position through two stages, a phase conjugate calculation step S64 is performed between the first and second stages of the two stages.

[0088] In the third aspect, in the first stage of the second complex amplitude image generating step S63, for each of the plurality of light irradiation directions, a complex amplitude image for each of the plurality of z-direction positions (z=z1, z3, z5 in this figure) is generated based on the complex amplitude image for the reference position (z=0) generated in the first complex amplitude image generating step S62 using the free propagation equations (1) and (2) above. Subsequently, in the phase conjugate calculation step S64, a phase conjugate calculation is performed on the complex amplitude images for each of the plurality of light irradiation directions, thereby generating a complex amplitude image for each of the plurality of light irradiation directions when the relationship between light irradiation and imaging of the observation object is reversed.

[0089] In the third aspect, furthermore, in the second stage of the second complex amplitude image generating step S63, for each of the plurality of light irradiation directions, a complex amplitude image for each z direction position (z=z2, z4, z6) is generated based on the complex amplitude image for the z direction position (z=z1, z3, z5) generated in the phase conjugate calculation step S64 by the free propagation equations (1) and (2) above. Then, in the two-dimensional phase image generating step S65, for each of the plurality of positions, a complex differential interference image is generated based on the complex amplitude images for each of the plurality of light irradiation directions generated in the second complex amplitude image generating step S63, and further a differential phase image is generated.

[0090] The first, second, and third modes differ in the number of phase conjugate operations performed on the complex amplitude image in the phase conjugate operation step S64. The overall processing time for the phase conjugate operation step S64 is shorter in the third mode than in the first mode, and even shorter in the second mode.

[0091] 15 is a diagram illustrating the order of the processes and images in the three-dimensional phase image generating step S66 and the refractive index distribution calculating step S67. In the three-dimensional phase image generating step S66, a three-dimensional phase image is generated based on the two-dimensional phase images for each of the multiple positions generated in the two-dimensional phase image generating step S65. At this time, for positions relatively close to the imaging unit, a two-dimensional phase image generated based on the complex amplitude image before the processing in the phase conjugate calculation step S64 (a two-dimensional phase image generated in the manner shown in FIG. 11) is primarily used. On the other hand, for positions relatively far from the imaging unit, a two-dimensional phase image generated based on the complex amplitude image after the processing in the phase conjugate calculation step S64 (a two-dimensional phase image generated in any of the manners shown in FIGS. 12 to 14) is primarily used. Subsequently, in the refractive index distribution calculating step S67, the three-dimensional refractive index distribution of the observation object is determined by deconvolution based on the three-dimensional phase image generated in the three-dimensional phase image generating step S66.

[0092] There are three modes for generating two-dimensional phase images at each position in the z direction: The phase image generated based on the complex amplitude image before the processing of phase conjugate calculation step S64 (the phase image generated in the mode of FIG. 11) is defined as the first phase image φ1; The phase image generated based on the complex amplitude image after the processing of phase conjugate calculation step S64 (the phase image generated in any of the modes of FIGS. 12 to 14) is defined as the second phase image φ2; A weighting function α is used whose differential coefficient with respect to variable z, which represents the distance from the imaging unit along the light propagation path, is 0 or less. The value of the weighting function is 0 or greater and 1 or less.

[0093] In the first embodiment, the weighting function α is set to a value when z is smaller than the threshold z th It is assumed that the value is a positive value (for example, 1) in the following range, and the value is 0 in other ranges. That is, the two-dimensional phase image is expressed by the following equation (8).

[0094]

number

[0095] In the second aspect, the weighting function α has a value that changes continuously in at least a part of the range in the z direction. That is, the two-dimensional phase image is expressed by the following equation (9).

[0096]

number

[0097] In the third aspect, the weighting function α has a value according to the position (x, y) on a plane perpendicular to the optical axis (z direction). That is, the two-dimensional phase image is expressed by the following equation (10).

[0098]

number

[0099] Next, the contents of the phase conjugate calculation in the phase conjugate calculation step S64 will be described with reference to FIGS.

[0100] FIG. 16 shows the input light U when the interference intensity image is captured by the imaging unit. in (k in ) and output light u out (r out ) is a diagram showing the U in (k in ) is the wave number k of the light irradiated onto the object being observed in represents the complex amplitude of u out (r out ) is the position r of the light emitted from the object being observed. out represents the complex amplitude of U in (k in ) and u out (r out The relationship between the column vector U in n-th element U in (k in n ) is the wave number k in n represents the complex amplitude of the plane wave. out n-th element u of out (r outn ) is at position r out n represents the complex amplitude of the light observed at the out ,k in ) is U in (k in ) and u out (r out ) and is called the transmission matrix. Such a transmission matrix can represent the scattering process of light in the observed object. The matrix T(r out ,k in Element T in the n1th row and n2th column of n1,n2 is the wave number k in n2 When a plane wave with amplitude 1 is input at position r out n1 represents the complex amplitude of the light observed at

[0101]

number

[0102] Figure 17 shows the input light U when the relationship between light irradiation and imaging is reversed. out (k out ) and output light u in (r in ) is shown in Fig. 1. In this case, U out (k out ) is the wave number k of the light irradiated onto the object being observed out represents the complex amplitude of u in (r in ) is the position r of the light emitted from the object being observed. in represents the complex amplitude of U out (k out ) and u in (r in The relationship between the column vector U out n-th element U out (k out n ) is the wave number k out n represents the complex amplitude of the plane wave. in n-th element u ofin (r in n ) is at position r in n represents the complex amplitude of the light observed at the in ,k out ) is U out (k out ) and u in (r in ) and is the transmission matrix when the relationship between light illumination and imaging is reversed.

[0103]

number

[0104] U in (k in ) is expressed as u in (r in ) is expressed as the Fourier transform of U out (k out ) is expressed as u out (r out ) is expressed as a Fourier transform of the equations (11) to (14). When the relationship between light irradiation and imaging is reversed, the transmission matrix S(r in ,k out ) is a matrix representing the inverse Fourier transform and a transmission matrix T(r out ,k in ) is expressed by the following equation (15).

[0105]

number

[0106]

number

[0107]

number

[0108] In the phase conjugate calculation step S64, first, the transmission matrix T(r out ,k in ) is calculated. Next, we calculate the transmission matrix T(r out ,k in ) and the transmission matrix S(r in ,k out ) is calculated. Then, this transmission matrix S(r in ,k out ) to obtain a complex amplitude image when the relationship between light irradiation and imaging is reversed.

[0109] The vector U of the input light in the n-th light irradiation direction when the imaging unit captures an interference intensity image for each of the multiple light irradiation directions. in n (k in ) is expressed by the following equation (16), where only the value of the n-th element is 1 and the values ​​of the other elements are 0. in n (k in ) for the output light u out n (r out ) is expressed by the following equation (17): This equation (17) corresponds to the complex amplitude obtained in the nth light irradiation direction.

[0110]

number

[0111]

number

[0112] From this equation (16) and the above equation (11), the following equation (18) is obtained. Then, by similarly calculating for each of the multiple light irradiation directions, the following equation (19) is obtained. In this way, the transmission matrix T(r out,k in ) can be obtained. Furthermore, from this equation (19) and the above equation (15), the transmission matrix S(r in ,k out ) can be obtained.

[0113]

number

[0114]

number

[0115] When the relationship between light irradiation and imaging is reversed, the input light U in the nth light irradiation direction among the multiple light irradiation directions is out n (k out ) is expressed by the following equation (20), where only the value of the n-th element is 1 and the values ​​of the other elements are 0. From this equation, the input light U out n (k out ) versus output light u in n (r in ) is expressed by the following equation (21). This equation (21) represents the complex amplitude when the relationship between light irradiation and imaging is reversed. In this way, a complex amplitude image when the relationship between light irradiation and imaging is reversed can be obtained.

[0116]

number

[0117]

number

[0118] The transmission matrix S(r in ,k out) is calculated using the transmission matrix T(r out ,k in ) must be calculated. Therefore, the transmission matrix T must be a square matrix in which the number of row elements is equal to the number of column elements. In other words, the matrix dimension in the light irradiation sideband space for the object of observation in the interference intensity image acquisition step S61 must be equal to the number of pixels in the complex amplitude image.

[0119] To make the two equal, the dimension of the matrix in the light irradiation side wavenumber space for the object to be observed in the interference intensity image acquisition step S61 can be made to match the number of pixels, or only a portion of the image obtained by the imaging unit can be used for subsequent processing. However, since the number of pixels in the image obtained by the imaging unit is generally, for example, 1024 × 1024, it is not easy to make the dimension of the matrix in the light irradiation side wavenumber space for the object to be observed the same as the number of pixels. Furthermore, using only a portion of the image obtained by the imaging unit for subsequent processing is not preferable because it leads to a decrease in resolution.

[0120] Therefore, as shown in Fig. 18, in the phase conjugate calculation step S64, it is preferable to divide the complex amplitude image into a plurality of partial images, each having the same number of pixels as the dimension of the matrix in the light irradiation sideband space for the object to be observed, perform the phase conjugate calculation on each of the plurality of partial images, and then combine the plurality of partial images. In this case, any two or more of the plurality of partial images may have a common area.

[0121] Next, the simulation results will be described. Simulations A to D described below were performed according to the procedures shown in FIGS.

[0122] In Simulation A, as shown in Figure 19, a transparent sphere was used as the object of observation. The dimensions of the matrix in the illumination side wavenumber space for the object of observation were set to various values, and a simulation was performed to examine the effect of the relationship between the dimensions of the matrix in the illumination side wavenumber space and the number of pixels in the image on the results of the phase conjugate operation. Figure 19 is a diagram schematically illustrating the layout used in the simulation. Here, the number of pixels was set to 108 x 108, and the dimensions of the matrix in the illumination side wavenumber space were set to 108 x 108, 54 x 54, 27 x 27, and 18 x 18, respectively. Figure 20(a) shows a phase image obtained by performing a phase conjugate operation when the dimensions of the matrix in the illumination side wavenumber space were set to 108 x 108. When the number of pixels in the image and the dimensions of the matrix in the illumination side wavenumber space were equal, no ghosts appeared in the phase image. Figure 20(b) shows a phase image obtained by performing a phase conjugate operation when the dimensions of the matrix in the illumination side wavenumber space were set to 54 x 54. When the dimension of the matrix in the optical illumination sideband space is small compared to the number of pixels in the image, ghosts appear in the phase image obtained by performing the phase conjugate operation. The fewer the dimension of the matrix in the optical illumination sideband space, the more ghosts appear in the phase image.

[0123] In Simulation B, as shown in FIG. 21, a phase image was used as the object of observation, and the dimension of the matrix in the illumination side frequency space for the object of observation was set to various values ​​to simulate the effect of the relationship between the dimension of the matrix in the illumination side frequency space and the number of pixels in the image on the results of the phase conjugate operation. FIG. 21 is a diagram schematically illustrating the layout used in the simulation. Here, the number of pixels was set to 108 × 108, and the dimensions of the matrix in the illumination side frequency space were set to 108 × 108, 54 × 54, 27 × 27, and 18 × 18, respectively, to perform the phase conjugate operation. FIG. 22(a) shows a phase image obtained by performing the phase conjugate operation when the dimension of the matrix in the illumination side frequency space was set to 108 × 108. When the number of pixels in the image and the dimension of the matrix in the illumination side frequency space were equal, no ghosts appeared in the phase image. FIG. 22(b) shows a phase image obtained by performing the phase conjugate operation when the dimension of the matrix in the illumination side frequency space was set to 54 × 54. When the dimension of the matrix in the optical illumination sideband space is small compared to the number of pixels in the image, ghosts appear in the phase image obtained by the phase conjugate operation. The fewer the dimension of the matrix in the optical illumination sideband space, the more ghosts appear in the phase image. In addition, multiple ghost images overlap each other.

[0124] As can be seen from the results of Simulations A and B, the dimension of the matrix in the light illumination side wavenumber space for the object under observation in the interference intensity image acquisition step S61 must be equal to the number of pixels of the complex amplitude image. Therefore, in the phase conjugate calculation step S64, it is preferable to divide the complex amplitude image into a plurality of partial images, each having the same number of pixels as the dimension of the matrix in the light illumination side wavenumber space for the object under observation, perform the phase conjugate calculation on each of these plurality of partial images, and then combine the plurality of partial images.

[0125] In Simulation C, a transparent sphere was used as the object of observation, and the dimensions of the matrix in the illumination sideband space for the object were set to various values ​​to simulate the effect of whether the imaging unit was focused on the object on the results of the phase conjugate calculation. Here, the number of pixels was set to 108 × 108, and the dimensions of the matrix in the illumination sideband space were set to 108 × 108 and 36 × 36, respectively, for the phase conjugate calculation. When the dimensions of the matrix in the illumination sideband space was set to 36 × 36, the complex amplitude image was divided into nine partial images for the phase conjugate calculation. Here, a phase image was generated from the complex amplitude image without passing through a complex differential interference contrast image.

[0126] Fig. 23(a) is a phase image obtained by performing a phase conjugate operation when the imaging unit is focused on the object of observation and the dimension of the matrix in the light-illumination side wavenumber space is 108 × 108. Fig. 23(b) is a phase image obtained by performing a phase conjugate operation when the imaging unit is focused on the object of observation and the dimension of the matrix in the light-illumination side wavenumber space is 36 × 36. Fig. 24(a) is a phase image obtained by performing a phase conjugate operation when the imaging unit is not focused on the object of observation and the dimension of the matrix in the light-illumination side wavenumber space is 108 × 108. Fig. 24(b) is a phase image obtained by performing a phase conjugate operation when the imaging unit is not focused on the object of observation and the dimension of the matrix in the light-illumination side wavenumber space is 36 × 36.

[0127] As can be seen from the results of Simulation C, even if the dimension of the matrix in the light-illumination sideband frequency space is smaller than the number of pixels, the appearance of ghosts in the phase image obtained by the phase conjugate operation can be suppressed by dividing the complex amplitude image into multiple partial images, each with the same number of pixels as the dimension of the matrix in the light-illumination sideband frequency space, and then performing the phase conjugate operation. However, when the imaging unit is not focused on the object being observed, noise appears in the phase image obtained by the phase conjugate operation, rather than ghosts. This noise is thought to be caused by scattered waves generated from the same scatterer that spill over from one partial image to another. This noise can be suppressed by generating a phase image from the complex amplitude image via a complex differential interference contrast image. By passing through a complex differential interference contrast image, light from positions other than the focus plane can be removed.

[0128] Figure 25(a) shows a differential phase image obtained by performing a phase conjugate operation when the dimension of the matrix in the light-irradiation sideband space is 108 × 108. Figure 25(b) shows a differential phase image obtained by performing a phase conjugate operation when the dimension of the matrix in the light-irradiation sideband space is 36 × 36. Each of Figures 25(a) and 25(b) shows five differential phase images at different z-direction positions. As shown in these figures, even when a complex amplitude image is divided into multiple partial images and phase conjugate operation is performed, the same differential phase images are obtained as when phase conjugate operation is performed without dividing the image.

[0129] In Simulation D, five types of phase images arranged in parallel at regular intervals were used as the observation object. Figure 26 is a diagram illustrating the layout used in the simulation. Here, the number of pixels was 360 × 360, and the dimension of the matrix in the light irradiation sideband space was 36 × 36. The complex amplitude image was divided into 100 partial images and subjected to phase conjugate calculation. Figure 27(a) is a phase differential image of the exact solution. Figure 27(b) is a phase differential image obtained without performing phase conjugate calculation. Figure 27(c) is a phase differential image obtained by performing phase conjugate calculation.

[0130] As shown in Figure 27(b), the phase differential image obtained without the phase conjugate operation is clearer the closer to the imaging unit and less clear the farther from the imaging unit. In contrast, as shown in Figure 27(c), the phase differential image obtained with the phase conjugate operation is less clear the closer to the imaging unit and more clear the farther from the imaging unit. Therefore, for positions relatively close to the imaging unit, phase images generated based on complex amplitude images before the phase conjugate operation are mainly used, and for positions relatively far from the imaging unit, phase images generated based on complex amplitude images after the phase conjugate operation are mainly used, thereby improving the penetration depth in observation of the object to be observed.

[0131] Next, an example will be described. In this example, the observation device 1A (FIG. 1) was used, and Fourier fringe analysis was adopted. In step S21, complex differential interference images were generated for each of two different shear directions (vertical shear and horizontal shear) on the complex amplitude image. The procedure shown in FIGS. 4 and 12 was followed. A spheroid of a three-dimensional culture of HepG2 derived from human liver cancer was used as the observation object. The number of pixels was 600 × 600. The dimension of the matrix in the light irradiation sideband frequency space for the observation object was 50 × 50. The complex amplitude image was divided into 144 partial images, and phase conjugate calculations were performed. FIGS. 28 to 44 show examples of images obtained at each step.

[0132] Fig. 28 is an interference intensity image (at normal irradiation) acquired in interference intensity image acquisition step S61. Fig. 29 is a complex amplitude image (real part, z=0) generated based on the interference intensity image (Fig. 28) in first complex amplitude image generation step S62. Fig. 30 is a complex amplitude image (real part, z=z) generated based on the complex amplitude image (Fig. 29) in second complex amplitude image generation step S63. n ) in the phase conjugate calculation step S64. n) is obtained by performing the phase conjugate calculation on each of the plurality of partial images (FIG. 31) in the phase conjugate calculation step S64. n )

[0133] FIG. 33 shows a complex amplitude image (real part, z=z) obtained by combining the multiple partial images (FIG. 32) after the phase conjugate calculation in the phase conjugate calculation step S64. n ) Fig. 34 shows a complex differential interference image (imaginary parts for the x-direction shear and the y-direction shear) generated based on the complex amplitude image (Fig. 33) in step S21 of the two-dimensional phase image generation step S65. Fig. 35 shows a differential phase image (x-direction shear and y-direction shear) generated based on the complex differential interference image (Fig. 34) in step S22 of the two-dimensional phase image generation step S65.

[0134] Fig. 36 shows differential phase images (x-direction shear and y-direction shear) generated in step S22 of two-dimensional phase image generation step S65 based on the complex amplitude image (Fig. 30) without performing phase conjugate calculation step S64. Fig. 37 shows differential phase images (x-direction shear and y-direction shear) obtained by combining the differential phase image (Fig. 35) obtained when phase conjugate calculation step S64 was performed and the differential phase image (Fig. 36) obtained when phase conjugate calculation step S64 was not performed. Fig. 38 shows the refractive index distribution generated based on the differential phase image in refractive index distribution calculation step S67.

[0135] FIG. 39 is a phase differential image at a position of z=10.4 μm. FIG. 40 is a phase differential image at a position of z=32.4 μm. FIG. 41 is a phase differential image at a position of z=54.0 μm. Each figure (a) is a phase differential image obtained without performing phase conjugate calculation. Each figure (b) is a phase differential image obtained with performing phase conjugate calculation. In the region indicated by the arrow in FIG. 39, close to the imaging unit, the presence of granules is clearly observed in the phase differential image (FIG. 39(a)) obtained without performing phase conjugate calculation step S64, whereas the presence of granules is not observed in the phase differential image (FIG. 39(b)) obtained with performing phase conjugate calculation. In contrast, at a position far from the imaging unit, in the region indicated by the arrow in Figure 41, the presence of granules is not observed in the phase differential image (Figure 41(a)) obtained when the phase conjugate calculation step S64 was not performed, whereas the presence of granules is clearly observed in the phase differential image (Figure 41(b)) obtained when the phase conjugate calculation was performed.

[0136] Figure 42 shows the refractive index distribution at a position z=10.4 μm. Figure 43 shows the refractive index distribution at a position z=54.0 μm. Each figure (a) shows the refractive index distribution obtained without performing the phase conjugate operation. Each figure (b) shows the refractive index distribution obtained by reconstructing by combining the phase differential images obtained with and without the phase conjugate operation. As shown in Figure 42, at a position close to the imaging unit, there is little difference between the refractive index distributions obtained without performing the phase conjugate operation (Figure 42(a)) and those obtained by reconstructing by combining the phase differential images obtained with and without the phase conjugate operation (Figure 42(b)). In contrast, in the region indicated by the arrow in Figure 43, at a position far from the imaging unit, the presence of granules is not observed in the refractive index distribution obtained when the phase conjugate operation was not performed (Figure 43(a)), whereas the presence of granules is clearly observed in the refractive index distribution obtained when the phase conjugate operation was reconstructed by combining the phase differential images obtained when the phase conjugate operation was performed and when it was not performed (Figure 43(b)).

[0137] In this way, for positions relatively close to the imaging unit, phase images generated based on complex amplitude images before phase conjugate calculation are primarily used, and for positions relatively far from the imaging unit, phase images generated based on complex amplitude images after phase conjugate calculation are primarily used, thereby improving the depth of penetration in observation of the object being observed.

[0138] Fig. 44(a) shows differential phase images at each z position obtained when no phase conjugate calculation was performed. Fig. 44(b) shows differential phase images at each z position obtained when the phase conjugate calculation step S64 was performed after the second complex amplitude image generation step S63 according to the procedure shown in Fig. 12. Fig. 44(c) shows differential phase images at each z position obtained when the phase conjugate calculation step S64 was performed before the second complex amplitude image generation step S63 according to the procedure shown in Fig. 13. Figs. 44(a) to (c) show differential phase images at z=-5.6 μm, -2.8 μm, 0 μm, 2.8 μm, and 5.6 μm, respectively.

[0139] When the phase conjugate calculation step S64 is performed before the second complex amplitude image generation step S63 (FIG. 44(c)), the same image quality is obtained in the phase differential images at z=-2.8 μm, 0 μm, and 2.8 μm compared to when the phase conjugate calculation step S64 is performed after the second complex amplitude image generation step S63 (FIG. 44(b)). However, in both cases where the phase conjugate calculation is performed (FIGS. 44(b) and (c)), the internal structure can be seen even at a position farther from the imaging unit, and the depth of penetration in the observation of the object can be improved, compared to when the phase conjugate calculation is not performed (FIG. 44(a)).

[0140] Furthermore, when the phase conjugate calculation step S64 is performed before the second complex amplitude image generation step S63 (FIG. 44(c)), the number of phase conjugate calculations, which are a bottleneck in calculation time, can be reduced compared to when the phase conjugate calculation step S64 is performed after the second complex amplitude image generation step S63 (FIG. 44(b)), thereby shortening the processing time. [Explanation of symbols]

[0141] 1A to 1C... observation device, 2... recording medium, 11... light source, 12... lens, 13... light input end, 14... optical fiber, 15... fiber coupler, 16, 17... optical fiber, 18, 19... light output end, 21... lens, 22... mirror, 23... lens, 24... condenser lens, 25... objective lens, 31... lens, 32... mirror, 33... drive unit, 34... lens, 41... beam splitter, 42... lens, 43... imaging unit, 60... analysis unit, 61... interference intensity image acquisition unit, 62... first complex amplitude image generation unit, 63... second complex amplitude image generation unit, 64... phase conjugate calculation unit, 65... two-dimensional phase image generation unit, 66... ​​three-dimensional phase image generation unit, 67... refractive index distribution calculation unit, 68... display unit, 69... memory unit.

Claims

1. an interference intensity image acquisition unit that acquires, from an imaging unit that captures an interference intensity image at a reference position due to interference between light that has been irradiated onto an observation object along each of a plurality of light irradiation directions and passed through the observation object and a reference light, the interference intensity image at the reference position in each of the plurality of light irradiation directions; a first complex amplitude image generating unit that generates a complex amplitude image of the reference position based on the interference intensity image of the reference position for each of the plurality of light irradiation directions; a second complex amplitude image generating unit configured to generate a complex amplitude image for each of a plurality of positions based on the complex amplitude image for the reference position for each of the plurality of light irradiation directions; a phase conjugate calculation unit that performs a phase conjugate calculation on the complex amplitude images in each of the plurality of light irradiation directions before, during, or after processing by the second complex amplitude image generation unit, to generate complex amplitude images in each of the plurality of light irradiation directions when a relationship between light irradiation and imaging of the observation object is reversed; a two-dimensional phase image generating unit that generates a complex differential interference image for each of the plurality of light irradiation directions based on the complex amplitude image for each of the plurality of light irradiation directions generated by the second complex amplitude image generating unit or the phase conjugate calculating unit, and generates a two-dimensional phase image based on the complex differential interference image for each of the plurality of light irradiation directions; a three-dimensional phase image generating unit that generates a three-dimensional phase image based on the two-dimensional phase images at each of the plurality of positions; Equipped with the two-dimensional phase image generating unit generates the two-dimensional phase image mainly based on the first phase image for positions relatively close to the imaging unit among the plurality of positions, and generates the two-dimensional phase image mainly based on the second phase image for positions relatively far from the imaging unit, when the phase image generated based on the complex amplitude image before the operation by the phase conjugate calculation unit is defined as a first phase image, and the phase image generated based on the complex amplitude image obtained by the operation by the phase conjugate calculation unit is defined as a second phase image; Observation equipment.

2. The observation device according to claim 1 , wherein the two-dimensional phase image generating unit generates the two-dimensional phase image based on a sum of the complex differential interference images for each of the plurality of light irradiation directions.

3. the phase conjugate calculation unit divides the complex amplitude image into a plurality of partial images, each of which has the same number of pixels as a dimension of a matrix in a light irradiation sideband space for the object to be observed, performs a phase conjugate calculation on each of the plurality of partial images, and then combines the plurality of partial images. The observation device according to claim 1 or 2.

4. the two-dimensional phase image generating unit uses a weighting function α whose differential coefficient with respect to a variable z representing a distance from the imaging unit along a light propagation path is equal to or less than 0 to obtain the two-dimensional phase image by multiplying the first phase image by α and the second phase image by (1-α). The observation device according to any one of claims 1 to 3.

5. the two-dimensional phase image generating unit uses, as the weighting function α, a function that is positive when the value of the variable z is equal to or less than a threshold value and has a value of 0 when the value of the variable z is other than a threshold value; The observation device according to claim 4.

6. the two-dimensional phase image generating unit uses, as the weighting function α, a function having a value according to a position on a plane orthogonal to an optical axis of the imaging unit; The observation device according to claim 4.

7. The two-dimensional phase image generating unit generating a complex differential interference image for each of the plurality of light irradiation directions for each of a plurality of shear directions different from one another on the image based on the complex amplitude image for each of the plurality of light irradiation directions; generating the two-dimensional phase image based on the complex differential interference images in the plurality of shear directions and the plurality of light irradiation directions; The observation device according to any one of claims 1 to 6.

8. a refractive index distribution calculation unit that calculates a three-dimensional refractive index distribution of the observation object based on the three-dimensional phase image. The observation device according to any one of claims 1 to 7.

9. an interference intensity image acquisition step of acquiring, from an imaging unit that captures interference intensity images at reference positions due to interference between light irradiated onto an observation object along each of a plurality of light irradiation directions and passing through the observation object, and a reference light, the interference intensity images at the reference positions in the plurality of light irradiation directions; a first complex amplitude image generating step of generating a complex amplitude image of the reference position based on the interference intensity image of the reference position for each of the plurality of light irradiation directions; a second complex amplitude image generating step of generating a complex amplitude image for each of a plurality of positions based on the complex amplitude image for the reference position for each of the plurality of light irradiation directions; a phase conjugate calculation step of performing a phase conjugate calculation on the complex amplitude images in each of the plurality of light irradiation directions before, during, or after the processing in the second complex amplitude image generation step, to generate complex amplitude images in each of the plurality of light irradiation directions when a relationship between light irradiation and imaging of the observation object is reversed; a two-dimensional phase image generating step of generating a complex differential interference image for each of the plurality of light irradiation directions based on the complex amplitude images for each of the plurality of light irradiation directions generated by the second complex amplitude image generating step or the phase conjugate calculation step, and generating a two-dimensional phase image based on the complex differential interference images for each of the plurality of light irradiation directions; a three-dimensional phase image generating step of generating a three-dimensional phase image based on the two-dimensional phase images at each of the plurality of positions; Equipped with In the two-dimensional phase image generating step, when a phase image generated based on the complex amplitude image before the operation in the phase conjugate operation step is defined as a first phase image and a phase image generated based on the complex amplitude image obtained by the operation in the phase conjugate operation step is defined as a second phase image, the two-dimensional phase image is generated mainly based on the first phase image for positions relatively close to the imaging unit among the plurality of positions, and the two-dimensional phase image is generated mainly based on the second phase image for positions relatively far from the imaging unit. Observation method.

10. 10. The observation method according to claim 9, wherein the two-dimensional phase image generating step generates the two-dimensional phase image based on a sum of the complex differential interference images for each of the plurality of light irradiation directions.

11. In the phase conjugate calculation step, the complex amplitude image is divided into a plurality of partial images, each of which has the same number of pixels as a dimension of a matrix in a light irradiation sideband space for the object to be observed, and a phase conjugate calculation is performed on each of the plurality of partial images, and then the plurality of partial images are combined. The observation method according to claim 9 or 10.

12. In the two-dimensional phase image generating step, a weighting function α having a differential coefficient of 0 or less with respect to a variable z representing a distance from the imaging unit along a light propagation path is used, and the two-dimensional phase image is obtained by multiplying the first phase image by α and the second phase image by (1-α). The observation method according to any one of claims 9 to 11.

13. In the two-dimensional phase image generating step, a function is used as the weighting function α, which is a positive value when the value of the variable z is equal to or less than a threshold value, and is 0 when the value of the variable z is other than a threshold value. The observation method according to claim 12.

14. In the two-dimensional phase image generating step, a function having a value according to a position on a plane perpendicular to an optical axis of the imaging unit is used as the weighting function α. The observation method according to claim 12.

15. In the two-dimensional phase image generating step, generating a complex differential interference image for each of the plurality of light irradiation directions for each of a plurality of shear directions different from one another on the image based on the complex amplitude image for each of the plurality of light irradiation directions; generating the two-dimensional phase image based on the complex differential interference images in the plurality of shear directions and the plurality of light irradiation directions; The observation method according to any one of claims 9 to 14.

16. a refractive index distribution calculation step of calculating a three-dimensional refractive index distribution of the object to be observed based on the three-dimensional phase image. The observation method according to any one of claims 9 to 15.

17. A program for causing a computer to execute each step of the observation method according to any one of claims 9 to 16.

18. A computer-readable recording medium on which the program according to claim 17 is recorded.

Images