Observation device and observation method

Abstract

An observation device (1A) is provided with a light source (11), a mirror (22), a condenser lens (24), an objective lens (25), a beam splitter (41), an imaging unit (43), and an analysis unit (50). An analysis unit (50) irradiates an object (S) to be observed with light in each of a plurality of light irradiation directions by changing the orientation of a reflection surface of a mirror (22), and acquires interference intensity images at a reference position from an imaging unit (43) for each of the plurality of light irradiation directions. The three-dimensional refractive index distribution of the object to be observed is obtained by performing necessary processing on the basis of the interference intensity images. As a result, it is possible to achieve an observation device capable of observing an object to be observed while reducing the influence of multiple scattered light even when the object to be observed is a multiple scatterer.

Description

Technical Field

[0001] The present disclosure relates to an observation device and an observation method. Background Art

[0002] In recent years, the technology for making three-dimensional cell tissues called cell spheroids or organoids has been continuously improved. In addition, research is underway to apply these three-dimensional cell tissues to drug development, regenerative medicine, etc. These three-dimensional cell tissues are optically transparent multiple scatterers. As a technology for imaging such optically transparent scatterers, various methods have been proposed so far.

[0003] Among them, imaging techniques using fluorescent probes include confocal microscopy, multiphoton microscopy, and light-sheet microscopy. On the other hand, optical coherence tomography (OCT) and other non-staining, non-invasive imaging techniques that do not use fluorescent probes are known.

[0004] While non-staining, non-invasive imaging is often desired for observation objects such as spheroids and organoids, there are few reports of OCT being used for imaging these objects. This is due to the low resolution of OCT imaging and the difficulty in interpreting the signals obtained using OCT imaging. Therefore, it can be said that a technology that could serve as the gold standard for non-staining three-dimensional cellular tissue imaging has not yet been established.

[0005] Quantitative phase imaging (QPI) is also known as a technology that can image the optical path length of an observation object non-invasively and without staining. QPI can obtain physical information such as the optical path length of an observation object (e.g., cells), and therefore has been widely used in the biological field. Images obtained using QPI can be used to generate other types of images, such as differential interference images and phase contrast microscopy images.

[0006] QPI is a technology that can produce images with a relatively high content of information, and is expected to be used for higher-resolution analysis than conventional analysis using brightfield images. Furthermore, with recent improvements in image recognition accuracy brought about by machine learning, high-resolution analysis using non-staining imaging techniques has been widely researched, and non-staining imaging of multiple scatterers is expected to play a significant role in the future. However, since the images obtained are merely two-dimensional projections of the optical path length, QPI cannot capture true three-dimensional structures.

[0007] Optical Diffraction Tomography (ODT), described in Patent Document 1, is also known as a technology capable of imaging the optical path length of an object under observation in a non-staining and non-invasive manner. ODT develops QPI into a technology capable of three-dimensional imaging, enabling three-dimensional refractive index tomography of the object under observation.

[0008] By using ODT for cell observation, it is possible to identify organelles such as the nucleus and mitochondria, and to track three-dimensional morphological changes, and it is expected to enable higher-level analysis than QPI.

[0009] Prior art literature

[0010] Patent Literature

[0011] Patent Document 1: Japanese Patent Application Laid-Open No. 2017-219826.

[0012] Non-Patent Literature 1: Pritam Pai et al., "Optical transmission matrix measurement sampled on a dense hexagonal lattice", OSA Continuum, Vol. 3, No. 3, pp. 637-648, 2020

[0013] Non-patent document 2: Duygu Akbulut et al., "Optical transmission matrix as a probe of the photonic strength", PHYSICAL REVIEW A 94, 043817, pp. 043817-1-043817-8, 2016

[0014] Non-patent document 3: Elbert G. van Putten et al., "The information age inoptics: Measuring the transmission matrix", Physics 3, 22, 2010,<URL:https: / / physics.aps.org / articles / v3 / 22?referer=apshome> Summary of the Invention

[0015] Problems to be solved by the invention

[0016] However, while conventional ODT can be applied to the observation of cells composed of multiple components, it is difficult to apply to the observation of multiple scatterers such as the three-dimensional cell tissue described above. This is because, with conventional ODT, when a large amount of multiple scattered light is generated in the observation object, the influence of the multiple scattered light appears significantly in the acquired image.

[0017] Light scattering refers to the phenomenon in which the direction of light travel is changed by its interaction with an object. In particular, when the spatial inhomogeneity of the refractive index of an object increases, light interacts with the object multiple times while passing through the object. Light that interacts with the object multiple times in this way is called multiply scattered light. In contrast, light that interacts with the object only once is called single scattered light. Multiply scattered light is known to cause the increase of speckle and the deterioration of the single-scattering to multi-scattering ratio (SMR), which can hinder measurement.

[0018] Speckle is caused by large spatial variations in intensity or phase due to interference of multiple scattered light when light is temporally and spatially coherent. To suppress speckle, a light source that outputs temporally or spatially incoherent light can be used. For example, conventional brightfield microscopes, such as phase contrast microscopes, use spatially and temporally incoherent light sources such as halogen lamps and light-emitting diodes to produce speckle-free images.

[0019] Deterioration in SMR occurs when multiply scattered light dominates over singly scattered light, causing the singly scattered light to be buried within the multiply scattered light. As the size of the observed object increases and the observation depth deepens, the single scattered light component decreases exponentially, while the multiply scattered light component increases in contrast.

[0020] The scattering direction of single scattered light directly corresponds to the structure of the object, making it easy to use for structural measurement. On the other hand, the relationship between multiple scattered light and the object's structure is complex, making it difficult to extract structural information. Therefore, in imaging techniques using single scattered light, it is known that if the single scattered light is buried in the multiple scattered light (i.e., SMR deteriorates), the measurement will fail.

[0021] SMR degradation can be suppressed by selectively detecting single scattered light from multiple scattered light, a technique called gating. Gating suppresses multiple scattered light, thereby simultaneously suppressing SMR degradation and speckle. Gating can be achieved using degrees of freedom, such as spatial, temporal, and polarization. Confocal microscopy is an example of spatial gating. OCT is an example of temporal and spatial gating.

[0022] Conventional ODT does not eliminate the effects of multiply scattered light. Therefore, when a large amount of multiply scattered light is generated in the observation object, speckle increases in the acquired image and SMR deteriorates. Therefore, while conventional ODT can be applied to observe cells composed of a few cells that generate little multiply scattered light, it is difficult to apply to the observation of multiply scattered objects such as three-dimensional cell tissues that generate a large amount of multiply scattered light.

[0023] An object of the embodiment is to provide an observation apparatus and an observation method capable of observing the observation object while reducing the influence of multiply scattered light even when the observation object is a multiply scatterer.

[0024] Technical means to solve the problem

[0025] The embodiment is an observation device. The observation device comprises: (1) an interference intensity image acquisition unit that acquires interference intensity images of an observation object irradiated with light along a plurality of light irradiation directions; (2) a first complex amplitude image generation unit that generates, for each of the plurality of light irradiation directions, a first complex amplitude image of light when a plane wave of light enters one of the first end face and the second end face and reaches the other, based on the interference intensity image; (3) a refractive index distribution image generation unit that generates a first refractive index distribution image of a section from the first end face to a first intermediate face between the first end face and the second end face, based on the first complex amplitude image of each of the plurality of light irradiation directions; (4) a second complex amplitude image (5) a transmission matrix generating unit, which generates a transmission matrix T based on the first complex amplitude image of each of the multiple light irradiation directions, and generates a transmission matrix T1 based on the second complex amplitude image of each of the multiple light irradiation directions; and (6) a third complex amplitude image generating unit, which generates a third complex amplitude image of the light when the plane wave light is incident on the first intermediate surface and reaches the second end surface for each of the multiple light irradiation directions, based on the product of the transmission matrix T and the inverse matrix of the transmission matrix T1.

[0026] The embodiment is an observation method. The observation method comprises: (1) an interference intensity image acquisition step, which acquires interference intensity images of an observation object irradiated with light along a plurality of light irradiation directions; (2) a first complex amplitude image generation step, which generates, for each of the plurality of light irradiation directions, a first complex amplitude image of light when a plane wave of light is incident on either the first end face or the second end face and reaches the other, based on the interference intensity image; (3) a refractive index distribution image generation step, which generates a first refractive index distribution image of a section from the first end face to a first intermediate face between the first end face and the second end face, based on the first complex amplitude image of each of the plurality of light irradiation directions; (4) a second complex amplitude image (5) a transmission matrix generating step, generating a transmission matrix T based on the first complex amplitude image of each of the multiple light irradiation directions, and generating a transmission matrix T1 based on the second complex amplitude image of each of the multiple light irradiation directions; and (6) a third complex amplitude image generating step, generating a third complex amplitude image of the light when the plane wave light is incident on the first intermediate surface and reaches the second end surface for each of the multiple light irradiation directions based on the product of the transmission matrix T and the inverse matrix of the transmission matrix T1.

[0027] The embodiment is a program. The program is a program for causing a computer to execute each step of the observation method having the above structure.

[0028] The embodiment is a recording medium. The recording medium is a computer-readable recording medium storing the program having the above structure.

[0029] Effects of the Invention

[0030] According to the observation apparatus and the observation method of the embodiment, even when the observation object is a multiple scatterer, the observation object can be observed while reducing the influence of multiple scattered light. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a diagram showing the configuration of an observation apparatus 1A.

[0032] Figure 2 It is a diagram showing the structure of the observation apparatus 1B.

[0033] Figure 3 It is a diagram showing the structure of the observation apparatus 1C.

[0034] Figure 4 It is a diagram showing the configuration of the analysis unit 50 of the observation apparatuses 1A to 1C.

[0035] Figure 5 These are diagrams for explaining an overview of an observation method using the observation apparatuses 1A to 1C.

[0036] Figure 6 It is a flow chart of the observation method.

[0037] Figure 7 This is a diagram explaining the order of generating each image in the observation method.

[0038] Figure 8 (a) to (e) are diagrams showing examples of scanning in the light irradiation direction toward the observation object S in the interference intensity image acquisition step S51.

[0039] Figure 9 This is a follow-up to the observation method. Figure 6 A flowchart of a processing example following the processing of the flowchart.

[0040] Figure 10 It is a diagram for explaining the steps of generating each image in the observation method.

[0041] Figure 11 This is a follow-up to the observation method. Figure 6 Flowchart of another processing example following the processing of the flowchart.

[0042] Figure 12 This is a diagram explaining the order of generating each image in the observation method.

[0043] Figure 13 is the input light U when the interference intensity image is captured by the imaging unit. in (k in ) and output light u out (r out ) picture.

[0044] Figure 14 4 is a flowchart of the refractive index distribution image generation process.

[0045] Figure 15 This is a diagram explaining the processing contents of BPM.

[0046] Figure 16 This is a flowchart of the second complex amplitude image generating step S54.

[0047] Figure 17 (a)k represents the case where the number of light irradiation directions is equal to the number of pixels of the complex amplitude image. x k y The distribution of light irradiation directions in wave number space and (b) the transmission matrix T when focusing light on a certain point r0 rr (r out ; r in = r0) in the xy space.

[0048] Figure 18(a)k represents the case where the number of light irradiation directions is smaller than the number of pixels of the complex amplitude image. x k y The distribution of light irradiation directions in wave number space and (b) the transmission matrix T' when focusing light on a certain point r0 rr (r out ; r in = r0) in the xy space.

[0049] Figure 19 This is a diagram explaining a method of obtaining the product of the entire transfer matrix T and the inverse matrix of the transfer matrix T1 when the number of light irradiation directions is smaller than the number of pixels of the complex amplitude image.

[0050] Figure 20 This is a diagram for explaining the order of generating each image in another aspect of the observation method.

[0051] Figure 21 A diagram schematically illustrating the configuration in simulation A.

[0052] Figure 22 Graphs showing the results of simulation A.

[0053] Figure 23 This is a diagram schematically illustrating the configuration in simulation B.

[0054] Figure 24 Graphs showing the results of simulation B.

[0055] Figure 25 Graphs showing the results of simulation B.

[0056] Figure 26 It is a diagram showing the configuration of the observation apparatus 100 .

[0057] Figure 27 1 is a diagram schematically showing the incidence of the first branched light and the second branched light on the observation object S in the observation apparatus 100 , and the incidence of the first branched light and the second branched light on the imaging unit 150 after passing through the observation object S. DETAILED DESCRIPTION

[0058] Embodiments of the observation device and observation method are described in detail below with reference to the accompanying drawings. In the description of the drawings, identical elements are denoted by the same reference numerals, and duplicate descriptions are omitted. The present invention is not limited to these examples and is intended to encompass all modifications within the meaning and scope of the claims and equivalents thereof.

[0059] Figure 11A is a diagram showing the configuration of an observation apparatus 1A, which includes a light source 11, a lens 12, a lens 21, a mirror 22, a lens 23, a condenser lens 24, an objective lens 25, a beam splitter 41, a lens 42, an imaging unit 43, and an analyzing unit 50.

[0060] Light source 11 outputs spatially and temporally coherent light, preferably a laser light source. Lens 12 is optically connected to light source 11 and focuses the light output from light source 11 onto light input end 13 of optical fiber 14, causing the light to enter light input end 13. Optical fiber 14 guides the light that has passed through lens 12 and entered light input end 13 to fiber coupler 15.

[0061] Fiber coupler 15 couples light between optical fiber 14 and optical fibers 16 and 17, branching the light guided by optical fiber 14 into two branches: one branched light is guided by optical fiber 16, and the other branched light is guided by optical fiber 17. The light guided by optical fiber 16 is emitted from light output end 18 as divergent light. The light guided by optical fiber 17 is emitted from light output end 19 as divergent light.

[0062] Lens 21 is optically connected to light output end 18 and collimates the light output as divergent light from light output end 18. Mirror 22 is optically connected to lens 21 and reflects light arriving from lens 21 toward lens 23. The orientation of the reflective surface of mirror 22 is variable. Lens 23 is optically connected to mirror 22. Condenser lens 24 is optically connected to lens 23. Lens 23 and condenser lens 24 preferably form a 4f optical system.

[0063] The lens 23 and the condenser lens 24 irradiate light onto the observation object S from a light irradiation direction corresponding to the orientation of the reflecting surface of the mirror 22. The objective lens 25 is optically connected to the condenser lens 24. The observation object S is arranged between the objective lens 25 and the condenser lens 24. The objective lens 25 receives light (object light) output from the condenser lens 24 and passing through the observation object S, and outputs the light to the beam splitter 41.

[0064] The beam splitter 41 is optically connected to the objective lens 25 and is also optically connected to the light output end 19. The beam splitter 41 combines the light (object light) output from the objective lens 25 and arriving with the light (reference light) output from the light output end 19 and arriving, and outputs the two lights to the lens 42. The lens 42 is optically connected to the beam splitter 41, and collimates the object light and reference light arriving from the beam splitter 41, respectively, and outputs them to the imaging unit 43.

[0065] The imaging unit 43 is optically connected to the lens 42 and captures an interference fringe image (interference intensity image) caused by the interference of the object light and the reference light arriving from the lens 42. The incident direction of the reference light is inclined relative to the incident direction of the object light onto the imaging surface of the imaging unit 43. The position where the object light and the reference light are combined by the beam splitter 41 can be further back than the imaging lens. However, considering the influence of aberrations, it is preferably between the objective lens 25 and the lens 42 as shown in the figure.

[0066] The analyzing unit 50 is electrically connected to the imaging unit 43, receives the interference intensity image captured by the imaging unit 43, and performs necessary processing based on the interference intensity image. Details of the analyzing unit 50 will be described later.

[0067] Figure 2 1B is a diagram showing the structure of the observation device 1B. Figure 2 The observation device 1B shown in FIG. Figure 1 In addition to the structure of the observation device 1A shown, a lens 31 , a mirror 32 , a lens 34 , and the like are further provided.

[0068] Lens 31 is optically connected to light output end 19 and collimates the light (reference light) output as divergent light from light output end 19. Mirror 32 is optically connected to lens 31 and reflects the light arriving from lens 31 toward lens 34. Lens 34 is optically connected to mirror 32 and outputs the light arriving from mirror 32 toward beam splitter 41.

[0069] The light output from lens 34 is temporarily focused in front of beam splitter 41 and then input into beam splitter 41 as divergent light. Beam splitter 41 combines the light output from objective lens 25 (object light) with the light output from lens 34 (reference light), and outputs both lights coaxially to lens 42. An imaging unit 43 captures an interference fringe image (interference intensity image) caused by the interference between the object light and the reference light that arrive from lens 42. The reference light is incident in a direction parallel to the direction of incidence of the object light on the imaging surface of imaging unit 43.

[0070] The drive unit 33 moves the mirror 32 in a direction perpendicular to the reflective surface of the mirror 32. The drive unit 33 is, for example, a piezoelectric actuator. The movement of the mirror 32 changes the difference (phase difference) in the optical path lengths between the object light and the reference light, from the optical branching in the fiber coupler 15 to the combined light in the beam splitter 41. If this optical path length difference varies, the interference intensity image captured by the imaging unit 43 will also vary.

[0071] Observation devices are not limited to Figure 1 and Figure 2 The structure example can be variously modified. Figure 1 ) and observation device 1B( Figure 2) is used as the object light, but the observation device 1C ( Figure 3 ) as the structure in which the light reflected by the observation object S is used as the object light.

[0072] Figure 3 1C is a diagram showing the structure of the observation device 1C. The observation device 1C includes a light source 11, a lens 12, a lens 21, a mirror 22, a lens 23, an objective lens 25, a beam splitter 41, a lens 42, a camera 43, and an analysis unit 50. Figure 1 ) Different points are explained.

[0073] Lens 21 is optically connected to light output end 18 of optical fiber 16 and collimates the light output as divergent light from light output end 18. Mirror 22 is optically connected to lens 21 and reflects light arriving from lens 21 toward lens 23. The orientation of the reflective surface of mirror 22 is variable. Lens 23 is optically connected to mirror 22. Objective lens 25 is optically connected to lens 23.

[0074] A beam splitter 41 is disposed between the lens 23 and the objective lens 25. The lens 23 and the objective lens 25 preferably form a 4f optical system. The lens 23 and the objective lens 25 irradiate the observation object S with light from a light irradiation direction corresponding to the orientation of the reflecting surface of the mirror 22. The objective lens 25 receives light (object light) reflected by the observation object S and outputs the light to the beam splitter 41.

[0075] The beam splitter 41 is optically connected to the objective lens 25 and is also optically connected to the light output end 19 of the optical fiber 17. The beam splitter 41 combines the light (object light) output from the objective lens 25 and arriving with the light (reference light) output from the light output end 19 and arriving, and outputs the two lights to the lens 42. The lens 42 is optically connected to the beam splitter 41, and collimates the object light and reference light arriving from the beam splitter 41, respectively, and outputs them to the imaging unit 43.

[0076] The imaging unit 43 is optically connected to the lens 42 and captures an interference fringe image (interference intensity image) caused by the interference of the object light and the reference light arriving from the lens 42. The incident direction of the reference light is inclined relative to the incident direction of the object light onto the imaging surface of the imaging unit 43. The position where the object light and the reference light are combined by the beam splitter 41 can be further back than the imaging lens. However, considering the influence of aberrations, it is preferably between the objective lens 25 and the lens 42 as shown in the figure.

[0077] In the observation device 1C ( Figure 3 ) structure, it is also possible to combine with the observation device 1B ( Figure 2 ) Similarly, a mechanism for changing the optical path length of the reference light is provided ( Figure 2By using the lens 31, mirror 32, drive unit 33, and lens 34 in the imaging unit 43, the difference (phase difference) in the optical path lengths of the object light and the reference light from the time the light is branched in the fiber coupler 15 to the time it is combined in the beam splitter 41 is changed. In this case, the incident direction of the reference light can be parallel to the incident direction of the object light onto the imaging surface of the imaging unit 43.

[0078] Figure 4 This figure shows the configuration of the analysis unit 50 of the observation apparatuses 1A to 1C. The analysis unit 50 includes an interference intensity image acquisition unit 51, a first complex amplitude image generation unit 52, a refractive index distribution image generation unit 53, a second complex amplitude image generation unit 54, a transfer matrix generation unit 55, a third complex amplitude image generation unit 56, a display unit 57, and a storage unit 58.

[0079] The analyzing unit 50 may be a computer and includes a processing device such as a CPU, a GPU, a DSP, or an FPGA, and a storage device such as a hard disk drive, a flash memory, a RAM, or a ROM.

[0080] The interference intensity image acquisition unit 51 irradiates the observation object S with light in a plurality of light irradiation directions by changing the orientation of the reflection surface of the mirror 22. The interference intensity image acquisition unit 51 also acquires an interference intensity image at a reference position from the imaging unit 43 for each of the plurality of light irradiation directions.

[0081] The interference intensity image acquisition unit 51 has an output port for outputting a control signal for changing the orientation of the reflecting surface of the mirror 22, and an input port for inputting the interference intensity image from the imaging unit 43. There is no need to move the objective lens 25 in the optical axis direction. The reference position is an image plane position that is conjugate with the imaging surface of the imaging unit 43.

[0082] The first complex amplitude image generator 52, the refractive index distribution image generator 53, the second complex amplitude image generator 54, the transfer matrix generator 55, and the third complex amplitude image generator 56 perform processing based on the interference intensity image. The display unit 57, which includes, for example, a liquid crystal display, displays images to be processed, images in progress, and images after processing.

[0083] The storage unit 58 stores various image data. The first complex amplitude image generator 52, the refractive index distribution image generator 53, the second complex amplitude image generator 54, the transfer matrix generator 55, the third complex amplitude image generator 56, the display unit 57, and the storage unit 58 may also be configured by cloud computing.

[0084] The storage unit 58 also stores a program for causing the interference intensity image acquisition unit 51, the first complex amplitude image generation unit 52, the refractive index distribution image generation unit 53, the second complex amplitude image generation unit 54, the transfer matrix generation unit 55, and the third complex amplitude image generation unit 56 to execute each process. This program may be stored in the storage unit 58 at the time of manufacture or shipment of the observation device, or may be stored in the storage unit 58 after shipment via a communication line. Alternatively, the program may be stored in the storage unit 58 after shipment from the factory, or may be stored in the storage unit 58 after being recorded on a computer-readable recording medium 2. The recording medium 2 may be any of a flexible disk, CD-ROM, DVD-ROM, BD-ROM, USB memory, and the like.

[0085] The first complex amplitude image generating unit 52, the second complex amplitude image generating unit 54, and the third complex amplitude image generating unit 56 all generate complex amplitude images of light, but differ in the content of the processing used to generate the complex amplitude images. Details of the processing performed by the interference intensity image acquiring unit 51, the first complex amplitude image generating unit 52, the refractive index distribution image generating unit 53, the second complex amplitude image generating unit 54, the transfer matrix generating unit 55, and the third complex amplitude image generating unit 56 will be described later.

[0086] Figure 5 This diagram outlines the observation method performed using observation apparatuses 1A to 1C. A first complex amplitude image generator 52 generates a first complex amplitude image based on the interference intensity images for each of the multiple light irradiation directions acquired by the interference intensity image acquirer 51. Furthermore, a transfer matrix generator 55 generates a transfer matrix T based on this first complex amplitude image.

[0087] The region containing the observation object is divided into multiple (three in the example shown) blocks by multiple surfaces perpendicular to the optical axis of lens 25 (in the example shown in the figure, the first end surface, the first intermediate surface, the second intermediate surface, and the second end surface). The transfer matrix for the first block between the first end surface and the first intermediate surface is T1, the transfer matrix for the second block between the first intermediate surface and the second intermediate surface is T2, and the transfer matrix for the third block between the second intermediate surface and the second end surface is T3. The overall transfer matrix T is approximately represented by the product of the transfer matrices T1, T2, and T3 (T3T2T1).

[0088] Utilizing this situation, the following process is repeated: First, for the first through third blocks, a refractive index distribution image is generated for the first block, which is then used to generate a complex amplitude image at the first intermediate plane. Next, a refractive index distribution image is generated for the second block, which is then used to generate a complex amplitude image at the second intermediate plane. This generates a refractive index distribution image with a high penetration depth.

[0089] Figure 6 This is a flow chart of an observation method. The observation method shown in this flow chart uses an observation device 1A ( Figure 1 ), Observation device 1B ( Figure 2 ) and observation device 1C( Figure 3 ) is acceptable in any of the cases. Figure 7 This is a diagram explaining the order of generating each image in the observation method.

[0090] This observation method generates a refractive index distribution image and includes an interference intensity image acquisition step S51, a first complex amplitude image generation step S52, a first refractive index distribution image generation step S53, a second complex amplitude image generation step S54, a transfer matrix generation step S55, and a third complex amplitude image generation step S56.

[0091] The interference intensity image acquisition step S51 is performed by the interference intensity image acquisition unit 51. The first complex amplitude image generation step S52 is performed by the first complex amplitude image generation unit 52. The first refractive index distribution image generation step S53 is performed by the refractive index distribution image generation unit 53. The second complex amplitude image generation step S54 is performed by the second complex amplitude image generation unit 54. The transfer matrix generation step S55 is performed by the transfer matrix generation unit 55. The third complex amplitude image generation step S56 is performed by the third complex amplitude image generation unit 56.

[0092] In the interference intensity image acquisition step S51, the interference intensity image acquisition unit 51 irradiates the observation object S along a plurality of light irradiation directions by changing the orientation of the reflection surface of the mirror 22. The interference intensity image acquisition unit 51 then acquires an interference intensity image at a reference position from the imaging unit 43 for each of the plurality of light irradiation directions.

[0093] exist Figure 1 、 Figure 2 and Figure 3 For the sake of convenience, the xyz orthogonal coordinate system is shown in FIG. The z axis is parallel to the optical axis of the objective lens 25. The reference position is the image plane position that is in a conjugate relationship with the imaging plane of the imaging unit 43. This position is set to z = 0. The direction of light irradiation to the observation object S can pass through the wave number vector (k x 、k y 、k z ) in k x and k y To express.

[0094] Figure 8 : is a diagram showing an example of scanning in the light irradiation direction toward the observation object S in the interference intensity image acquisition step S51. x , the vertical axis is set to k y K x k yIn the plane, the position of each circle represents the direction of light exposure.

[0095] The scanning direction of light irradiation is k x k y In a plane, the scan may be (a) discretely and periodically arranged in a rectangular lattice, (b) discretely and periodically arranged in a honeycomb shape, (c) discretely and periodically arranged in a hexagonal lattice, (d) discretely arranged on the circumference of multiple concentric circles, or (e) discretely arranged in a spiral shape.

[0096] In any case, as long as Figure 1 and Figure 2 The condenser lens 24 or Figure 3 As long as the NA of the objective lens 25 in the structure allows, scanning in the light irradiation direction can be performed. It can be any of raster scanning and random scanning. In the case of raster scanning, return scanning may be present or absent.

[0097] In the first complex amplitude image generating step S52 , the first complex amplitude image generating unit 52 generates a first complex amplitude image for each of the plurality of light irradiation directions based on the interference intensity image acquired by the interference intensity image acquiring unit 51 .

[0098] In the observation device 1A ( Figure 1 ) and observation device 1C( Figure 3 ), the first complex amplitude image generating unit 52 can generate the first complex amplitude image based on one interference intensity image by Fourier fringe analysis. Figure 2 ), the first complex amplitude image generating unit 52 can generate a first complex amplitude image based on three or more interference intensity images having different optical path length differences (phase differences) between the object light and the reference light using a phase shift method. The first complex amplitude image is a complex amplitude image of light when plane wave light is incident on one of the first end face and the second end face and reaches the other.

[0099] In the first refractive index distribution image generation step S53, the refractive index distribution image generation unit 53 generates a first refractive index distribution image for the section from the first end surface to the first intermediate surface based on the first complex amplitude images for each of the plurality of light irradiation directions generated by the first complex amplitude image generation unit 52. The details of the process for generating the refractive index distribution image will be described later.

[0100] In the second complex amplitude image generation step S54, the second complex amplitude image generation unit 54 generates, for each of the plurality of light irradiation directions, a second complex amplitude image of light when the plane wave light enters the first end facet and reaches the first intermediate facet, based on the first refractive index distribution image generated by the refractive index distribution image generation unit 53. The details of the process for generating the second complex amplitude image will be described later.

[0101] In the transfer matrix generation step S55, the transfer matrix generation unit 55 generates a transfer matrix T based on the first complex amplitude images for each of the plurality of light irradiation directions generated by the first complex amplitude image generation unit 52. Furthermore, the transfer matrix generation unit 55 generates a transfer matrix T1 based on the second complex amplitude images for each of the plurality of light irradiation directions generated by the second complex amplitude image generation unit 54. Details of the transfer matrix will be described later.

[0102] In the third complex amplitude image generation step S56, the third complex amplitude image generation unit 56 generates a complex amplitude image based on the product (TT1) of the transfer matrix T generated by the transfer matrix generation unit 55 and the inverse matrix of the transfer matrix T1. -1 =T3T2), for each of the plurality of light irradiation directions, a third complex amplitude image of light is generated when the plane wave light is incident on the first intermediate surface and reaches the second end surface.

[0103] After the third complex amplitude image generation step S56, the refractive index distribution image generation unit 53 may generate a refractive index distribution image for the section from the first intermediate surface to the second end surface based on the third complex amplitude images for each of the plurality of light irradiation directions generated by the third complex amplitude image generation unit 56. Alternatively, Figures 9 to 12 To illustrate, the refractive index distribution image generator 53 generates a second refractive index distribution image for the interval from the first intermediate surface to the second intermediate surface, and then generates a third refractive index distribution image for the interval from the second intermediate surface to the second end surface. By sequentially generating refractive index distribution images from the blocks on one end surface side in this manner, a refractive index distribution image with a high penetration depth can be generated.

[0104] Figure 9 This is a follow-up to the observation method. Figure 6 A flowchart of a processing example following the processing of the flowchart. Figure 10 This is a diagram explaining the order of generating each image in the observation method.

[0105] In the second refractive index distribution image generating step S63 , the refractive index distribution image generating unit 53 generates a second refractive index distribution image of the section from the first intermediate plane to the second intermediate plane based on the third complex amplitude images for each of the plurality of light irradiation directions generated by the third complex amplitude image generating unit 56 .

[0106] In the fourth complex amplitude image generating step S64 , the second complex amplitude image generating unit 54 generates, based on the second refractive index distribution image, a fourth complex amplitude image of light when the plane wave light enters the first intermediate surface and reaches the second intermediate surface for each of the plurality of light irradiation directions.

[0107] In the transfer matrix generation step S65, the transfer matrix generation unit 55 generates a transfer matrix T based on the third complex amplitude image of each of the plurality of light irradiation directions. 32 (=T3T2), and further, a transmission matrix T2 is generated based on the fourth complex amplitude images of each of the plurality of light irradiation directions.

[0108] In the fifth complex amplitude image generating step S66, the third complex amplitude image generating unit 56 generates the image based on the transfer matrix T 32 The product of the inverse matrix of the transmission matrix T2 (T 32 T2 -1 = T3), for each of the plurality of light irradiation directions, a fifth complex amplitude image of light is generated when the plane wave light is incident on the second intermediate surface and reaches the second end surface.

[0109] In the third refractive index distribution image generating step S67 , the refractive index distribution image generating unit 53 generates a third refractive index distribution image for the section from the second intermediate surface to the second end surface based on the fifth complex amplitude images for each of the plurality of light irradiation directions generated by the third complex amplitude image generating unit 56 .

[0110] Figure 11 This is a follow-up to the observation method. Figure 6 Flowchart of another processing example following the processing of the flowchart. Figure 12 This is a diagram explaining the order of generating each image in the observation method.

[0111] In the second refractive index distribution image generating step S73 , the refractive index distribution image generating unit 53 generates a second refractive index distribution image of the section from the first intermediate plane to the second intermediate plane based on the third complex amplitude images for each of the plurality of light irradiation directions generated by the third complex amplitude image generating unit 56 .

[0112] In the sixth complex amplitude image generating step S74, the second complex amplitude image generating unit 54 generates, based on the second refractive index distribution image, a sixth complex amplitude image of light when the light, represented by the second complex amplitude image generated by the second complex amplitude image generating unit 54, enters the first intermediate surface and reaches the second intermediate surface for each of the plurality of light irradiation directions.

[0113] In the transfer matrix generation step S75, the transfer matrix generation unit 55 generates a transfer matrix T based on the sixth complex amplitude image in each of the plurality of light irradiation directions. 21 (=T2T1).

[0114] In the seventh complex amplitude image generating step S76, the third complex amplitude image generating unit 56 generates the image based on the transfer matrix T and the transfer matrix T 21 The product of the inverse matrix (TT 21 -1 = T3), for each of the plurality of light irradiation directions, a seventh complex amplitude image of light is generated when the plane wave light is incident on the second intermediate surface and reaches the second end surface.

[0115] In the third refractive index distribution image generating step S77 , the refractive index distribution image generating unit 53 generates a third refractive index distribution image for the section from the second intermediate surface to the second end surface based on the seventh complex amplitude images for each of the plurality of light irradiation directions generated by the third complex amplitude image generating unit 56 .

[0116] Next, the details of the transmission matrix will be described. Figure 13 is the input light U when the interference intensity image is captured by the imaging unit. in (k in ) and output light u out (r out ) diagram. in (k in ) represents the wave number k of the light irradiating the observation object as a plane wave in The complex amplitude of u out (r out ) represents the position r of the light output from the observation object out The complex amplitude of .

[0117] U in (k in ) and u out (r out The relationship between ) is expressed by the following formula (1). in The nth element U in (k in n ) represents the wave number k in n The complex amplitude of the plane wave. The column vector u out The nth element u out (r out n ) indicates that at position r out n The complex amplitude of the observed light. The matrix T with N rows and N columns in this formula rk (r out ;k in ) represents U in (k in ) and u out (r outThe linear relationship between the two is called the transfer matrix. Through such a transfer matrix, the scattering process of light in the observed object can be represented. The matrix T rk (r out ;k in )'s element T at row n1 and column n2 n1 、 n2 Indicates that at wave number k in n2 When a plane wave with amplitude 1 is input at position r out n1 The complex amplitude of the observed light.

[0118] [Number 1]

[0119]

[0120] When the interference intensity image is captured by the imaging unit in each of the plurality of light irradiation directions, the vector U of the input light in the n-th light irradiation direction is in n (k in ) is represented by the following formula (2), where only the value of the nth element is 1 and the values ​​of the other elements are 0. in n (k in ), output light u out n (r out ) is represented by the following formula (3). This formula (3) corresponds to the complex amplitude obtained when the n-th light is irradiated in the direction.

[0121] [Number 2]

[0122]

[0123] [Number 3]

[0124]

[0125] Based on this equation (2) and the above equation (1), the following equation (4) is obtained. Then, the same equation is obtained for each of the multiple light irradiation directions, and the following equation (5) is obtained. In this way, the transfer matrix T can be obtained. rk (r out ;k in ).

[0126] [Number 4]

[0127]

[0128] [Number 5]

[0129]

[0130] The transmission matrix Trk (r out ;k in ) is the wave number basis and the output is the position basis. rk (r out ;k in ) is Fourier transformed, thereby generating a transfer matrix T with both input and output as position bases rr (r out ; r in ).

[0131] [Number 6]

[0132] T rr (r out ; r in )=∫T rk (r out ;k in )·exp(ik in r in )dk in (6)

[0133] The resulting transmission matrix T rk (r out ;k in ), T rr (r out ; r in ) has the following optical meaning (see non-patent documents 1 to 3).

[0134] The transfer matrix T whose input is a wavenumber basis and output is a position basis rk (r out ;k in ) is the wave number k on the first surface when there is no observation object (input) in The wavefront of the light is given to each position r on the second surface relative to the case where the observation object exists (output) out The transmittance matrix of the light wavefront.

[0135] The input and output sides are the transfer matrix T of the position basis rr (r out ; r in ) is the position r on the first surface when there is no observation object (input) in The wavefront of the light is given to each position r on the second surface relative to the presence of the observation object (output) out The transmittance matrix of the light wavefront.

[0136] When the third complex amplitude image generating unit 56 calculates the product of a certain transfer matrix and the inverse matrix of another certain transfer matrix, both are previously set to have a transfer matrix T with a wave number basis as input and a position basis as output. rk , or both are pre-set as the transfer matrix T with position basis for both input and output rr For example, when calculating the product of the transfer matrix T and the inverse matrix of the transfer matrix T1, the transfer matrix T1 is used as the transfer matrix at the first intermediate position based on the numerical calculation of the free propagation of light. The same applies to the product of other transfer matrices and their inverse matrices.

[0137] Complex differential interference image W(r out ) can be used using the transmission matrix T rr , is generated by the following formula (7) or (8). rr (r out ; r in ) will r out and r in T after shearing δr on both sides rr (r out -δr; r in -δr) is set as T rr * (r out -δr; r in -δr).

[0138] [Number 7]

[0139]

[0140] [Number 8]

[0141]

[0142] (7) for each position r on the second surface out , by finding T in the first face rr (r out ; r in ) and T rr * (r out -δr; r in The sum of the products of each element of -δr) generates the complex differential interference image W(r out ). This summation generates the output side spatially resolved complex differential interference image W(r out ). That is, there is no spatial resolution with respect to the input, but there is spatial resolution with respect to the output, and the sum is calculated with respect to the basis of the input.

[0143] This is similar to a system that simultaneously illuminates the entire object under observation (i.e., lacks spatial resolution in terms of input) and acquires a two-dimensional image of the object (i.e., has spatial resolution in terms of output). In this system, the acquired image is significantly affected by scatterers between the object under observation and the light-receiving unit.

[0144] (8) for each position r on the first surface in , by finding T in the second face rr (r out ; r in ) and T rr * (r out -δr; r in The sum of the products of each element of -δr) generates the complex differential interference image W(r in ). This summation generates the input-side spatially resolved complex differential interference image W(r in ). That is, with respect to the input, there is spatial resolution, but with respect to the output, there is no spatial resolution, and the sum is calculated for the basis of the output.

[0145] This is similar to a system that focuses light onto various locations on an object and scans the focused points (i.e., with spatial resolution for input), and then measures the total amount of light passing through the object at each focused point (i.e., with no spatial resolution for output). This system can capture an image of the object while suppressing the influence of scatterers between the object and the light-receiving unit.

[0146] At least one of the x component δx and the y component δy of δr is non-zero. If δx≠0 and δy=0, a complex differential interference image is obtained with the x direction as the shearing direction. If δx=0 and δy≠0, a complex differential interference image is obtained with the y direction as the shearing direction. If δx≠0 and δy≠0, a complex differential interference image is obtained with the direction corresponding to the ratio of δx to δy as the shearing direction.

[0147] Based on the output side spatially resolved complex differential interference image W n (r out ), the phase differential image is generated by the following formula (9). Alternatively, it can be based on the complex differential interference image W of the input side spatial resolution n (r in ), the phase differential image is generated by the following formula (10). Complex differential interference image W n (r out ), W n (r in ) is equivalent to the phase differential image.

[0148] [Number 9]

[0149] ∠W n (r out ) (9)

[0150] [Number 10]

[0151] ∠W n (r in ) (10)

[0152] Next, the details of the refractive index distribution image generation process performed by the refractive index distribution image generation unit 53 will be described. Figure 14 4 is a flowchart of the refractive index distribution image generation process.

[0153] In step S81 , the light wavefront is allowed to freely propagate at position z for each of a plurality of light irradiation directions, and a complex amplitude image u at position z is obtained.

[0154] For example, assuming that the light wavefront propagates freely from the position z = 0 to the position z = d, if the complex amplitude image at the position z = 0 is set to u(x, y, 0), and the two-dimensional Fourier transform of u(x, y, 0) is set to U(k x , k y , 0), let the complex amplitude image at the position z=d be u(x, y, d), and let the two-dimensional Fourier transform of u(x, y, d) be U(k x , k y , d), the free propagation can be calculated by processing including the following equations (11) and (12). i is an imaginary unit, and k0 is the wave number of light in the observation object.

[0155] [Number 11]

[0156]

[0157] [Number 12]

[0158] u n (r, d) = ∫U n (k in ,d)exp(-ik in r)dk in (12)

[0159] In step S82, a complex differential interference image W(r) is generated by the following equation (13). δr represents shearing. In step S83, a phase differential image represented by the phase of the complex differential interference image W(r) is generated.

[0160] [Number 13]

[0161]

[0162] In step S84, the value obtained by adding δz to z is set as the new value z. In step S85, it is determined whether the new value z has reached the final value z. end In step S85, it is determined that the new z has not reached the final value z end In the case of , return to step S81. In step S85, it is determined that the new z has reached the final value z end In this case, go to step S86.

[0163] From the initial value of z to the final value z end The processes from step S81 to step S85 are repeated at the δz scale. By this repeated process, the initial value of z to the final value z is generated. end The phase differential image at each z position of the δz scale, ie, the three-dimensional phase differential image, is generated. In step S86 proceeding from step S85, a refractive index distribution image is generated by deconvolution based on the three-dimensional phase differential image.

[0164] Next, the details of the complex amplitude image generation process performed by the second complex amplitude image generator 54 will be described. The first complex amplitude image generator 52 generates a complex amplitude image based on the interference intensity image, which serves as measured data. The third complex amplitude image generator 56 utilizes the transfer matrix to represent the complex amplitude image and generates the complex amplitude image based on calculations of the transfer matrix. In contrast, the second complex amplitude image generator 54 uses the refractive index distribution image generated by the refractive index distribution image generator 53 to numerically calculate the propagation of light in the medium represented by the refractive index distribution image, thereby generating a complex amplitude image, as described below.

[0165] By numerical calculation taking into account the refractive index distribution of each block, the first position (z=z j-1 ) propagates inside the block, and the second position (z = z j That is, for each of the plurality of light irradiation directions, based on the first position of the j-th block (z = z j-1 ) and the refractive index distribution of the jth block, and generate the second position (z = z j ) complex amplitude image.

[0166] This processing uses a method that numerically calculates the propagation of light wavefronts, taking into account the refractive index distribution of the medium. Known numerical calculation methods for propagation in inhomogeneous media include the Beam Propagation Method (BPM), the Split-Step Non-Paraxial (SSNP) Method, and the Multi-layer Born multiple-scattering model. The following describes processing using the BPM.

[0167] Figure 15 This diagram illustrates BPM processing. It shows a block. As shown, the block is divided into M slices (seven in this diagram) (the first through Mth slices) based on the distance from the imaging unit along the light propagation path (z direction). Each slice has a thickness of approximately the same wavelength.

[0168] The thickness of each slice can be constant. Here, the thickness of each slice is set to a constant value Δz. The mth slice among the first to Mth slices is the slice from position (z j-1 +(m-1)Δz) to position (z j-1 +mΔz). From the first position (z=z j-1 ) to the second position (z=z j ) sequentially imparts phase changes corresponding to the refractive index distribution in the first to Mth slices to propagate the light wavefront Δz.

[0169] The phase change o(x, y, z) imparted to the light wavefront when propagating through a slice of thickness Δz at position z is expressed by the following equation (14). v is the wave number of light in a vacuum. δn(x, y, z) is the difference between the refractive index distribution n(x, y, z) of the object under observation at position z and the refractive index n of the background (medium). b The difference of is expressed by the following formula (15). In addition, cosθ is expressed by the following formula (16).

[0170] [Number 14]

[0171]

[0172] [Number 15]

[0173] δn(x,y,z)=n((x,y,z)-n b (15)

[0174] [Number 16]

[0175]

[0176] If the position of the mth slice (z=z j-1 If the complex amplitude of light at position (z+(m-1)Δz) is u(x, y, z), then the complex amplitude of light at position (z+Δz) after the light propagates inside the mth slice is expressed by the following formula (17). P(k x , k y ;Δz) is expressed by the following formula (18). Formula (17) expresses the Fourier transform of the product of the complex amplitude u(x, y, z) of light and the phase change o(x, y, z), and the Fourier transform result is compared with P(k x , k y ; Δz) is inversely Fourier transformed to obtain the complex amplitude u(x, y, z+Δz) of the light at the position (z+Δz) after propagation in the slice of thickness Δz. Δz It is a function that performs calculations on light propagation in Δz.

[0177] [Number 17]

[0178]

[0179] [Number 18]

[0180]

[0181] The propagation of the light wavefront in each slice of the block is expressed by the following equations (19) to (21). That is, if the first position of the block (z = z j-1 ) is set as u(x, y, z j-1 ), then the complex amplitude u(x, y, z j-1 +Δz) is expressed by the following equation (19). If the complex amplitude of the light after propagating through the (m-1)th slice of the block is u(x, y, z j-1 +(m-1)Δz), then the complex amplitude u(x, y, z) of the light after propagation in the mth slice of the block is j-1 +mΔz) is expressed by the following equation (20). If the complex amplitude of the light after propagating through the (M-1)th slice of the block is u(x, y, z j-1 +(M-1)Δz), then the second position after propagation in the Mth slice of the block (z=z j ) is the complex amplitude u(x, y, z j ) is represented by the following formula (21).

[0182] [Number 19]

[0183] u(x,y,z k-1 +Δz)=P Δz [o(x,y,zj-1 )·u(x,y,z j-1 )] (19)

[0184] [Number 20]

[0185] u(x,y,z j-1 +mΔz)=P Δz [o(x,y,z j-1 +(m-1)Δz)·u(x,y,z j-1 +(m-1)Δz)] (20)

[0186] [Number 21]

[0187] u(x,y,z j )=P Δz [o(x,y,z j-1 +(M-1)Δz)·u(x,y,z j-1 +(M-1)Δz)] (20)

[0188] Thus, by numerical calculation taking into account the refractive index distribution of the block, the first position (z=z j-1 ) propagates sequentially within the block for each slice, and thus the second position (z = z j ) of the light wavefront.

[0189] Figure 16 This is a flowchart of the second complex amplitude image generation step S54. In step S91, the position z is initialized to the first position of the block (z=z j-1 In step S92, the interaction between the complex amplitude u(x, y, z) of the light at position z and the phase change o(x, y, z) is calculated. In step S93, the wavefront of the light after the interaction propagates a distance Δz, and the complex amplitude u(x, y, z+Δz) of the light at position z+Δz is calculated.

[0190] In step S94, z obtained by adding Δz is set as the new z. In step S95, if it is determined that the position z has not yet reached the second position of the block (z=z j ), then return to step S92 and repeat steps S92 to S94. In step S95, if it is determined that position z has reached the second position of the block (z=z j ), the processing of the second complex amplitude image generation step S54 is terminated. The complex amplitude of light obtained at the end becomes the second position of the block (z=z j ) is the complex amplitude at .

[0191] However, it is ideal that the number of light irradiation directions directed toward the observed object during the interference intensity image acquisition step S51 is equal to the number of pixels of the complex amplitude image. However, in reality, the number of light irradiation directions is less than the number of pixels of the complex amplitude image (undersampling). For example, if the number of pixels of the complex amplitude image is 1024×1024, it is not easy, even if possible, to achieve the same number of light irradiation directions. Alternatively, it is also possible to use only a portion of the image obtained by the imaging unit (the same number of pixels as the number of light irradiation directions) for subsequent processing, but this leads to a decrease in resolution and is therefore not preferred.

[0192] Find the transfer matrix T with both input and output as position basis rr (r out ; r in ) is calculated as the sum of the equation (22) in actual numerical calculation. When the number of light irradiation directions is less than the number of pixels in the complex amplitude image, if the scanning direction of the light irradiation direction to the observed object is as follows: Figure 8 (a) to (c) are shown in k x k y If it is discrete and periodic in the plane, then it is not an ideal transmission matrix T rr (r out ; r in ), instead, we get the transmission matrix T' shown in the following formula (23) rr (r out ; r in ). (23) In the formula, r p The periodic distribution of the positions of multiple light irradiation directions in the wave number space is equivalent to the inverse of its period. T' in formula (23) rr (r out ; r in ) is equivalent to making the ideal T rr (r out ; r in ) Repeat offset r p And add the values ​​obtained.

[0193] [Number 22]

[0194]

[0195] [Number 23]

[0196]

[0197] Figure 17 (a)k represents the case where the number of light irradiation directions is equal to the number of pixels of the complex amplitude image. x k yThe distribution of light irradiation directions in wave number space and (b) the transmission matrix T when focusing light on a certain point r0 rr (r out ; r in = r0) in the xy space. Figure 18 (a)k represents the case where the number of light irradiation directions is smaller than the number of pixels of the complex amplitude image. x k y The distribution of light irradiation directions in wave number space and (b) the transmission matrix T' when focusing light on a certain point r0 rr (r out ; r in = r0) in the xy space. Figure 18 r in px 、r py The periodic distribution of positions corresponding to a plurality of light irradiation directions in the wave number space is equivalent to the inverse of the period.

[0198] As shown in the figure, when the number of light irradiation directions is less than the number of pixels of the complex amplitude image, the output becomes multi-point irradiation. When using the transfer matrix T' of formula (23) rr (r out ; r in ) When the subsequent steps are processed, the results of multiple points in space are added together, and the complex differential interference image, phase differential image and refractive index distribution image cannot be correctly obtained.

[0199] Therefore, in the case of such undersampling, in the interference intensity image acquisition step S51, the interference intensity image acquisition unit 51 performs the following operations: Figure 8 As shown in (a) to (c), the photographing unit captures interference intensity images when light is irradiated to the observation object from multiple light irradiation directions whose positions of wave number vectors representing the light irradiation directions are discretely and periodically distributed in the wave number space, and obtains interference intensity images of each of the multiple light irradiation directions.

[0200] Then, in the third complex amplitude image generation step S56, the third complex amplitude image generation unit 56 calculates the product of the transfer matrix T and the inverse matrix of the transfer matrix T1 in a plurality of areas divided based on the periodic distribution of the positions of the wave number vectors of each of the plurality of light irradiation directions in the wave number space, and generates a third complex amplitude image for each of the plurality of light irradiation directions based on the products of each of the plurality of areas.

[0201] That is, in the case of undersampling, in the third complex amplitude image generating step S56, as shown in FIG. Figure 19 As shown, the transmission matrix T' rr (r out ; r in= r0) is divided into regions D(r in -r p Then, in these multiple regions D(r in -r p ) After respectively finding the product of the transfer matrix T and the inverse matrix of the transfer matrix T1, the in -r p ) are combined to obtain the product of the overall transfer matrix T and the inverse matrix of the transfer matrix T1 (the following formula (24)).

[0202] [Number 24]

[0203]

[0204] The observation device and observation method of this embodiment can divide the observation object into blocks by multiple surfaces perpendicular to the optical axis of the objective lens, and sequentially generate the refractive index distribution image of each block from the irradiation side or the shooting side. In the above description, the method of sequentially generating the refractive index distribution image of each block from the irradiation side is described. However, using Figure 20 For illustration, a method of sequentially generating refractive index distribution images of each block from the imaging side may be employed. Alternatively, a method of using a refractive index distribution image generated based on measurement data from a first period and generating a third complex amplitude image based on measurement data from another second period may be employed.

[0205] Figure 20 This diagram illustrates the order in which each image is generated in another embodiment of the observation method. During each of the first and second different periods, the interference intensity image acquisition unit 51 acquires interference intensity images for each of the multiple light irradiation directions from the imaging unit. The first complex amplitude image generation unit 52 generates first complex amplitude images for each of the multiple light irradiation directions based on the interference intensity images for each of the multiple light irradiation directions acquired by the interference intensity image acquisition unit 51.

[0206] The refractive index distribution image generating unit 53 generates a first refractive index distribution image for the section from the first end face to the first intermediate face based on the first complex amplitude images for each of the plurality of light irradiation directions during the first period generated by the first complex amplitude image generating unit 52. The second complex amplitude image generating unit 54 generates a second complex amplitude image for each of the plurality of light irradiation directions when plane wave light is incident on the first end face and reaches the first intermediate face based on the first refractive index distribution image generated by the refractive index distribution image generating unit 53.

[0207] The transfer matrix generator 55 generates a transfer matrix T based on the first complex amplitude images of each of the plurality of light irradiation directions in the second period. The transfer matrix generator 55 also generates a transfer matrix T1 based on the second complex amplitude images of each of the plurality of light irradiation directions in the first period.

[0208] The third complex amplitude image generator 56 generates, for each of the plurality of light irradiation directions, a third complex amplitude image of the plane wave light incident on the first intermediate surface and reaching the second end surface, based on the product of the transmission matrix T during the second period and the inverse matrix of the transmission matrix T1 during the first period. The refractive index distribution image generator 53 generates a second refractive index distribution image for the section from the first intermediate surface to the second intermediate surface, based on the third complex amplitude images for each of the plurality of light irradiation directions generated by the third complex amplitude image generator 56.

[0209] This observation method is effective, for example, when rapid time-delay scanning can be performed on the central region of the observation object, assuming that the overall refractive index distribution does not significantly change. By generating a refractive index distribution image for the region between the first end face and the first intermediate face based on measurement data from the first period, and precalculating the transfer matrix T1, a third amplitude image can be quickly generated based on the first complex amplitude image generated from measurement data from the second period and the transfer matrix T1.

[0210] Next, the simulation results are described. Figure 2 The measurement system shown is based on Figure 6 、 Figure 11 、 Figure 12 in the order shown.

[0211] In simulation A, if Figure 21 As shown in FIG. 1 , a simulation was performed using an object in which five types of phase images were arranged side by side at intervals as an observation object. Figure 21 Schematically illustrates the arrangement in simulation A. The first complex amplitude image generated by the first complex amplitude image generating unit 52 is a complex amplitude image of light that has passed through all five types of phase images.

[0212] The second complex amplitude image generated by the second complex amplitude image generating unit 54 is a complex amplitude image of light at a position having passed through three phase images on the light-irradiation side of the five types of phase images. The third complex amplitude image generated by the third complex amplitude image generating unit 56 is generated by calculation based on the transfer matrix of the first and second complex amplitude images described above, and corresponds to a complex amplitude image of light having passed through two phase images on the imaging side of the five types of phase images.

[0213] Figure 22 is a graph showing the results of simulation A. Figure 22In the figure, the top section represents the phase differential image of the exact solution, the second section represents the input side spatially resolved phase differential image generated based on the first complex amplitude image, the third section represents the input side spatially resolved phase differential image generated based on the second complex amplitude image, and the bottom section represents the input side spatially resolved phase differential image generated based on the third complex amplitude image.

[0214] The input-side spatially resolved phase differential image generated based on the first or second complex amplitude image produces a sharper phase differential image the further away from the imaging unit it is. Conversely, the input-side spatially resolved phase differential image generated based on the third complex amplitude image produces a sharper phase differential image the closer it is to the imaging unit. Simulation A demonstrates that generating a phase differential image based on any of the first, second, and third complex amplitude images enables observation of an object with a high penetration depth.

[0215] In simulation B, if Figure 23 As shown, the simulation was performed using an object simulating a cell aggregate as an observation object. Figure 23 This is a diagram schematically illustrating the configuration in simulation B. Figure 24 and Figure 25 These are graphs showing the results of simulation B.

[0216] exist Figure 24 In FIG, the upper part shows the input side spatially resolved phase differential image generated based only on the first complex amplitude image, and the lower part shows the input side spatially resolved phase differential image generated by the method of this embodiment. Figure 24 In FIG, the left end shows the phase differential image in the cross section parallel to the z axis, and the others show the phase differential images in the xy cross section at three positions in the z direction.

[0217] exist Figure 25 In the figure, the upper part shows the refractive index distribution image of the exact solution, the middle part shows the refractive index distribution image generated based on the input side spatially resolved phase differential image generated based only on the first complex amplitude image, and the lower part shows the refractive index distribution image generated based on the input side spatially resolved phase differential image generated by the method of this embodiment. Figure 25 In FIG. 1 , the left end represents the maximum value projection image in the y direction, and the others represent the refractive index distribution images in the xy cross section at three positions in the z direction.

[0218] Simulation B also shows that by generating a phase differential image based on any of the first complex amplitude image, the second complex amplitude image, and the third complex amplitude image, an observation object with a high penetration depth can be observed.

[0219] As described above, according to this embodiment, even when the observation object is a multiple scatterer, it is possible to observe the observation object with a good penetration depth and reduce the influence of multiple scattered light.

[0220] In the observation device and observation method described so far, the first branch light of one of the first branch light and the second branch light output from the light source and branched into two is received by the imaging unit as reference light without passing through the observation object, and the other second branch light is received by the imaging unit as object light after passing through the observation object. For each of the multiple light irradiation directions of the second branch light (object light), an interference intensity image caused by the interference between the first branch light (reference light) and the second branch light (object light) is captured, and a complex amplitude image is generated based on the interference intensity image.

[0221] Not limited to this, such as the following use Figure 26 and Figure 27 As described, the structure can also be such that, for both the first branch light and the second branch light output from the light source and branched into two, after passing through the observation object, the light is received by the shooting unit, and for each of the multiple light irradiation directions of the second branch light, an interference intensity image caused by the interference between the first branch light and the second branch light is shot, and a complex amplitude image is generated based on the interference intensity image.

[0222] Figure 26 1 is a diagram showing the structure of an observation device 100. The observation device 100 includes a light source 110, an irradiation unit 131, and an imaging unit 150. The light source 110 outputs spatially coherent light. The light output from the light source 110 may be temporally coherent or incoherent.

[0223] The light source 110 may be a laser light source, or may be a light source such as an SLD (Super Luminescent Diode), an SC (Super Continuum) light source, or an optical frequency comb light source. Alternatively, the light source 110 may be configured to enhance spatial coherence by passing spatially incoherent light output from an LED (Light Emitting Diode), a mercury lamp, or the like through a pinhole.

[0224] Lens 121 is optically connected to light source 110 and focuses light output from light source 110 onto light input end 122 of optical fiber 123, causing the light to enter light input end 122. Optical fiber 123 guides the light incident on light input end 122 to light output end 124. The light guided by optical fiber 123 is emitted from light output end 124 as divergent light. Lens 125 is optically connected to light output end 124 and receives and collimates the light output as divergent light from light output end 124, then outputs the collimated light to irradiation unit 131.

[0225] The irradiation unit 131 receives light output from the light source 110 and passing through the lens 121, the optical fiber 123, and the lens 125, and splits the input light into two, namely, first branched light and second branched light. The irradiation unit 131 overlaps the first branched light and the second branched light, and then irradiates the object S. The irradiation unit 131 irradiates the object S with the first branched light along a specific light irradiation direction and irradiates the object S with the second branched light along multiple light irradiation directions.

[0226] The irradiation unit 131 includes a beam splitter 311 , a phase modulation type spatial light modulator 313 , a polarizing plate 314 , a half wavelength plate 315 , a polarizing plate 316 , a lens 318 , and an objective lens 319 .

[0227] The beam splitter 311 reflects light arriving via the polarizer 314 and the half-wave plate 315 provided between the lens 125 toward the spatial light modulator 313 . The beam splitter 311 also receives light arriving from the spatial light modulator 313 and outputs the light toward the polarizer 316 .

[0228] The spatial light modulator 313 does not phase-modulate the linearly polarized light of the first orientation, while selectively phase-modulating the linearly polarized light of the second orientation, among the linearly polarized light of the first and second orientations that are orthogonal to each other and that are incident on the modulation surface. The polarizer 314 and the half-wave plate 315 set the polarization state of the light so that the light incident on the modulation surface of the spatial light modulator 313 from the beam splitter 311 contains the linearly polarized light components of the first and second orientations to the same extent.

[0229] The polarizer 316 receives light arriving from the spatial light modulator 313 via the beam splitter 311 and causes interference between the first and second linearly polarized lights contained in the light. The lens 318 and the objective lens 319 illuminate the first and second branched lights output from the polarizer 316 as plane waves, respectively, toward the observation object S.

[0230] The irradiation unit 131 having such a structure can use the linearly polarized light of the first orientation that has not been phase-modulated by the spatial light modulator 313 as the first branched light, and irradiate the observation object S along a certain light irradiation direction with the first branched light. The irradiation unit 131 can use the linearly polarized light of the second orientation that has been phase-modulated by the spatial light modulator 313 as the second branched light, and irradiate the observation object S along a plurality of light irradiation directions with the second branched light.

[0231] The direction in which the second branched light is irradiated onto the observation object S can be set by the orientation and spacing of the phase modulation pattern on the modulation surface of the spatial light modulator 313. Furthermore, the phase difference between the first branched light and the second branched light can be set by shifting the phase modulation pattern on the modulation surface of the spatial light modulator 313.

[0232] Alternatively, the phase difference may be adjusted according to the position of the spatial light modulator 313. However, setting the phase difference by shifting the phase modulation pattern on the modulation plane of the spatial light modulator 313 is preferable in that no components are mechanically moved.

[0233] The objective lens 141 receives light (first branched light and second branched light) emitted from the irradiation unit 131 toward the observation object S and passes through the observation object S, and outputs the light toward the mirror 142. The lens 143 receives light output from the objective lens 141 and reflected by the mirror 142, and causes the light to enter the imaging surface of the imaging unit 150.

[0234] The imaging unit 150 receives both the first branched light and the second branched light that reach the imaging surface from the lens 143 and captures an interference intensity image resulting from the interference between the first branched light and the second branched light. For each of the multiple light irradiation directions of the second branched light, the imaging unit 150 captures the interference intensity image when the phase difference between the first branched light and the second branched light is set to a plurality of phase differences. By performing desired processing based on the interference intensity image captured by the imaging unit 150, a complex amplitude image, etc., can be generated.

[0235] Using the observation device 100 ( Figure 26 ) is as follows. Figure 27 1 is a diagram schematically showing the incidence of the first branched light and the second branched light on the observation object S in the observation apparatus 100 , and the incidence of the first branched light and the second branched light on the imaging unit 150 after passing through the observation object S.

[0236] The irradiation unit 131 irradiates the observation object S with the first branched light and the second branched light in an overlapping manner. At this time, the irradiation direction of the first branched light relative to the observation object S is fixed, and the irradiation direction of the second branched light relative to the observation object S is each of a plurality of irradiation directions, and the phase difference between the first branched light and the second branched light is adjusted. Set to various values.

[0237] The wavefront of the first branched light incident on the observation object S is represented as u 0，in (r) The wavefront of the second branched light incident on the observation object S along the nth light irradiation direction (n=1 to N) among the plurality of (N) light irradiation directions of the second branched light is expressed as r is a variable representing position. The phase difference between the first branched light and the second branched light is expressed as u0(r) on the imaging plane or focal plane (optically conjugate to the imaging plane) of the imaging unit 150, and the wavefront of the second branched light is expressed as

[0238] The interference intensity image obtained by imaging by the imaging unit 150 By u0(r) and The square of the absolute value of the sum is expressed. Interference intensity image The phase difference between the first branched light and the second branched light is set to The interference intensity image is acquired by imaging by the imaging unit 150 when the first branched light is incident on the observation object S along a certain light irradiation direction relative to the observation object and the second branched light is incident on the observation object along the n-th light irradiation direction.

[0239] The focal plane (a plane optically conjugate to the imaging plane) may be located on the observation object S, closer to the imaging unit 150 than the observation object S, or closer to the irradiation unit 131 than the observation object S.

[0240] The multiple light irradiation directions of the second branch light are respectively based on the multiple phase differences set as The interference intensity image obtained by the imaging unit 150 at each time is used to obtain the interference term C by the phase shift method. n (r)=u0 * (r)·u n (r). The interference term u0(r)·u can be obtained n * (r) The interference term C is calculated for each of the plurality of light irradiation directions of the second branched light (ie, for each n (=1 to N)). n (r).

[0241] Based on the interference term C obtained for each of the multiple light irradiation directions of the second branch light n (r), generates a complex amplitude image of the first branched light. The phase of the complex amplitude u0(r) of the first branched light (r) After correcting the phase tilt (difference in light incident direction) between the first branch light and the second branch light, the coherent sum C of the corrected interference term can be obtained. sum (r), through the coherence and C sum The phase of (r) is approximately expressed.

[0242] The amplitude A0(r) of the complex amplitude u0(r) of the first branched light can illuminate the observation object S without irradiating the second branched light, and only the first branched light can illuminate the observation object S. According to the intensity image |u0(r)| captured by the imaging unit 150 2 Alternatively, the amplitude A0(r) of the complex amplitude u0(r) of the first branch light can be obtained by using the interference term C n (r) intensity and I sum The square root of (r) is approximately expressed.

[0243] Based on the phase of the complex amplitude u0(r) of the first branched light obtained as above and amplitude A0(r), a complex amplitude image u0(r) of the first branched light can be generated. Then, based on the complex amplitude image u0(r) of the first branched light and the interference term C n (r) It is possible to generate a complex amplitude image u of the second branched light in each of the multiple light irradiation directions n (r) The subsequent processing is the same as that already explained.

[0244] The observation apparatus and the observation method are not limited to the above-described embodiments and configuration examples, and various modifications are possible.

[0245] The observation device of the first mode of the above-mentioned embodiment comprises: (1) an interference intensity image acquisition unit, which acquires interference intensity images of the observation object irradiated with light along a plurality of light irradiation directions; (2) a first complex amplitude image generation unit, which generates, for each of the plurality of light irradiation directions, a first complex amplitude image of light when the plane wave light is incident on either the first end face or the second end face and reaches the other, based on the interference intensity image; (3) a refractive index distribution image generation unit, which generates a first refractive index distribution image of the interval from the first end face to the first intermediate face between the first end face and the second end face, based on the first complex amplitude image of each of the plurality of light irradiation directions; (4) a first complex amplitude image generation unit, which generates, for each of the plurality of light irradiation directions, a first complex amplitude image of light when the plane wave light is incident on either the first end face or the second end face and reaches the other; (2) a complex amplitude image generating unit, which generates, based on the first refractive index distribution image, a second complex amplitude image of the light when the plane wave light is incident on the first end surface and reaches the first intermediate surface for each of the multiple light irradiation directions; (5) a transmission matrix generating unit, which generates a transmission matrix T based on the first complex amplitude image of each of the multiple light irradiation directions, and generates a transmission matrix T1 based on the second complex amplitude image of each of the multiple light irradiation directions; and (6) a third complex amplitude image generating unit, which generates, based on the product of the transmission matrix T and the inverse matrix of the transmission matrix T1, a third complex amplitude image of the light when the plane wave light is incident on the first intermediate surface and reaches the second end surface for each of the multiple light irradiation directions.

[0246] In the observation device of the second mode, it can also be constructed as follows: in the structure of the first mode, the interference intensity image acquisition unit obtains the interference intensity images of each of the multiple light irradiation directions from the shooting unit that shoots the interference intensity images generated by the interference of light that irradiates the observation object along multiple light irradiation directions and passes through the observation object and the reference light.

[0247] In the observation device of the third type, it can also be constructed that, in the structure of the first type, the interference intensity image acquisition unit obtains the interference intensity image produced by the interference of light that is irradiated to the observation object along multiple light irradiation directions and passes through the observation object and light that is irradiated to the observation object along a certain light irradiation direction and passes through the observation object.

[0248] In the observation device of the fourth aspect, in any of the configurations of the first to third aspects, the refractive index distribution image generator may generate a refractive index distribution image of the section from the first intermediate surface to the second end surface based on the third complex amplitude images in each of the plurality of light irradiation directions.

[0249] In the observation device of the fifth embodiment, in any one of the structures of the first to fourth embodiments, the refractive index distribution image generating unit generates a second refractive index distribution image for the interval from the first intermediate surface to the second intermediate surface between the first intermediate surface and the second end surface based on the third complex amplitude image for each of the plurality of light irradiation directions; the second complex amplitude image generating unit generates, for each of the plurality of light irradiation directions, a fourth complex amplitude image of light when plane wave light is incident on the first intermediate surface and reaches the second intermediate surface based on the second refractive index distribution image; and the transfer matrix generating unit generates a transfer matrix T based on the third complex amplitude image for each of the plurality of light irradiation directions. 32 , and generates a transmission matrix T2 based on the fourth complex amplitude image of each of the plurality of light irradiation directions, and the third complex amplitude image generating unit generates a transmission matrix T based on the fourth complex amplitude image of each of the plurality of light irradiation directions. 32 The product of the inverse matrix of the transmission matrix T2 generates, for each of the plurality of light irradiation directions, a fifth complex amplitude image of light when the plane wave light enters the second intermediate surface and reaches the second end surface.

[0250] In the observation device of the sixth embodiment, it may be configured such that, in any of the structures of the first to fourth embodiments, the refractive index distribution image generating unit generates a second refractive index distribution image for a section from the first intermediate surface to the second intermediate surface between the first intermediate surface and the second end surface based on the third complex amplitude images for each of the plurality of light irradiation directions; the second complex amplitude image generating unit generates, based on the second refractive index distribution image, a sixth complex amplitude image of light when the light, represented by the second complex amplitude image, enters the first intermediate surface and reaches the second intermediate surface for each of the plurality of light irradiation directions; and the transfer matrix generating unit generates a transfer matrix T based on the sixth complex amplitude image for each of the plurality of light irradiation directions. 21The third complex amplitude image generating unit is based on the transfer matrix T and the transfer matrix T 21 The product of the inverse matrix of generates, for each of the plurality of light irradiation directions, a seventh complex amplitude image of light when the plane wave light enters the second intermediate surface and reaches the second end surface.

[0251] In the observation device of the seventh embodiment, it can also be configured so that, in any of the structures of the first to sixth embodiments, the interference intensity image acquisition unit acquires the interference intensity images of each of the multiple light irradiation directions in the first period and the second period respectively, the first complex amplitude image generation unit generates the first complex amplitude images of each of the multiple light irradiation directions based on the interference intensity images of each of the multiple light irradiation directions for the first period and the second period respectively, the refractive index distribution image generation unit generates the first refractive index distribution image based on the first complex amplitude images of each of the multiple light irradiation directions in the first period, the second complex amplitude image generation unit generates the second complex amplitude images of each of the multiple light irradiation directions based on the first refractive index distribution image, the transfer matrix generation unit generates the transfer matrix T based on the first complex amplitude images of each of the multiple light irradiation directions in the second period, and generates the transfer matrix T1 based on the second complex amplitude images of each of the multiple light irradiation directions in the first period, and the third complex amplitude image generation unit generates the third complex amplitude image of each of the multiple light irradiation directions based on the product of the transfer matrix T of the second period and the inverse matrix of the transfer matrix T1 of the first period.

[0252] In the observation device of the eighth embodiment, it can also be constructed as follows: in the structure of the second embodiment, the interference intensity image acquisition unit obtains interference intensity images for each of the multiple light irradiation directions from the capturing unit that captures interference intensity images when irradiating light to the observation object from multiple light irradiation directions whose positions are discretely and periodically distributed along the wave number vectors representing the light irradiation directions in the wave number space; the third complex amplitude image generation unit calculates the product of the transfer matrix T and the inverse matrix of the transfer matrix T1 in each of the multiple regions divided based on the periodic distribution of the positions of the wave number vectors of the multiple light irradiation directions in the wave number space, and generates third complex amplitude images for each of the multiple light irradiation directions based on the products of each of these multiple regions.

[0253] The observation method of the first mode of the above-mentioned embodiment comprises: (1) an interference intensity image acquisition step of acquiring interference intensity images of the observation object irradiated with light along a plurality of light irradiation directions; (2) a first complex amplitude image generation step of generating, for each of the plurality of light irradiation directions, a first complex amplitude image of light when a plane wave of light is incident on either the first end face or the second end face and reaches the other, based on the interference intensity image; (3) a refractive index distribution image generation step of generating a first refractive index distribution image of the interval from the first end face to the first intermediate face between the first end face and the second end face, based on the first complex amplitude image of each of the plurality of light irradiation directions; (4) a first complex amplitude image generation step of generating, for each of the plurality of light irradiation directions, a first complex amplitude image of light when the plane wave of light is incident on either the first end face or the second end face and reaches the other; (2) a step of generating a complex amplitude image, based on the first refractive index distribution image, generating, for each of a plurality of light irradiation directions, a second complex amplitude image of the light when the plane wave light is incident on the first end face and reaches the first intermediate face; (5) a step of generating a transmission matrix, based on the first complex amplitude image of each of the plurality of light irradiation directions, generating a transmission matrix T, and based on the second complex amplitude image of each of the plurality of light irradiation directions, generating a transmission matrix T1; and (6) a step of generating a third complex amplitude image, based on the product of the transmission matrix T and the inverse matrix of the transmission matrix T1, generating, for each of the plurality of light irradiation directions, a third complex amplitude image of the light when the plane wave light is incident on the first intermediate face and reaches the second end face.

[0254] In the observation method of the second embodiment, it can also be constructed as follows: in the structure of the first embodiment, in the interference intensity image acquisition step, interference intensity images of each of the multiple light irradiation directions are obtained from the shooting unit that shoots the interference intensity images generated by the interference of light that irradiates the observation object along multiple light irradiation directions and passes through the observation object and the reference light.

[0255] In the third-type observation method, it can also be constructed as follows: in the structure of the first type, in the interference intensity image acquisition step, an interference intensity image produced by the interference of light that is irradiated onto the observation object along multiple light irradiation directions and passes through the observation object and light that is irradiated onto the observation object along a certain light irradiation direction and passes through the observation object is acquired.

[0256] In the observation method of the fourth embodiment, it can also be configured that, in any structure of the first to third embodiments, in the refractive index distribution image generation step, a refractive index distribution image of the interval from the first intermediate surface to the second end surface is generated based on the third complex amplitude image of each of the multiple light irradiation directions.

[0257] In the observation method of the fifth embodiment, it may be configured such that, in any of the structures of the first to fourth embodiments, in the refractive index distribution image generating step, a second refractive index distribution image of the interval from the first intermediate surface to the second intermediate surface between the first intermediate surface and the second end surface is generated based on the third complex amplitude images for each of the plurality of light irradiation directions; in the second complex amplitude image generating step, a fourth complex amplitude image of light when plane wave light is incident on the first intermediate surface and reaches the second intermediate surface is generated for each of the plurality of light irradiation directions based on the second refractive index distribution image; and in the transfer matrix generating step, a transfer matrix T is generated based on the third complex amplitude images for each of the plurality of light irradiation directions. 32 , and based on the fourth complex amplitude images of each of the plurality of light irradiation directions, a transmission matrix T2 is generated. In the third complex amplitude image generation step, based on the transmission matrix T 32 The product of the inverse matrix of the transmission matrix T2 generates, for each of the plurality of light irradiation directions, a fifth complex amplitude image of light when the plane wave light enters the second intermediate surface and reaches the second end surface.

[0258] In the observation method of the sixth aspect, it may be configured such that, in any of the structures of the first to fourth aspects, in the refractive index distribution image generating step, a second refractive index distribution image of the interval from the first intermediate surface to the second intermediate surface between the first intermediate surface and the second end surface is generated based on the third complex amplitude images for each of the plurality of light irradiation directions; in the second complex amplitude image generating step, based on the second refractive index distribution image, a sixth complex amplitude image of light when the light enters the first intermediate surface and reaches the second intermediate surface, represented by the second complex amplitude image, is generated for each of the plurality of light irradiation directions; and in the transfer matrix generating step, a transfer matrix T is generated based on the sixth complex amplitude image for each of the plurality of light irradiation directions. 21 In the third complex amplitude image generation step, based on the transfer matrix T and the transfer matrix T 21 The product of the inverse matrix of generates, for each of the plurality of light irradiation directions, a seventh complex amplitude image of light when the plane wave light enters the second intermediate surface and reaches the second end surface.

[0259] In the observation method of the seventh embodiment, it can also be configured as follows: in any of the structures of the first to sixth embodiments, in the interference intensity image acquiring step, interference intensity images for each of the plurality of light irradiation directions are acquired in the first period and the second period, respectively; in the first complex amplitude image generating step, first complex amplitude images for each of the plurality of light irradiation directions are generated based on the interference intensity images for each of the plurality of light irradiation directions for the first period and the second period, in the refractive index distribution image generating step, a first refractive index distribution image is generated based on the first complex amplitude images for each of the plurality of light irradiation directions in the first period, in the second complex amplitude image generating step, a second complex amplitude image for each of the plurality of light irradiation directions is generated based on the first refractive index distribution image, in the transfer matrix generating step, a transfer matrix T is generated based on the first complex amplitude images for each of the plurality of light irradiation directions in the second period, and a transfer matrix T1 is generated based on the second complex amplitude images for each of the plurality of light irradiation directions in the first period, and in the third complex amplitude image generating step, a third complex amplitude image for each of the plurality of light irradiation directions is generated based on the product of the transfer matrix T for the second period and the inverse matrix of the transfer matrix T1 for the first period.

[0260] In the observation method of the eighth embodiment, it can also be constructed as follows: in the structure of the second embodiment, in the interference intensity image acquisition step, interference intensity images of each of the multiple light irradiation directions are obtained from a capturing unit that captures interference intensity images when irradiating light to the observation object in multiple light irradiation directions respectively distributed discretely and periodically along the positions of the wave number vectors representing the light irradiation directions in the wave number space; in the third complex amplitude image generation step, the product of the transfer matrix T and the inverse matrix of the transfer matrix T1 is calculated in each of the multiple regions divided based on the periodic distribution of the positions in the wave number space of the wave number vectors of the multiple light irradiation directions; and the third complex amplitude image of each of the multiple light irradiation directions is generated based on the products of each of these multiple regions.

[0261] The program of the above-described embodiment is a program for causing a computer to execute each step of the observation method having the above-described structure.

[0262] The recording medium of the above embodiment is a computer-readable recording medium storing the program having the above configuration.

[0263] Industrial applicability

[0264] The embodiment can be utilized as an observation apparatus and an observation method that can observe the observation object while reducing the influence of multiply scattered light even when the observation object is a multiply scatterer.

[0265] Description of Reference Signs

[0266] 1A to 1C…Observation device, 2…Recording medium, 11…Light source, 12…Lens, 13…Light incident end, 14…Optical fiber, 15…Fiber coupler, 16, 17…Optical fiber, 18, 19…Light output end, 21…Lens, 22…Mirror, 23…Lens, 24…Converging lens, 25…Objective lens, 31…Lens, 32…Mirror, 33…Drive unit, 34…Lens, 41…Beam splitter, 42…Lens, 43…Image capture unit, 50…Analysis unit, 51…Interference intensity image acquisition unit, 52…First complex amplitude image generation unit, 53…Refractive index distribution image generation unit, 54…Second complex amplitude image generation unit, 55…Transmission matrix generation unit, 56…Third complex amplitude image generation unit, 57…Display unit, 58…Storage unit, 100…Observation device, 110…Light source, 131…Illumination unit, 150…Image capture unit

Claims

1. An observation device, wherein: have: an interference intensity image acquisition unit that acquires interference intensity images of an observation object irradiated with light along a plurality of light irradiation directions; a first complex amplitude image generating unit for generating, for each of a plurality of light irradiation directions, a first complex amplitude image of light when plane wave light is incident on one of the first end face and the second end face and reaches the other; a refractive index distribution image generating unit configured to generate a first refractive index distribution image of a section from the first end surface to a first intermediate surface between the first end surface and the second end surface based on the first complex amplitude images in each of a plurality of light irradiation directions; a second complex amplitude image generating unit for generating, based on the first refractive index distribution image, a second complex amplitude image of light when plane wave light is incident on the first end face and reaches the first intermediate face for each of a plurality of light irradiation directions; a transmission matrix generating unit configured to generate a transmission matrix T based on the first complex amplitude image in each of a plurality of light irradiation directions, and to generate a transmission matrix T1 based on the second complex amplitude image in each of a plurality of light irradiation directions; and A third complex amplitude image generating unit generates, for each of a plurality of light irradiation directions, a third complex amplitude image of light when plane wave light is incident on the first intermediate surface and reaches the second end surface, based on the product of the transfer matrix T and the inverse matrix of the transfer matrix T1.

2. The observation device according to claim 1, wherein The interference intensity image acquisition unit acquires the interference intensity images for each of the plurality of light irradiation directions from a capturing unit that captures interference intensity images generated by interference between light irradiating the observation object along the plurality of light irradiation directions and passing through the observation object and reference light.

3. The observation device according to claim 1, wherein The interference intensity image acquisition unit acquires an interference intensity image generated by interference between light irradiated onto the observation object along a plurality of light irradiation directions and passing through the observation object and light irradiated onto the observation object along a certain light irradiation direction and passing through the observation object.

4. The observation device according to any one of claims 1 to 3, wherein The refractive index distribution image generating unit generates a refractive index distribution image of a section from the first intermediate surface to the second end surface based on the third complex amplitude image in each of a plurality of light irradiation directions.

5. The observation device according to any one of claims 1 to 4, wherein The refractive index distribution image generating unit generates a second refractive index distribution image of a section from the first intermediate surface to a second intermediate surface between the first intermediate surface and the second end surface based on the third complex amplitude image in each of a plurality of light irradiation directions. The second complex amplitude image generating unit generates, based on the second refractive index distribution image, a fourth complex amplitude image of light when plane wave light is incident on the first intermediate surface and reaches the second intermediate surface for each of a plurality of light irradiation directions. The transfer matrix generation unit generates a transfer matrix T based on the third complex amplitude image in each of the plurality of light irradiation directions. 32 , and generating a transmission matrix T2 based on the fourth complex amplitude image of each of the plurality of light illumination directions, The third complex amplitude image generating unit generates a complex amplitude image based on the transfer matrix T 32 The product of the inverse matrix of the transmission matrix T2 and the inverse matrix of the transmission matrix T2 generates a fifth complex amplitude image of the light when the plane wave light enters the second intermediate surface and reaches the second end surface for each of the plurality of light irradiation directions.

6. The observation device according to any one of claims 1 to 4, wherein The refractive index distribution image generating unit generates a second refractive index distribution image of a section from the first intermediate surface to a second intermediate surface between the first intermediate surface and the second end surface based on the third complex amplitude image in each of a plurality of light irradiation directions. The second complex amplitude image generating unit generates, based on the second refractive index distribution image, a sixth complex amplitude image of light when the light enters the first intermediate surface and reaches the second intermediate surface, for each of a plurality of light irradiation directions, represented by the second complex amplitude image. The transfer matrix generation unit generates a transfer matrix T based on the sixth complex amplitude image in each of the plurality of light irradiation directions. 21 , The third complex amplitude image generating unit generates a complex amplitude image based on the transfer matrix T and the transfer matrix T 21 The seventh complex amplitude image of the light when the plane wave light enters the second intermediate surface and reaches the second end surface is generated for each of the plurality of light irradiation directions by taking the product of the inverse matrices of .

7. The observation device according to any one of claims 1 to 6, wherein: The interference intensity image acquisition unit acquires the interference intensity images in each of a plurality of light irradiation directions during a first period and a second period, respectively. The first complex amplitude image generating unit generates the first complex amplitude image for each of the plurality of light irradiation directions based on the interference intensity image for each of the plurality of light irradiation directions for each of the first period and the second period. The refractive index distribution image generating unit generates the first refractive index distribution image based on the first complex amplitude images in each of the plurality of light irradiation directions during the first period. The second complex amplitude image generating unit generates the second complex amplitude image for each of a plurality of light irradiation directions based on the first refractive index distribution image. The transfer matrix generating unit generates a transfer matrix T based on the first complex amplitude image of each of the plurality of light irradiation directions during the second period, and generates a transfer matrix T1 based on the second complex amplitude image of each of the plurality of light irradiation directions during the first period. The third complex amplitude image generating unit generates the third complex amplitude image for each of a plurality of light irradiation directions based on the product of the transfer matrix T during the second period and the inverse matrix of the transfer matrix T1 during the first period.

8. The observation device according to claim 2, wherein The interference intensity image acquisition unit acquires the interference intensity image for each of the plurality of light irradiation directions from a capturing unit that captures the interference intensity image when irradiating the observation object with light in the plurality of light irradiation directions that are discretely and periodically distributed along wave number vectors representing the light irradiation directions in the wave number space. The third complex amplitude image generating unit calculates the product of the transfer matrix T and the inverse matrix of the transfer matrix T1 in each of a plurality of regions divided based on the periodic distribution of the positions of the wave number vectors of each of the plurality of light irradiation directions in the wave number space, and generates the third complex amplitude image for each of the plurality of light irradiation directions based on the product for each of these plurality of regions.

9. An observation method, wherein: have: an interference intensity image obtaining step of obtaining interference intensity images of the observation object irradiated with light along a plurality of light irradiation directions; a first complex amplitude image generating step of generating, for each of a plurality of light irradiation directions, a first complex amplitude image of light when plane wave light is incident on one of the first end face and the second end face and reaches the other; a refractive index distribution image generating step of generating a first refractive index distribution image of a section from the first end face to a first intermediate face between the first end face and the second end face based on the first complex amplitude images in each of a plurality of light irradiation directions; a second complex amplitude image generating step of generating, based on the first refractive index distribution image, a second complex amplitude image of light when plane wave light is incident on the first end surface and reaches the first intermediate surface for each of a plurality of light irradiation directions; a transmission matrix generating step of generating a transmission matrix T based on the first complex amplitude images of each of the plurality of light irradiation directions, and generating a transmission matrix T1 based on the second complex amplitude images of each of the plurality of light irradiation directions; and A third complex amplitude image generating step is performed, based on the product of the transmission matrix T and the inverse matrix of the transmission matrix T1, to generate a third complex amplitude image of the light when the plane wave light is incident on the first intermediate surface and reaches the second end surface for each of the multiple light irradiation directions.

10. The observation method according to claim 9, wherein: In the interference intensity image acquisition step, the interference intensity images of each of the multiple light irradiation directions are acquired from a shooting unit that captures interference intensity images generated by the interference of light that irradiates the observation object along multiple light irradiation directions and passes through the observation object and the reference light.

11. The observation method according to claim 9, wherein: In the interference intensity image acquisition step, an interference intensity image is obtained which is generated by the interference of light irradiated onto the observation object along multiple light irradiation directions and passing through the observation object and light irradiated onto the observation object along a certain light irradiation direction and passing through the observation object.

12. The observation method according to any one of claims 9 to 11, wherein In the refractive index distribution image generating step, a refractive index distribution image of a section from the first intermediate surface to the second end surface is generated based on the third complex amplitude image in each of a plurality of light irradiation directions.

13. The observation method according to any one of claims 9 to 12, wherein: In the refractive index distribution image generating step, a second refractive index distribution image of a section from the first intermediate surface to a second intermediate surface between the first intermediate surface and the second end surface is generated based on the third complex amplitude image in each of the plurality of light irradiation directions. In the second complex amplitude image generating step, based on the second refractive index distribution image, a fourth complex amplitude image of light when plane wave light is incident on the first intermediate surface and reaches the second intermediate surface is generated for each of a plurality of light irradiation directions. In the transfer matrix generation step, a transfer matrix T is generated based on the third complex amplitude image of each of the plurality of light irradiation directions. 32 , and generating a transmission matrix T2 based on the fourth complex amplitude image of each of the plurality of light illumination directions, In the third complex amplitude image generation step, based on the transfer matrix T 32 The product of the inverse matrix of the transmission matrix T2 and the inverse matrix of the transmission matrix T2 generates a fifth complex amplitude image of the light when the plane wave light enters the second intermediate surface and reaches the second end surface for each of the plurality of light irradiation directions.

14. The observation method according to any one of claims 9 to 12, wherein In the refractive index distribution image generating step, a second refractive index distribution image of a section from the first intermediate surface to a second intermediate surface between the first intermediate surface and the second end surface is generated based on the third complex amplitude image in each of the plurality of light irradiation directions. In the second complex amplitude image generating step, based on the second refractive index distribution image, a sixth complex amplitude image of light is generated for each of a plurality of light irradiation directions, which is represented by the second complex amplitude image and is obtained when the light enters the first intermediate surface and reaches the second intermediate surface. In the transfer matrix generation step, a transfer matrix T is generated based on the sixth complex amplitude image of each of the plurality of light irradiation directions. 21 , In the third complex amplitude image generation step, based on the transfer matrix T and the transfer matrix T 21 The seventh complex amplitude image of the light when the plane wave light enters the second intermediate surface and reaches the second end surface is generated for each of the plurality of light irradiation directions by taking the product of the inverse matrices of .

15. The observation method according to any one of claims 9 to 14, wherein In the interference intensity image acquisition step, the interference intensity images of each of the plurality of light irradiation directions are acquired in a first period and a second period respectively. In the first complex amplitude image generating step, the first complex amplitude images for each of the plurality of light irradiation directions are generated based on the interference intensity images for each of the plurality of light irradiation directions for each of the first period and the second period. In the refractive index distribution image generating step, the first refractive index distribution image is generated based on the first complex amplitude images in each of the plurality of light irradiation directions during the first period. In the second complex amplitude image generating step, the second complex amplitude images for each of a plurality of light irradiation directions are generated based on the first refractive index distribution image. In the transfer matrix generating step, a transfer matrix T is generated based on the first complex amplitude images of each of the plurality of light irradiation directions during the second period, and a transfer matrix T1 is generated based on the second complex amplitude images of each of the plurality of light irradiation directions during the first period. In the third complex amplitude image generating step, the third complex amplitude image for each of a plurality of light irradiation directions is generated based on the product of the transmission matrix T during the second period and the inverse matrix of the transmission matrix T1 during the first period.

16. The observation method according to claim 10, wherein: In the interference intensity image acquisition step, the interference intensity images for each of the plurality of light irradiation directions are acquired from a photographing unit that photographs the interference intensity images when irradiating light to the observation object in the plurality of light irradiation directions, each of which is discretely and periodically distributed along the positions of the wave number vectors representing the light irradiation directions in the wave number space. In the third complex amplitude image generation step, in each of a plurality of regions divided based on the periodic distribution of the positions of the wave number vectors of each of the plurality of light irradiation directions in the wave number space, the product of the transfer matrix T and the inverse matrix of the transfer matrix T1 is calculated, and the third complex amplitude image of each of the plurality of light irradiation directions is generated based on the product of each of these plurality of regions.

17. A program, wherein It is used to make a computer execute each step of the observation method according to any one of claims 9 to 16.

18. A computer-readable recording medium, wherein: The program according to claim 17 is stored.

Images