Observation device and observation method

Abstract

An observation device (1A) is provided with a light source (11), a mirror (22), a condenser lens (24), an objective lens (25), a beam splitter (41), an imaging unit (43), and an analysis unit (50). An analysis unit (50) applies light to an object (S) to be observed along each of a plurality of light irradiation directions by changing the orientation of a reflection surface of a mirror (22), acquires interference intensity images at a reference position from an imaging unit (43) for each of the plurality of light irradiation directions, and obtains a phase differential image of the object to be observed by performing a necessary process on the basis of the interference intensity images. As a result, it is possible to easily observe an object to be observed by reducing the influence of multiple scattered light even when the object to be observed is a multiple scatterer.

Description

Technical Field

[0001] The present invention relates to an observation device and an observation method. Background Art

[0002] In recent years, the technology for making three-dimensional cell tissues called spheroids and organoids has been advancing. In addition, research is underway to apply these three-dimensional cell tissues to drug development, regenerative medicine, etc. These three-dimensional cell tissues are optically transparent multiple scatterers. As a technology for imaging such optically transparent scatterers, various methods have been proposed so far.

[0003] Among them, examples of imaging techniques using fluorescent probes include confocal microscopy, multiphoton microscopy, and light sheet microscopy. Meanwhile, known non-staining / non-invasive imaging techniques that do not use fluorescent probes include optical coherence tomography (OCT).

[0004] While non-staining, non-invasive imaging is often required for observation objects such as spheroids and organoids, there are few reports of OCT being used for imaging these objects. This is due to the low resolution of OCT imaging and the difficulty in interpreting the signals obtained. Therefore, it can be said that a gold-standard non-staining three-dimensional tissue imaging technology has not yet been established.

[0005] Quantitative phase imaging (QPI) is a technology that can image the optical path length of an object in a non-staining and non-invasive manner. QPI can obtain physical information such as the optical path length of an object (e.g., cells), and its application in the biological field is growing. Images obtained through QPI can be used to generate other types of images, such as differential interference images and phase contrast microscopy images.

[0006] QPI is a technology that can produce images with a relatively high content of information, and is expected to be applicable to higher-content analysis than conventional analysis using brightfield images. Furthermore, with recent improvements in image recognition accuracy based on machine learning, research is actively underway into high-content analysis using non-staining imaging techniques, and non-staining imaging of multiple scatterers is expected to play a significant role in the future. However, since the images obtained by QPI are merely two-dimensional projections of the optical path length, they cannot capture the true three-dimensional structure.

[0007] Another known technology for imaging the optical path length of an object under observation in a non-staining, non-invasive manner is optical diffraction tomography (ODT), described in Patent Document 1. ODT develops QPI into a technology capable of three-dimensional imaging, enabling three-dimensional refractive index tomography of the object under observation.

[0008] By using ODT for cell observation, it is possible to identify cell organelles such as the nucleus and mitochondria, and to track three-dimensional morphological changes, which is expected to enable higher-level analysis than QPI.

[0009] The technology described in Non-Patent Document 1 can acquire a phase differential image of an observation object in a non-staining and non-invasive manner using point scanning, a Wollaston prism, and a phase shift method.

[0010] Prior art literature

[0011] Patent Literature

[0012] Patent Document 1: Japanese Patent Application Laid-Open No. 2017-219826

[0013] Non-patent literature

[0014] Non-patent literature 1: Xi Chen et al., "Artificial confocal microscopy for deeplabel-free imaging", arXiv:2110.14823, 2021,<URL:https: / / arxiv.org / ftp / arxiv / papers / 2110 / 2110.14823.pdf>

[0015] Non-patent literature 2: Pritam Pai et al., "Optical transmission matrix measurement sampled on a dense hexagonal lattice", OSA Continuum, Vol. 3, No. 3, pp. 637-648, 2020

[0016] Non-patent document 3: Duygu Akbulut et al., "Optical transmission matrix as a probe of the photonic strength", PHYSICAL REVIEW A 94, 043817, pp. 043817-1-043817-8, 2016

[0017] Non-patent document 4: Elbert G. van Putten et al., "The information age inoptics: Measuring the transmission matrix", Physics 3, 22, 2010,<URL:https: / / physics.aps.org / articles / v3 / 22?referer=apshome> Summary of the Invention

[0018] Technical problem to be solved by the invention

[0019] However, while conventional ODT can be applied to the observation of cells composed of multiple components, it is difficult to apply to the observation of multiple scatterers such as the three-dimensional cell tissues described above. This is because, with conventional ODT, when the observed object generates a large amount of multiple scattered light, the effect of the multiple scattered light is significantly reflected in the acquired image.

[0020] Light scattering refers to the phenomenon in which light interacts with an object, changing its direction of travel. In particular, when the spatial inhomogeneity of the object's refractive index increases, light interacts with the object multiple times while passing through it. Light that interacts with the object multiple times in this way is called multiply scattered light. Light that interacts with the object only once is called single scattered light. Multiply scattered light is known to increase speckle and deteriorate the single-scattering to multi-scattering ratio (SMR), hindering measurement.

[0021] When light is temporally and spatially coherent, interference from multiple scattered light causes large spatial variations in intensity or phase, resulting in speckle. To suppress speckle, a light source that outputs temporally or spatially incoherent light can be used. For example, conventional bright-field microscopes, such as phase-contrast microscopes, use spatially and temporally incoherent light sources, such as halogen lamps and light-emitting diodes, to produce speckle-free images.

[0022] SMR deterioration occurs because multiply scattered light dominates over single scattered light, burying it in the multiply scattered light. As the object becomes larger and the observation depth increases, the single scattered light component decreases exponentially, while the multiply scattered light component increases in contrast.

[0023] Because the scattering direction of single scattered light directly corresponds to the structure of the object, it is easy to use for measuring the object's structure. On the other hand, the relationship between multiple scattered light and the object's structure is complex, making it difficult to extract information about the object's structure. Therefore, in imaging techniques that utilize single scattered light, it is known that if the single scattered light is buried in the multiple scattered light (i.e., if the SMR deteriorates), the measurement will fail.

[0024] SMR degradation can be suppressed using a technique called light gating, which selectively detects single scattered light versus multiple scattered light. Because light gating can suppress multiple scattered light, speckle can also be suppressed while simultaneously suppressing SMR degradation. Light gating is achieved by exploiting spatial, temporal, and polarization degrees of freedom. Confocal microscopy is an example of spatial light gating. OCT is an example of temporal and spatial light gating.

[0025] Conventional ODT does not eliminate the effects of multiply scattered light. Therefore, when the observed object generates a large amount of multiply scattered light, speckle increases in the acquired image and deteriorates the SMR. Therefore, while conventional ODT can be used to observe cells composed of a few cells that generate little multiply scattered light, it is difficult to apply to the observation of multiple scatterers such as three-dimensional cell tissues that generate a large amount of multiply scattered light.

[0026] Although the technology described in Non-Patent Document 1 is applicable to the observation of multiple scatterers such as three-dimensional cell tissues that generate a large amount of multiple scattered light, it has the problem of requiring mechanical scanning in the optical axis direction of the objective lens to obtain a three-dimensional image, making measurement difficult.

[0027] An object of the embodiment is to provide an observation apparatus and an observation method that can reduce the influence of multiply scattered light and easily observe the observation object even when the observation object is a multiply scatterer.

[0028] Technical means to solve the problem

[0029] The embodiment is an observation device. The observation device includes: (1) an interference intensity image acquisition unit that acquires interference intensity images of an observation object irradiated with light along a plurality of light irradiation directions; (2) a complex amplitude image generation unit that generates a complex amplitude image based on the interference intensity image for each of the plurality of light irradiation directions; (3) a transfer matrix generation unit that generates a transfer matrix relative to each position r on the first surface when the observation object is not present, based on the complex amplitude image for each of the plurality of light irradiation directions. in The wavefront of the light is such that each position r on the second surface which is the same as the first surface when observing the object exists out The light wavefront establishes the associated transmission matrix T rr (rout ; r in ); (4) a complex differential interference image generating unit based on the transmission matrix T rr (r out ; r in ), for each position r on the first surface in , find T on the second side rr (r out ; r in ) and T rr *(r out -δr; r in -δr) to generate a complex differential interference image; and (5) a phase differential image generating unit that generates a phase differential image based on the complex differential interference image.

[0030] The embodiment is an observation method. The observation method includes: (1) an interference intensity image acquisition step, acquiring interference intensity images of the observation object irradiated with light along multiple light irradiation directions; (2) a complex amplitude image generation step, generating a complex amplitude image based on the interference intensity image for each of the multiple light irradiation directions; (3) a transfer matrix generation step, generating a transfer matrix relative to each position r on the first surface when the observation object is not present, based on the complex amplitude image of each of the multiple light irradiation directions. in The wavefront of the light is such that each position r on the second surface which is the same as the first surface when observing the object exists out The light wavefront establishes the associated transmission matrix T rr (r out ; r in ); (4) Complex differential interference image generation step, based on the transfer matrix T rr (r out ; r in ), for each position r on the first surface in , find T on the second side rr (r out ; r in ) and T rr *(r out -δr; r in -δr) to generate a complex differential interference image; and (5) a phase differential image generating step of generating a phase differential image based on the complex differential interference image.

[0031] The embodiment is a program. The program is a program for causing a computer to execute each step of the observation method having the above structure.

[0032] The embodiment is a recording medium. The recording medium is a computer-readable recording medium having the program having the above structure recorded thereon.

[0033] Effects of the Invention

[0034] According to the observation apparatus and the observation method of the embodiment, even when the observation object is a multiple scatterer, the influence of multiple scattered light can be reduced and the observation object can be easily observed. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a diagram showing the configuration of an observation apparatus 1A.

[0036] Figure 2 It is a diagram showing the structure of the observation apparatus 1B.

[0037] Figure 3 It is a diagram showing the structure of the observation apparatus 1C.

[0038] Figure 4 It is a diagram showing the configuration of the analysis unit 50 of the observation apparatuses 1A to 1C.

[0039] Figure 5 It is a flow chart of the observation method.

[0040] Figure 6 (a) to (e) are diagrams showing examples of scanning in the light irradiation direction toward the observation object S in the interference intensity image acquisition step S51.

[0041] Figure 7 is the input light U when the imaging unit captures the interference intensity image. in (k in ) and output light u out (r out ) picture.

[0042] Figure 8 is another flow chart of the observation method.

[0043] Figure 9 (a)k represents the case where the number of light irradiation directions is equal to the number of pixels of the complex amplitude image. x k y The distribution of light irradiation directions in the wave number space, and (b) the transmission matrix T when the light is focused on a certain point r0 rr (r out ; r in = r0) in the xy space.

[0044] Figure 10 (a)k represents the case where the number of light irradiation directions is smaller than the number of pixels of the complex amplitude image. x k y The distribution of light irradiation directions in the wave number space, and (b) the transmission matrix T' when the light is focused on a certain point r0 rr (r out ; rin = r0) in the xy space.

[0045] Figure 11 This is to explain the generation of a complex differential interference image W(r in ) method.

[0046] Figure 12 A diagram schematically illustrating the configuration in simulation A.

[0047] Figure 13 Graphs showing the results of simulation A.

[0048] Figure 14 Graph showing the results of simulation A.

[0049] Figure 15 This is a diagram schematically illustrating the configuration in simulation B.

[0050] Figure 16 Graphs showing the results of simulation B.

[0051] Figure 17 Graphs showing the results of simulation B.

[0052] Figure 18 It is a diagram showing the configuration of the observation apparatus 100 .

[0053] Figure 19 The diagram schematically shows the incidence of the first branched light and the second branched light on the observation object S in the observation apparatus 100 , and the incidence of the first branched light and the second branched light on the imaging unit 150 after passing through the observation object S. DETAILED DESCRIPTION

[0054] Embodiments of the observation device and observation method are described in detail below with reference to the accompanying drawings. In the description of the drawings, identical elements are denoted by identical reference numerals, and duplicate descriptions are omitted. The present invention is not limited to these examples and is intended to encompass all modifications within the meaning and scope of the claims and equivalents thereof.

[0055] Figure 1 1A is a diagram showing the configuration of an observation apparatus 1A, which includes a light source 11, a lens 12, a lens 21, a mirror 22, a lens 23, a condenser lens 24, an objective lens 25, a beam splitter 41, a lens 42, an imaging unit 43, and an analyzing unit 50.

[0056] Light source 11 outputs spatially and temporally coherent light, preferably a laser light source. Lens 12 is optically connected to light source 11 and focuses the light output from light source 11 onto light input end 13 of optical fiber 14, causing the light to enter light input end 13. Optical fiber 14 guides the light that has passed through lens 12 and entered light input end 13 to fiber coupler 15.

[0057] Fiber coupler 15 couples light between optical fiber 14 and optical fibers 16 and 17, branching light 2 guided by optical fiber 14. One branch of light is guided by optical fiber 16, and the other branch of light is guided by optical fiber 17. The light guided by optical fiber 16 is emitted as divergent light from light output end 18. The light guided by optical fiber 17 is emitted as divergent light from light output end 19.

[0058] Lens 21 is optically connected to light output end 18 and collimates the light output as divergent light from light output end 18. Mirror 22 is optically connected to lens 21 and reflects light arriving from lens 21 toward lens 23. The orientation of the reflective surface of mirror 22 is variable. Lens 23 is optically connected to mirror 22. Condenser lens 24 is optically connected to lens 23. Lens 23 and condenser lens 24 preferably form a 4f optical system.

[0059] The lens 23 and the condenser lens 24 irradiate light onto the observation object S from a light irradiation direction corresponding to the orientation of the reflection surface of the mirror 22. The objective lens 25 is optically connected to the condenser lens 24. The observation object S is arranged between the objective lens 25 and the condenser lens 24. The objective lens 25 receives light (object light) output from the condenser lens 24 and having passed through the observation object S, and outputs the light to the beam splitter 41.

[0060] The beam splitter 41 is optically connected to the objective lens 25 and is also optically connected to the light output end 19. The beam splitter 41 combines the light (object light) output from the objective lens 25 and the light (reference light) output from the light output end 19, and outputs the two lights to the lens 42. The lens 42 is optically connected to the beam splitter 41, and collimates the object light and reference light that have arrived from the beam splitter 41, respectively, and outputs them to the imaging unit 43.

[0061] The imaging unit 43 is optically connected to the lens 42 and captures the interference fringe image (interference intensity image) produced by the interference of the object light and the reference light arriving from the lens 42. The incident direction of the reference light is inclined relative to the incident direction of the object light onto the imaging surface of the imaging unit 43. The position where the object light and the reference light are combined by the beam splitter 41 can be later than the imaging lens, but considering the influence of aberrations, it is preferably between the objective lens 25 and the lens 42 as shown in the figure.

[0062] The analyzing unit 50 is electrically connected to the imaging unit 43, receives the interference intensity image captured by the imaging unit 43, and performs necessary processing based on the interference intensity image. Details of the analyzing unit 50 will be described later.

[0063] Figure 2 1B is a diagram showing the structure of the observation device 1B. Figure 2 The observation device 1B shown in FIG. Figure 1 The observation device 1A shown in the figure also includes a lens 31 , a mirror 32 , a lens 34 , and the like.

[0064] Lens 31 is optically connected to light output end 19 and collimates the light (reference light) output as divergent light from light output end 19. Mirror 32 is optically connected to lens 31 and reflects the light arriving from lens 31 toward lens 34. Lens 34 is optically connected to mirror 32 and outputs the light arriving from mirror 32 toward beam splitter 41.

[0065] The light output from lens 34 is temporarily focused in front of beam splitter 41 and then input into beam splitter 41 as divergent light. Beam splitter 41 combines the light output from objective lens 25 (object light) and the light output from lens 34 (reference light), and outputs the two lights coaxially to lens 42. An imaging unit 43 captures an interference fringe image (interference intensity image) produced by the interference of the object light and the reference light that have arrived from lens 42. The reference light is incident in a direction parallel to the direction of incidence of the object light on the imaging surface of imaging unit 43.

[0066] The drive unit 33 moves the mirror 32 in a direction perpendicular to the reflective surface of the mirror 32. The drive unit 33 is, for example, a piezoelectric actuator. The movement of the mirror 32 changes the difference (phase difference) in the optical path lengths of the object light and the reference light from the optical branching in the fiber coupler 15 to the combined light in the beam splitter 41. When this optical path length difference varies, the interference intensity image captured by the imaging unit 43 also varies.

[0067] Observation devices are not limited to Figure 1 and Figure 2 The structure example can be variously modified. Figure 1 ) and observation device 1B( Figure 2 ) is used as the object light, but the observation device 1C ( Figure 3 ) as the structure of the device, the light reflected by the observed object S is used as the object light.

[0068] Figure 31C is a diagram showing the structure of the observation device 1C. The observation device 1C includes a light source 11, a lens 12, a lens 21, a mirror 22, a lens 23, an objective lens 25, a beam splitter 41, a lens 42, a camera 43, and an analysis unit 50. The following mainly describes the observation device 1A ( Figure 1 ) are explained.

[0069] Lens 21 is optically connected to light output end 18 of optical fiber 16 and collimates the light output as divergent light from light output end 18. Mirror 22 is optically connected to lens 21 and reflects light arriving from lens 21 toward lens 23. The orientation of the reflective surface of mirror 22 is variable. Lens 23 is optically connected to mirror 22. Objective lens 25 is optically connected to lens 23.

[0070] A beam splitter 41 is disposed between the lens 23 and the objective lens 25. The lens 23 and the objective lens 25 preferably form a 4f optical system. The lens 23 and the objective lens 25 irradiate the observation object S with light from a light irradiation direction corresponding to the orientation of the reflecting surface of the mirror 22. The objective lens 25 receives light (object light) reflected by the observation object S and outputs the light to the beam splitter 41.

[0071] The beam splitter 41 is optically connected to the objective lens 25 and is also optically connected to the light output end 19 of the optical fiber 17. The beam splitter 41 combines the light (object light) output from the objective lens 25 and the light (reference light) output from the light output end 19, and outputs the two lights to the lens 42. The lens 42 is optically connected to the beam splitter 41, and collimates the object light and reference light that have arrived from the beam splitter 41, respectively, and outputs them to the imaging unit 43.

[0072] The imaging unit 43 is optically connected to the lens 42 and captures the interference fringe image (interference intensity image) produced by the interference of the object light and the reference light arriving from the lens 42. The incident direction of the reference light is inclined relative to the incident direction of the object light onto the imaging surface of the imaging unit 43. The position where the object light and the reference light are combined by the beam splitter 41 can also be later than the imaging lens, but considering the influence of aberrations, it is preferably between the objective lens 25 and the lens 42 as shown in the figure.

[0073] In the observation device 1C ( Figure 3 ) structure, it is also possible to combine with the observation device 1B ( Figure 2 ) Similarly, a mechanism for changing the optical path length of the reference light is provided ( Figure 2 By using the lens 31, mirror 32, drive unit 33, and lens 34 in the optical fiber coupler 15, the difference in optical path length (phase difference) between the object light and the reference light from the time the light is branched in the optical fiber coupler 15 to the time it is combined in the beam splitter 41 is changed. In this case, the incident direction of the reference light can be parallel to the incident direction of the object light onto the imaging surface of the imaging unit 43.

[0074] Figure 4 This figure shows the configuration of the analysis unit 50 of the observation apparatuses 1A to 1C. The analysis unit 50 includes an interference intensity image acquisition unit 51, a complex amplitude image generation unit 52, a transfer matrix generation unit 53, a complex differential interference image generation unit 54, a phase differential image generation unit 55, a refractive index distribution image generation unit 56, a display unit 57, and a storage unit 58.

[0075] The analyzing unit 50 may be a computer and includes a processing device such as a CPU, a GPU, a DSP, or an FPGA, and a storage device such as a hard disk drive, a flash memory, a RAM, or a ROM.

[0076] The interference intensity image acquisition unit 51 irradiates the observation object S with light in each of a plurality of light irradiation directions by changing the orientation of the reflection surface of the mirror 22. The interference intensity image acquisition unit 51 also acquires interference intensity images at the reference position from the imaging unit 43 for each of the plurality of light irradiation directions.

[0077] The interference intensity image acquisition unit 51 has an output port for outputting a control signal for changing the orientation of the reflecting surface of the mirror 22, and an input port for inputting the interference intensity image from the imaging unit 43. There is no need to move the objective lens 25 in the optical axis direction. The reference position is an image plane position that is conjugate with the imaging surface of the imaging unit 43.

[0078] The complex amplitude image generator 52, transfer matrix generator 53, complex differential interference image generator 54, phase differential image generator 55, and refractive index distribution image generator 56 perform processing based on the interference intensity image. The display unit 57, which includes, for example, a liquid crystal display, displays the image to be processed, an image in progress, and the image after processing.

[0079] The storage unit 58 stores various image data. The complex amplitude image generator 52, the transfer matrix generator 53, the complex differential interference image generator 54, the phase differential image generator 55, the refractive index distribution image generator 56, the display unit 57, and the storage unit 58 may also be configured by cloud computing.

[0080] The storage unit 58 also stores programs for causing the interference intensity image acquisition unit 51, the complex amplitude image generation unit 52, the transfer matrix generation unit 53, the complex differential interference image generation unit 54, the phase differential image generation unit 55, and the refractive index distribution image generation unit 56 to execute their respective processes. These programs may be stored in the storage unit 58 at the time of manufacture or shipment of the observation device, or may be stored in the storage unit 58 after shipment via a communication line. Alternatively, the programs recorded on the computer-readable recording medium 2 may be stored in the storage unit 58. The recording medium 2 may be any recording medium such as a floppy disk, CD-ROM, DVD-ROM, BD-ROM, or USB memory.

[0081] Details of the processing by the interference intensity image acquisition unit 51 , complex amplitude image generation unit 52 , transfer matrix generation unit 53 , complex differential interference image generation unit 54 , phase differential image generation unit 55 , and refractive index distribution image generation unit 56 will be described later.

[0082] Figure 5 This is a flow chart of an observation method. The observation method shown in this flow chart uses an observation device 1A ( Figure 1 ), Observation device 1B ( Figure 2 ) and observation device 1C( Figure 3 ) can be performed in any of the cases.

[0083] The observation method generates a phase differential image, including an interference intensity image acquisition step S51, a complex amplitude image generation step S52, a transfer matrix generation step S53, a complex differential interference image generation step S54, and a phase differential image generation step S55.

[0084] The interference intensity image acquisition step S51 is performed by the interference intensity image acquisition unit 51. The complex amplitude image generation step S52 is performed by the complex amplitude image generation unit 52. The transfer matrix generation step S53 is performed by the transfer matrix generation unit 53. The complex differential interference image generation step S54 is performed by the complex differential interference image generation unit 54. The phase differential image generation step S55 is performed by the phase differential image generation unit 55.

[0085] In the interference intensity image acquisition step S51, the interference intensity image acquisition unit 51 irradiates the observation object S with light in each of a plurality of light irradiation directions by changing the orientation of the reflection surface of the mirror 22. The interference intensity image acquisition unit 51 then acquires an interference intensity image at a reference position from the imaging unit 43 for each of the plurality of light irradiation directions.

[0086] exist Figure 1 、 Figure 2 and Figure 3 In each of the figures, for the sake of convenience, an xyz orthogonal coordinate system is shown. The z axis is parallel to the optical axis of the objective lens 25. The reference position is the image plane position that is in a conjugate relationship with the imaging plane of the imaging unit 43. This position is set to z = 0. The direction of light irradiation to the observation object S can be determined by the wave number vector (k x , k y , k in kz) x and k y express.

[0087] Figure 6 : is a diagram showing an example of scanning in the light irradiation direction toward the observation object S in the interference intensity image acquisition step S51.x , the vertical axis is set to k y K x k y In the plane, the position of each circle represents the direction of light exposure.

[0088] The scanning direction of light irradiation is k x k y In the plane, (a) can be a scan that is discretely and periodically arranged in a rectangular lattice shape, (b) can be a scan that is discretely and periodically arranged in a honeycomb shape, (c) can be a scan that is discretely and periodically arranged in a hexagonal lattice shape, (d) can be a scan that is discretely arranged on the circumference of each of multiple concentric circles, and (e) can also be a scan that is discretely arranged in a spiral shape.

[0089] In either case, as long as Figure 1 and Figure 2 The condenser lens 24 or Figure 3 The NA of the objective lens 25 in the structure allows scanning in the light irradiation direction. It can be either raster scanning or random scanning. In the case of raster scanning, return scanning may or may not exist.

[0090] In the complex amplitude image generation step S52, the complex amplitude image generation unit 52 generates a complex amplitude image for each of the plurality of light irradiation directions based on the interference intensity image acquired by the interference intensity image acquisition unit 51. Figure 1 ) and observation device 1C( Figure 3 ), the complex amplitude image generating unit 52 can generate a complex amplitude image based on one interference intensity image using the Fourier fringe analysis method. Figure 2 ), the complex amplitude image generating unit 52 can generate a complex amplitude image based on three or more interference intensity images having different optical path length differences (phase differences) between the object light and the reference light using a phase shift method.

[0091] In the transfer matrix generation step S53 , the transfer matrix generation unit 53 generates a transfer matrix as follows based on the complex amplitude images for each of the plurality of light irradiation directions generated by the complex amplitude image generation unit 52 .

[0092] Figure 7 is the input light U when the imaging unit captures the interference intensity image. in (k in ) and output light u out (r out ) diagram. in (k in ) represents the wave number k of the light irradiating the observation object as a plane wave in The complex amplitude of u out(r out ) represents the position r of the light output from the observation object out The complex amplitude of .

[0093] U in (k in ) and u out (r out The relationship between ) is expressed by the following formula (1). in The nth element U in (k in n ) represents the wave number k in n The complex amplitude of the plane wave. The column vector u out The nth element u out (r out n ) means at position r out n The complex amplitude of the observed light. The matrix T with N rows and N columns in this formula rk (r out ;k in ) represents U in (k in ) and u out (r out The linear relationship between the two is called the transfer matrix. Using such a transfer matrix, the scattering process of light in the object under observation can be represented. rk (r out ;k in )'s element T at row n1 and column n2 n1,n2 Indicates that at wave number k in n2 When a plane wave with amplitude 1 is input at position r out n1 The complex amplitude of the observed light.

[0094] [Number 1]

[0095]

[0096] For each of the plurality of light irradiation directions, the vector U of the input light in the n-th light irradiation direction when the interference intensity image is captured by the imaging unit is in n (k in ) is represented by the following formula (2), where only the value of the nth element is 1 and the values ​​of the other elements are 0. in n (k in ), output light u out n (r out) is represented by the following formula (3). This formula (3) corresponds to the complex amplitude obtained when the n-th light is irradiated in the direction.

[0097] [Number 2]

[0098]

[0099] [Number 3]

[0100]

[0101] The following equation (4) is obtained from equation (2) and equation (1). Then, if the same equation is obtained for each of the multiple light irradiation directions, the following equation (5) is obtained. In this way, the transfer matrix T can be obtained. rk (r out ;k in ).

[0102] [Number 4]

[0103]

[0104] [Number 5]

[0105]

[0106] The transmission matrix T rk (r out ;k in ) is the wave number basis and the output is the position basis. According to the following formula (6), the input to the transmission matrix T rk (r out ;k in ) is Fourier transformed, thereby generating a transmission matrix T with both input and output as position basis rr (r out ; r in ).

[0107] [Number 6]

[0108] T rr (r out ; r in )=∫T rk (r out ;k in )·exp(ik in r in )dk in (6)

[0109] The resulting transmission matrix T rk (r out ;k in ), T rr (r out ; rin ) has the following optical meaning (refer to non-patent documents 2 to 4).

[0110] The transfer matrix T whose input is a wavenumber basis and output is a position basis rk (r out ;k in ), is the wave number k on the first surface when there is no observation object (input) in The wavefront of the light is given relative to each position r on the second surface when the observation object exists (output) out The matrix of the transmittance of the light wavefront.

[0111] The input and output are both position-based transmission matrices T rr (r out ; r in ), is the position r on the first surface when there is no observation object (input) in The wavefront of the light is given relative to each position r on the second surface when the observation object exists (output) out The matrix of the transmittance of the light wavefront.

[0112] Complex differential interference image W(r out ) can be generated by the following formula (7). rr (r out ; r in ) will r out and r in Both cut the T of δr rr (r out -δr; r in -δr) is set as T rr *(r out -δr; r in -δr). For each position r on the second surface out , find T on the first face rr (r out ; r in ) and T rr *(r out -δr; r in The complex differential interference image W(r) is generated by summing the products of each element of out ).

[0113] [Number 7]

[0114]

[0115] This summation generates the spatially resolved complex differential interference image W(r out). That is, there is no spatial resolution with respect to the input, but there is spatial resolution with respect to the output, and the sum is taken with respect to the basis of the input.

[0116] This is similar to a system that simultaneously irradiates the entire object under observation with light (i.e., has no spatial resolution for the input) and obtains a two-dimensional image of the object under observation (i.e., has spatial resolution for the output). In this system, the influence of scatterers between the object under observation and the light receiving unit is greatly apparent in the obtained image.

[0117] In this embodiment, in the complex differential interference image generation step S54, the complex differential interference image generation unit 54 generates the complex differential interference image based on the transfer matrix T generated by the transfer matrix generation unit 53. rr (r out ; r in ), using the following formula (8), for each position r on the first surface in , find T on the second side rr (r out ; r in ) and T rr *(r out -δr; r in The complex differential interference image W(r) is generated by summing the products of each element of in ).

[0118] [Number 8]

[0119]

[0120] At least one of the x component δx and the y component δy of δr is non-zero. If δx≠0, δy=0, a complex differential interference image W(r) with the x direction as the shearing direction can be obtained. in ). If δx=0, δy≠0, then the complex differential interference image W(r) with the y direction as the shear direction can be obtained. in If δx≠0, δy≠0, a complex differential interference image W(r) can be obtained with the direction corresponding to the ratio of δx and δy as the shear direction. in ).

[0121] The summation calculation in the complex differential interference image generation step S54 obtains the input side spatially resolved complex differential interference image W(r in ). That is, there is spatial resolution with respect to the input but no spatial resolution with respect to the output, and the sum is taken over the basis of the output.

[0122] This is similar to a system that focuses light onto various locations on an object and scans the focused points (i.e., has spatial resolution for input), and then measures the total amount of light that has passed through the object at each focused point (i.e., has no spatial resolution for output). This system can capture an image of the object while suppressing the influence of scatterers between the object and the light-receiving unit.

[0123] In the phase differential image generation step S55, the phase differential image generation unit 55 generates a phase differential image based on the complex differential interference image W(r in ), the phase differential image is generated using the following formula (9). Complex differential interference image W(r in ) is equivalent to the phase differential image.

[0124] [Number 9]

[0125] ∠W(r in ) (9)

[0126] Figure 8 This is another flow chart of the observation method. The observation method shown in this flow chart uses the observation device 1A ( Figure 1 ), Observation device 1B ( Figure 2 ) and observation device 1C( Figure 3 ) can be performed in any of the cases.

[0127] This observation method generates phase differential images of multiple positions in the z direction, and then generates a three-dimensional refractive index distribution image. In addition to the interference intensity image acquisition step S51, the complex amplitude image generation step S52, the transfer matrix generation step S53, the complex differential interference image generation step S54 and the phase differential image generation step S55, it also includes a refractive index distribution image generation step S56 and steps S57 to S59.

[0128] After the interference intensity image acquisition step S51 and the complex amplitude image generation step S52 , in step S57 , the light wavefront is allowed to freely propagate at position z for each of the plurality of light irradiation directions, and a complex amplitude image u at position z is obtained.

[0129] For example, assuming that the light wavefront propagates freely from the position z = 0 to the position z = d, if the complex amplitude image at the position z = 0 is set to u(x, y, 0), and the two-dimensional Fourier transform of u(x, y, 0) is set to U(k x , k y , 0), let the complex amplitude image at the position z=d be u(x, y, d), and let the two-dimensional Fourier transform of u(x, y, d) be U(k x , k y, d), the free propagation can be calculated by processing including the following equations (10) and (11). i is an imaginary unit, and k0 is the wave number of light in the observation object.

[0130] [Number 10]

[0131]

[0132] [Number 11]

[0133] u(x,y,d)=∫U(k x ,k y ,d)exp(-ik x x-ik y y)dk x dk y (11)

[0134] After step S57, the processes of the transfer matrix generation step S53 and the complex differential interference image generation step S54 are performed. Then, in step S58, the value obtained by adding δz to z is used as the new value z. In step S59, it is determined whether the new value z has reached the final value z. end In step S59, it is determined that the new z has not reached the final value z. end In the case of , return to step S57. In step S59, it is determined that the new z has reached the final value z end In the case of , proceed to the phase differential image generation step S55.

[0135] Each process from step S57 to step S59 is from the initial value of z to the final value z end This process is repeated at δz scale. By this repeated process, the complex differential interference image generation unit 54 generates the image from the initial value of z to the final value z end The complex differential interference image W(r in , z), that is, three-dimensional complex differential interference image.

[0136] In the phase differential image generation step S55 proceeding from step S59, the phase differential image generation unit 55 generates a phase differential image from the initial value of z to the final value z. end Then, in the refractive index distribution image generating step S56, the refractive index distribution image generating unit 56 generates a three-dimensional refractive index distribution image of the observation object by deconvolution based on the three-dimensional phase differential image generated by the phase differential image generating unit 55.

[0137] In addition, it is ideal that the number of light irradiation directions directed toward the observed object during the interference intensity image acquisition step S51 is equal to the number of pixels in the complex amplitude image. However, in reality, the number of light irradiation directions is less than the number of pixels in the complex amplitude image (undersampling). For example, if the number of pixels in the complex amplitude image is 1024×1024, even if the same number of light irradiation directions can be achieved, it is not easy to achieve. Alternatively, it is also considered to use only a portion of the image obtained by the imaging unit in subsequent processing (the same number of pixels as the number of light irradiation directions), but this will lead to a decrease in resolution and is not preferred.

[0138] Obtain the transfer matrix T with both input and output as position basis rr (r out ; r in ) is calculated as the sum of the equation (12) in actual numerical calculation. When the number of light irradiation directions is less than the number of pixels in the complex amplitude image, if the scanning direction of the light irradiation to the object of observation is as follows: Figure 6 (a) to (c) are shown in k x k y The plane is discrete and periodic, so it is not the ideal transmission matrix T rr (r out ; r in ), and instead obtain the transmission matrix T' represented by the following formula (13): rr (r out ; r in ). (13) In the formula, r p It corresponds to the periodic distribution of the positions of multiple light irradiation directions in the wave number space and is equivalent to the inverse of the period. rr (r out ; r in ) is equivalent to making the ideal T rr (r out ; r in ) Repeated offset r p And the value obtained by adding them together.

[0139] [Number 12]

[0140]

[0141] [Number 13]

[0142]

[0143] Figure 9 (a)k represents the case where the number of light irradiation directions is equal to the number of pixels of the complex amplitude image. x k yThe distribution of light irradiation directions in wave number space, and (b) the transmission matrix T when the light is focused on a certain point r0 rr (r out ; r in = r0) in the xy space.

[0144] Figure 10 (a)k represents the case where the number of light irradiation directions is smaller than the number of pixels of the complex amplitude image. x k y The distribution of light irradiation directions in wave number space, and (b) the transmission matrix T' when the light is focused on a certain point r0 rr (r out ; r in = r0) in the xy space. Figure 10 r in px 、r py It corresponds to the periodic distribution of the positions of a plurality of light irradiation directions in the wave number space and is equivalent to the inverse of the period.

[0145] As shown in the figure, when the number of light irradiation directions is less than the number of pixels of the complex amplitude image, the output becomes multi-point irradiation. rr (r out ; r in ) is used for the subsequent steps, the results of multiple points in space are added together, and the complex differential interference image, phase differential image and refractive index distribution image cannot be correctly obtained.

[0146] Therefore, in the case of such undersampling, in the interference intensity image acquisition step S51, the interference intensity image acquisition unit 51 obtains the image from the image. Figure 6 (a) to (c) show a photographing unit that captures interference intensity images when light is irradiated to an observation object along multiple light irradiation directions in which the positions of wave number vectors representing the light irradiation directions are discrete and periodically distributed in the wave number space, thereby obtaining interference intensity images for each of the multiple light irradiation directions.

[0147] Then, in the complex differential interference image generating step S54, the complex differential interference image generating unit 54 generates a complex differential interference image W(r ) by calculating the sum of products in each of the plurality of regions of the second surface divided based on the periodic distribution of the positions of the wave number vectors of the plurality of light irradiation directions in the wave number space. in ).

[0148] That is, in the case of undersampling, in the complex differential interference image generation step S54, as shown in FIG. Figure 11 As shown, the transmission matrix T' rr (r out ; r in= r0) is divided into regions D(r in -r p Then, by in -r p ) are summed up to generate the complex differential interference image W(r in )(Formula (14) below).

[0149] [Number 14]

[0150]

[0151] Next, the simulation results are described. Figure 2 The measurement system shown is based on Figure 8 in the order shown.

[0152] In simulation A, if Figure 12 As shown in FIG. 1 , a simulation was performed using an object in which five types of phase images were arranged side by side with intervals between them as an observation object. Figure 12 A diagram schematically illustrating the configuration in simulation A. Figure 13 : is a diagram showing simulation results when the number of light irradiation directions is the same as the number of pixels of the complex amplitude image. Figure 14 : is a diagram showing simulation results when the number of light irradiation directions is smaller than the number of pixels of the complex amplitude image.

[0153] exist Figure 13 and Figure 14 In each of , the upper layer represents the phase differential image of the strict solution, the middle layer represents the phase differential image generated in the case of output side spatial resolution (the above (7) formula), and the lower layer represents the phase differential image generated in the case of input side spatial resolution (the above (8) formula). Figure 13 and Figure 14 The phase differential images obtained respectively have the same degree of clarity.

[0154] exist Figure 13 and Figure 14 In either case, in the case of output-side spatial resolution, a clearer phase differential image is obtained as the position is closer to the imaging unit, and in the case of input-side spatial resolution, a clearer phase differential image is obtained as the position is farther from the imaging unit.

[0155] In simulation B, if Figure 15 As shown, a simulation was performed using an object simulating a cell cluster as an observation object. Figure 15 This is a diagram schematically illustrating the configuration in simulation B. Figure 16 and Figure 17 Each of these is a diagram showing simulation results when the number of light irradiation directions is smaller than the number of pixels of the complex amplitude image.

[0156] exist Figure 16 In , the upper layer represents the phase differential image generated when the output side is spatially resolved (the above formula (7)), and the lower layer represents the phase differential image generated when the input side is spatially resolved (the above formula (8)). Figure 16 In FIG, the left end shows a phase differential image in a cross section parallel to the z axis, and the others show phase differential images in xy cross sections at three positions in the z direction.

[0157] exist Figure 17 In the figure, the upper layer represents the refractive index distribution image of the strict solution, the middle layer represents the refractive index distribution image generated when the output side is spatially resolved (the above formula (7)), and the lower layer represents the refractive index distribution image generated when the input side is spatially resolved (the above formula (8)). Figure 17 In FIG. 1 , the left end shows the maximum value projection image in the y direction, and the others show the refractive index distribution images in the xy cross-section at each of the three positions in the z direction.

[0158] Even in this simulation B, in the case of output-side spatial resolution, the closer to the imaging part, the clearer the phase differential image and refractive index distribution image obtained, while in the case of input-side spatial resolution, the farther away from the imaging part, the clearer the phase differential image and refractive index distribution image obtained.

[0159] Based on the results of simulations A and B, in the phase differential image generation step S55, the phase differential image generation unit 55 preferably generates a phase differential image for each position r on the first surface at a position relatively far from the imaging unit. in , find T on the second side rr (r out ; r in ) and T rr *(r out -δr; r in The complex differential interference image is generated by summing the products of each element of -δr). On the contrary, for a position relatively close to the imaging unit, it is preferable to sum the products of each element of the second surface r out , find T on the first face rr (r out ; r in ) and T rr *(r out -δr; r in -δr) to generate a complex differential interference image.

[0160] As described above, according to this embodiment, even when the observation object is a multiple scatterer, a good penetration depth is achieved, and the observation object can be observed while reducing the influence of multiple scattered light. Furthermore, even when obtaining a three-dimensional phase differential image, mechanical scanning of the objective lens in the optical axis direction is not required, making measurement easier.

[0161] In the observation device and observation method described above, for one of the first branch light and the second branch light output from the light source and branched into two, the first branch light is received by the imaging unit as reference light without passing through the observation object, and for the other second branch light, the second branch light is received by the imaging unit as object light after passing through the observation object. For each of the multiple light irradiation directions of the second branch light (object light), an interference intensity image generated by the interference between the first branch light (reference light) and the second branch light (object light) is captured, and a complex amplitude image is generated based on the interference intensity image.

[0162] Not limited to this, such as the following use Figure 18 and Figure 19 As described, the structure can also be such that, for both the first branch light and the second branch light output from the light source and branched into two, after passing through the observation object, the light is received by the shooting unit, and for each of the multiple light irradiation directions of the second branch light, an interference intensity image generated by the interference of the first branch light and the second branch light is shot, and a complex amplitude image is generated based on the interference intensity image.

[0163] Figure 18 1 is a diagram showing the structure of an observation device 100. The observation device 100 includes a light source 110, an irradiation unit 131, and an imaging unit 150. The light source 110 outputs spatially coherent light. The light output from the light source 110 may be temporally coherent or incoherent.

[0164] Light source 110 may be a laser light source, or may be a light source such as an SLD (Super Luminescent Diode), an SC (Super Continuum) light source, or an optical frequency comb light source. Alternatively, spatial coherence may be enhanced by passing spatially incoherent light output from an LED (Light Emitting Diode), a mercury lamp, or the like through a pinhole.

[0165] Lens 121 is optically connected to light source 110 and focuses light output from light source 110 onto light input end 122 of optical fiber 123, causing the light to enter light input end 122. Optical fiber 123 guides the light incident on light input end 122 to light output end 124. The light guided by optical fiber 123 is emitted from light output end 124 as divergent light. Lens 125 is optically connected to light output end 124 and receives and collimates the light output as divergent light from light output end 124, outputting the collimated light toward irradiation unit 131.

[0166] The irradiation unit 131 receives light output from the light source 110 and passed through the lens 121, the optical fiber 123, and the lens 125, and branches the input light 2 into a first branched light and a second branched light. The irradiation unit 131 overlaps the first branched light and the second branched light and irradiates the object S. The irradiation unit 131 irradiates the object S with the first branched light along a specific light irradiation direction and irradiates the object S with the second branched light along each of a plurality of light irradiation directions.

[0167] The irradiation section 131 includes a beam splitter 311 , a phase modulation type spatial light modulator 313 , a polarizer 314 , a half wavelength plate 315 , a polarizer 316 , a lens 318 , and an objective lens 319 .

[0168] The beam splitter 311 reflects light arriving via the polarizer 314 and the half-wavelength plate 315 disposed between the lens 125 toward the spatial light modulator 313 . The beam splitter 311 also receives light arriving from the spatial light modulator 313 and outputs the light toward the polarizer 316 .

[0169] The spatial light modulator 313 does not phase-modulate the linearly polarized light of the first orientation, while selectively phase-modulating the linearly polarized light of the second orientation, among the linearly polarized light of the first and second orientations that are orthogonal to each other and that are incident on the modulation surface. The polarizer 314 and the half-wave plate 315 set the polarization state of light so that the light incident on the modulation surface of the spatial light modulator 313 from the beam splitter 311 contains linearly polarized light components of the first and second orientations to equal degrees.

[0170] The polarizer 316 receives light arriving from the spatial light modulator 313 via the beam splitter 311 and causes interference between the first and second linearly polarized lights contained in the light. The lens 318 and the objective lens 319 illuminate the first and second branched lights output from the polarizer 316 as plane waves, respectively, toward the observation object S.

[0171] The irradiation unit 131 having such a structure can use the linearly polarized light of the first orientation that has not been phase-modulated by the spatial light modulator 313 as the first branched light, and irradiate the observation object S with the first branched light along a certain light irradiation direction. The irradiation unit 131 can use the linearly polarized light of the second orientation that has been phase-modulated by the spatial light modulator 313 as the second branched light, and irradiate the observation object S with the second branched light along each of a plurality of light irradiation directions.

[0172] The direction in which the second branched light is irradiated onto the observation object S can be set by the orientation and spacing of the phase modulation pattern on the modulation surface of the spatial light modulator 313. Furthermore, the phase difference between the first branched light and the second branched light can be set by shifting the phase modulation pattern on the modulation surface of the spatial light modulator 313.

[0173] The phase difference can also be adjusted by the position of the spatial light modulator 313. However, setting the phase difference by shifting the phase modulation pattern on the modulation plane of the spatial light modulator 313 is preferable because there is no mechanical movement of the components.

[0174] The objective lens 141 receives light (first branched light and second branched light) emitted from the irradiation unit 131 toward the observation object S and passes through the observation object S, and outputs the light toward the mirror 142. The lens 143 receives light output from the objective lens 141 and reflected by the mirror 142, and causes the light to enter the imaging surface of the imaging unit 150.

[0175] The imaging unit 150 receives both the first branched light and the second branched light that reach the imaging surface from the lens 143 and captures an interference intensity image generated by the interference between the first branched light and the second branched light. The imaging unit 150 captures the interference intensity image for each of the multiple phase differences between the first branched light and the second branched light, for each of the multiple light irradiation directions of the second branched light. By performing desired processing based on the interference intensity image captured by the imaging unit 150, a complex amplitude image, etc., can be generated.

[0176] Using the observation device 100 ( Figure 18 ) in the case of the complex amplitude image generating unit is as follows. Figure 19 1 is a diagram schematically showing the incidence of the first branched light and the second branched light on the observation object S in the observation apparatus 100 , and the incidence of the first branched light and the second branched light on the imaging unit 150 after passing through the observation object S.

[0177] The irradiation unit 131 irradiates the observation object S with the first branched light and the second branched light in an overlapping manner. At this time, the irradiation direction of the first branched light relative to the observation object S is set to a constant, the irradiation direction of the second branched light relative to the observation object S is set to a plurality of irradiation directions, and the phase difference φ between the first branched light and the second branched light is set to various values.

[0178] The wavefront of the first branched light incident on the observation object S is represented as u 0,in (r) The wavefront of the second branched light incident on the observation object S along the nth light irradiation direction (n=1 to N) among the plurality of (N) light irradiation directions of the second branched light is represented by u n,in (r)exp(iφ). r is a variable indicating position. φ is the phase difference between the first branched light and the second branched light. The wavefront of the first branched light in the imaging plane or focal plane (optically conjugate to the imaging plane) of the imaging unit 150 is represented as u0(r), and the wavefront of the second branched light is represented as u n (r)exp(iφ).

[0179] The interference intensity image I obtained by imaging by the imaging unit 150 n (r, φ), by u0(r) and u n (r)exp(iφ) is expressed as the square of the absolute value of the sum. Interference intensity image I n (r, φ) is the interference intensity image obtained by photographing the photographing unit 150 when the phase difference between the first branch light and the second branch light is set to φ, the first branch light is incident on the observation object S along a certain light irradiation direction relative to the observation object, and the second branch light is incident on the observation object along the nth light irradiation direction relative to the observation object.

[0180] The focal plane (a plane optically conjugate to the imaging plane) may be located on the observation object S, on the imaging unit 150 side relative to the observation object S, or on the irradiation unit 131 side relative to the observation object S.

[0181] For each of the plurality of light irradiation directions of the second branched light, the interference term C is obtained by the phase shift method based on the interference intensity image obtained by the imaging unit 150 when the plurality of phase differences φ are set. n (r)=u0*(r)·u n (r). The interference term u0(r)·u can also be obtained n *(r). The interference term C is calculated for each of the plurality of light irradiation directions of the second branched light (ie, for each n (=1 to N)). n (r).

[0182] Based on the interference terms (C n(r)) to generate a complex amplitude image of the first branched light. The phase φ0(r) of the complex amplitude u0(r) of the first branched light can be used to calculate the coherent sum C of the corrected interference terms after correcting the phase slope (difference in light incident direction) between the first branched light and the second branched light. sum (r), using the coherence and C sum The phase of (r) is approximately expressed.

[0183] The amplitude A0(r) of the complex amplitude u0(r) of the first branched light can be obtained from the intensity image |u0(r)| captured by the imaging unit 150 when the observation object S is irradiated with only the first branched light without irradiating the second branched light. 2 Alternatively, the amplitude A0(r) of the complex amplitude u0(r) of the first branched light can be obtained by using the interference term C n (r) intensity and I sum The square root of (r) is approximately expressed.

[0184] Based on the phase φ0(r) and amplitude A0(r) of the complex amplitude u0(r) of the first branched light obtained as described above, the complex amplitude image u0(r) of the first branched light can be generated. n (r) It is possible to generate a complex amplitude image u of the second branched light in each of the multiple light irradiation directions n (r) The subsequent processing is the same as that already explained.

[0185] The observation apparatus and the observation method are not limited to the above-described embodiments and configuration examples, and various modifications are possible.

[0186] The observation device of the first mode of the above-mentioned embodiment includes: (1) an interference intensity image acquisition unit that acquires interference intensity images of the observation object irradiated with light along each of the plurality of light irradiation directions; (2) a complex amplitude image generation unit that generates a complex amplitude image for each of the plurality of light irradiation directions based on the interference intensity image; (3) a transfer matrix generation unit that generates a transfer matrix with respect to each position r on the first surface when the observation object is not present, based on the complex amplitude image of each of the plurality of light irradiation directions. in The wavefront of the light is such that each position r on the second surface which is the same as the first surface when observing the object exists out The light wavefront establishes the associated transmission matrix T rr (r out ; r in ); (4) a complex differential interference image generating unit based on the transmission matrix T rr (r out ; r in ), for each position r on the first surface in , generate T on the second siderr (r out ; r in ) and T rr *(r out -δr; r in -δr) to generate a complex differential interference image; and (5) a phase differential image generating unit that generates a phase differential image based on the complex differential interference image.

[0187] In the observation device of the second mode, it can also be constructed as follows: in the structure of the first mode, the interference intensity image acquisition unit obtains the interference intensity images of each of the multiple light irradiation directions from the shooting unit that shoots the interference intensity images generated by the interference of light irradiated to the observation object along the multiple light irradiation directions and passing through the observation object and the reference light.

[0188] In the observation device of the third embodiment, it can also be constructed as follows: in the structure of the second embodiment, the interference intensity image acquisition unit obtains interference intensity images of each of the multiple light irradiation directions from the capturing unit that captures the interference intensity image when light is irradiated to the observation object along the multiple light irradiation directions respectively distributed discretely and periodically along the wave number vectors representing the light irradiation directions in the wave number space, and the complex differential interference image generation unit obtains the sum of the products in each of the multiple areas of the second surface divided based on the periodic distribution of the positions of the wave number vectors of the multiple light irradiation directions in the wave number space, thereby generating a complex differential interference image.

[0189] In the observation device of the fourth mode, it can also be constructed as follows: in the structure of the first mode, the interference intensity image acquisition unit acquires an interference intensity image produced by the interference of light irradiated on the observation object and passing through the observation object along multiple light irradiation directions and light irradiated on the observation object and passing through the observation object along a certain light irradiation direction.

[0190] In the observation device of the fifth aspect, in any one of the configurations of the first to fourth aspects, the phase differential image generating unit may generate a phase differential image for each position r on the first surface at a position relatively far from the imaging unit. in , find T on the second side rr (r out ; r in ) and T rr *(r out -δr; r in -δr) is summed to generate a complex differential interference image. For a position relatively close to the imaging unit, for each position r on the second surface out , find T on the first face rr (r out ; r in ) and T rr *(rout -δr; r in -δr) to generate a differential complex interference image.

[0191] In the observation device of the sixth aspect, in any one of the configurations of the first to fifth aspects, the phase differential image generating unit may generate a three-dimensional phase differential image by generating a phase differential image at each position along the optical axis of the imaging unit.

[0192] The observation apparatus of the seventh aspect may be configured so that, in any one of the configurations of the first to sixth aspects, it further includes a refractive index distribution image generator that generates a three-dimensional refractive index distribution image of the observation object based on the three-dimensional phase differential image.

[0193] The observation method of the first mode of the above-mentioned embodiment includes: (1) an interference intensity image acquisition step of acquiring interference intensity images of the observation object irradiated with light along a plurality of light irradiation directions; (2) a complex amplitude image generation step of generating a complex amplitude image based on the interference intensity image for each of the plurality of light irradiation directions; (3) a transfer matrix generation step of generating a transfer matrix relative to each position r on the first surface when the observation object is not present, based on the complex amplitude images of each of the plurality of light irradiation directions. in The wavefront of the light is such that each position r on the second surface which is the same as the first surface when observing the object exists out The light wavefront establishes the associated transmission matrix T rr (r out ; r in ); (4) Complex differential interference image generation step, based on the transfer matrix T rr (r out ; r in ), for each position r on the first surface in , generate T on the second side rr (r out ; r in ) and T rr *(r out -δr; r in -δr) to generate a complex differential interference image; and (5) a phase differential image generating step of generating a phase differential image based on the complex differential interference image.

[0194] In the observation method of the second embodiment, it can also be constructed as follows: in the structure of the first embodiment, in the interference intensity image acquisition step, interference intensity images of each of the multiple light irradiation directions are obtained from a shooting unit that captures interference intensity images generated by interference between light irradiated onto the observation object along multiple light irradiation directions and passing through the observation object and the reference light.

[0195] In the observation method of the third embodiment, it can also be constructed as follows: in the structure of the second embodiment, in the interference intensity image acquisition step, interference intensity images of each of the multiple light irradiation directions are obtained from a capturing unit that captures interference intensity images when light is irradiated to the observation object along multiple light irradiation directions respectively distributed discretely and periodically along the positions of wave number vectors representing the light irradiation directions in the wave number space; in the complex differential interference image generation step, the sum of products is calculated in each of the multiple areas of the second surface divided based on the periodic distribution of the positions of the wave number vectors of each of the multiple light irradiation directions in the wave number space, thereby generating a complex differential interference image.

[0196] In the observation method of the fourth mode, it can also be constructed as follows: in the structure of the first mode, in the interference intensity image acquisition step, an interference intensity image produced by the interference of light irradiated on the observation object and passing through the observation object along multiple light irradiation directions and light irradiated on the observation object and passing through the observation object along a certain light irradiation direction is acquired.

[0197] In the observation method of the fifth aspect, in any one of the configurations of the first to fourth aspects, in the phase differential image generating step, for each position r on the first surface, for a position relatively far from the imaging unit, in , find T on the second side rr (r out ; r in ) and T rr *(r out -δr; r in -δr) is summed to generate a complex differential interference image. For a position relatively close to the imaging unit, for each position r on the second surface out , find T on the first face rr (r out ; r in ) and T rr *(r out -δr; r in -δr) to generate a differential complex interference image.

[0198] In the observation method of the sixth aspect, in any one of the configurations of the first to fifth aspects, in the phase differential image generating step, a three-dimensional phase differential image is generated by generating phase differential images at respective positions along the optical axis of the imaging unit.

[0199] The observation method of the seventh aspect may be configured so that, in any one of the configurations of the first to sixth aspects, further includes a refractive index distribution image generating step of generating a three-dimensional refractive index distribution image of the observation object based on the three-dimensional phase differential image.

[0200] The program of the above-described embodiment is a program for causing a computer to execute each step of the observation method having the above-described structure.

[0201] The recording medium of the above embodiment is a computer-readable recording medium having the program having the above configuration recorded thereon.

[0202] Industrial applicability

[0203] The embodiment can be used as an observation apparatus and an observation method that can reduce the influence of multiply scattered light and easily observe the observation object even when the observation object is a multiply scatterer.

[0204] Description of Reference Signs

[0205] 1A to 1C…Observation device, 2…Recording medium, 11…Light source, 12…Lens, 13…Light incident end, 14…Optical fiber, 15…Fiber coupler, 16, 17…Optical fiber, 18, 19…Light exit end, 21…Lens, 22…Mirror, 23…Lens, 24…Converging lens, 25…Objective lens, 31…Lens, 32…Mirror, 33…Drive unit, 34…Lens, 41…Beam splitter, 42…Lens, 43…Image capture unit, 50…Analyzer, 51…Interference intensity image acquisition unit, 52…Complex amplitude image generation unit, 53…Transfer matrix generation unit, 54…Complex differential interference image generation unit, 55…Phase differential image generation unit, 56…Refractive index distribution image generation unit, 57…Display unit, 58…Storage unit, 100…Observation device, 110…Light source, 131…Illumination unit, 150…Image capture unit

Claims

1. An observation device, characterized in that: include: an interference intensity image acquisition unit that acquires interference intensity images of the observation object irradiated with light along a plurality of light irradiation directions; a complex amplitude image generating unit configured to generate a complex amplitude image based on the interference intensity image for each of the plurality of light irradiation directions; A transfer matrix generating unit for generating, based on the complex amplitude images of the plurality of illumination directions, a transmission matrix for each position r on the first surface when the observation object is not present. in The wavefront of the light is such that each position r on the second surface which is the same as the first surface when the observation object exists out The light wavefront establishes the associated transmission matrix T rr (r out ; r in ); A complex differential interference image generating unit based on the transmission matrix T rr (r out ; r in ), for each position r on the first surface in , find T on the second surface rr (r out ; r in ) and T rr *(r out -δr; r in -δr), thereby generating a complex differential interference image; and A phase differential image generating unit generates a phase differential image based on the complex differential interference image.

2. The observation device according to claim 1, wherein The interference intensity image acquisition unit acquires the interference intensity images in each of the plurality of light irradiation directions from a capturing unit that captures interference intensity images generated by interference between light irradiated along each of the plurality of light irradiation directions and passing through the observation object and reference light.

3. The observation device according to claim 2, wherein The interference intensity image acquisition unit acquires the interference intensity image for each of the plurality of light irradiation directions from a capturing unit that captures the interference intensity image when irradiating light to the observation object along the plurality of light irradiation directions, each of which is discretely and periodically distributed along a wave number vector representing the light irradiation direction in a wave number space. The complex differential interference image generating unit generates the complex differential interference image by calculating the sum of the products in each of a plurality of regions of the second surface divided based on the periodic distribution of the positions of the wave number vectors of the respective light irradiation directions in the wave number space.

4. The observation device according to claim 1, wherein The interference intensity image acquisition unit acquires an interference intensity image generated by interference between light irradiated on the observation object and passing through the observation object along each of a plurality of light irradiation directions and light irradiated on the observation object and passing through the observation object along a certain light irradiation direction.

5. The observation device according to any one of claims 1 to 4, wherein The phase differential image generating unit, For a position relatively far from the imaging unit, for each position r on the first surface in , find T on the second surface rr (r out ; r in ) and T rr *(r out -δr; r in -δr), thereby generating the complex differential interference image, For the position relatively close to the imaging unit, for each position r on the second surface out , find T on the first surface rr (r out ; r in ) and T rr *(r out -δr; r in The complex differential interference image is generated by summing the products of each element of the complex differential interference image.

6. The observation device according to any one of claims 1 to 5, wherein: The phase differential image generating unit generates a three-dimensional phase differential image by generating a phase differential image at each position along the optical axis of the imaging unit.

7. The observation device according to claim 6, wherein The system further includes a refractive index distribution image generating unit configured to generate a three-dimensional refractive index distribution image of the observation object based on the three-dimensional phase differential image.

8. An observation method, characterized in that: include: an interference intensity image acquisition step of acquiring interference intensity images of the observation object irradiated with light along a plurality of light irradiation directions; a complex amplitude image generating step of generating a complex amplitude image based on the interference intensity image for each of the plurality of light irradiation directions; A transfer matrix generating step of generating a transfer matrix with respect to each position r on the first surface when the observation object is not present, based on the complex amplitude images of each of the plurality of illumination directions. in The wavefront of the light is such that each position r on the second surface which is the same as the first surface when the observation object exists out The light wavefront establishes the associated transmission matrix T rr (r out ; r in ); The complex differential interference image generation step is based on the transmission matrix T rr (r out ; r in ), for each position r on the first surface in , find T on the second surface rr (r out ; r in ) and T rr *(r out -δr; r in -δr), thereby generating a complex differential interference image; and A phase differential image generating step is to generate a phase differential image based on the complex differential interference image.

9. The observation method according to claim 8, wherein: In the interference intensity image acquisition step, the interference intensity images of each of the multiple light irradiation directions are acquired from a capturing unit that captures interference intensity images generated by interference between light irradiated onto the observation object along each of the multiple light irradiation directions and passing through the observation object and reference light.

10. The observation method according to claim 9, wherein: In the interference intensity image acquisition step, the interference intensity images for each of the plurality of light irradiation directions are acquired from a photographing unit that photographs the interference intensity images when irradiating light to the observation object along the plurality of light irradiation directions, each of which is discretely and periodically distributed along the wave number vectors representing the light irradiation directions in the wave number space. In the complex differential interference image generation step, the complex differential interference image is generated by calculating the sum of the products in each of the plurality of regions of the second surface divided based on the periodic distribution of the positions of the wave number vectors of each of the plurality of light irradiation directions in the wave number space.

11. The observation method according to claim 8, wherein In the interference intensity image acquisition step, an interference intensity image is obtained which is generated by the interference of light irradiated onto and passing through the observation object along multiple light irradiation directions and light irradiated onto and passing through the observation object along a certain light irradiation direction.

12. The observation method according to any one of claims 8 to 11, wherein: In the phase differential image generating step, For a position relatively far from the imaging unit, for each position r on the first surface in , find T on the second surface rr (r out ; r in ) and T rr *(r out -δr; r in -δr), thereby generating the complex differential interference image, For the position relatively close to the imaging unit, for each position r on the second surface out , find T on the first surface rr (r out ; r in ) and T rr *(r out -δr; r in -δr) to generate the differential complex interference image.

13. The observation method according to any one of claims 8 to 12, wherein: In the phase differential image generating step, a three-dimensional phase differential image is generated by generating a phase differential image at each position along the optical axis of the imaging unit.

14. The observation method according to claim 13, wherein: The method further includes a refractive index distribution image generating step of generating a three-dimensional refractive index distribution image of the observation object based on the three-dimensional phase differential image.

15. A program, characterized in that It is used to make a computer execute each step of the observation method according to any one of claims 8 to 14.

16. A computer-readable recording medium, characterized in that The program according to claim 15 is recorded.

Images