Observation apparatus and observation method

Abstract

To provide an observation device capable of acquiring three dimensional function information of an observation object while suppressing an increase in the number of images to be acquired.SOLUTION: The observation apparatus includes an image acquisition unit 61, a wavefront propagation calculator 62, and an analyzer 63. The image acquiring unit 61 causes light under the n-th application condition among the first to N-th application conditions to be applied to the observation target at times tn and tn + T, acquires a complex-amplitude image u1, n of the light on the focal plane at the time tn, and acquires a complex-amplitude image u2, n of the light on the focal plane at the time (tn + T). The wavefront propagation calculator 62 propagates the wavefront of light represented by the complex-amplitude images u1, n (r) and u2, n (r) on the focal plane to another plane, and calculates the complex-amplitude images u1, n (r) and u2, n (r) representing the wavefront of light on the other plane at that time. The analysis unit 63 extracts temporal fluctuation components of the image of the observation object based on the complex magnitude images u1, 1 to u1, N and u2, 1 to u2, N on the focal plane and the other planes, and analyzes the extracted temporal fluctuation components.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] This invention relates to an observation device and an observation method. [Background technology]

[0002] Non-staining and non-invasive imaging techniques for objects such as cells include phase-contrast microscopy, differential interference contrast microscopy, optical coherence tomography (OCT), and optical diffraction tomography (ODT). These imaging techniques can acquire information about the morphology of an object by utilizing the contrast related to its refractive index and reflectivity. However, these imaging techniques cannot acquire information about the function of the object.

[0003] If it becomes possible to obtain information on the function of cells as objects of observation, it is expected that it will be possible to perform highly accurate evaluations of cell quality in regenerative medicine, efficacy and toxicity of drugs in drug discovery, and quality evaluation of sperm, eggs, embryos, etc. in assisted reproductive technology.

[0004] Non-staining imaging techniques intended to enable the acquisition of information regarding the function of an object being observed have been proposed in Patent Document 1 and Non-Patent Documents 1 and 2. The imaging techniques described in Patent Document 1 and Non-Patent Document 1 acquire a large number of OCT images at a certain time interval in a single focal plane. The imaging technique described in Non-Patent Document 2 acquires a large number of approximate two-dimensional refractive index images at a certain time interval in a single focal plane. These imaging techniques can then visualize metabolic contrast information based on the motility of the object being observed by analyzing the temporal variation of the object's image based on the large number of images acquired at a certain time interval. These imaging techniques enable the acquisition of biologically meaningful functional information, such as metabolic contrast, which cannot be obtained with conventional non-staining imaging. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] U.S. Patent No. 11543641 [Non-patent literature]

[0006] [Non-Patent Document 1] Hugonnet, H. et. al, "Improving specificity and axial spatial resolution of refractive indeximaging by exploiting uncorrelated subcellular dynamics," ACS Photonics. https: / / doi.org / 10.1021 / acsphotonics.3c01236 [Non-Patent Document 2] Apelian, C. et.al, "Dynamic full field optical coherence tomography: subcellular metabolic contrast revealed in tissues by interferometric signals temporal analysis," Biomedical Optics Express, 7(4), 1511-1524. (2016). [Non-Patent Document 3] Tomohiro Shirai, "Phase Information Hidden in Intensity Distribution," Optics, Vol. 38, No. 10, pp. 496-502 (2009). [Overview of the project] [Problems that the invention aims to solve]

[0007] Using the imaging techniques described in Patent Document 1 and Non-Patent Documents 1 and 2, by analyzing the temporal variation of the image of the observation object based on a large number of images acquired at a certain time interval for each of a plurality of focal planes, it is possible to obtain functional information on the three-dimensional metabolic contrast of the observation object. However, in this case, the number of images to be acquired becomes extremely large, and for example, it is necessary to acquire tens of thousands to hundreds of thousands of images. This involves various practical problems such as an increase in the measurement time required for image acquisition, an increase in the storage capacity required for image storage, and an increase in the time required for image processing.

[0008] The present invention has been made to solve the above problems, and an object thereof is to provide an observation apparatus and an observation method capable of acquiring three-dimensional functional information of an observation object while suppressing an increase in the number of images to be acquired.

Means for Solving the Problems

[0009] The first aspect of the observation apparatus of the present invention is as follows: (1) Light under the n-th irradiation condition (n is an integer of 1 or more and N or less) among the first to N-th irradiation conditions (N is an integer of 2 or more) that are different from each other with respect to the spatial intensity or phase distribution of light is irradiated to the observation object at time t n and time (t n +T), and the complex amplitude image u n of light at the focal plane is acquired at time t 1,n , and the complex amplitude image u n of light at the focal plane is acquired at time (t 2,n +T). An image acquisition unit; (2) For each n, a wavefront propagation calculation unit that calculates, by calculation, the complex amplitude images u 1,n , u 2,n representing the wavefronts of light at the focal plane, respectively, and the complex amplitude images u 1,n , u 2,n representing the wavefronts of light at another plane when the wavefronts of light at the focal plane are propagated to one or more other planes; (3) At each of the focal plane and other planes, an analysis unit that extracts the temporal variation component of the image of the observation object based on the complex amplitude images u 1,1 ~u 1,N , u 2,1 ~u 2,N and analyzes the temporal variation component.

[0010] In a second aspect of the observation device of the present invention, in addition to the first aspect, the analysis unit analyzes a complex amplitude image u 1,n and complex amplitude image u 2,n The product of the complex conjugate image of one complex amplitude image and the other complex amplitude image, the complex amplitude image u 1,n and complex amplitude image u 2,n The result obtained by dividing one of the complex amplitude images by the other complex amplitude image, or the complex amplitude image u 1,n and complex amplitude image u 2,n The difference between these two values ​​is extracted as the temporal variation component of the image of the observed object.

[0011] In a third aspect of the observation device of the present invention, in addition to the first or second aspect, the analysis unit analyzes a complex amplitude image u 1,n Phase and complex amplitude image u 2,n The difference between this phase and the observed object's image is extracted as the temporal variation component.

[0012] In a fourth aspect of the observation apparatus of the present invention, in addition to any of the first to third aspects, the analysis unit performs analysis based on the results obtained by performing a Fourier transform with respect to the time variable on the temporal variation component of the image of the object being observed.

[0013] In a fifth aspect of the observation apparatus of the present invention, in addition to any of the first to fourth aspects, the analysis unit performs an evaluation of the function of cells or cell aggregates by analyzing the temporal variation component of the image of the cells or cell aggregates as the object of observation.

[0014] In the sixth aspect of the observation apparatus of the present invention, in addition to any of the first to fifth aspects, the image acquisition unit irradiates the object to be observed with plane waves having different irradiation directions, as the first to Nth irradiation conditions.

[0015] The observation method of the present invention involves (1) irradiating an object with light under the nth irradiation condition (n is an integer between 1 and N) from among the first to Nth irradiation conditions (N is an integer of 2 or more) which are different from each other in terms of the spatial intensity or phase distribution of light, at time t n and time (t nIrradiate at +T) and time t n The complex amplitude image of light at the focal plane u 1,n Get the time (t n The complex amplitude image of light at the focal plane (+T) u 2,n (1) An image acquisition step to obtain the complex amplitude image u at the focal plane for each n. 1,n ,u 2,n A complex amplitude image u representing the wavefront of light on one or more other surfaces when the wavefront of light represented by each is propagated to one or more other surfaces. 1,n ,u 2,n (3) A wavefront propagation calculation step in which the complex amplitude image u is obtained by calculation, and (4) in the focal plane and other planes respectively 1,1 ~u 1,N ,u 2,1 ~u 2,N The system includes an analysis step of extracting the temporal variation component of the image of the object being observed based on the above, and analyzing that temporal variation component.

[0016] The program of the present invention causes a computer to perform each step of the observation method of the present invention described above. The recording medium of the present invention is a computer-readable recording on which the program of the present invention described above is stored. [Effects of the Invention]

[0017] According to the present invention, it is possible to acquire three-dimensional functional information of an object being observed while suppressing an increase in the number of images to be acquired. [Brief explanation of the drawing]

[0018] [Figure 1] Figure 1 shows the configuration of observation device 1A. [Figure 2] Figure 2 shows the configuration of observation device 1B. [Figure 3] Figure 3 shows the configuration of observation device 1C. [Figure 4] Figure 4 schematically shows the incidence of the first branched light and the second branched light onto the object to be observed S in the observation device 1C, as well as the incidence of the first branched light and the second branched light onto the imaging unit 50 after passing through the object to be observed S. [Figure 5] Figure 5 shows the configuration of the processing unit 60 of the observation device. [Figure 6] Figure 6 is a flowchart of the observation method. [Figure 7] Figure 7 shows the simulation conditions. [Figure 8] Figure 8 shows the complex amplitude image obtained from the simulation. [Figure 9] Figure 9 shows the simulation results for the first extraction method. [Figure 10] Figure 10 shows the simulation results for the first extraction method. [Figure 11] Figure 11 shows the simulation results for the first extraction method. [Figure 12] Figure 12 shows the exact solution obtained by calculation from the precise phase information of the observed object. [Figure 13] Figure 13 shows the exact solution obtained by calculation from the exact phase information of the observed object. [Figure 14] Figure 14 shows the simulation results for the third extraction method. [Figure 15] Figure 15 shows the simulation results for the third extraction method. [Figure 16] Figure 16 shows the simulation results for the third extraction method. [Figure 17] Figure 17 shows the simulation results for the fourth extraction method. [Figure 18] Figure 18 shows the simulation results for the fourth extraction method. [Figure 19] Figure 19 shows the simulation results for the fourth extraction method. [Figure 20] Figure 20 shows other simulation results for the first extraction method. [Figure 21] Figure 21 shows yet another result of the simulation for the first extraction method. [Figure 22] Figure 22 shows yet another result of the simulation for the first extraction method. [Figure 23] Figure 23 shows the simulation results on the surface z=7.8λ0. [Figure 24] Figure 24 shows the simulation results on the surface z=7.8λ0. [Figure 25] Figure 25 shows the simulation results on the surface z=7.8λ0. [Figure 26] Figure 26 shows the simulation results on the surface z=7.8λ0. [Figure 27] Figure 27 shows the simulation results for the surface z = 23.4λ0. [Figure 28] Figure 28 shows the simulation results on the surface z = 23.4λ0. [Figure 29] Figure 29 shows the simulation results on the surface z = 23.4λ0. [Figure 30] Figure 30 shows the simulation results on the surface z = 23.4λ0. [Figure 31] Figure 31 shows the simulation results for the surface z = 33.8λ0. [Figure 32] Figure 32 shows the simulation results on the surface z = 33.8λ0. [Figure 33] Figure 33 shows the simulation results on the surface z = 33.8λ0. [Figure 34] Figure 34 shows the simulation results on the surface z = 33.8λ0. [Figure 35] Figure 35 shows the experimental results of HepG2 at time 0 min on the z=10.0 μm surface. [Figure 36] Figure 36 shows the experimental results for the HepG2 plane at z=20.0 μm at time 0 min. [Figure 37] Figure 37 shows the experimental results of HepG2 at a z=10.0 μm plane at time 2 min. [Figure 38] Figure 38 shows the experimental results of HepG2 at a z=20.0 μm plane at time 2 min. [Modes for carrying out the invention]

[0019] Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the attached drawings. In the description of the drawings, the same elements will be denoted by the same reference numerals, and redundant descriptions will be omitted. The present invention is not limited to these examples, but is indicated by the claims, and all modifications within the meaning and scope equivalent to the claims are intended to be included.

[0020] The observation apparatus and observation method of this embodiment acquire a complex amplitude image at a certain focal plane and perform necessary processing based on the acquired complex amplitude image. Various configurations are possible for acquiring the complex amplitude image, and observation apparatuses 1A to 1C are described below as examples, but are not limited to these.

[0021] Figure 1 shows the configuration of observation device 1A. This observation device 1A includes a light source 10, lens 11, lens 21, mirror 22, lens 23, condenser lens 24, objective lens 41, beam splitter 42, lens 43, imaging unit 50, and processing unit 60, etc.

[0022] The light source 10 outputs spatially and temporally coherent light, and is preferably a laser light source. The lens 11 is optically connected to the light source 10 and focuses the light output from the light source 10 onto the optical incident end 12 of the optical fiber 13, causing the light to enter the optical incident end 12. The optical fiber 13 guides the light that entered the optical incident end 12 via the lens 11 to the fiber coupler 15. The fiber coupler 15 couples the light between the optical fiber 13 and optical fibers 16 and 17, splitting the light that has been guided by the optical fiber 13 into two, with one branch guided by the optical fiber 16 and the other branched light guided by the optical fiber 17. The light guided by the optical fiber 16 is emitted as divergent light from the optical exit end 18. The light guided by the optical fiber 17 is emitted as divergent light from the optical exit end 19.

[0023] Lens 21 is optically connected to the light output end 18 and collimates the light output from the light output end 18 as divergent light. Mirror 22 is optically connected to lens 21 and reflects the light arriving from lens 21 to lens 23. The orientation of the reflective surface of mirror 22 is variable. Lens 23 is optically connected to mirror 22. Condenser lens 24 is optically connected to lens 23. Lens 23 and condenser lens 24 preferably constitute a 4f optical system. Lens 23 and condenser lens 24 irradiate the object to be observed S with light from a light irradiation direction corresponding to the orientation of the reflective surface of mirror 22. Objective lens 41 is optically connected to condenser lens 24. The object to be observed S is placed between objective lens 41 and condenser lens 24. Objective lens 41 receives light (object light) output from condenser lens 24 and passed through the object to be observed S, and outputs that light to beam splitter 42.

[0024] The beam splitter 42 is optically connected to the objective lens 41 and also to the light output end 19. The beam splitter 42 combines the light output from the objective lens 41 (object light) and the light output from the light output end 19 (reference light), and outputs both lights to the lens 43. The lens 43 is optically connected to the beam splitter 42 and collimates the object light and reference light that arrive from the beam splitter 42, respectively, and outputs them to the imaging unit 50.

[0025] The imaging unit 50 is optically connected to the lens 43 and captures an interference fringe image (interference intensity image) due to the interference between object light and reference light arriving from the lens 43. The incident direction of the reference light is inclined with respect to the incident direction of the object light onto the imaging surface of the imaging unit 50. The position where the object light and reference light are combined by the beam splitter 42 may be after the imaging lens, but considering the effects of aberrations, it is desirable that it be between the objective lens 41 and the lens 43, as shown in the figure.

[0026] The processing unit 60 has an output port that outputs a control signal to control the orientation of the reflective surface of the mirror 22, and controls the light irradiation conditions for the object S being observed using this control signal. The processing unit 60 also has an input port for receiving interference intensity image data captured by the imaging unit 50, and generates a complex amplitude image using Fourier fringe analysis based on this interference intensity image, and performs the necessary processing based on this complex amplitude image. Details of the processing unit 60 will be described later.

[0027] Figure 2 shows the configuration of observation device 1B. This observation device 1B includes a light source 10, lens 11, lens 21, mirror 22, lens 23, condenser lens 24, objective lens 41, lens 43, mirror 44, focal plane position setting unit 45, imaging unit 50, and processing unit 60, etc.

[0028] The optical system from the light source 10 to the optical fiber 13 in the configuration of observation device 1B (Figure 2) is the same as that of observation device 1A (Figure 1). In observation device 1B, the optical fiber 13 guides the light incident at the optical incident end 12 to the optical output end 14 via the lens 11. The light guided by the optical fiber 13 is emitted as divergent light from the optical output end 14 to the lens 21. The lens 21 is optically connected to the optical output end 14 and collimates the light output as divergent light from the optical output end 14 and outputs it to the mirror 22.

[0029] The optical system from lens 21 to objective lens 41 in the configuration of observation device 1B (Figure 2) is the same as that of observation device 1A (Figure 1). In observation device 1B, the objective lens 41 receives light output from the condenser lens 24 that has passed through the object to be observed S, and outputs that light to the mirror 44. Lens 43 collimates the light output from the objective lens 41 that has been reflected by the mirror 44 and outputs it to the imaging unit 50.

[0030] The focal plane position setting unit 45 sets the position of the focal plane (a plane optically conjugate to the imaging plane of the imaging unit 50) by adjusting the position of the objective lens 41 along a direction parallel to the optical axis of the objective lens 41. The focal plane position setting unit 45 can be configured to include, for example, a movable stage or a piezo actuator. The imaging unit 50 is optically connected to the lens 43 and captures an intensity image at the focal plane set by the focal plane position setting unit 45 based on the light that reaches from the lens 43.

[0031] The processing unit 60 has an output port that outputs a control signal to control the orientation of the reflective surface of the mirror 22, and this control signal controls the light irradiation conditions for the object S to be observed. The processing unit 60 also has an output port that outputs a control signal to control the position of the focal plane set by the focal plane position setting unit 45, and this control signal controls the position of the focal plane on which the imaging unit 50 should capture an intensity image. Furthermore, the processing unit 60 has an input port for receiving data of the intensity image captured by the imaging unit 50, and generates a complex amplitude image based on this intensity image, and performs the necessary processing based on this complex amplitude image. The processing unit 60 can generate a complex amplitude image from intensity images at two different focal planes based on the Transport of Intensity Equation (TIE) (Non-Patent Literature 3).

[0032] Figure 3 shows the configuration of the observation device 1C. This observation device 1C includes a light source 10, lens 11, lens 21, illumination unit 31, objective lens 41, lens 43, mirror 44, imaging unit 50, and processing unit 60, among others.

[0033] The optical system from the light source 10 to the lens 21 in the configuration of observation device 1C (Figure 3) is the same as that in the configuration of observation device 1B (Figure 2). In observation device 1C, the lens 21 collimates the light output as divergent light from the light output end 14 and outputs it to the illumination unit 31.

[0034] The irradiation unit 31 receives light output from the light source 10, which has passed through lens 11, optical fiber 13, and lens 21, and generates a first beam of light and a second beam of light from the input light. The irradiation unit 31 then superimposes these first and second beams of light onto the object to be observed S. The irradiation unit 31 irradiates the object to be observed S with the first beam of light along a certain light irradiation direction, and irradiates the object to be observed S with the second beam of light along each of several light irradiation directions.

[0035] The irradiation unit 31 includes a beam splitter 311, a phase-modulated spatial light modulator 313, a polarizer 314, a half-wave plate 315, a polarizer 316, a lens 318, and an objective lens 319.

[0036] The beam splitter 311 reflects the light that has arrived via the polarizer 314 and half-wave plate 315, which are placed between it and the lens 21, to the spatial light modulator 313. The beam splitter 311 also receives the light that has arrived from the spatial light modulator 313 and outputs this light to the polarizer 316.

[0037] The spatial light modulator 313 selectively phase-modulates the linearly polarized light of the second direction, without phase-modulating the linearly polarized light of the first direction, from among the linearly polarized light of the first and second directions, which are orthogonal to each other, that is incident on the modulation plane. The polarizer 314 and the half-wave plate 315 set the polarization state of the light so that the light incident from the beam splitter 311 to the modulation plane of the spatial light modulator 313 contains linearly polarized components of the first and second directions to an equal degree.

[0038] The polarizer 316 receives light that has arrived from the spatial light modulator 313 via the beam splitter 311 and makes it possible to interfere with the linearly polarized light of the first and second directions contained in that light. The polarizer 316 has an optical axis that is 45 degrees different in direction from the polarization direction of the light that has arrived from the spatial light modulator 313 via the beam splitter 311 (linearly polarized light of the first and second directions), and selectively transmits the polarization component of the input light in the direction of the optical axis. The lens 318 and the objective lens 319 irradiate the object S to be observed with the first and second light outputs from the polarizer 316 as plane waves.

[0039] The illumination unit 31 having such a configuration can emit linearly polarized light of a first direction, which is not phase-modulated by the spatial light modulator 313, as the first light, and emit this first light along a certain light irradiation direction to the object to be observed S. The illumination unit 31 can emit linearly polarized light of a second direction, which is phase-modulated by the spatial light modulator 313, as the second light, and emit this second light along each of a plurality of light irradiation directions to the object to be observed S. The direction of the light irradiation of the second light to the object to be observed S can be set by the orientation and spacing of the phase modulation pattern on the modulation surface of the spatial light modulator 313. Furthermore, the phase difference between the first light and the second light can be set by the shift of the phase modulation pattern on the modulation surface of the spatial light modulator 313.

[0040] The optical system from the objective lens 41 to the imaging unit 50 in the configuration of observation device 1C (Figure 3) is the same as that in the configuration of observation device 1B (Figure 2). In observation device 1C, the objective lens 41 receives light (first light, second light) that has been irradiated onto the object to be observed S by the illumination unit 31 and passed through the object to be observed S, and outputs that light to the mirror 44. Lens 43 receives the light output from the objective lens 41 and reflected by the mirror 44, and causes that light to incident on the imaging surface of the imaging unit 50.

[0041] The imaging unit 50 receives both the first and second light rays that have reached the imaging surface from the lens 43 and captures an interference intensity image due to the interference between the first and second light rays. For each of the multiple light irradiation directions of the second light rays, the imaging unit 50 captures an interference intensity image when the phase difference between the first and second light rays is set to each of the multiple phase differences.

[0042] The processing unit 60 has an output port that outputs a control signal for controlling the modulation of light by the spatial light modulator 313, and controls the light irradiation conditions for the object S to be observed using this control signal. The processing unit 60 also has an input port for receiving interference intensity image data captured by the imaging unit 50, and generates a complex amplitude image based on this interference intensity image, and performs the necessary processing based on this complex amplitude image.

[0043] The complex amplitude image generation process in this observation device 1C is as follows. Figure 4 is a schematic diagram showing the incidence of the first branched light and the second branched light onto the object to be observed S, and the incidence of the first branched light and the second branched light onto the imaging unit 50 after passing through the object to be observed S, in the observation device 1C. The illumination unit 31 illuminates the object to be observed S with the first branched light and the second branched light superimposed. At this time, the direction of light irradiation of the first branched light onto the object to be observed S is kept constant, and the direction of light irradiation of the second branched light onto the object to be observed S is set to each of the multiple light irradiation directions, and the phase difference φ between the first branched light and the second branched light is set to each value.

[0044] The wavefront of the first branched light incident on the object S is u 0,in (r) is denoted as (r). The wavefront of the second branched light incident on the object S along the nth light irradiation direction (n=1~N) out of the multiple (N) light irradiation directions of the second branched light is u n,in This is expressed as (r)exp(iφ). r is a variable representing position. φ is the phase difference between the first branched light and the second branched light. The wavefront of the first branched light at the imaging plane or focal plane (a plane optically conjugate to the imaging plane) of the imaging unit 50 is represented as u0(r), and the wavefront of the second branched light is represented as u n This is expressed as (r)exp(iφ).

[0045] Interference intensity image I obtained by imaging by the imaging unit 50 n (r,φ) is u0(r) and u n It is expressed as the square of the absolute value of the sum with (r)exp(iφ). Interferometry image I n (r,φ) is an interference intensity image acquired by imaging by the imaging unit 50 when the phase difference between the first branched light and the second branched light is set to φ, the first branched light is incident on the object S along a certain light irradiation direction, and the second branched light is incident on the object S along the nth light irradiation direction. The focal plane (a plane optically conjugate to the imaging plane) may be on the object S, on the imaging unit 50 side of the object S, or on the irradiation unit 31 side of the object S.

[0046] For each of the multiple light irradiation directions of the second branched beam, the interference term C is calculated by the phase shift method based on the interference intensity image acquired by the imaging unit 50 when each of the multiple phase difference φ is set. n (r=u0 * (r)·u n Find (r). Interference term u0(r)·u n * (r) can also be calculated. This interference term C n (r) is determined for each of the multiple light irradiation directions of the second branched light (i.e., for each n (=1 to N)).

[0047] Interference term C obtained for each of the multiple light irradiation directions of the second branched beam n Based on (r), a complex amplitude image of the first branched light is generated. The phase φ0(r) of the complex amplitude u0(r) of the first branched light is calculated by correcting for the phase slope (difference in the direction of incident light) between the first branched light and the second branched light, and then calculating the coherent sum C of the corrected interference terms. sum Find (r), and this coherent sum C sum It can be approximately expressed in terms of the phase of (r). The amplitude A0(r) of the complex amplitude u0(r) of the first branched light is the intensity image |u0(r)| captured by the imaging unit 50 when only the first branched light is irradiated onto the object S without irradiating the object S with the second branched light. 2 It can be determined from this. Alternatively, the amplitude A0(r) of the complex amplitude u0(r) of the first branched light is the interference term C n (r) Sum of Intensities I sum It can be approximately expressed as the square root of (r).

[0048] Based on the phase φ0(r) and amplitude A0(r) of the complex amplitude u0(r) of the first branched light obtained as described above, a complex amplitude image u0(r) of the first branched light can be generated. Then, the complex amplitude image u0(r) of the first branched light and the interference term C n Based on (r), the complex amplitude image u of the second branch of light from each of the multiple light irradiation directions n (r) can be generated.

[0049] Furthermore, in this observation device 1C, after generating a complex amplitude image of the second branched light for each of the multiple focal planes, the complex amplitude image of the other focal plane may be updated based on the complex amplitude image representing the wavefront of light when the wavefront of light represented by the complex amplitude image of any of the multiple focal planes is propagated to any other focal plane. By repeating this update process among the multiple complex amplitude images, the complex amplitude images can be improved.

[0050] Next, the processing unit 60 will be described. The processing unit 60 may be a computer. The processing unit 60 includes a processing unit such as a CPU, GPU, DSP, or FPGA, and a storage unit such as a hard disk drive, flash memory, RAM, or ROM. The storage unit stores various images and programs for processing execution.

[0051] Figure 5 shows the configuration of the processing unit 60 of the observation device. The processing unit 60 comprises an image acquisition unit 61, a wavefront propagation calculation unit 62, an analysis unit 63, a display unit 64, and a storage unit 65. Figure 6 is a flowchart of the observation method. The observation method of this embodiment comprises an image acquisition step S1 performed by the image acquisition unit 61, a wavefront propagation calculation step S2 performed by the wavefront propagation calculation unit 62, and an analysis step S3 performed by the analysis unit 63.

[0052] The image acquisition unit 61 captures light from the first to the nth irradiation condition (where n is an integer between 1 and N) among the first to the Nth irradiation conditions (where N is an integer of 2 or more) that differ from each other in terms of the spatial intensity or phase distribution of light, onto the object to be observed at time t n and time (t n Irradiate at +T) and time t n The complex amplitude image of light at the focal plane u 1,n Get the time (t n The complex amplitude image of light at the focal plane (+T) u 2,n The image is obtained (image acquisition step S1). Time t n and time (t n +T) Each of the nth irradiation conditions is the same. For each of the first to nth irradiation conditions, plane waves with different irradiation directions may be irradiated onto the object to be observed. In the following, time tn The complex amplitude image obtained is u 1,n (r) is used to represent the time (t n The complex amplitude image obtained at +T) u 2,n This is denoted as (r). r is a variable representing the position at the focal plane. Complex amplitude image u 1,1 ~u 1,N The period for acquiring the complex amplitude image u 2,1 ~u 2,N The period for obtaining the data may overlap in some way.

[0053] The wavefront propagation calculation unit 62 calculates the complex amplitude image u at the focal plane for each n. 1,n (r),u 2,n (r) The wavefronts of light represented by each are propagated to one or more other surfaces, and a complex amplitude image u representing the wavefronts of light at those other surfaces is obtained. 1,n (r),u 2,n (r) is calculated (wavefront propagation calculation step S2). The analysis unit 63 calculates the complex amplitude image u at the focal plane and other planes, respectively. 1,1 ~u 1,N ,u 2,1 ~u 2,N Based on this, the temporal variation component of the image of the object being observed is extracted, and the extracted temporal variation component is analyzed (analysis step S3). The analysis step S3 may be performed after the wavefront propagation calculation to all other surfaces has been completed in wavefront propagation calculation step S2. Alternatively, the analysis step S3 may be performed for some of the surfaces after a portion of the wavefront propagation calculation step S2 has been completed (wavefront propagation calculation to some of the other surfaces).

[0054] The display unit 64 displays images to be processed, images in the process of processing, and images after processing, and includes, for example, a liquid crystal display. The storage unit 65 stores data for various images. The storage unit 65 also stores a program for executing the image acquisition step S1, the wavefront propagation calculation step S2, and the analysis step S3. This program may be stored in the storage unit at the time of manufacture or shipment of the observation device, or it may be acquired via a communication line after shipment and stored in the storage unit 65, or it may be recorded on a computer-readable recording medium 2 and stored in the storage unit 65. The recording medium 2 can be any flexible disk, CD-ROM, DVD-ROM, BD-ROM, USB memory, etc.

[0055] Next, the specific processing details of the analysis unit 63 in analysis step S3 will be explained. There are four types of extraction methods for the analysis unit 63 to extract the temporal variation component of the image of the object being observed, as follows:

[0056] In the first extraction mode, the analysis unit 63 analyzes the complex amplitude image u 1,n (r) and complex amplitude image u 2,n The product of the complex conjugate image of one of the complex amplitude images in (r) and the other complex amplitude image is the temporal variation component c of the image of the observed object. n Extract as (r) (see formula (1) below).

[0057]

number

[0058] In the second extraction mode, the analysis unit 63 analyzes the complex amplitude image u 1,n (r) and complex amplitude image u 2,n The result obtained by dividing one of the complex amplitude images in (r) by the other complex amplitude image is the temporal variation component c of the observed object's image. n Extract as (r) (see equation (2) below). In equation (2), when the value of the denominator on the right side is 0, c n Since the value of (r) is indeterminate, replace it with c in equation (3) n(r) may be the time-varying component. ε is a positive small value. When ε = 0, Equation (3) coincides with Equation (2).

[0059]

Number

[0060]

Number

[0061] In the first extraction mode and the second extraction mode, the analysis unit 63 can obtain the difference between the phase of the complex amplitude image u 1,n (r) and the phase of the complex amplitude image u 2,n (r) as c n (r), and this phase difference g n (r) may be used as the time-varying component of the image of the observation object (Equation (4) below). Represent the complex amplitude image u 1,n (r) by Equation (5) below, and assume that the complex amplitude image u 2,n (r) is represented by Equation (6) below. Then, g n (r) in Equation (4) can be represented by Equation (7) below. j is the imaginary unit, a(r, t n ) is the amplitude, and φ(r, t n ) is the phase. If there is a partial region where the phase cannot be obtained due to the influence of noise, in order to reduce the influence of the noise, it is preferable to create a mask m n (r) based on c n (r) and let this m n (r) act on g n (r).

[0062]

Number

[0063]

Number

[0064]

number

[0065]

number

[0066] In the third extraction mode, the analysis unit 63 analyzes the complex amplitude image u 1,n and complex amplitude image u 2,n The difference between this and the temporal variation component g of the image of the observed object. n Extract as (r) (see formula (8) below).

[0067]

number

[0068] In the fourth extraction mode, the analysis unit 63 analyzes the complex amplitude image u 1,n Transmission matrix T based on 1,n We will obtain the complex amplitude image u 2,n Transmission matrix T based on 2,n We find the transmission matrix T 1,n ,T 2,n The matrix product (T) of the inverse of one transmission matrix and the other transmission matrix. 2,n -1 T 1,n , T 2,n T 1,n -1 , T 1,n -1 T 2,n or T 1,n T 2,n -1 By calculating the matrix product and separating it for each incident wavenumber, the temporal variation component c of the image of the observed object can be obtained. n Extract (r). This c n (r) Phase g n (r) may be the temporal variation component of the image of the object being observed.

[0069] The transmission matrix describes the input-output relationship of light for the entire system, so T2,n -1 T 1,n The calculation of the matrix product of T 1,n After forward propagation of light waves in the system represented by T 2,n This corresponds to the operation of backpropagating light waves in a system represented by T. 2,n -1 T 1,n From the matrix product of T 1,n and T 2,n Only the portion corresponding to the difference can be obtained. Furthermore, if the object being observed is deemed to have no absorption, the inverse of the transmission matrix can be expressed as the Hermitian conjugate of that transmission matrix. The fourth extraction method is effective when the phase change is not large.

[0070] In the first to fourth extraction embodiments, the analysis unit 63 may perform the analysis based on the results obtained by performing a Fourier transform with respect to the time variable on the temporal variation component of the image of the object being observed. g expressed by equation (7) above n The Fourier transform G(r,ω) of (r) is φ(r,t n Using the Fourier transform Φ(r,ω) of ), it is expressed by equation (9) below. ω is the angular frequency. τ is the time interval for acquiring the complex amplitude image. This Fourier transform gives {g n (r) n=1,…,N Since these are added together, when using light with different irradiation conditions from the 1st to the Nth, this summation process extracts a tomographic image at the focal plane.

[0071]

number

[0072] The analysis unit 63 may use G(r,ω) from equation (9) to determine the average frequency (equation (10) below), the sharpness of the frequency distribution (equation (11) below), or the total vibration energy (equation (12) below), and perform the analysis based on these values. f = 2πω, where π is pi. Alternatively, these average frequency, sharpness of the frequency distribution, and total vibration energy may be mapped to the HSV color space, and the analysis may be performed based on this. For example, let H be the average frequency, S be the sharpness of the frequency distribution, and V be the total vibration energy.

[0073]

number

[0074]

number

[0075]

number

[0076] The analysis unit 63 calculates g with respect to n (i.e., with respect to the time variable). n (r) Moving average g' n (r) is calculated (Equation (13) below), and this moving average g' n Alternatively, the variance h(r) of (r) can be calculated (using equation (14) below), and the analysis can be performed based on this variance h(r). Equation (13) below represents a 2M+1 point moving average, where M is an integer greater than or equal to 1. By taking the variance of the moving average in this way, the tomographic image at the focal plane can be extracted.

[0077]

number

[0078]

number

[0079] Next, we will explain the simulation results. Below, we will describe the conditions and results of the simulation performed on one surface for the extraction and analysis of the temporal variation component of the image of the observed object, and then describe the conditions and results of the simulation performed on multiple surfaces.

[0080] Figure 7 shows the simulation conditions. Here, the object under observation is assumed to be a two-dimensional object, and the phase φ of each of the 8x8 circular regions arranged in two dimensions within the object is assumed to vary at various frequencies ω within the range of 0Hz to 24.8Hz. The start time of acquiring the complex amplitude image is set to t=0. The time interval for acquiring the complex amplitude image is τ(=t). n+1 -t n The time interval was set to 0.02 s. N=341. With T=Nτ, T=6.82 s. For each of the first to Nth irradiation conditions, a plane wave with a different irradiation direction was irradiated onto the object. This figure shows the phase distribution of the object at times t=0.00 s, 2.40 s, 4.80 s, 6.82 s, 9.22 s, and 11.62 s.

[0081] Figure 8 shows the complex amplitude image obtained from the simulation. The upper part of this figure shows the complex amplitude image u 1,1 (r)~u 1,N (r) Complex amplitude image u at time t=0.00s 1,1 (r), complex amplitude image u at time t=2.40s 1,121 (r), and the complex amplitude image u at time t=4.80s 1,241 (r) represents the real part of each. The lower part of this figure shows the complex amplitude image u 2,1 (r)~u 2,N (r) Complex amplitude image u at time t=6.82s 2,1 (r) Complex amplitude image u at time t=9.22s 2,121 (r), and the complex amplitude image u at time t=11.62s 2,241 (r) represents the real part of each element.

[0082] Figures 9 to 11 show the simulation results for the first extraction method. The upper part of Figure 9 shows u 1,1 (r)~u 1,N (r) and u 2,1 (r)~u 2,N Based on (r), c1(r)~c obtained in equation (1) above N (r) of c1(r), c 121 (r) and c 241 (r) Represents the real part of each image. The lower part of Figure 9 shows c1(r)~c N Based on (r), g1(r)~g obtained in equation (4) above N (r) g1(r), g 121 (r) and g 241 (r) represents the real part of each image. Figure 10 shows g1(r)~g N The images of the square of the absolute value of G(r,ω) obtained by equation (9) above based on (r) are shown for frequencies f=0.44Hz, 8.80Hz, and 18.33Hz, respectively. Figure 11 shows the distributions of (a) the average frequency obtained by equation (10) above based on G(r,ω), (b) the sharpness of the frequency distribution obtained by equation (11) above based on G(r,ω), and (c) the total vibration energy obtained by equation (12) above based on G(r,ω).

[0083] Figures 12 and 13 show the exact solutions calculated from the exact phase information of the observed object based on the simulation conditions shown in Figure 7. Figure 12 shows images of the square of the absolute value of G(r,ω) at frequencies f=0.44Hz, 8.80Hz, and 18.33Hz, respectively. Figure 13 shows the distributions of (a) average frequency, (b) sharpness of the frequency distribution, and (c) total vibration energy. The simulation results for the first extraction method (Figures 10 and 11) agree well with the exact solutions (Figures 12 and 13).

[0084] Figures 14 to 16 show the simulation results for the third extraction method. Figure 14 shows u 1,1 (r)~u 1,N (r) and u 2,1 (r)~u2,N Based on (r), g1(r)~g obtained in equation (8) above N (r) g1(r), g 121 (r) and g 241 (r) represents the real part of each image. Figure 15 shows g1(r)~g N The images of the square of the absolute value of G(r,ω) obtained by equation (9) above based on (r) are shown for frequencies f=0.44Hz, 8.80Hz, and 18.33Hz, respectively. Figure 16 shows the distributions of (a) the average frequency obtained by equation (10) above based on G(r,ω), (b) the sharpness of the frequency distribution obtained by equation (11) above based on G(r,ω), and (c) the total vibration energy obtained by equation (12) above based on G(r,ω). The simulation results for the third extraction method (Figures 15 and 16) also agree well with the exact solution (Figures 12 and 13).

[0085] Figures 17 to 19 show the simulation results for the fourth extraction method. The upper part of Figure 17 shows u 1,1 (r)~u 1,N (r) and u 2,1 (r)~u 2,N Based on (r), the c1(r)~c obtained by calculating the transmission matrix N (r) of c1(r), c 121 (r) and c 241 (r) Represents the real part of each image. The lower part of Figure 17 shows c1(r)~c N Based on (r), g1(r)~g obtained in equation (4) above N (r) g1(r), g 121 (r) and g 241 (r) represents the real part of each image. Figure 18 shows g1(r)~g NThe images of the square of the absolute value of G(r,ω) obtained by equation (9) above based on (r) are shown for frequencies f=0.44Hz, 8.80Hz, and 18.33Hz, respectively. Figure 19 shows the distribution of (a) the average frequency obtained by equation (10) above based on G(r,ω), (b) the sharpness of the frequency distribution obtained by equation (11) above based on G(r,ω), and (c) the total vibration energy obtained by equation (12) above based on G(r,ω).

[0086] Comparing the simulation results for the fourth extraction method (Figures 18 and 19) with the exact solution (Figures 12 and 13), the average frequency shows good agreement between the two, but the sharpness of the frequency distribution and the total energy of the oscillations do not. This is thought to be because the transmission matrix is ​​effective in a steady state but not in a time-varying state. When the object of observation is a cell, the center frequency of time variation is thought to differ depending on the size of the organelle, so it is expected that organelles can be discriminated by frequency discrimination even in this fourth extraction method.

[0087] Figure 20 shows other simulation results for the first sampling method. This figure shows the distribution of the variance h(r) obtained from equations (13) and (14) above. The value of M was set to 2 when calculating the moving average in equation (13).

[0088] Figures 21 and 22 show yet another result of the simulation for the first extraction method. Here, the complex amplitude image u described in Figure 8 1,1 (r)~u 1,N (r) and complex amplitude image u 2,1 (r)~u 2,N (r) in addition to the complex amplitude image u 3,1 (r)~u 3,N (r) was obtained. Time t n , t n +T, t n The irradiation conditions to the object being observed are the same for both +2T. Figure 21 shows the complex amplitude image obtained from the simulation. This figure shows the complex amplitude image u3,1 (r)~u 3,N (r) Complex amplitude image u at time t=13.64s 3,1 (r) Complex amplitude image u at time t=16.04s 3,121 (r), and the complex amplitude image u at time t=18.44s 3,241 (r) represents the real part of each. Figure 22 shows the distribution of (a) the average frequency obtained by equation (10) above, (b) the sharpness of the frequency distribution obtained by equation (11) above, and (c) the total energy of the vibration obtained by equation (12) above. This figure is a complex amplitude image u 1,1 (r)~u 1,N (r) and complex amplitude image u 3,1 (r)~u 3,N This was obtained from (r).

[0089] Comparing Figure 11 and Figure 22, the frequency information that was missing in Figure 11 is available in Figure 22. This is thought to be due to the following reason: In equation (9) above, G(r,ω), the right-hand side is φ(r,t n Since the Fourier transform Φ(r,ω) of ) is multiplied by the window function (sin(ωT / 2)), some frequency components are lost. However, the window function of G(r,ω) used in Figure 11 is sin(ωT / 2), while the window function of G(r,ω) used in Figure 22 is sin(ωT). Therefore, it is thought that the frequencies at which information is lost are different between Figure 11 and Figure 22. It is expected that a more accurate analysis will be possible by analyzing Figure 11 and Figure 22 together.

[0090] The simulation results described so far involved the extraction and analysis of the temporal variation component of the observed object's image across a single plane. Below, we will describe the conditions and results of simulations in which the extraction and analysis of the temporal variation component of the observed object's image across multiple planes.

[0091] The simulation conditions are as follows: The object under observation is assumed to be a three-dimensional object, and the phase φ of each of the multiple spherical regions arranged in three dimensions within the object is assumed to vary at various frequencies ω within the range of 0Hz to 24.8Hz. The start time of acquiring the complex amplitude image is set to t=0. The time interval for acquiring the complex amplitude image is τ(=t). n+1 -t n The interval was set to 0.02 s. N=341. With T=Nτ, T=6.82 s. For each of the first to N irradiation conditions, a plane wave with a different irradiation direction was irradiated onto the object being observed.

[0092] Complex amplitude image at one focal plane u 1,1 (r)~u 1,N (r),u 2,1 (r)~u 2,N (r) is obtained, and based on these, complex amplitude images u on multiple other surfaces are obtained by wavefront propagation calculation. 1,1 (r)~u 1,N (r),u 2,1 (r)~u 2,N (r) was generated. The position of the focal plane from which the original complex amplitude image was obtained was set to z=0. Analysis was performed on the focal plane and several other planes using the first extraction method. Below, the z position is expressed as a ratio to the wavelength of light λ0.

[0093] Figures 23 to 26 show the simulation results on the surface z = 7.8λ0. The upper part of Figure 23 shows the complex amplitude image u 1,1 (r)~u 1,N (r) Complex amplitude image u at time t=0.00s 1,1 (r), complex amplitude image u at time t=2.40s 1,121 (r), and the complex amplitude image u at time t=4.80s 1,241 (r) represents the real part of each. The lower part of Figure 23 shows the complex amplitude image u 2,1 (r)~u 2,N (r) Complex amplitude image u at time t=6.82s 2,1 (r) Complex amplitude image u at time t=9.22s 2,121 (r), and the complex amplitude image u at time t=11.62s 2,241(r) represents the real part of each. The upper part of Figure 24 is u 1,1 (r)~u 1,N (r) and u 2,1 (r)~u 2,N Based on (r), c1(r)~c obtained in equation (1) above N (r) of c1(r), c 121 (r) and c 241 (r) Represents the real part of each image. The lower part of Figure 24 shows c1(r)~c N Based on (r), g1(r)~g obtained in equation (4) above N (r) g1(r), g 121 (r) and g 241 (r) represents the real part of each image. Figure 25 shows g1(r)~g N The images of the absolute values ​​of G(r,ω) obtained by equation (9) above based on (r) are shown for frequencies f=0.44Hz, 8.80Hz, and 18.33Hz, respectively. Figure 26 shows the distributions of (a) the average frequency obtained by equation (10) above based on G(r,ω), (b) the sharpness of the frequency distribution obtained by equation (11) above based on G(r,ω), and (c) the total vibration energy obtained by equation (12) above based on G(r,ω).

[0094] Figures 27 to 30 show the simulation results on the surface z = 23.4λ0. The upper part of Figure 27 shows the complex amplitude image u 1,1 (r)~u 1,N (r) Complex amplitude image u at time t=0.00s 1,1 (r), complex amplitude image u at time t=2.40s 1,121 (r), and the complex amplitude image u at time t=4.80s 1,241 (r) represents the real part of each. The lower part of Figure 27 shows the complex amplitude image u 2,1 (r)~u 2,N (r) Complex amplitude image u at time t=6.82s 2,1 (r) Complex amplitude image u at time t=9.22s 2,121 (r), and the complex amplitude image u at time t=11.62s 2,241 (r) represents the real part of each. The upper part of Figure 28 is u 1,1 (r)~u1,N (r) and u 2,1 (r)~u 2,N Based on (r), c1(r)~c obtained in equation (1) above N (r) of c1(r), c 121 (r) and c 241 (r) Represents the real part of each image. The lower part of Figure 28 shows c1(r)~c N Based on (r), g1(r)~g obtained in equation (4) above N (r) g1(r), g 121 (r) and g 241 (r) represents the real part of each image. Figure 29 shows g1(r)~g N The images of the absolute values ​​of G(r,ω) obtained by equation (9) above based on (r) are shown for frequencies f=0.44Hz, 8.80Hz, and 18.33Hz, respectively. Figure 30 shows the distributions of (a) the average frequency obtained by equation (10) above based on G(r,ω), (b) the sharpness of the frequency distribution obtained by equation (11) above based on G(r,ω), and (c) the total vibration energy obtained by equation (12) above based on G(r,ω).

[0095] Figures 31 to 34 show the simulation results on the surface z = 33.8λ0. The upper part of Figure 31 shows the complex amplitude image u 1,1 (r)~u 1,N (r) Complex amplitude image u at time t=0.00s 1,1 (r), complex amplitude image u at time t=2.40s 1,121 (r), and the complex amplitude image u at time t=4.80s 1,241 (r) represents the real part of each. The lower part of Figure 31 shows the complex amplitude image u 2,1 (r)~u 2,N (r) Complex amplitude image u at time t=6.82s 2,1 (r) Complex amplitude image u at time t=9.22s 2,121 (r), and the complex amplitude image u at time t=11.62s 2,241 (r) represents the real part of each. The upper part of Figure 32 is u 1,1 (r)~u 1,N (r) and u 2,1 (r)~u 2,NBased on (r), c1(r)~c obtained in equation (1) above N (r) of c1(r), c 121 (r) and c 241 (r) Represents the real part of each image. The lower part of Figure 32 shows c1(r)~c N Based on (r), g1(r)~g obtained in equation (4) above N (r) g1(r), g 121 (r) and g 241 (r) represents the real part of each image. Figure 33 shows g1(r)~g N The images of the absolute values ​​of G(r,ω) obtained by equation (9) above based on (r) are shown for frequencies f=0.44Hz, 8.80Hz, and 18.33Hz, respectively. Figure 34 shows the distributions of (a) the average frequency obtained by equation (10) above based on G(r,ω), (b) the sharpness of the frequency distribution obtained by equation (11) above based on G(r,ω), and (c) the total vibration energy obtained by equation (12) above based on G(r,ω).

[0096] As can be seen from Figures 23 to 34, a complex amplitude image is obtained by measurement at one focal plane, the wavefront of light represented by that complex amplitude image is propagated to one or more other planes, and the complex amplitude image representing the wavefront of light at those other planes is calculated. In this way, the temporal variation component of the image of the object being observed can be extracted based on the complex amplitude image at the other planes as well, and the extracted temporal variation component can be analyzed.

[0097] Next, I will explain the experimental results. Living HepG2 (human liver cancer-derived cell line) was used as the object of observation. The time interval for acquiring complex amplitude images was τ(=t). n+1 -t nThe time interval was set to 16.6 ms, N=333, and T=Nτ. For each of the first to Nth irradiation conditions, plane waves with different irradiation directions were irradiated onto the object being observed. The position of the focal plane from which the complex amplitude image was acquired was set to z=0. At time 0, the temperature of HepG2 was set to 0°C, and thereafter the temperature of HepG2 was gradually increased until it reached 37°C at time 20 min. Analysis was performed on the focal plane and several other planes using the first extraction method.

[0098] Figure 35 shows the experimental results for the HepG2 surface at z=10.0 μm at time 0 min. Figure 36 shows the experimental results for the HepG2 surface at z=20.0 μm at time 0 min. Figure 37 shows the experimental results for the HepG2 surface at z=10.0 μm at time 2 min. Figure 38 shows the experimental results for the HepG2 surface at z=20.0 μm at time 2 min. These figures represent the distributions of (a) refractive index, (b) mean frequency, (c) sharpness of the frequency distribution, and (d) total vibration energy.

[0099] As can be seen from these figures, the difference in the HepG2 image due to temperature differences is small in the refractive index distribution (Figure (a)). However, in the mean frequency distribution (Figure (b)), the sharpness distribution of the frequency distribution (Figure (c)), and the total vibrational energy distribution (Figure (d)), the difference in the HepG2 image due to temperature differences is large, indicating that HepG2 metabolism is enhanced at higher temperatures. The difference in the HepG2 image due to temperature differences is particularly large in the total vibrational energy distribution (Figure (d)).

[0100] As described above, in this embodiment, the image acquisition unit 61 illuminates the object of observation with light from the nth irradiation condition among the first to Nth different irradiation conditions at time t n and time (t n Irradiate at +T) and time t n The complex amplitude image of light at the focal plane u 1,n Get the time (t n The complex amplitude image of light at the focal plane (+T) u 2,n The wavefront propagation calculation unit 62 obtains the complex amplitude image u at the focal plane for each n.1,n (r),u 2,n (r) The wavefronts of light represented by each are propagated to one or more other surfaces, and a complex amplitude image u representing the wavefronts of light at those other surfaces is obtained. 1,n (r),u 2,n (r) is calculated. Then, the analysis unit 63 calculates the complex amplitude image u for the focal plane and other planes respectively. 1,1 ~u 1,N ,u 2,1 ~u 2,N Based on this, the temporal variation component of the image of the observed object is extracted, and this extracted temporal variation component is analyzed.

[0101] This suppresses the increase in the number of images that the image acquisition unit 61 must acquire through measurement, and enables the acquisition of three-dimensional functional information of the object being observed. Therefore, it is possible to suppress the increase in measurement time required for image acquisition, suppress the increase in storage capacity required for image storage, and suppress the increase in time required for image processing.

[0102] When the object being observed is a cell (or cell aggregate), the analysis unit 63 can evaluate the function of the cell by analyzing the temporal variation component of the cell image. For example, the analysis unit 63 can determine whether a cell is alive or dead, evaluate the quality of fertilized eggs, and evaluate the aging state of cells. It is expected that such evaluations will enable highly accurate evaluation of cell quality in regenerative medicine, efficacy and toxicity of drugs in drug discovery, and quality of sperm, eggs, and embryos in assisted reproductive technology.

[0103] Furthermore, in this embodiment, the image acquisition unit 61 illuminates the object of observation with light from the nth irradiation condition among the first to Nth different irradiation conditions at time t n and time (t n Irradiate at +T) and time t n The complex amplitude image of light at the focal plane u 1,n Get the time (t n The complex amplitude image of light at the focal plane (+T) u 2,nSince it acquires these images, the analysis unit 63 can generate a phase image or a phase differential image at the focal plane and other planes, and can also generate a refractive index distribution image. By analyzing the temporal variation component of the image of the object being observed together with these images, the analysis unit 63 can evaluate the object being observed with higher accuracy.

[0104] The object to be observed may be irradiated with light that induces temporal variations in its image. For example, a protein capable of manipulating the refractive index by light irradiation can be embedded in a cell, and the refractive index distribution of the cell can be changed by irradiating the cell with light. According to this embodiment, such a protein can be evaluated, and the temporal change in the refractive index distribution of a cell in which such a protein is embedded can also be evaluated.

[0105] Furthermore, for example, a mid-infrared thermal microscope irradiates an object to be observed with mid-infrared light, causing the object to absorb the mid-infrared light through molecular vibrations, generating heat. This expansion of the object's volume causes a local change in refractive index, which can then be detected using visible light. According to this embodiment, it is also possible to perform mid-infrared spectroscopic imaging of such cells. [Explanation of symbols]

[0106] 1A~1C... Observation device, 2... Recording medium, 10... Light source, 11... Lens, 12... Incident light end, 13... Optical fiber, 14... Outgoing light end, 15... Fiber coupler, 16,17... Optical fiber, 18,19... Outgoing light end, 21... Lens, 22... Mirror, 23... Lens, 24... Condenser lens, 41... Objective lens, 42... Beam splitter, 43... Lens, 44... Mirror, 45... Focal plane position setting unit, 50... Imaging unit, 60... Processing unit, 61... Image acquisition unit, 62... Wavefront propagation calculation unit, 63... Analysis unit, 64... Display unit, 65... Memory unit.

Claims

1. Light under the nth irradiation condition (where n is an integer between 1 and N) from among the first to nth irradiation conditions (where N is an integer of 2 or more) which differ from each other in terms of the spatial intensity or phase distribution of light, is directed at the object of observation at time t ｎ and time (t ｎ Irradiate at +T) and time t ｎ The complex amplitude image of light at the focal plane u 1,n Get the time (t ｎ +T) is the complex amplitude image of light at the focal plane u 2,n An image acquisition unit that acquires an image, For each n, the complex amplitude image u on the focal plane 1,n , u 2,n A complex amplitude image u representing the optical wavefront on one or more other surfaces when the optical wavefronts represented by each are propagated to the one or more other surfaces 1,n , u 2,n A wavefront propagation calculation unit that obtains by calculation In the focal plane and the other plane, a complex amplitude image u 1,1 ~u 1,N , u 2,1 ~u 2,N An analysis unit extracts the temporal variation component of the image of the object being observed based on this, and analyzes that temporal variation component. An observation device equipped with the following features.

2. The analysis unit analyzes the complex amplitude image u 1,n and complex amplitude image u 2,n The product of the complex conjugate image of one complex amplitude image and the other complex amplitude image, the complex amplitude image u 1,n and complex amplitude image u 2,n The result obtained by dividing one of the complex amplitude images by the other complex amplitude image, or the complex amplitude image u 1,n and complex amplitude image u 2,n The difference between this and the above is extracted as the temporal variation component of the image of the object being observed. The observation apparatus according to claim 1.

3. The analysis unit analyzes the complex amplitude image u 1,n Phase and complex amplitude image u 2,n The difference from the phase is extracted as the temporal variation component of the image of the object being observed. The observation apparatus according to claim 1.

4. The analysis unit performs the analysis based on the results obtained by performing a Fourier transform on the temporal variation component of the image of the object being observed with respect to the time variable. The observation apparatus according to claim 1.

5. The analysis unit performs an evaluation of the function of the cells or cell aggregates by analyzing the temporal variation component of the image of the cells or cell aggregates that are the objects of observation. The observation apparatus according to claim 1.

6. The image acquisition unit irradiates the object to be observed with plane waves having different irradiation directions, for each of the first to N irradiation conditions. The observation apparatus according to claim 1.

7. Light under the nth irradiation condition (where n is an integer between 1 and N) from among the first to nth irradiation conditions (where N is an integer of 2 or more) which differ from each other in terms of the spatial intensity or phase distribution of light, is directed onto the object of observation at time t ｎ and time (t ｎ Irradiate at +T) and time t ｎ The complex amplitude image of light at the focal plane u 1,n Get the time (t ｎ +T) is the complex amplitude image of light at the focal plane u 2,n Image acquisition step to obtain, For each n, the complex amplitude image u at the focal plane 1,n , u 2,n A complex amplitude image u representing the wavefront of light on one or more other surfaces when the wavefronts of light represented by each are propagated to those other surfaces. 1,n , u 2,n A wavefront propagation calculation step that calculates the following, In the focal plane and the other plane, a complex amplitude image u 1,1 ~u 1,N , u 2,1 ~u 2,N An analysis step in which the temporal variation component of the image of the object being observed is extracted based on the above, and the temporal variation component is analyzed, An observation method that includes the following features.

8. A program for causing a computer to perform each step of the observation method described in claim 7.

9. A computer-readable recording medium on which the program described in claim 8 is recorded.

Images