Observation device and observation method

Abstract

This observation device includes an image acquisition unit 61, a wavefront propagation calculation unit 62, and an analysis unit 63. The image acquisition unit 61 irradiates an observation object with light of a n-th irradiation condition among the 1st to N-th irradiation conditions at time tn and time tn + T, acquires a complex amplitude image u1,n of light at a focal plane at time tn, and acquires a complex amplitude image u2,n of light at the focal plane at time (tn + T). The wavefront propagation calculation unit 62 causes a wavefront of light represented by complex amplitude images u1,n (r) and u2,n (r) on the focal plane to propagate to another plane, and obtains complex amplitude images u1,n (r) and u2,n (r) representing the wavefront of light on the other plane at that time by calculation. The analysis unit 63 extracts a temporal variation component of the image of the observation object on the basis of the complex amplitude images u1,1 to u1,N and u2,1 to u2,N on the focal plane and the other plane, and analyzes the temporal variation component. Thus, an observation device capable of acquiring three-dimensional function information of an observation object by suppressing an increase in the number of images to be acquired is realized.

Description

Observation device and observation method

[0001] The present disclosure relates to an observation device and an observation method.

[0002] Known techniques for non-staining and non-invasive imaging of observation objects such as cells include phase contrast microscopes, differential interference microscopes, optical coherence tomography (OCT), and optical diffraction tomography (ODT).

[0003] These imaging techniques can obtain information about the morphology of an object by utilizing the contrast of the refractive index or reflectance of the object, but they cannot obtain information about the function of the object.

[0004] If it becomes possible to obtain information about the function of cells as the object of observation, it is expected that it will be possible to evaluate with high accuracy the quality of cells in regenerative medicine, the efficacy and toxicity of drugs in drug discovery, and the quality of sperm, eggs, embryos, etc. in assisted reproductive technology.

[0005] Non-staining imaging techniques intended to enable acquisition of information about the function of an observed object have been proposed in Patent Document 1 and Non-Patent Documents 1 and 2. The imaging techniques described in Patent Document 1 and Non-Patent Document 1 acquire a large number of OCT images at a certain time interval in one focal plane. The imaging technique described in Non-Patent Document 2 acquires a large number of approximate two-dimensional refractive index images at a certain time interval in one focal plane.

[0006] These imaging techniques can visualize metabolic contrast information based on the motility of an object by analyzing the temporal fluctuation of the image of the object based on multiple images acquired at a certain time interval. These imaging techniques enable the acquisition of biologically meaningful functional information, namely metabolic contrast, which cannot be obtained by conventional non-staining imaging.

[0007] U.S. Patent No. 1,543,641

[0008] H. Hugonnet et al., "Improving specificity and axial spatial resolution of refractive index imaging by exploiting uncorrelated subcellular dynamics," ACS Photonics, 2023, https: / / doi.org / 10.1021 / acsphotonics.3c01236C. Apelian et al., "Dynamic full-field optical coherence tomography: subcellular metabolic contrast revealed in tissues by interferometric signals temporal analysis," Biomedical Optics Express, Vol. 7, No. 4, pp. 1511-1524, 2016. Tomohiro Shirai, "Phase information hidden in intensity distributions," Optics, Vol. 38, No. 10, pp. 496-502, 2009.

[0009] Using the imaging techniques described in Patent Document 1 and Non-Patent Documents 1 and 2, it is possible to obtain functional information on the three-dimensional metabolic contrast of an object by analyzing the temporal fluctuations of the image of the object based on a large number of images acquired at certain time intervals at multiple focal planes. However, in this case, the number of images to be acquired becomes enormous, for example, tens of thousands to hundreds of thousands of images must be acquired. This leads to various practical problems, such as an increase in the measurement time required to acquire the images, an increase in the memory capacity required to store the images, and an increase in the time required for image processing.

[0010] The embodiments aim to provide an observation device and an observation method that can acquire three-dimensional functional information of an observation object while suppressing an increase in the number of images to be acquired.

[0011] The embodiment is an observation device. The observation device (1) irradiates an object to be observed with light under an n-th irradiation condition (n is an integer of 1 to N) among first to N-th irradiation conditions (N is an integer of 2 or more) that are different from one another in terms of spatial intensity or phase distribution of the light, at a time t ｎ and time (t ｎ +T) and at time t ｎ The complex amplitude image u of light at the focal plane 1,n Obtain the time (t ｎ +T) is the complex amplitude image u of light at the focal plane 2,n (2) an image acquisition unit that acquires a complex amplitude image u at the focal plane for each n; 1,n , u 2,n A complex amplitude image u representing the wavefront of light at one or more other surfaces when the wavefront of light represented by each of the images propagates to the other surfaces. 1,n , u 2,n (3) a wavefront propagation calculation unit that calculates a complex amplitude image u 1,1 ~u 1,N , u 2,1 ~u 2,N and an analysis unit that extracts a time-varying component of the image of the object to be observed based on the time-varying component and analyzes the time-varying component.

[0012] The embodiment is an observation method. The observation method includes: (1) irradiating an object with light under an n-th irradiation condition (n is an integer of 1 to N) among first to N-th irradiation conditions (N is an integer of 2 or more) that are different from each other in terms of spatial intensity or phase distribution of light, at a time t ｎ and time (t ｎ +T) and at time t ｎ The complex amplitude image u of light at the focal plane 1,n Obtain the time (t ｎ +T) is the complex amplitude image u of light at the focal plane 2,n (2) for each n, a complex amplitude image u at the focal plane is acquired; 1,n , u 2,n A complex amplitude image u representing the wavefront of light at one or more other surfaces when the wavefront of light represented by each of the images propagates to the other surfaces. 1,n , u 2,n(3) a wavefront propagation calculation step for calculating a complex amplitude image u 1,1 ~u 1,N , u 2,1 ~u 2,N and an analyzing step of extracting a time-varying component of the image of the object to be observed based on the extracted time-varying component and analyzing the time-varying component.

[0013] The embodiment is a program for causing a computer to execute each step of the observation method having the above-described configuration.

[0014] The embodiment is a recording medium, which is a computer-readable recording medium having the program configured as described above recorded thereon.

[0015] According to the observation device and observation method of the embodiment, it is possible to acquire three-dimensional functional information of an observation object while suppressing an increase in the number of images to be acquired.

[0016] FIG. 1 is a diagram showing the configuration of the observation apparatus 1A. FIG. 2 is a diagram showing the configuration of the observation apparatus 1B. FIG. 3 is a diagram showing the configuration of the observation apparatus 1C. FIG. 4 is a diagram showing the incidence of the first branched light and the second branched light on the observation object S in the observation apparatus 1C, and the incidence of the first branched light and the second branched light on the imaging unit 50 after passing through the observation object S. FIG. 5 is a diagram showing the configuration of the processing unit 60 of the observation apparatus. FIG. 6 is a flowchart of the observation method. FIG. 7 is a diagram showing the conditions of the simulation. FIG. 8 is a diagram showing a complex amplitude image obtained by the simulation. FIG. 9 is a diagram showing the simulation results for the first extraction mode. FIG. 10 is a diagram showing the simulation results for the first extraction mode. FIG. 11 is a diagram showing the simulation results for the first extraction mode (a) to (c). FIG. 12 is a diagram showing an exact solution calculated from the exact phase information of the observation object. FIG. 13 is a diagram showing an exact solution calculated from the exact phase information of the observation object (a) to (c). FIG. 14 is a diagram showing simulation results for the third extraction mode. FIG. 15 is a diagram showing simulation results for the third extraction mode. FIG. 16 is a diagram showing simulation results for the third extraction mode (a) to (c). FIG. 17 is a diagram showing simulation results for the fourth extraction mode. FIG. 18 is a diagram showing simulation results for the fourth extraction mode. FIG. 19 is a diagram showing simulation results for the fourth extraction mode (a) to (c). FIG. 20 is a diagram showing other simulation results for the first extraction mode. FIG. 21 is a diagram showing still other simulation results for the first extraction mode. FIG. 22 is a diagram showing still other simulation results for the first extraction mode (a) to (c). FIG. 23 is a diagram showing simulation results for z=7.8λ ０ 24 shows the simulation results for the plane z=7.8λ. ０ 25 shows the simulation results for the plane z=7.8λ. ０ 26A to 26C are diagrams showing the simulation results for the plane z=7.8λ. ０ 27 shows the simulation results for the plane z=23.4λ. ０28 shows the simulation results for the plane z=23.4λ. ０ 29 shows the simulation results for the plane z=23.4λ. ０ 30A to 30C are diagrams showing the simulation results for the plane z=23.4λ. ０ 31 shows the simulation results for the plane z=33.8λ. ０ 32 shows the simulation results for the plane z=33.8λ. ０ 33 shows the simulation results for the plane z=33.8λ. ０ 34A to 34C are diagrams showing the simulation results for the plane z=33.8λ. ０ 3 shows the simulation results for the z=10.0 μm surface of HepG2 at time 0 min (a) to (d). FIG. 36 shows the experimental results for the z=20.0 μm surface of HepG2 at time 0 min (a) to (d). FIG. 37 shows the experimental results for the z=10.0 μm surface of HepG2 at time 2 min (a) to (d). FIG. 38 shows the experimental results for the z=20.0 μm surface of HepG2 at time 2 min (a) to (d).

[0017] Hereinafter, embodiments of the observation device and observation method will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicate explanations will be omitted. The present invention is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

[0018] The observation device and observation method of this embodiment acquire a complex amplitude image at a certain focal plane and perform required processing based on the acquired complex amplitude image. Various configurations are possible for acquiring the complex amplitude image, and the following description will use observation devices 1A to 1C as examples, but the present invention is not limited to these.

[0019] 1 is a diagram showing the configuration of an observation device 1A, which includes a light source 10, a lens 11, a lens 21, a mirror 22, a lens 23, a condenser lens 24, an objective lens 41, a beam splitter 42, a lens 43, an imaging unit 50, a processing unit 60, and the like.

[0020] The light source 10 outputs spatially and temporally coherent light, and is preferably a laser light source. The lens 11 is optically connected to the light source 10 and focuses the light output from the light source 10 onto a light input end 12 of an optical fiber 13, causing the light to enter the light input end 12. The optical fiber 13 guides the light input to the light input end 12 by the lens 11 to a fiber coupler 15.

[0021] The fiber coupler 15 couples light between the optical fiber 13 and the optical fibers 16 and 17, and splits the light guided by the optical fiber 13 into two beams, one of which is guided by the optical fiber 16 and the other by the optical fiber 17. The light guided by the optical fiber 16 is emitted as divergent light from a light emitting end 18. The light guided by the optical fiber 17 is emitted as divergent light from a light emitting end 19.

[0022] The lens 21 is optically connected to the light emitting end 18 and collimates the light output as divergent light from the light emitting end 18. The mirror 22 is optically connected to the lens 21 and reflects the light reaching it from the lens 21 to the lens 23. The orientation of the reflecting surface of the mirror 22 is variable. The lens 23 is optically connected to the mirror 22. The condenser lens 24 is optically connected to the lens 23. The lens 23 and the condenser lens 24 preferably constitute a 4f optical system.

[0023] The lens 23 and the condenser lens 24 irradiate the observation object S with light from a light irradiation direction that corresponds to the orientation of the reflecting surface of the mirror 22. The objective lens 41 is optically connected to the condenser lens 24. The observation object S is disposed between the objective lens 41 and the condenser lens 24. The objective lens 41 receives light (object light) that has been output from the condenser lens 24 and passed through the observation object S, and outputs the light to the beam splitter 42.

[0024] The beam splitter 42 is optically connected to the objective lens 41 and also to the light emitting end 19. The beam splitter 42 combines the light (object light) output from the objective lens 41 and reaching it and the light (reference light) output from the light emitting end 19 and reaching it, and outputs both beams to the lens 43. The lens 43 is optically connected to the beam splitter 42, and collimates the object light and the reference light that have reached it from the beam splitter 42, and outputs them to the imaging unit 50.

[0025] The imaging unit 50 is optically connected to the lens 43 and captures an interference fringe image (interference intensity image) resulting from the interference between the object light and the reference light arriving from the lens 43. The direction of incidence of the reference light is tilted relative to the direction of incidence of the object light on the imaging surface of the imaging unit 50. The position where the object light and the reference light are combined by the beam splitter 42 may be after the imaging lens, but considering the influence of aberration, it is preferably between the objective lens 41 and the lens 43 as shown in the figure.

[0026] The processing unit 60 has an output port that outputs a control signal for controlling the orientation of the reflecting surface of the mirror 22, and controls the conditions for irradiating light onto the observation object S using this control signal. The processing unit 60 also has an input port for inputting data of the interference intensity image captured by the imaging unit 50, and generates a complex amplitude image based on this interference intensity image using Fourier fringe analysis, and performs required processing based on this complex amplitude image. Details of the processing unit 60 will be described later.

[0027] 2 is a diagram showing the configuration of the observation device 1B, which includes a light source 10, a lens 11, a lens 21, a mirror 22, a lens 23, a condenser lens 24, an objective lens 41, a lens 43, a mirror 44, a focal plane position setting unit 45, an imaging unit 50, a processing unit 60, and the like.

[0028] The optical system from the light source 10 to the optical fiber 13 in the configuration of the observation device 1B ( FIG. 2 ) is the same as the configuration of the observation device 1A ( FIG. 1 ). In the observation device 1B, the optical fiber 13 guides light that has been incident on the light incident end 12 by the lens 11 to the light emitting end 14. The light guided by the optical fiber 13 is output from the light emitting end 14 to the lens 21 as divergent light. The lens 21 is optically connected to the light emitting end 14, and collimates the light that has been output from the light emitting end 14 as divergent light and outputs it to the mirror 22.

[0029] The optical system from the lens 21 to the objective lens 41 in the configuration of the observation device 1B ( FIG. 2 ) is the same as the configuration of the observation device 1A ( FIG. 1 ). In the observation device 1B, the objective lens 41 inputs light that has been output from the condenser lens 24 and passed through the observation target S, and outputs the light to the mirror 44. The lens 43 collimates the light that has been output from the objective lens 41 and reflected by the mirror 44, and outputs the light to the imaging unit 50.

[0030] The focal plane position setting unit 45 sets the position of the focal plane (a plane optically conjugate with respect to the imaging plane of the imaging unit 50) by adjusting the position of the objective lens 41 along a direction parallel to the optical axis of the objective lens 41. The focal plane position setting unit 45 may include, for example, a movable stage or a piezoelectric actuator. The imaging unit 50 is optically connected to the lens 43, and captures an intensity image at the focal plane set by the focal plane position setting unit 45 based on light reaching from the lens 43.

[0031] The processing unit 60 has an output port that outputs a control signal for controlling the orientation of the reflecting surface of the mirror 22, and this control signal controls the conditions for irradiating the observation object S with light. The processing unit 60 also has an output port that outputs a control signal for controlling the position of the focal plane set by the focal plane position setting unit 45, and this control signal controls the position of the focal plane at which the imaging unit 50 should capture an intensity image.

[0032] The processing unit 60 also has an input port for inputting data of the intensity image captured by the imaging unit 50, generates a complex amplitude image based on this intensity image, and performs required processing based on this complex amplitude image. The processing unit 60 can generate a complex amplitude image from the intensity images at two different focal planes based on the transport of intensity equation (TIE) (Non-Patent Document 3).

[0033] 3 is a diagram showing the configuration of the observation device 1C. This observation device 1C includes a light source 10, a lens 11, a lens 21, an irradiation unit 31, an objective lens 41, a lens 43, a mirror 44, an imaging unit 50, a processing unit 60, and the like.

[0034] The optical system from the light source 10 to the lens 21 in the configuration of the observation device 1C (FIG. 3) is the same as the configuration of the observation device 1B (FIG. 2). In the observation device 1C, the lens 21 collimates the light output as divergent light from the light output end 14 and outputs it to the irradiation unit 31.

[0035] The illumination unit 31 receives light output from the light source 10 and transmitted through the lens 11, the optical fiber 13, and the lens 21, and generates first light and second light from the received light. The illumination unit 31 also superimposes the first light and second light on each other and illuminates the observation object S. The illumination unit 31 illuminates the observation object S with the first light along a fixed light illumination direction, and illuminates the observation object S with the second light along each of a plurality of light illumination directions.

[0036] The irradiation unit 31 includes a beam splitter 311 , a phase-modulating spatial light modulator 313 , a polarizer 314 , a half-wave plate 315 , a polarizer 316 , a lens 318 and an objective lens 319 .

[0037] The beam splitter 311 reflects light that has reached it via a polarizer 314 and a half-wave plate 315, which are provided between the beam splitter 311 and the lens 21, to the spatial light modulator 313. The beam splitter 311 also receives light that has reached it from the spatial light modulator 313, and outputs this light to a polarizer 316.

[0038] The spatial light modulator 313 selectively phase-modulates the linearly polarized light in the second direction, without phase-modulating the linearly polarized light in the first direction, among the linearly polarized light in the first and second directions that are orthogonal to each other and that is incident on the modulation surface of the spatial light modulator 313. The polarizer 314 and the half-wave plate 315 set the polarization state of the light so that the light that is incident on the modulation surface of the spatial light modulator 313 from the beam splitter 311 contains linearly polarized components in the first and second directions to the same extent.

[0039] The polarizer 316 inputs the light that has arrived from the spatial light modulator 313 via the beam splitter 311 and enables interference between the linearly polarized light in the first and second directions contained in the light. The polarizer 316 has an optical axis whose orientation is 45 degrees different from the polarization orientation of the light (linearly polarized light in the first and second directions) that has arrived from the spatial light modulator 313 via the beam splitter 311, and selectively transmits the polarized component of the input light in the orientation of the optical axis. The lens 318 and the objective lens 319 irradiate the first light and the second light output from the polarizer 316 onto the observation object S as plane waves.

[0040] The irradiation unit 31 having such a configuration can treat linearly polarized light in a first direction that has not been phase-modulated by the spatial light modulator 313 as first light, and can irradiate this first light along a fixed light irradiation direction onto the observation object S. The irradiation unit 31 can treat linearly polarized light in a second direction that has been phase-modulated by the spatial light modulator 313 as second light, and can irradiate this second light along each of a plurality of light irradiation directions onto the observation object S.

[0041] The direction of irradiation of the second light onto the observation object S can be set by the orientation and spacing of the phase modulation pattern on the modulation surface of the spatial light modulator 313. Furthermore, the phase difference between the first light and the second light can be set by shifting the phase modulation pattern on the modulation surface of the spatial light modulator 313.

[0042] The optical system from the objective lens 41 to the imaging unit 50 in the configuration of the observation device 1C ( FIG. 3 ) is the same as the configuration of the observation device 1B ( FIG. 2 ). In the observation device 1C, the objective lens 41 inputs light (first light and second light) that has been irradiated onto the observation object S by the irradiation unit 31 and passed through the observation object S, and outputs the light to the mirror 44. The lens 43 inputs the light that has been output from the objective lens 41 and reflected by the mirror 44, and causes the light to be incident on the imaging surface of the imaging unit 50.

[0043] The imaging unit 50 receives both the first light and the second light that have reached the imaging surface from the lens 43, and captures an interference intensity image resulting from interference between the first light and the second light. The imaging unit 50 captures interference intensity images when the phase difference between the first light and the second light is set to each of a plurality of phase differences for each of a plurality of light irradiation directions of the second light.

[0044] The processing unit 60 has an output port that outputs a control signal for controlling the modulation of light by the spatial light modulator 313, and uses this control signal to control the conditions for irradiating light onto the observation object S. The processing unit 60 also has an input port for inputting data of the interference intensity image captured by the imaging unit 50, generates a complex amplitude image based on this interference intensity image, and performs required processing based on this complex amplitude image.

[0045] The details of the complex amplitude image generation process in this observation device 1C are as follows: Fig. 4 is a diagram schematically showing the incidence of the first branched light and the second branched light on the observation object S, and the incidence of the first branched light and the second branched light on the imaging unit 50 after passing through the observation object S, in the observation device 1C.

[0046] The irradiating unit 31 irradiates the first branched light and the second branched light in an overlapping manner onto the observation object S. At this time, the light irradiation direction of the first branched light onto the observation object S is constant, the light irradiation direction of the second branched light onto the observation object S is set to each of a plurality of light irradiation directions, and the phase difference φ between the first branched light and the second branched light is set to each value.

[0047] The wavefront of the first branched light incident on the observation object S is denoted by u ０,inThe wavefront of the second branched light incident on the observation object S along the n-th light irradiation direction (n=1 to N) among the multiple (N) light irradiation directions of the second branched light is represented as u ｎ,in (r)exp(iφ), where r is a variable representing the position. φ is the phase difference between the first branched light and the second branched light. The wavefront of the first branched light on the imaging plane or focal plane (a plane optically conjugate to the imaging plane) of the imaging unit 50 is expressed as u ０ (r), and the wavefront of the second branched light is represented as u ｎ (r) is expressed as exp(iφ).

[0048] An interference intensity image I obtained by imaging using the imaging unit 50 ｎ (r, φ) is u ０ (r) and u ｎ (r) exp(iφ) is expressed as the square of the absolute value of the sum. ｎ (r, φ) is an interference intensity image acquired by imaging using the imaging unit 50 when the phase difference between the first branched light and the second branched light is φ, the first branched light is incident on the observation object S along a certain light irradiation direction relative to the observation object, and the second branched light is incident on the observation object along the nth light irradiation direction.

[0049] The focal plane (a plane optically conjugate to the imaging plane) may be on the observation object S, on the imaging unit 50 side of the observation object S, or on the irradiation unit 31 side of the observation object S.

[0050] For each of the plurality of light irradiation directions of the second branched light, the interference term C is calculated by the phase shift method based on the interference intensity images acquired by the imaging unit 50 when each of the plurality of phase differences φ is set. ｎ (r) = u ０ ＊ (r) u ｎ (r) is found. Interference term u ０ (r) u ｎ ＊ (r) can be obtained. ｎ (r) is found for each of the plurality of light irradiation directions of the second split light (that is, for each n (=1 to N)).

[0051] The interference term C obtained for each of the plurality of light irradiation directions of the second branched lightｎ A complex amplitude image of the first branched light is generated based on (r). ０ Phase φ of (r) ０ (r) is the coherent sum C of the interference terms after correcting the phase gradient (difference in the light incident direction) between the first branched light and the second branched light. sum (r) is calculated, and the coherent sum C sum It can be approximately expressed by the phase of (r).

[0052] The complex amplitude u of the first branched light ０ Amplitude A of (r) ０ (r) is an intensity image |u ０ (r) | ２ Alternatively, the complex amplitude u of the first branched light can be calculated from ０ Amplitude A of (r) ０ (r) is the interference term C ｎ (r) Intensity sum I sum It can be approximately expressed as the square root of (r).

[0053] The complex amplitude u of the first branched light obtained as described above ０ Phase φ of (r) ０ (r) and amplitude A ０ (r), a complex amplitude image u of the first branched light is obtained. ０ Then, a complex amplitude image u of the first branched light can be generated. ０ (r) and the interference term C ｎ (r), a complex amplitude image u of the second branched light in each of the plurality of light irradiation directions is obtained. ｎ (r) can be generated.

[0054] Furthermore, in this observation device 1C, after generating a complex amplitude image of the second branched light for each of the multiple focal planes, the complex amplitude image of any one of the multiple focal planes may be updated based on a complex amplitude image representing the wavefront of light when the wavefront of light represented by the complex amplitude image of the other focal plane is propagated to the other focal plane. By repeating this updating process between the multiple complex amplitude images, the complex amplitude images can be improved.

[0055] Next, the processing unit 60 will be described. The processing unit 60 may be a computer. The processing unit 60 includes a processing device such as a CPU, GPU, DSP, or FPGA, and a storage unit such as a hard disk drive, flash memory, RAM, or ROM. The storage unit stores various images, programs for executing processing, and the like.

[0056] Fig. 5 is a diagram showing the configuration of a processing unit 60 of the observation device. The processing unit 60 includes an image acquisition unit 61, a front propagation calculation unit 62, an analysis unit 63, a display unit 64, and a storage unit 65. Fig. 6 is a flowchart of the observation method. The observation method of this embodiment includes an image acquisition step S1 performed by the image acquisition unit 61, a front propagation calculation step S2 performed by the front propagation calculation unit 62, and an analysis step S3 performed by the analysis unit 63.

[0057] The image acquisition unit 61 applies light under an n-th irradiation condition (n is an integer of 1 to N) out of first to N-th irradiation conditions (N is an integer of 2 or more) that are different from one another in terms of the spatial intensity or phase distribution of light to an object to be observed at time t ｎ and time (t ｎ +T), and at time t ｎ The complex amplitude image u of light at the focal plane 1,n Obtain the time (t ｎ +T) is the complex amplitude image u of light at the focal plane 2,n (image acquisition step S1). ｎ and time (t ｎ +T) the n-th irradiation conditions are the same.

[0058] The object to be observed may be irradiated with plane waves having different irradiation directions as the first to Nth irradiation conditions. ｎ The complex amplitude image acquired at u 1,n (r) and time (t ｎ +T) is taken as the complex amplitude image u 2,n (r), where r is a variable representing the position on the focal plane. 1,1 ~u 1,N and the period for acquiring the complex amplitude image u 2,1 ~u 2,NThe periods for acquiring the data may partially overlap with each other.

[0059] The wavefront propagation calculation unit 62 calculates a complex amplitude image u 1,n (r), u 2,n (r) The wavefront of light represented by each of the two is propagated to one or more other surfaces, and a complex amplitude image u representing the wavefront of light at the other surface is obtained. 1,n (r), u 2,n (r) is calculated (wavefront propagation calculation step S2). The analysis unit 63 calculates the complex amplitude image u 1,1 ~u 1,N , u 2,1 ~u 2,N The time-varying component of the image of the object to be observed is extracted based on the above, and the extracted time-varying component is analyzed (analysis step S3).

[0060] The processing of the analysis step S3 may be performed after the front propagation calculation up to all other surfaces is completed in the front propagation calculation step S2. After a part of the processing of the front propagation calculation step S2 (front propagation calculation up to some of the multiple other surfaces) is completed, the processing of the analysis step S3 may be performed for those some surfaces.

[0061] The display unit 64 displays the image to be processed, the image in the middle of processing, the image after processing, etc., and includes, for example, a liquid crystal display. The storage unit 65 stores data of various images.

[0062] The storage unit 65 also stores a program for executing each process of the image acquisition step S1, the wavefront propagation calculation step S2, and the analysis step S3. This program may be stored in the storage unit 65 at the time of manufacture or shipment of the observation device, or may be acquired via a communication line after shipment and stored in the storage unit 65, or may be recorded on a computer-readable recording medium 2 and then stored in the storage unit 65. The recording medium 2 may be any medium such as a flexible disk, CD-ROM, DVD-ROM, BD-ROM, or USB memory.

[0063] Next, a description will be given of the specific processing contents of the analysis unit 63 in the analysis step S3. The analysis unit 63 may extract the time-varying component of the image of the object to be observed in the following first to fourth extraction modes.

[0064] In the first extraction mode, the analysis unit 63 extracts the complex amplitude image u 1,n (r) and the complex amplitude image u 2,n The product of the complex conjugate image of one of the complex amplitude images (r) and the other complex amplitude image is defined as the time variation component c of the image of the object to be observed. n (r) (see formula (1) below).

[0065] In the second extraction mode, the analysis unit 63 extracts the complex amplitude image u 1,n (r) and the complex amplitude image u 2,n The result obtained by dividing one of the complex amplitude images (r) by the other complex amplitude image is used as the time-varying component c of the image of the object to be observed. n (r) (see equation (2) below). In equation (2), when the denominator on the right side is 0, n Since the value of (r) is indefinite, instead, c in the following equation (3) is used. n (r) may be a time-varying component. ε is a small positive value. When ε=0, equation (3) is equal to equation (2).

[0066] In the first and second extraction modes, the analysis unit 63 extracts the complex amplitude image u 1,n Phase of (r) and complex amplitude image u 2,n The difference in phase with (r) is c n (r), and this phase difference g ｎ (r) may be the time-varying component of the image of the object to be observed (the following equation (4)).

[0067] Complex amplitude image u 1,n (r) is expressed by the following equation (5), and the complex amplitude image u 2,n If (r) is expressed by the following equation (6), then g in equation (4) ｎ (r) can be expressed by the following equation (7): j is the imaginary unit, and a(r, t ｎ) is the amplitude, and φ(r, t ｎ ) is the phase. If there is a partial region where the phase cannot be determined due to the influence of noise, in order to reduce the influence of the noise, n Based on (r), a mask m ｎ (r) and create this m ｎ (r) to g ｎ It is preferable to act on (r).

[0068] In the third extraction mode, the analysis unit 63 extracts the complex amplitude image u 1,n and the complex amplitude image u 2,n The difference between these is the time variation component g of the image of the object being observed. n (r) (see formula (8) below).

[0069] In the fourth extraction mode, the analysis unit 63 extracts the complex amplitude image u 1,n The transmission matrix T based on 1,n and the complex amplitude image u 2,n The transmission matrix T based on 2,n and calculate the transmission matrix T 1,n , T 2,n The matrix product (T 2,n -1 T 1,n , T 2,n T 1,n -1 , T 1,n -1 T 2,n or T 1,n T 2,n -1 ) and dividing this matrix product by incident wave number, the time fluctuation component c n (r) is extracted. n Phase g of (r) ｎ (r) may be the time-varying component of the image of the object being observed.

[0070] The transmission matrix describes the optical input / output relationship of the entire system, so T 2,n -1 T1,n The matrix multiplication of T 1,n After forward propagation of the light wave in the system represented by 2,n This corresponds to the operation of propagating a light wave backward through the system represented by T 2,n -1 T 1,n From the matrix product of T 1,n and T 2,n Only the part corresponding to the difference between the two can be obtained.

[0071] If the object to be observed is considered to have no absorption, the inverse matrix of the transmission matrix can be expressed as the Hermitian conjugate of the transmission matrix. The fourth extraction mode is effective when the phase change is not large.

[0072] In the first to fourth extraction modes, the analysis unit 63 may perform the analysis based on the results obtained by Fourier transforming the temporal fluctuation component of the image of the observation object with respect to the time variable. ｎ The Fourier transform G(r,ω) of (r) is φ(r,t ｎ Using the Fourier transform Φ(r,ω) of the complex amplitude image, the following equation (9) is obtained: ω is the angular frequency, and τ is the time interval between acquisitions of the complex amplitude images. ｎ (r)} n=1,…,N are added together, so when light beams with different first to Nth irradiation conditions are used, a tomographic image on the focal plane is extracted by this addition process.

[0073] The analysis unit 63 may use G(r, ω) in equation (9) to calculate the average frequency (equation (10) below), the sharpness of the frequency distribution (equation (11) below), or the total energy of vibration (equation (12) below), and perform analysis based on this. f = 2πω, where π is the constant of the circumference of a circle. Alternatively, the average frequency, the sharpness of the frequency distribution, and the total energy of vibration may be mapped to the HSV color space, and analysis may be performed based on this. For example, let H be the average frequency, S be the sharpness of the frequency distribution, and V be the total energy of vibration.

[0074] The analysis unit 63 calculates g with respect to n (i.e., with respect to the time variable). ｎ Moving average g' of (r) ｎ (r) is calculated (the following formula (13)), and the moving average g' is calculated. ｎ The variance h(r) of (r) may be calculated (Equation (14) below), and analysis may be performed based on this variance h(r). Equation (13) below represents a 2M+1 point moving average, where M is an integer equal to or greater than 1. By calculating the variance of the moving average in this way, a tomographic image on the focal plane is extracted.

[0075] Next, we will explain the simulation results. In the following, we will explain the conditions and results of a simulation performed on one surface regarding the extraction and analysis of the temporal variation components of the image of the observed object, and then explain the conditions and results of simulations performed on each of multiple surfaces.

[0076] 7 shows the simulation conditions. Here, the object to be observed is assumed to be a two-dimensional object, and the phase φ of each of the 8 × 8 circular regions arranged two-dimensionally within the object to be observed varies with various frequencies ω within the range of 0 Hz to 24.8 Hz.

[0077] The start time of complex amplitude image acquisition was set to t = 0. The time interval τ (= t ｎ＋１ -t ｎ ) was set to 0.02 s. N = 341. T = Nτ, and T = 6.82 s. Plane waves with different irradiation directions were irradiated onto the object under irradiation conditions 1 to N. This figure shows the phase distribution of the object under observation at times t = 0.00 s, 2.40 s, 4.80 s, 6.82 s, 9.22 s, and 11.62 s.

[0078] 8 shows a complex amplitude image obtained by simulation. The upper part of this figure shows the complex amplitude image u 1,1 (r) ~u 1,N (r) Complex amplitude image u at time t = 0.00 s 1,1 (r) Complex amplitude image u at time t = 2.40 s 1,121 (r), and the complex amplitude image u at time t = 4.80 s1,241 (r), and the real part of each. The bottom part of this figure shows the complex amplitude image u 2,1 (r) ~u 2,N (r) Complex amplitude image u at time t = 6.82 s 2,1 (r) Complex amplitude image u at time t = 9.22 s 2,121 (r), and the complex amplitude image u at time t = 11.62 s 2,241 (r), which represent the real parts of each.

[0079] 9 to 11 are diagrams showing the simulation results for the first extraction mode. 1,1 (r) ~u 1,N (r) and u 2,1 (r) ~u 2,N Based on (r), the c obtained by the above formula (1) １ (r) to c Ｎ (r) c １ (r), c 121 (r) and c 241 (r) represents the real part of each image. １ (r) to c Ｎ Based on (r), g obtained by the above formula (4) １ (r) to g Ｎ g of (r) １ (r), g 121 (r) and g 241 (r) represents the real part of each image.

[0080] FIG. 10 shows the g １ (r) to g Ｎ The squared images of the absolute value of G(r,ω) obtained from equation (9) based on (r) are shown at frequencies f = 0.44 Hz, 8.80 Hz, and 18.33 Hz. Figure 11 shows the distributions of (a) the average frequency obtained from equation (10) based on G(r,ω), (b) the sharpness of the frequency distribution obtained from equation (11) based on G(r,ω), and (c) the total energy of vibration obtained from equation (12) based on G(r,ω).

[0081] Figures 12 and 13 show the exact solution calculated from the exact phase information of the object under the simulation conditions shown in Figure 7. Figure 12 shows the squared images of the absolute value of G(r, ω) at frequencies f = 0.44 Hz, 8.80 Hz, and 18.33 Hz. Figure 13 shows the distributions of (a) the average frequency, (b) the sharpness of the frequency distribution, and (c) the total vibration energy. The simulation results for the first extraction mode (Figures 10 and 11) are in good agreement with the exact solution (Figures 12 and 13).

[0082] 14 to 16 are diagrams showing the simulation results for the third extraction mode. 1,1 (r) ~u 1,N (r) and u 2,1 (r) ~u 2,N Based on (r), g obtained by the above formula (8) １ (r) to g Ｎ g of (r) １ (r), g 121 (r) and g 241 (r) represents the real part of each image.

[0083] FIG. 15 shows the g １ (r) to g Ｎ The squared images of the absolute value of G(r,ω) obtained using equation (9) based on (r) are shown at frequencies f = 0.44 Hz, 8.80 Hz, and 18.33 Hz. Figure 16 shows the distributions of (a) the average frequency obtained using equation (10) based on G(r,ω), (b) the sharpness of the frequency distribution obtained using equation (11) based on G(r,ω), and (c) the total vibration energy obtained using equation (12) based on G(r,ω). The simulation results for the third extraction mode ( Figures 15 and 16 ) also closely match the exact solutions ( Figures 12 and 13 ).

[0084] 17 to 19 are diagrams showing the simulation results for the fourth extraction mode. 1,1 (r) ~u 1,N (r) and u 2,1 (r) ~u 2,N(r) based on the transmission matrix calculation １ (r) to c Ｎ (r) c １ (r), c 121 (r) and c 241 (r) represents the real part of each image. １ (r) to c Ｎ Based on (r), g obtained by the above formula (4) １ (r) to g Ｎ g of (r) １ (r), g 121 (r) and g 241 (r) represents the real part of each image.

[0085] FIG. 18 shows the g １ (r) to g Ｎ The squared images of the absolute value of G(r,ω) obtained by the above formula (9) based on (r) are shown at frequencies f = 0.44 Hz, 8.80 Hz, and 18.33 Hz. Figure 19 shows the distributions of (a) the average frequency obtained by the above formula (10) based on G(r,ω), (b) the sharpness of the frequency distribution obtained by the above formula (11) based on G(r,ω), and (c) the total energy of vibration obtained by the above formula (12) based on G(r,ω).

[0086] Comparing the simulation results (Figs. 18 and 19) for the fourth extraction mode with the exact solution (Figs. 12 and 13) reveals that the two agree well in terms of average frequency, but the sharpness of the frequency distribution and the total energy of vibration do not. This is thought to be because the transmission matrix is ​​effective in a steady state but not in a time-varying state. When the object of observation is a cell, the central frequency of the time variation is thought to differ depending on the size of the organelle, so it is expected that the fourth extraction mode will also be able to distinguish organelles by frequency discrimination.

[0087] 20 is a diagram showing other results of the simulation for the first extraction mode. This diagram shows the distribution of the variance h(r) obtained from the above equations (13) and (14). The value of M when calculating the moving average of equation (13) was set to 2.

[0088] 21 and 22 are diagrams showing further results of the simulation in the first extraction mode. Here, the complex amplitude image u 1,1 (r) ~u 1,N (r) and the complex amplitude image u 2,1 (r) ~u 2,N (r) plus the complex amplitude image u 3,1 (r) ~u 3,N (r) was acquired. At time t ｎ , t ｎ +T, t ｎ The irradiation conditions for the observation object in each of +2T are the same.

[0089] 21 is a diagram showing a complex amplitude image obtained by simulation. This diagram shows the complex amplitude image u 3,1 (r) ~u 3,N (r) Complex amplitude image u at time t = 13.64 s 3,1 (r) Complex amplitude image u at time t = 16.04 s 3,121 (r), and the complex amplitude image u at time t = 18.44 s 3,241 (r), and the real part of each. Figure 22 shows the distribution of (a) the average frequency obtained by the above formula (10), (b) the sharpness of the frequency distribution obtained by the above formula (11), and (c) the total energy of the vibration obtained by the above formula (12). This figure shows the distribution of the complex amplitude image u 1,1 (r) ~u 1,N (r) and the complex amplitude image u 3,1 (r) ~u 3,N This was calculated from (r).

[0090] Comparing Fig. 11 with Fig. 22, it is clear that frequency information that is missing in Fig. 11 is available in Fig. 22. This is thought to be due to the following reasons.

[0091] In the above formula (9), G(r, ω) is expressed as φ(r, t ｎ) is multiplied by a window function (sin(ωT / 2)), some frequency components are missing. However, while the window function of G(r,ω) used in FIG. 11 is sin(ωT / 2), the window function of G(r,ω) used in FIG. 22 is sin(ωT), so it is thought that the frequencies where information is missing are different between FIG. 11 and FIG. 22. It is expected that more accurate analysis will be possible by analyzing FIG. 11 and FIG. 22 together.

[0092] The simulation results explained so far were obtained by extracting and analyzing the time-varying components of the image of the observed object from a single plane. Below, we will explain the conditions and results of a simulation in which the time-varying components of the image of the observed object were extracted and analyzed from multiple planes.

[0093] The simulation conditions were as follows: the object to be observed was a three-dimensional object, and the phase φ of each of the multiple spherical regions arranged three-dimensionally in the object to be observed was set to change at various frequencies ω within the range of 0 Hz to 24.8 Hz. The start time of acquisition of the complex amplitude image was set to t = 0. The time interval τ (= t ｎ＋１ -t ｎ ) was set to 0.02 s. N=341. T=Nτ was set to 6.82 s. Plane waves with different irradiation directions were irradiated onto the observation object under each of the first to Nth irradiation conditions.

[0094] Complex amplitude image u at one focal plane 1,1 (r) ~u 1,N (r), u 2,1 (r) ~u 2,N (r), and based on these, complex amplitude images u at multiple other planes are calculated by wavefront propagation calculation. 1,1 (r) ~u 1,N (r), u 2,1 (r) ~u 2,N The position of the focal plane where the original complex amplitude image was acquired was set to z=0. Analysis was performed using the first extraction method on the focal plane and on each of the multiple other planes. In the following, the z position is defined as the wavelength λ of light. ０ It is expressed as a ratio to.

[0095] 23 to 26 show the results for z=7.8λ. ０ The upper part of FIG. 23 shows the simulation results for the complex amplitude image u 1,1 (r) ~u 1,N (r) Complex amplitude image u at time t = 0.00 s 1,1 (r) Complex amplitude image u at time t = 2.40 s 1,121 (r), and the complex amplitude image u at time t = 4.80 s 1,241 (r), and the real part of each. The bottom part of Fig. 23 shows the complex amplitude image u 2,1 (r) ~u 2,N (r) Complex amplitude image u at time t = 6.82 s 2,1 (r) Complex amplitude image u at time t = 9.22 s 2,121 (r), and the complex amplitude image u at time t = 11.62 s 2,241 (r) represents the real part of each.

[0096] The upper part of Figure 24 shows u 1,1 (r) ~u 1,N (r) and u 2,1 (r) ~u 2,N Based on (r), the c obtained by the above formula (1) １ (r) to c Ｎ (r) c １ (r), c 121 (r) and c 241 (r) represents the real part of each image. １ (r) to c Ｎ Based on (r), g obtained by the above formula (4) １ (r) to g Ｎ g of (r) １ (r), g 121 (r) and g 241 (r) represents the real part of each image.

[0097] FIG. 25 shows the g １ (r) to g ＮThe images of the absolute value of G(r,ω) obtained by the above formula (9) based on (r) are shown at frequencies f = 0.44 Hz, 8.80 Hz, and 18.33 Hz. Figure 26 shows the distributions of (a) the average frequency obtained by the above formula (10) based on G(r,ω), (b) the sharpness of the frequency distribution obtained by the above formula (11) based on G(r,ω), and (c) the total energy of vibration obtained by the above formula (12) based on G(r,ω).

[0098] 27 to 30 show the results for z=23.4λ. ０ The upper part of FIG. 27 shows the simulation results for the complex amplitude image u 1,1 (r) ~u 1,N (r) Complex amplitude image u at time t = 0.00 s 1,1 (r) Complex amplitude image u at time t = 2.40 s 1,121 (r), and the complex amplitude image u at time t = 4.80 s 1,241 (r), and the real part of each. The bottom part of Fig. 27 shows the complex amplitude image u 2,1 (r) ~u 2,N (r) Complex amplitude image u at time t = 6.82 s 2,1 (r) Complex amplitude image u at time t = 9.22 s 2,121 (r), and the complex amplitude image u at time t = 11.62 s 2,241 (r) represents the real part of each.

[0099] The upper part of Figure 28 shows u 1,1 (r) ~u 1,N (r) and u 2,1 (r) ~u 2,N Based on (r), the c obtained by the above formula (1) １ (r) to c Ｎ (r) c １ (r), c 121 (r) and c 241 (r) represents the real part of each image. １ (r) to c Ｎ Based on (r), g obtained by the above formula (4) １ (r) to g Ｎ g of (r) １ (r), g 121 (r) and g 241(r) represents the real part of each image.

[0100] FIG. １ (r) to g Ｎ The images of the absolute value of G(r,ω) obtained by the above formula (9) based on (r) are shown at frequencies f = 0.44 Hz, 8.80 Hz, and 18.33 Hz. Figure 30 shows the distributions of (a) the average frequency obtained by the above formula (10) based on G(r,ω), (b) the sharpness of the frequency distribution obtained by the above formula (11) based on G(r,ω), and (c) the total energy of vibration obtained by the above formula (12) based on G(r,ω).

[0101] 31 to 34 show the results for z=33.8λ. ０ The upper part of FIG. 31 shows the simulation results for the complex amplitude image u 1,1 (r) ~u 1,N (r) Complex amplitude image u at time t = 0.00 s 1,1 (r) Complex amplitude image u at time t = 2.40 s 1,121 (r), and the complex amplitude image u at time t = 4.80 s 1,241 (r), and the real part of each. The bottom part of Fig. 31 shows the complex amplitude image u 2,1 (r) ~u 2,N (r) Complex amplitude image u at time t = 6.82 s 2,1 (r) Complex amplitude image u at time t = 9.22 s 2,121 (r), and the complex amplitude image u at time t = 11.62 s 2,241 (r) represents the real part of each.

[0102] The upper part of Figure 32 shows u 1,1 (r) ~u 1,N (r) and u 2,1 (r) ~u 2,N Based on (r), the c obtained by the above formula (1) １ (r) to c Ｎ (r) c １ (r), c 121 (r) and c 241 (r) represents the real part of each image. １ (r) to c Ｎ Based on (r), g obtained by the above formula (4)１ (r) to g Ｎ g of (r) １ (r), g 121 (r) and g 241 (r) represents the real part of each image.

[0103] FIG. 33 shows the g １ (r) to g Ｎ The images of the absolute value of G(r,ω) obtained by the above formula (9) based on (r) are shown at frequencies f = 0.44 Hz, 8.80 Hz, and 18.33 Hz. Figure 34 shows the distributions of (a) the average frequency obtained by the above formula (10) based on G(r,ω), (b) the sharpness of the frequency distribution obtained by the above formula (11) based on G(r,ω), and (c) the total energy of vibration obtained by the above formula (12) based on G(r,ω).

[0104] As can be seen from Figures 23 to 34, a complex amplitude image at one focal plane is obtained by measurement, the wavefront of light represented by that complex amplitude image is propagated to one or more other planes, and complex amplitude images representing the wavefront of light at those other planes are obtained by calculation.This makes it possible to extract the temporal fluctuation components of the image of the object to be observed at those other planes based on the complex amplitude image, and to analyze the extracted temporal fluctuation components.

[0105] Next, the experimental results will be explained. A living HepG2 (a cell line derived from human liver cancer) was used as the observation object. The time interval τ (= t ｎ＋１ -t ｎ ) was set to 16.6 ms, N = 333, and T = Nτ. Plane waves with different irradiation directions were irradiated onto the object under the first to Nth irradiation conditions. The position of the focal plane where the complex amplitude image was acquired was set to z = 0. The temperature of HepG2 was set to 0°C at time 0, and then gradually increased until the temperature of HepG2 reached 37°C at time 20 min. Analysis was performed using the first extraction mode at the focal plane and multiple other planes.

[0106] Fig. 35 shows the experimental results for the z = 10.0 μm surface of HepG2 at time 0 min. Fig. 36 shows the experimental results for the z = 20.0 μm surface of HepG2 at time 0 min. Fig. 37 shows the experimental results for the z = 10.0 μm surface of HepG2 at time 2 min. Fig. 38 shows the experimental results for the z = 20.0 μm surface of HepG2 at time 2 min. These figures show the distributions of (a) refractive index, (b) average frequency, (c) sharpness of frequency distribution, and (d) total vibration energy.

[0107] As can be seen from these figures, the difference in the HepG2 image due to temperature differences is small in the refractive index distribution (each figure (a)). However, the difference in the HepG2 image due to temperature differences is large in the average frequency distribution (each figure (b)), the frequency distribution sharpness distribution (each figure (c)), and the total vibration energy distribution (each figure (d)). It can be seen that the metabolism of HepG2 improves when the temperature is high. The difference in the HepG2 image due to temperature differences is particularly large in the total vibration energy distribution (each figure (d)).

[0108] As described above, in this embodiment, the image acquisition unit 61 irradiates the object under the n-th irradiation condition among the first to N-th irradiation conditions at time t ｎ and time (t ｎ +T), and at time t ｎ The complex amplitude image u of light at the focal plane 1,n Obtain the time (t ｎ +T) is the complex amplitude image u of light at the focal plane 2,n Get.

[0109] The wavefront propagation calculation unit 62 calculates a complex amplitude image u 1,n (r), u 2,n (r) The wavefront of light represented by each of the two is propagated to one or more other surfaces, and a complex amplitude image u representing the wavefront of light at the other surface is obtained. 1,n (r), u 2,n Then, the analysis unit 63 calculates the complex amplitude image u 1,1 ~u 1,N , u 2,1 ~u 2,NThe time-varying component of the image of the object to be observed is extracted based on the above, and the extracted time-varying component is analyzed.

[0110] This makes it possible to acquire three-dimensional functional information of the object of observation while suppressing an increase in the number of images that the image acquisition unit 61 must acquire through measurement. Therefore, it is possible to suppress an increase in the measurement time required to acquire images, an increase in the storage capacity required to store images, and an increase in the time required to process images.

[0111] When the object of observation is a cell (or cell mass), the analysis unit 63 can evaluate the function of the cell by analyzing the temporal fluctuation components of the cell image. For example, the analysis unit 63 can determine whether a cell is alive or dead, evaluate the quality of a fertilized egg, or evaluate the aging state of a cell. By enabling such evaluations, it is expected that highly accurate evaluations of cell quality in regenerative medicine, evaluation of drug efficacy and toxicity in drug discovery, evaluation of the quality of sperm, eggs, embryos, and the like in assisted reproductive technology, etc. can be performed.

[0112] In this embodiment, the image acquisition unit 61 irradiates the object under the n-th irradiation condition among the first to N-th irradiation conditions at time t ｎ and time (t ｎ +T), and at time t ｎ The complex amplitude image u of light at the focal plane 1,n Obtain the time (t ｎ +T) is the complex amplitude image u of light at the focal plane 2,n Since the analysis unit 63 acquires the phase image or the phase differential image at each of the focal plane and other planes, the analysis unit 63 can also generate a refractive index distribution image. The analysis unit 63 can analyze the temporal fluctuation component of the image of the observed object together with these images, thereby enabling evaluation of the observed object with higher accuracy.

[0113] The object to be observed may be irradiated with light that induces a temporal variation in the image of the object to be observed. For example, a protein that can manipulate the refractive index by light irradiation may be embedded in a cell, and the refractive index distribution of the cell may be changed by irradiating the cell with light. According to this embodiment, such a protein can be evaluated, and the temporal change in the refractive index distribution of the cell in which such a protein is embedded can be evaluated.

[0114] Furthermore, for example, a mid-infrared photothermal microscope can irradiate an object to be observed with mid-infrared light, which is absorbed by the object through molecular vibrations, generating heat, causing a localized change in refractive index due to volume expansion of the object, and detecting this change in refractive index or phase using visible light. According to this embodiment, mid-infrared spectroscopic imaging of such cells can also be performed.

[0115] The observation device and observation method are not limited to the above-described embodiment and configuration example, and various modifications are possible.

[0116] The observation device of the first aspect according to the above embodiment (1) irradiates an object under an n-th irradiation condition (n is an integer of 1 to N) among first to N-th irradiation conditions (N is an integer of 2 or more) that are different from one another in terms of spatial intensity or phase distribution of light at a time t ｎ and time (t ｎ +T) and at time t ｎ The complex amplitude image u of light at the focal plane 1,n Obtain the time (t ｎ +T) is the complex amplitude image u of light at the focal plane 2,n (2) an image acquisition unit that acquires a complex amplitude image u at the focal plane for each n; 1,n , u 2,n A complex amplitude image u representing the wavefront of light at one or more other surfaces when the wavefront of light represented by each of the images propagates to the other surfaces. 1,n , u 2,n (3) a wavefront propagation calculation unit that calculates a complex amplitude image u 1,1 ~u 1,N , u 2,1 ~u 2,Nand an analysis unit that extracts a time-varying component of the image of the object to be observed based on the time-varying component and analyzes the time-varying component.

[0117] In the observation device of the second aspect, in the configuration of the first aspect, the analysis unit analyzes the complex amplitude image u 1,n and the complex amplitude image u 2,n the product of the complex conjugate image of one of the complex amplitude images and the other complex amplitude image, 1,n and the complex amplitude image u 2,n or the result obtained by dividing one complex amplitude image by the other complex amplitude image, u 1,n and the complex amplitude image u 2,n The difference between the two may be extracted as a time-varying component of the image of the object being observed.

[0118] In the observation device of the third aspect, in the configuration of the first or second aspect, the analysis unit analyzes the complex amplitude image u 1,n The phase and complex amplitude image u 2,n The phase difference between the two may be extracted as a time-varying component of the image of the object being observed.

[0119] In the observation device of the fourth aspect, in the configuration of any one of the first to third aspects, the analysis unit may be configured to perform analysis based on the results obtained by Fourier transforming the temporal fluctuation component of the image of the observation object with respect to the time variable.

[0120] In the observation device of the fifth aspect, in the configuration of any of the first to fourth aspects, the analysis unit may be configured to evaluate the function of the cells or cell clusters by analyzing the temporal fluctuation components of the image of the cells or cell clusters as the observation object.

[0121] In the observation device of the sixth aspect, in the configuration of any one of the first to fifth aspects, the image acquisition unit may be configured to irradiate the observation object with plane waves having different irradiation directions as first to Nth irradiation conditions.

[0122] The observation method according to the above embodiment includes: (1) irradiating an object with light under an n-th irradiation condition (n is an integer of 1 to N) among first to N-th irradiation conditions (N is an integer of 2 or more) that are different from one another in terms of the spatial intensity or phase distribution of light at a time t ｎand time (t ｎ +T) and at time t ｎ The complex amplitude image u of light at the focal plane 1,n Obtain the time (t ｎ +T) is the complex amplitude image u of light at the focal plane 2,n (2) for each n, a complex amplitude image u at the focal plane is acquired; 1,n , u 2,n A complex amplitude image u representing the wavefront of light at one or more other surfaces when the wavefront of light represented by each of the images propagates to the other surfaces. 1,n , u 2,n (3) a wavefront propagation calculation step for calculating a complex amplitude image u 1,1 ~u 1,N , u 2,1 ~u 2,N and an analyzing step of extracting a time-varying component of the image of the object to be observed based on the extracted time-varying component and analyzing the time-varying component.

[0123] The program according to the above embodiment is a program for causing a computer to execute each step of the observation method having the above configuration.

[0124] The recording medium according to the above embodiment is a computer-readable recording medium on which the program having the above configuration is recorded.

[0125] The embodiments can be used as an observation device and an observation method that can acquire three-dimensional functional information of an observation object while suppressing an increase in the number of images to be acquired.

[0126] 1A to 1C...observation device, 2...recording medium, 10...light source, 11...lens, 12...light input end, 13...optical fiber, 14...light output end, 15...fiber coupler, 16, 17...optical fiber, 18, 19...light output end, 21...lens, 22...mirror, 23...lens, 24...condenser lens, 41...objective lens, 42...beam splitter, 43...lens, 44...mirror, 45...focal plane position setting unit, 50...imaging unit, 60...processing unit, 61...image acquisition unit, 62...wavefront propagation calculation unit, 63...analysis unit, 64...display unit, 65...memory unit, 31...irradiation unit.

Claims

1. Light under the nth irradiation condition (n is an integer of 1 to N) among the first to Nth irradiation conditions (N is an integer of 2 or more) that differ from each other in terms of the spatial intensity or phase distribution of light is irradiated onto the object to be observed at time t ｎ and time (t ｎ +T) and at time t ｎ The complex amplitude image u of light at the focal plane 1,n Obtain the time (t ｎ +T) a complex amplitude image u of light at the focal plane 2,n an image acquisition unit for acquiring a complex amplitude image u at the focal plane for each n; 1,n , u 2,n A complex amplitude image u representing the wavefront of light at one or more other surfaces when the wavefront of light represented by each of the images propagates to the other surfaces. 1,n , u 2,n a wavefront propagation calculation unit that calculates a complex amplitude image u 1,1 ~u 1,N , u 2,1 ~u 2,N and an analysis unit that extracts a time-varying component of an image of the object to be observed based on the time-varying component and analyzes the time-varying component.

2. The analysis unit generates a complex amplitude image u 1,n and the complex amplitude image u 2,n the product of the complex conjugate image of one of the complex amplitude images and the other complex amplitude image, 1,n and the complex amplitude image u 2,n or the result obtained by dividing one complex amplitude image by the other complex amplitude image, u 1,n and the complex amplitude image u 2,n 2. The observation device according to claim 1, wherein a difference between the image of the observation object and the image of the observation object is extracted as a time-varying component.

3. The analysis unit generates a complex amplitude image u 1,n Phase and complex amplitude image u 2,n 3. The observation device according to claim 1, wherein a difference in phase between the first and second light components is extracted as a time-varying component of the image of the observation object.

4. An observation device according to any one of claims 1 to 3, wherein the analysis section performs analysis based on the results obtained by Fourier transforming the time-varying components of the image of the observation object with respect to the time variable.

5. An observation device according to any one of claims 1 to 4, wherein the analysis unit evaluates the function of the cells or cell clusters by analyzing the temporal fluctuation components of the image of the cells or cell clusters as the observation object.

6. An observation device according to any one of claims 1 to 5, wherein the image acquisition unit irradiates the observation object with plane waves having mutually different irradiation directions as first to Nth irradiation conditions.

7. Light under the nth irradiation condition (n is an integer of 1 to N) among the first to Nth irradiation conditions (N is an integer of 2 or more) which are different from each other in terms of the spatial intensity or phase distribution of light is projected onto the object to be observed at time t ｎ and time (t ｎ +T) and at time t ｎ The complex amplitude image u of light at the focal plane 1,n Obtain the time (t ｎ +T) a complex amplitude image u of light at the focal plane 2,n and for each n, acquiring a complex amplitude image u at said focal plane. 1,n , u 2,n A complex amplitude image u representing the wavefront of light at one or more other surfaces when the wavefront of light represented by each of the images propagates to the other surfaces. 1,n , u 2,n a wavefront propagation calculation step for calculating a complex amplitude image u 1,1 ~u 1,N , u 2,1 ~u 2,N and an analysis step of extracting a time-varying component of the image of the object to be observed based on the above-mentioned formula and analyzing the time-varying component.

8. A program for causing a computer to execute each step of the observation method according to claim 7.

9. A computer-readable recording medium on which the program according to claim 8 is recorded.

Images