Observation device and observation method

Abstract

To provide a device that allows easy observation of a moving observation object.SOLUTION: An observation device 1A is a device for observing a moving observation object S, and comprises a light source 11, a beam splitter 12, mirrors 13, 14, a beam splitter 15, a modulation unit 21, lenses 31-34, an imaging unit 41, and a processing unit 42. The modulation unit 21 receives input of first branch light L1 arriving from the mirror 13, and applies spatial amplitude modulation to the input first branch light L1 on the basis of a modulation pattern to obtain first branch light including a plurality of light components different from each other in wavenumber components in the direction of movement of the observation object S. The processing unit 42 creates complex amplitude images when the observation object S is irradiated with the plurality of light components on the basis of detection signals repeatedly output from the imaging unit 41.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an apparatus and method for observing a moving object. [Background technology]

[0002] Patent Document 1 discloses an invention of an apparatus and method for observing a moving observation object. The observation apparatus described in this document splits light output from a light source into a first branched light and a second branched light, combines the first branched light, which has been Doppler-shifted by passing through a moving observation object, with the second branched light, whose optical frequency has been shifted by the heterodyne frequency, and causes heterodyne interference between the first branched light and the second branched light. This observation apparatus can obtain time-series data of a complex amplitude image of the first branched light on the imaging plane based on time-series data of an intensity image (two-dimensional interference image) of the interference light that has reached the imaging plane of a camera. This observation apparatus can image a moving observation object non-stained and non-invasively, and can be used, for example, to observe moving cells with a flow cytometer. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2013 / 065796 [Non-patent literature]

[0004] [Non-Patent Document 1] W. Choi, C. Fang-Yen, K. Badizadegan, S. Oh, N. Lue, RR Dasari, and MS Feld, "Tomographicphase microscopy," Nat Meth 4(9), 717-719 (2007). [Non-Patent Document 2] Y. Sung, W. Choi, C. Fang-Yen, K. Badizadegan, RR Dasari, and MS Feld, "Optical diffraction tomography for high resolution live cell imaging," Optics Express 17(1), 266 (2009). Summary of the Invention [Problem to be solved by the invention]

[0005] In the invention disclosed in Patent Document 1, since the imaging plane is different from the object plane, it is not easy to adjust the optical system, and focus adjustment is also difficult, so that it is not easy to observe the observation target.

[0006] The present invention has been made to solve the above problems, and has an object to provide an apparatus and method that can easily observe a moving observation target object. [Means for solving the problem]

[0007] The observation apparatus of the present invention is an apparatus for observing a moving observation target. A first aspect of the observation device of the present invention includes: (1) a light source that outputs light; (2) a branching unit that branches the light into a first branched light and a second branched light; (3) a modulation unit that spatially amplitude-modulates the first branched light based on a modulation pattern to form the first branched light including a plurality of light components having different wave number components in the moving direction of an object to be observed; (4) an irradiation unit that irradiates an object to be observed located at a position optically conjugate with the first branched light after being amplitude-modulated by the modulation unit; (5) a combining unit that combines the first branched light and the second branched light after passing through the object to output combined light; (6) an imaging unit that has an imaging surface that is located at a position optically conjugate with the object to be observed, receives the combined light at the imaging surface, and repeatedly outputs a detection signal representing a two-dimensional interference image; and (7) a processing unit that generates a complex amplitude image when each of the plurality of light components is irradiated onto the object to be observed based on the detection signal repeatedly output from the imaging unit. The processing unit generates an image of the object to be observed at each time by the first branched light that has reached the imaging surface based on time series data of the two-dimensional interference image generated from the detection signals repeatedly output from the imaging unit, generates an image of the object to be observed at each time in a coordinate system fixed to the object to be observed, extracts components that change over time at a frequency corresponding to the wave number component in the direction of movement of each of the multiple light components, and generates a complex amplitude image when each of the multiple light components is irradiated onto the object to be observed.

[0008] In a second aspect of the observation device of the present invention, in addition to the first aspect, the modulation section generates a first branched light including a plurality of light components whose wave numbers are distributed two-dimensionally in wave number space.

[0009] In a third aspect of the observation device of the present invention, in addition to the first or second aspect, the modulation unit generates a first branched light including a plurality of light components having different surpluses when the wave number component in the moving direction is divided by the frame rate of the imaging unit.

[0010] In a fourth aspect of the observation device of the present invention, in addition to any of the first to third aspects, the modulation section spatially amplitude-modulates the first branched light based on a modulation pattern in which values ​​of two positions that are symmetrical with respect to the origin in a Fourier transform pattern of the modulation pattern are in a complex conjugate relationship.

[0011] In a fifth aspect of the observation device of the present invention, in addition to any one of the first to fourth aspects, the modulation section includes a spatial light modulator with a fixed modulation pattern.

[0012] In a sixth aspect of the observation device of the present invention, in addition to any one of the first to fourth aspects, the modulation section includes a spatial light modulator capable of variably setting the modulation pattern.

[0013] In a seventh aspect of the observation device of the present invention, in addition to any of the first to sixth aspects, the processing unit generates a phase differential image of the object to be observed based on a complex amplitude image obtained when each of the multiple light components is irradiated onto the object to be observed.

[0014] In an eighth aspect of the observation apparatus of the present invention, in addition to the seventh aspect, the processing unit generates a refractive index distribution image of the observation object based on the phase differential image.

[0015] The observation method of the present invention is a method for observing a moving observation object. A first aspect of the observation method of the present invention includes: (1) a branching step of branching light output from a light source into a first branched light and a second branched light; (2) a modulation step of using a modulation unit that spatially amplitude-modulates the first branched light based on a modulation pattern to generate the first branched light including a plurality of light components having different wave number components in the moving direction of an object to be observed; (3) an irradiation step of irradiating an object to be observed located at a position optically conjugate with the first branched light after being amplitude-modulated by the modulation unit; (4) a combining step of combining the first branched light and the second branched light after passing through the object to be observed and outputting a combined light; (5) an imaging step of receiving the combined light at an imaging surface using an imaging unit having an imaging surface arranged at a position optically conjugate with the object to be observed and repeatedly outputting a detection signal representing a two-dimensional interference image; and (6) a processing step of generating a complex amplitude image when each of the plurality of light components is irradiated onto the object to be observed based on the detection signal repeatedly output from the imaging unit. In the processing step, an image of the object to be observed at each time is generated by the first branched light that has reached the imaging surface based on time series data of a two-dimensional interference image generated from detection signals repeatedly output from the imaging unit, an image of the object to be observed at each time in a coordinate system fixed to the object to be observed is generated, a component that changes over time at a frequency corresponding to the wave number component in the movement direction of each of the multiple light components is extracted, and a complex amplitude image when each of the multiple light components is irradiated onto the object to be observed is generated.

[0016] In a second aspect of the observation method of the present invention, in addition to the first aspect, the modulation step generates a first branched light including a plurality of light components whose wave numbers are distributed two-dimensionally in wave number space.

[0017] In a third aspect of the observation method of the present invention, in addition to the first or second aspect, in the modulation step, the first branched light includes a plurality of light components whose surpluses when the wave number component in the moving direction is divided by the frame rate of the imaging unit are different from each other.

[0018] In a fourth aspect of the observation method of the present invention, in addition to any one of the first to third aspects, in the modulation step, the first branched light is spatially amplitude-modulated based on a modulation pattern in which values ​​of two positions that are symmetrical with respect to the origin in a Fourier transform pattern of the modulation pattern are in a complex conjugate relationship.

[0019] In a fifth aspect of the observation method of the present invention, in addition to any one of the first to fourth aspects, in the modulation step, the first split light is spatially amplitude-modulated using a modulation section including a spatial light modulator having a fixed modulation pattern.

[0020] In a sixth aspect of the observation method of the present invention, in addition to any one of the first to fourth aspects, in the modulation step, the first branched light is spatially amplitude-modulated using a modulation section including a spatial light modulator capable of variably setting a modulation pattern.

[0021] In a seventh aspect of the observation method of the present invention, in addition to any one of the first to sixth aspects, in the processing step, a phase differential image of the object to be observed is generated based on complex amplitude images obtained when each of a plurality of light components is irradiated onto the object to be observed.

[0022] In an eighth aspect of the observation method of the present invention, in addition to the seventh aspect, a refractive index distribution image of the observation object is generated based on the phase differential image in the processing step. Effect of the Invention

[0023] According to the present invention, a moving observation target can be easily observed. [Brief description of the drawings]

[0024] [Figure 1] FIG. 1 is a diagram showing the configuration of an observation apparatus 1A. [Diagram 2] FIG. 2 is a diagram showing the configuration of the observation device 1B. [Diagram 3] FIG. 3 is a diagram showing the configuration of the observation apparatus 1C. [Figure 4]FIG. 4 is a diagram showing an example of the distribution of wave number vectors {(kx,n,ky,n)}n in wave number space. [Diagram 5] FIG. 5 is a diagram showing a modulation pattern m(x, y) obtained from M(kx, ky) created according to equation (4) based on the distribution of wave number vectors {(kx, n, ky, n)}n in the wave number space shown in FIG. [Figure 6] FIG. 6 is a diagram showing a modulation pattern m(x,y) obtained from M(kx,ky) created according to equation (5) based on the distribution of wave number vectors {(kx,n,ky,n)}n in the wave number space shown in FIG. [Figure 7] FIG. 7 is a diagram showing another example of the distribution of wave number vectors {(kx,n,ky,n)}n in wave number space. [Figure 8] FIG. 8 is a diagram showing another example of the distribution of wave number vectors {(kx,n,ky,n)}n in wave number space. [Figure 9] FIG. 9 is a diagram showing the distribution of wave vectors {(kx,n,ky,n)}n used in the simulation. [Figure 10] FIG. 10 is a diagram showing a modulation pattern m(x, y) created based on the distribution of wave vectors {(kx, n, ky, n)} n shown in FIG. [Figure 11] FIG. 11 is a diagram showing the real part of the image of uout(x, y, t) at each time generated by the simulation. [Figure 12] FIG. 12 is a diagram showing the real part of the image of u'out(x', y, t) after coordinate transformation at each time generated in the simulation. [Figure 13] FIG. 13 is a diagram showing complex amplitude images in each irradiation direction generated by the simulation. [Figure 14] FIG. 14 is a diagram showing a phase differential image of an object to be observed generated by simulation. [Figure 15] FIG. 15 is a diagram showing a refractive index distribution image of an object to be observed generated by simulation. [Figure 16]FIG. 16 is a diagram showing the distribution of wave vectors {(kx,n,ky,n)}n used in the simulation. [Figure 17] FIG. 17 is a diagram showing a modulation pattern m(x, y) created based on the distribution of wave vectors {(kx, n, ky, n)} n shown in FIG. [Figure 18] FIG. 18 is a diagram showing the correspondence relationship between the wave number component kx,n in the x direction of each light component and the frequency fn. [Figure 19] FIG. 19 is a diagram showing frequencies fn after aliasing arranged in order from smallest to largest. [Figure 20] FIG. 20 is a diagram showing the real part of the image of uout(x, y, t) at each time generated in the simulation. [Figure 21] FIG. 21 is a diagram showing the real part of the image of u'out(x', y, t) after coordinate transformation at each time generated in the simulation. [Figure 22] FIG. 22 is a diagram showing complex amplitude images in each irradiation direction generated by simulation. [Figure 23] FIG. 23 is a diagram showing a phase differential image of the object to be observed generated by the simulation. [Figure 24] FIG. 24 is a diagram showing a refractive index distribution image of an object to be observed generated by simulation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] Hereinafter, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicated descriptions are omitted. The present invention is not limited to these examples, but is indicated by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

[0026] 1 is a diagram showing the configuration of an observation device 1A. The observation device 1A is a device for observing a moving observation target S, and includes a light source 11, a beam splitter 12, mirrors 13 and 14, a beam splitter 15, a modulation section 21, lenses 31 to 34, an imaging section 41, and a processing section 42. An observation method using this observation device 1A includes a branching step, a modulation step, an irradiation step, a combining step, an imaging step, and a processing step.

[0027] For ease of explanation, an xyz Cartesian coordinate system is shown in this figure. The x-axis is parallel to the direction of movement of the observation object S. The z-axis is parallel to the optical axis of the lenses 32 and 33.

[0028] The moving observation object S is, for example, a cell or cell mass flowing in a flow cell, or a biological sample placed on a slide glass that is being moved by a transport unit. In addition, the observation object S may be moved relative to the observation device 1A, and the observation device 1A (for example, an optical system) may be configured to move relative to the stationary observation object S.

[0029] The light source 11 outputs light. Preferably, the light source 11 is a laser light source that outputs laser light having a single optical frequency, such as a He-Ne laser light source or a semiconductor laser light source. The light source 11 may include a beam expander that expands the beam diameter of the light. The light source 11 may be an incoherent light source such as an SLD (Superluminescent Diode) light source or an SC (Super Continuum) light source. The light source 11 is preferably a light source with a long temporal coherence length, but may be a spatially incoherent LED or halogen lamp combined with a pinhole or the like that improves spatial coherence.

[0030] The beam splitter 12 is a branching section optically connected to the light source 11. The beam splitter 12 receives the light output from the light source 11 and branches the received light into a first branched light L1 and a second branched light L2 (branching step). The beam splitter 12 outputs the first branched light L1 to a mirror 13 and outputs the second branched light L2 to a mirror 14. The beam splitter 12 may be a half mirror.

[0031] The mirror 13 is optically connected to the beam splitter 12. The mirror 13 receives the first branched light L1 output from the beam splitter 12, reflects the received first branched light L1, and outputs the reflected light to the modulation unit 21.

[0032] The mirror 14 is optically connected to the beam splitter 12. The mirror 14 receives the second branched light L2 output from the beam splitter 12, reflects the received second branched light L2, and outputs the reflected light to the beam splitter 15.

[0033] The beam splitter 15 inputs the first branched light L1 that has been reflected by the mirror 13 and arrived via the modulation section 21 and the observation object S, etc., and also inputs the second branched light L2 that has been reflected by the mirror 14 and arrived. The beam splitter 15 is a multiplexing section, which multiplexes the input first branched light L1 and second branched light L2 and outputs the multiplexed light to the imaging section 41 (multiplexing step). The traveling directions of the first branched light L1 and the second branched light L2 from the beam splitter 15 to the imaging section 41 are not parallel to each other but form a certain angle. The beam splitter 15 may be a half mirror.

[0034] The optical system from the beam splitter 12 through the mirrors 13 and 14 to the beam splitter 15 constitutes a Mach-Zehnder interferometer. On the optical path of the first split light L1 from the mirror 13 to the beam splitter 15, the modulation unit 21, the lens 31, the lens 32, the lens 33, and the lens 34 are arranged in this order. The lenses 31 to 34 may be spherical lenses. The observation object S moves in the x-axis direction between the lenses 32 and 33. The observation object S is, for example, a cell moving in the x-axis direction along a flow path in a flow cytometer.

[0035] The modulation unit 21 is optically connected to the mirror 13. The modulation unit 21 inputs the first branched light L1 arriving from the mirror 13, spatially amplitude-modulates the input first branched light L1 based on a modulation pattern, and outputs the modulated first branched light L1 to the lens 31 (modulation step).

[0036] The lenses 31 and 32 are an irradiation unit optically connected to the modulation unit 21. The lenses 31 and 32 input the first branched light L1 that has been amplitude-modulated by the modulation unit 21 and then output and reaches the first branched light L1, and irradiate the observation object S located at a position optically conjugate with the modulation unit 21 (irradiation step). The optical axis of the lens 32 is parallel to the z-axis.

[0037] The lenses 33 and 34 allow the first branched light L1 after passing through the observation object S to be incident on the imaging surface of the imaging unit 41, so that an image of the observation object S can be formed on the imaging surface of the imaging unit 41. The lens 33 is an objective lens, and the optical axis of the lens 33 is parallel to the z-axis.

[0038] The imaging unit 41 is optically connected to the beam splitter 15. The imaging unit 41 is an area sensor such as a CCD sensor or a CMOS sensor. The imaging surface of the imaging unit 41 is disposed at a position optically conjugate with the observation object S. The imaging unit 41 receives the combined light output from the beam splitter 15 on the imaging surface, and repeatedly outputs a detection signal representing a two-dimensional interference image due to interference between the first branched light L1 and the second branched light L2 (imaging step).

[0039] The processing unit 42 is electrically connected to the imaging unit 41. The processing unit 42 inputs detection signals repeatedly output from the imaging unit 41, and processes time series data of a two-dimensional interference image generated from the detection signals (processing step). The processing unit 42 creates a complex amplitude image of the observation object S through this processing. Furthermore, the processing unit 42 can also create a phase differential image of the observation object S, and can also create a refractive index distribution image of the observation object S.

[0040] The processing unit 42 is, for example, a calculation device (such as a computer) incorporating a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), a storage medium such as a RAM (Random Access Memory) or a ROM (Read Only Memory), a communication module, and an input / output module, etc. Also, for example, the processing unit 42 may be configured with an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0041] The modulation section 21 shown in Fig. 1 is a transmissive spatial light modulator that spatially modulates the amplitude of the first split light L1 based on a modulation pattern and transmits the light. The modulation pattern in the modulation section 21 may be fixed or may be variably settable. The modulation section 21 having a fixed modulation pattern may be, for example, a grayscale photomask or a photosensitive film (negative film). The modulation section 21 may have a modulation pattern according to the density distribution of minute light blocking regions.

[0042] The modulation section that spatially modulates the amplitude of the first split light can be in various forms. The modulation section may be configured to include a transmissive spatial light modulator, or may be configured to include a reflective spatial light modulator. The modulation pattern in the modulation section may be fixed, or may be variably settable. The modulation section may be configured to include a phase-modulation spatial light modulator and a polarizer. Figures 2 and 3 are diagrams showing the configuration of an observation device including a modulation section of another configuration example.

[0043] FIG. 2 is a diagram showing the configuration of the observation device 1B. Compared with the configuration of the observation device 1A (FIG. 1), the observation device 1B (FIG. 2) is different in that it includes a beam splitter 16 and a modulation unit 22 instead of the mirror 13 and the modulation unit 21. The beam splitter 16 reflects the first branched light L1 arriving from the beam splitter 12 to the modulation unit 22, and transmits the first branched light L1 arriving from the modulation unit 22 to the lens 31. The modulation unit 22 is a reflective spatial light modulator that can spatially modulate the amplitude of light. The modulation unit 22 inputs the first branched light L1 arriving from the beam splitter 16, spatially modulates the amplitude of the first branched light L1 based on a modulation pattern, and reflects it to the beam splitter 16. The modulation unit 22 may be a DMD (Digital Mirror Device), and in this case, it can have a modulation pattern according to the density distribution of pixels that reflect light to the beam splitter 16.

[0044] FIG. 3 is a diagram showing the configuration of the observation device 1C. Compared with the configuration of the observation device 1B (FIG. 2), the observation device 1C (FIG. 3) is different in that it includes a modulation section including a phase modulation type spatial light modulator 23 and a polarizer 24 instead of the modulation section 22 (amplitude modulation type spatial light modulator). The beam splitter 16 reflects the first branched light L1 arriving from the beam splitter 12 to the spatial light modulator 23, and transmits the first branched light L1 arriving from the spatial light modulator 23 to the polarizer 24. The spatial light modulator 23 is of a reflective type that can spatially phase-modulate light. The spatial light modulator 23 inputs the first branched light L1 arriving from the beam splitter 16, spatially phase-modulates the first branched light L1 based on the modulation pattern, and reflects it to the beam splitter 16. The polarizer 24 inputs the first branched light L1 that has been reflected by the spatial light modulator 23 and has reached the polarizer 24 via the beam splitter 16, and selectively transmits a linearly polarized component of the first branched light L1 in a certain direction to the lens 31.

[0045] The spatial light modulator 23 is, for example, a reflective, electrically addressed spatial light phase modulator, LCOS-SLM (Liquid Crystal on Silicon Spatial Light Modulator). The LCOS-SLM can perform highly accurate and fast-response phase modulation by directly controlling the voltage of the liquid crystal by applying CMOS technology to the address section. The spatial light modulator 23 has a modulation surface on which a plurality of pixel regions are arranged, each of which performs different modulation depending on the polarization direction of the input light and outputs the light. Linearly polarized light in a specific direction (for example, 45 degrees with respect to the orientation direction of the liquid crystal in the spatial light modulator 23) is input to the spatial light modulator 23. The polarizer 24 outputs linearly polarized light in a direction different from the specific direction (for example, 135 degrees with respect to the orientation direction of the liquid crystal in the spatial light modulator 23) from the light output from the spatial light modulator 23.

[0046] With this configuration, even if the spatial light modulator 23 is of a phase modulation type, the spatial light modulator 23 and the polarizer 24 can form a modulation section that spatially amplitude-modulates the first split light L1. The observation object S is located at a position optically conjugate with the modulation surface of the spatial light modulator 23.

[0047] Next, the amplitude modulation of the first branched light by the modulation unit (including the modulation unit 21 in FIG. 1, the modulation unit 22 in FIG. 2, and the spatial light modulator 23 and the polarizer 24 in FIG. 3) in the modulation step will be described. The modulation unit spatially amplitude-modulates the first branched light based on a modulation pattern. By this amplitude modulation, the modulation unit modulates the wave number component k x,n The first branched light u output from the modulator is a multiple (N) optical component with different wavelengths. Each optical component is a plane wave. in (x, y) is expressed as the coherent sum of N optical components as shown in the following equation (1). The wave vector of the nth optical component among the N optical components is (k x,n ,k y,n ), and the amplitude of the nth light component is U in (k x,n ,k y,n ).

[0048]

number

[0049] The modulation pattern of the modulation section for generating the first branched light expressed by the formula (1) needs to satisfy the following conditions. Since the magnitude of the amplitude modulation is a real number between 0 and 1, when the modulation pattern representing the distribution of the magnitude of the amplitude modulation is m(x, y), m(x, y) satisfies the relationship of the formula (2) below, and the Fourier transform of m(x, y) is M(k x ,k y ) satisfies the relationship of the following equation (3). x ,k y ) represents the distribution of wave vectors in the wave number space of the light that is amplitude modulated and output by the modulator, and therefore has a discrete distribution. x ,k y ) in the discrete distribution of the wave number component k in the moving direction (x direction) of the observed object S x,n are required to be different values ​​as a necessary condition for processing in the processing unit 42. sWhen K images are taken, the frequency resolution when they are Fourier transformed on the time axis is f s Since it is given by / K, the wave number component k x,n It is sufficient that m(x,y) is separated by this amount or more. m(x,y) is designed to satisfy the above conditions.

[0050]

number

[0051]

number

[0052] As a specific example of the design, a modulation pattern m(x,y) can be designed by performing the following steps 1 to 3. In step 1, the wave number component k in the moving direction (x direction) of the observation target S is x,n In order to make the k x >0, so that there are no overlaps in the range {k x,n} n Determine each k x,n For any k y,n and assign the wave vector {(k x,n ,k y,n )} n Determine.

[0053] Next, in step 2, in order to satisfy the condition of the above equation (3), k in the wave number space is x <0, centered at the origin, k x >0, the wave vector {(k x,n ,k y,n )} n Also, M(k x,n ,k y,n Determine the amplitude and phase of φ in Eq. (5). n is an arbitrary real number. In the wave number space, the wave number vector {(k x,n ,k y,n )}n In the range that does not correspond to M(k x ,k y )=0.

[0054]

number

[0055]

number

[0056] Furthermore, in step 3, M(k x ,k y ) is Fourier transformed to obtain m(x, y), and then m(x, y) is scaled by the following equation (6) so as to satisfy the condition of equation (2) above, thereby obtaining the modulation pattern m(x, y) of the modulation section for generating the first branched light expressed by equation (1) above.

[0057]

number

[0058] Figure 4 shows the wave vector {(k x,n ,k y,n )} n In this figure, the wave vector {(k x,n ,k y,n )} n The positions corresponding to the wave vectors {(k x,n ,k y,n )} n Based on the distribution of M(k x ,k y 6 is a diagram showing a modulation pattern m(x,y) obtained from the wave number vector {(k x,n ,k y,n )} n Based on the distribution of M(k x ,ky 5 and 6 show a modulation pattern m(x, y) obtained from the amplitude modulation shown in FIG. 5. In each of FIG. 5 and FIG. 6, the magnitude of the amplitude modulation is shown by shading, and a part of FIG. 5(a) is enlarged and shown in FIG. 5(b). In order to effectively utilize the dynamic range of the image captured by the image capture unit 41, it is preferable that the light intensity distribution is as flat as possible, and from this point of view, the modulation pattern m(x, y) shown in FIG. 5 is more preferable.

[0059] Figures 7 and 8 show the wave vector {(k x,n ,k y,n )} n 13A and 13B are diagrams showing other examples of the distribution of wave vectors {(k x,n ,k y,n )} n The corresponding positions are indicated by black circles.

[0060] The example shown in Fig. 7(a) is a square lattice in wave number space, k x After shearing in the direction, each lattice point is assigned a wave vector {(k x,n ,k y,n )} n (The same applies to the example shown in Fig. 4.) In the example shown in Fig. 7(b), the wave vector {(k x,n ,k y,n )} n The example shown in FIG. 7(c) is a non-periodic discrete wave number space with a wave number vector {(k x,n ,k y,n )} n In these examples, the wave vector {(k x,n ,k y,n )} n In x,n However, no identical items are included.

[0061] The example shown in FIG. 8(a) is a modification of the example shown in FIG. 7(a), and the wave vector {(k x,n ,k y,n )} nk of some of the wave vectors (three wave vectors enclosed by ellipses in the figure) x,n During processing by the processing unit 42, k x,n Wave vectors with the same k cannot be separated from each other, but other wave vectors can be separated from each other, so we have the wave vectors {(k x,n ,k y,n )} n It may be a distribution of

[0062] The example shown in FIG. 8(b) shows that the wave vector {(k x,n ,k y,n )} n In the example shown in this figure, k y,n =0. In this example, the wave vector {(k x,n ,k y,n )} n In x,n However, the wave vector {(k x,n ,k y,n )} n When is distributed one-dimensionally (Figure 8(b)), the wave vector {(k x,n ,k y,n )} n In contrast to the case where the wave vector {(k x,n ,k y,n )} n The number N is small.

[0063] In order to improve the accuracy of the 3D image reconstruction of the observation object S, the configurable wave vector {(k x,n ,k y,n )} n The larger the number N of the wave number vectors, the more preferable it is. In order to reduce the anisotropy of the resolution of the three-dimensional image of the object S to be observed, the wave number vector {(k x,n ,k y,n )} n From this viewpoint, it is preferable that the wave vector {(k x,n ,k y,n )}n When is distributed one-dimensionally (Figure 8(b)), the wave vector {(k x,n ,k y,n )} n is distributed two-dimensionally (FIGS. 7(a) to (c), FIG. 8(a)).

[0064] Next, acquisition of a two-dimensional interference image by the imaging unit 41 in the imaging step will be described. in When the light (x, y) is irradiated onto the observation object S, the first branched light u out (x, y, t) are expressed by the following formula (7). In this formula, o(x-Vt, y) represents an image of the observation object S at a position optically conjugate with respect to the imaging plane of the imaging unit 41. V is the moving speed of the observation object S moving in the x direction. t is a time variable. The imaging unit 41 receives the first split light u out The optical axis of the optical fiber is (x, y, t) and the second branched light incident as a plane wave along a direction inclined to the incident direction of the first branched light are received on an imaging plane, and a detection signal representing a two-dimensional interference image due to interference between the first branched light and the second branched light is repeatedly output.

[0065]

number

[0066] Next, the content of the processing by the processing unit 42 in the processing step will be described. The processing unit 42 inputs the detection signal repeatedly output from the imaging unit 41 and performs the following processing. Based on the detection signal at each time, a two-dimensional interference image at each time (i.e., time-series data of the two-dimensional interference image) is generated. Then, based on the two-dimensional interference image at each time t, the first branched light u that has reached the imaging plane of the imaging unit 41 at each time t is calculated by a fringe analysis method. out An image of the observation object S by (x, y, t), that is, an image of the observation object S located at a position optically conjugate with respect to the imaging plane, is generated.

[0067] Considering that the observation object S is moving in the x direction at a speed V, the x coordinate in the xyz coordinate system fixed to the measurement system is converted to an x' coordinate fixed to the observation object S. In other words, a coordinate conversion as expressed by the following formula (8) is performed. The first split light u' after such a coordinate conversion is out The image of (x', y, t) is expressed as the following equation (9). Furthermore, by using the above equation (1), this equation (9) can be expressed as the following equation (10).

[0068]

number

[0069]

number

[0070]

number

[0071] This equation (10) is expressed by the following equation (11) n (The wave number component k in the direction of movement of each of the N light components x,n The u' expressed by this equation (10) is the sum of components that change over time at a frequency that depends on the out (x',y,t) is expressed as a discrete time interval {t i} i Then, we sample at frequency f n When the time-varying components are extracted, as shown in the following equation (12), in the x'yz coordinate system fixed to the observation object S, each light component (wave vector (k x,n ,k y,n ), amplitude U in (k x,n ,k y,n )), that is, complex amplitude images for each of the N irradiation directions, are obtained.

[0072]

number

[0073]

number

[0074] Also, a phase differential image of the observation object S can be generated based on these N irradiation directions. Furthermore, a three-dimensional refractive index distribution image of the observation object S can be generated based on this phase differential image. In generating these images, the method described in Non-Patent Document 1 or Non-Patent Document 2 can be used. The method described in Non-Patent Document 1 uses the Fourier Slice theorem and is applicable when light scattering in the observation object S can be ignored. The method described in Non-Patent Document 2 uses the Fourier Diffraction theorem and generates a three-dimensional refractive index distribution image of the observation object S taking into account light scattering in the observation object S.

[0075] Next, the simulation results will be described with reference to Figs. 9 to 15. The numerical aperture of each of the lenses 32 and 33 was set to 1.0. The wavelength in a vacuum of the light output from the light source 11 was set to λ0, and the moving speed V of the observation object S moving in the x direction was set to 1000 λ0 / s. The imaging frame rate of the imaging unit 41 was set to 3000 fps. The number of two-dimensional interference images captured by the imaging unit 41 was set to 312.

[0076] Figure 9 shows the wave vectors {(k x,n ,k y,n )} n FIG. 10 shows the distribution of the wave vector {(k x,n ,k y,n )} n 10 is a diagram showing a modulation pattern m(x, y) created based on the distribution of . In FIG. 10, the magnitude of the amplitude modulation is indicated by shading.

[0077] 11 to 15 are diagrams showing various images generated in the simulation. Figure 11 shows the u out The real part of the image of (x, y, t). (a) shows the real part of the image of u at t = 0 ms. out (b) shows the real part of the image of (x,y,t). out (c) shows the real part of the image of (x,y,t). out Denote the real part of the image of (x,y,t).

[0078] Figure 12 shows the u' after coordinate transformation at each time point generated by the simulation. out The real part of the image of (x', y, t). (a) shows u' at t=0 ms. out (b) shows the real part of the image of (x', y, t). (b) shows u' at t=50ms. out (c) shows the real part of the image of (x', y, t). out Denote the real part of the image of (x',y,t).

[0079] FIG. 13 is a diagram showing complex amplitude images for each irradiation direction generated by simulation. (a) shows the incident angle (θ x ,θ y )=(-35.2°,-25.6°) light component (f n (b) shows the real part of the complex amplitude image when the observation object is irradiated with the incident angle (θ x ,θ y )=(-11.6°,-46.2°) light component (f n (c) shows the real part of the complex amplitude image when the observation object is irradiated with the incident angle (θ x ,θ y )=(0.8°,-16.7°) light component (f n The figure shows the real part of the complex amplitude image when the observation object is illuminated with a frequency of 19.2 Hz.

[0080] 14 is a diagram showing phase differential images of an object to be observed generated by simulation. The leftmost image shows a phase differential image in an xz cross section, and the others show phase differential images in an xy cross section at three positions in the z direction.

[0081] 15 is a diagram showing refractive index distribution images of an observation object generated by simulation. The upper part shows a refractive index distribution image of an exact solution, and the lower part shows a refractive index distribution image generated by simulation. The leftmost part shows a maximum intensity projection image in the y direction, and the others show refractive index distribution images in the xy cross section at three positions in the z direction.

[0082] As shown in the simulation results, according to this embodiment, a moving observation object can be observed. Furthermore, according to this embodiment, spherical lenses can be used as the lenses 31 to 34 instead of cylindrical lenses. Both the optical system between the modulation unit and the observation object and the optical system between the observation object and the imaging unit are not special, and can be normal imaging optical systems. Therefore, according to this embodiment, the optical system can be easily adjusted, and a moving observation object can be easily observed.

[0083] In the processing by the processing unit 42 described above, the frequency f n In order to extract the time-varying components using equation (11), it was assumed that the imaging frame rate of the imaging unit 41 was sufficiently fast. The Nyquist theorem states that in order to accurately reconstruct the waveform of a signal, it is necessary to sample the signal at a speed at least twice as fast as the maximum frequency component of the signal. In other words, the numerical aperture of the lens 32, which is the objective lens on the irradiation side, is set to NA in Let the wavelength of light be λ0 and the wave number of light be k0. Then, k x,n The maximum value of k x,max is expressed by the following equation (13), and the frequency f n The maximum value of f max According to the Nyquist theorem, the image capturing frame rate f sis expressed as 2f max It is necessary that the above is true.

[0084]

number

[0085]

number

[0086]

number

[0087] However, the above formula (10) does not need to be reproduced accurately without distortion, and the imaging frame rate f s is 2F max That is, each light component (wave vector (k x,n ,k y,n ), amplitude U in (k x,n ,k y,n )) is irradiated to the object. s 2F max If it is less than this (undersampling condition), aliasing is exploited as described below.

[0088] Aliasing is the process of reducing the Nyquist rate (2f max ) sampling rate f s When a continuous-time signal is sampled, the discrete-time Fourier transform of the discrete-time signal obtained by sampling is [-f s / 2,f s This refers to the phenomenon in which the frequency f is folded back into the frequency range f / 2. n A signal with sampling rate f sIf we sample at , the frequency f observed due to aliasing n,alias is expressed by the following equation (16). mod f s f s is a modulo operation, and f s Gives the remainder when divided by f. For example, n = 80Hz, f s = 100Hz, then f n,alias =-20Hz.

[0089]

number

[0090] Due to aliasing, the observed frequency will differ from the actual frequency. However, if the observed frequency and the actual frequency correspond one-to-one, each light component (wave vector (k x,n ,k y,n ), amplitude U in (k x,n ,k y,n )) can be used to obtain a complex amplitude image (the right side of the above equation (12)). In order to ensure that the observed frequency and the actual frequency correspond one-to-one, the wave number component k x,n The frame rate f of the imaging unit 41 is s All we need to do is make sure that the remainders when divided by are different.

[0091] When aliasing is used, the frequency f is calculated by the following equation (17) instead of the above equation (12). n The frequency f n,alias When the time-varying components are extracted, each light component (wave vector (k x,n ,k y,n ), amplitude U in (k x,n ,k y,n)), that is, complex amplitude images for each of the N irradiation directions, are obtained.

[0092]

number

[0093] Next, the results of a simulation using aliasing will be described with reference to Figs. 16 to 24. The numerical aperture of each of the lenses 32 and 33 was set to 1.0. The wavelength in a vacuum of the light output from the light source 11 was set to λ0, and the moving speed V of the observation object S moving in the x direction was set to 1000λ0 / s. The imaging frame rate of the imaging unit 41 was set to 1000 fps. The number of two-dimensional interference images captured by the imaging unit 41 was set to 117.

[0094] Figure 16 shows the wave vector {(k x,n ,k y,n )} n FIG. 17 shows the distribution of the wave vector {(k x,n ,k y,n )} n 17 is a diagram showing a modulation pattern m(x, y) created based on the distribution of . In FIG. 17, the magnitude of the amplitude modulation is shown by shading.

[0095] Figure 18 shows the wave number component k in the x direction of each light component. x,n and frequency f n 19 shows the correspondence between the frequency f n As shown in these figures, the wave number component k in the moving direction (x direction) of the observation target S is x,n The frame rate f of the imaging unit 41 is s When divided by, the remainders are different.

[0096] 20 to 24 are diagrams showing various images generated in a simulation in which aliasing is utilized. Figure 20 shows the u out The real part of the image of (x, y, t). (a) shows the real part of the image of u at t = 0 ms. out (b) shows the real part of the image of (x,y,t). out (c) shows the real part of the image of (x,y,t). out Denote the real part of the image of (x,y,t).

[0097] Figure 21 shows the u' after coordinate transformation at each time point generated by the simulation. out The real part of the image of (x', y, t). (a) shows u' at t=0 ms. out (b) shows the real part of the image of (x', y, t). (b) shows u' at t=50ms. out (c) shows the real part of the image of (x', y, t). out Denote the real part of the image of (x',y,t).

[0098] FIG. 22 shows complex amplitude images for each irradiation direction generated by simulation. (a) shows the incident angle (θ x ,θ y )=(-1.1°,10.8°) light component (f n (b) shows the real part of the complex amplitude image when the observation object is irradiated with the incident angle (θ x ,θ y )=(43.0°,-10.8°) light component (f n (c) shows the real part of the complex amplitude image when the observation object is illuminated with the incident angle (θ x ,θ y )=(24.9°,-10.8°) light component (f n The figure shows the real part of the complex amplitude image when the observation object is illuminated with a frequency of 100 Hz (=230.7 Hz).

[0099] 23 is a diagram showing phase differential images of an object to be observed generated by simulation. The leftmost image shows a phase differential image in an xz cross section, and the others show phase differential images in an xy cross section at three positions in the z direction.

[0100] 24 is a diagram showing refractive index distribution images of an observation object generated by simulation. The upper part shows a refractive index distribution image of an exact solution, and the lower part shows a refractive index distribution image generated by simulation. The leftmost part shows a maximum intensity projection image in the y direction, and the others show refractive index distribution images in the xy cross section at three positions in the z direction.

[0101] As shown in the simulation results, even when aliasing is used, a moving observation target can be observed. Moreover, by using aliasing, even when the moving speed V of the observation target S is fast or the imaging frame rate of the imaging unit 41 is slow, a moving observation target can be observed. [Explanation of symbols]

[0102] Reference numerals 1A to 1C refer to an observation device, 11 a light source, 12 a beam splitter (branching section), 13, 14 a mirror, 15 a beam splitter (combining section), 16 a beam splitter, 21, 22 a modulation section, 23 a spatial light modulator, 24 a polarizer, 31 to 34 a lenses, 41 a imaging section, 42 a processing section.

Claims

1. An apparatus for observing a moving observation object, comprising: A light source that outputs light; a branching unit that branches the light into a first branched light and a second branched light; a modulation unit that spatially amplitude-modulates the first branched light based on a modulation pattern to generate a first branched light including a plurality of light components having different wave number components in a moving direction of the observation object; an illumination unit that illuminates the observation object located at a position optically conjugate with the first split light that has been amplitude-modulated by the modulation unit; a multiplexing unit that multiplexes the first split light and the second split light that have passed through the observation object and outputs multiplexed light; an imaging unit having an imaging surface arranged at a position optically conjugate with the object to be observed, receiving the combined light at the imaging surface, and repeatedly outputting a detection signal representing a two-dimensional interference image; a processing unit that generates a complex amplitude image when the object to be observed is irradiated with each of the plurality of light components based on the detection signals repeatedly output from the imaging unit; Equipped with the processing unit generates an image of the object of observation at each time by the first branched light that has reached the imaging surface based on time series data of the two-dimensional interference image generated from a detection signal repeatedly output from the imaging unit, generates an image of the object of observation at each time in a coordinate system fixed to the object of observation, extracts a component that changes over time at a frequency corresponding to a wave number component in the moving direction of each of the plurality of light components, and generates a complex amplitude image when the object of observation is irradiated with each of the plurality of light components. Observation equipment.

2. The modulation unit generates a first branched light including a plurality of optical components whose wave numbers are two-dimensionally distributed in a wave number space. The observation device according to claim 1.

3. the modulation unit generates a first branched light including a plurality of light components having different surpluses when a wave number component in the moving direction is divided by a frame rate of the imaging unit, The observation device according to claim 1.

4. the modulation unit spatially amplitude-modulates the first split light based on a modulation pattern in which values ​​at two positions that are symmetric with respect to an origin in a Fourier transform pattern of the modulation pattern have a complex conjugate relationship. The observation device according to claim 1.

5. The modulation unit includes a spatial light modulator in which the modulation pattern is fixed. The observation device according to claim 1.

6. The modulation unit includes a spatial light modulator capable of variably setting the modulation pattern. The observation device according to claim 1.

7. The processing unit generates a phase differential image of the observation object based on a complex amplitude image obtained when the observation object is irradiated with each of the plurality of light components. The observation device according to claim 1.

8. The processing unit generates a refractive index distribution image of the observation object based on the phase differential image. The observation device according to claim 7.

9. A method for observing a moving observation object, comprising the steps of: a branching step of branching light output from a light source into a first branched light and a second branched light; a modulation step of spatially amplitude-modulating the first branched light based on a modulation pattern to obtain a first branched light including a plurality of light components having different wave number components in a moving direction of the observation object; an irradiation step of irradiating the object of observation located at a position optically conjugate with the first split light that has been amplitude-modulated by the modulation unit; a combining step of combining the first split light and the second split light that have passed through the observation object and outputting a combined light; an imaging step of receiving the combined light on an imaging surface of an imaging unit arranged at a position optically conjugate with the object to be observed, and repeatedly outputting a detection signal representing a two-dimensional interference image; a processing step of generating a complex amplitude image when the object to be observed is irradiated with each of the plurality of light components based on the detection signals repeatedly output from the imaging unit; Equipped with In the processing step, an image of the object of observation at each time is generated by the first branched light that has reached the imaging surface based on time series data of the two-dimensional interference image generated from the detection signal repeatedly output from the imaging unit, an image of the object of observation at each time in a coordinate system fixed to the object of observation is generated, and a component that changes over time at a frequency corresponding to a wave number component in the moving direction of each of the plurality of light components is extracted to generate a complex amplitude image when the object of observation is irradiated with each of the plurality of light components. Observation method.

10. In the modulating step, the first branched light includes a plurality of optical components whose wave numbers are two-dimensionally distributed in a wave number space. The observation method according to claim 9.

11. In the modulating step, a remainder obtained by dividing the wave number component in the moving direction by a frame rate of the imaging unit is set to a first branched light including a plurality of light components different from each other. The observation method according to claim 9.

12. In the modulating step, the first split light is spatially amplitude-modulated based on a modulation pattern in which values ​​at two positions that are symmetric with respect to an origin in a Fourier transform pattern of the modulation pattern have a complex conjugate relationship. The observation method according to claim 9.

13. In the modulating step, the first split light is spatially amplitude-modulated using the modulating unit including a spatial light modulator having a fixed modulation pattern. The observation method according to claim 9.

14. In the modulating step, the first split light is spatially amplitude-modulated using the modulating unit including a spatial light modulator capable of variably setting the modulation pattern. The observation method according to claim 9.

15. In the processing step, a phase differential image of the observation object is generated based on a complex amplitude image obtained when the observation object is irradiated with each of the plurality of light components. The observation method according to claim 9.

16. generating a refractive index distribution image of the object to be observed based on the phase differential image in the processing step; The observation method according to claim 15.

Images