Microscope device

Abstract

This invention provides a microscope device that can more accurately determine the three-dimensional refractive index distribution in a sample. [Solution] The microscope device includes a first microscope unit that irradiates a sample with first illumination light directed in a first direction and detects light from the sample corresponding to the irradiation of the first illumination light; a second microscope unit that irradiates the sample with second illumination light directed in a second direction different from the first direction and detects light from the sample corresponding to the irradiation of the second illumination light; and a data processing unit that generates a three-dimensional refractive index distribution in the sample based on the detection signal of light detected by the first microscope unit and the detection signal of light detected by the second microscope unit.

Description

Technical Field

[0001] The present invention relates to a microscope apparatus.

Background Art

[0002] In recent years, methods for obtaining a three-dimensional refractive index distribution in a sample such as a phase object have been devised (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0004] The microscope apparatus according to the present invention includes a first microscope unit that irradiates a sample with first illumination light directed in a first direction and detects light from the sample in response to the irradiation of the first illumination light, a second microscope unit that irradiates the sample with second illumination light directed in a second direction different from the first direction and detects light from the sample in response to the irradiation of the second illumination light, and a data processing unit that generates a three-dimensional refractive index distribution in the sample based on a detection signal of the light detected by the first microscope unit and a detection signal of the light detected by the second microscope unit.

Brief Description of the Drawings

[0005] [Figure 1] It is a schematic diagram of a microscope apparatus according to a first embodiment. [Figure 2] It is a schematic configuration diagram showing a microscope apparatus according to a first embodiment. [Figure 3] It is an enlarged view of a stage in a first embodiment. [Figure 4] It is a graph showing a distribution of light transmittance of a modulation element. [Figure 5] It is a schematic diagram showing a method of performing deconvolution based on image data of a plurality of cross-sections of a sample. [Figure 6] This figure shows an image of a sample obtained by bright-field observation using a conventional method. [Figure 7] This figure shows an image of the refractive index distribution of a sample produced by a conventional method. [Figure 8] This figure shows the conventional distribution of POTF. [Figure 9] This figure shows the spectral distribution obtained by estimating and supplementing the missing cone region of the conventional POTF. [Figure 10] This figure shows an image of a sample obtained by bright-field observation. [Figure 11] This figure shows an image of the refractive index distribution of the sample. [Figure 12] This is a diagram showing the distribution of POTF. [Figure 13] This figure shows the spectral distribution obtained by estimating and filling in the missing cone region of the POTF. [Figure 14] This is a flowchart showing the data generation method according to the first embodiment. [Figure 15] This is a schematic diagram showing a modified example of the microscope apparatus according to the first embodiment. [Figure 16] This is a schematic diagram showing a microscope apparatus according to the second embodiment. [Figure 17] This is an enlarged view of the stage in the second embodiment. [Figure 18] This is a schematic diagram showing a modified example of the microscope apparatus according to the second embodiment. [Figure 19] This is a schematic diagram showing a microscope apparatus according to the third embodiment. [Figure 20] This is an enlarged view of the stage in the third embodiment. [Figure 21] This is a schematic diagram showing a modified example of the microscope apparatus according to the third embodiment. [Figure 22] This is a schematic diagram of a microscope apparatus according to the fourth embodiment. [Figure 23] This is a schematic diagram showing a microscope apparatus according to the fourth embodiment. [Figure 24] This is an enlarged view of the stage in the fourth embodiment. [Figure 25] It is a schematic diagram of a microscope apparatus according to the fifth embodiment. [Figure 26] It is a schematic diagram of a microscope apparatus according to the sixth embodiment. [Figure 27] It is a schematic configuration diagram showing a first microscope unit according to a modification.

Embodiments for Carrying Out the Invention

[0006] Hereinafter, the microscope apparatus according to each embodiment will be described. In the figures used in the following description, for the sake of easy understanding of the features, the constituent parts may be shown enlarged for convenience, and the dimensional ratios and the like of each constituent part are not necessarily the same as the actual ones.

[0007] <First Embodiment> First, the microscope apparatus 1 according to the first embodiment will be described with reference to FIGS. 1 and 2. As shown in FIGS. 1 and 2, the microscope apparatus 1 according to the first embodiment includes a first microscope unit 10 and a second microscope unit 50. Further, as shown in FIG. 2, the microscope apparatus 1 includes a stage 2, a control unit 90, and an image processing unit 91. The stage 2 supports a sample SA. The sample SA is a phase object such as a cell, for example. A stage driving unit (not shown) is provided on the stage 2. The stage driving unit moves the stage 2 along the optical axis AX1 of the first microscope unit 10.

[0008] As shown in FIG. 2, a coordinate axis extending in the optical axis direction (vertical direction) of the first microscope unit 10 is defined as the z-axis, and the coordinate axes perpendicular to the z-axis are defined as the x-axis and the y-axis. By moving the stage 2 in the z direction by the stage driving unit, as shown in FIG. 3, it is possible to acquire image data of the cross section of the sample SA at a predetermined position Z0, a position Z0 + Δz separated from the position Z0 by +Δz, a position Z0 - Δz separated from the position Z0 by -Δz, a position Z0 + 2Δz separated from the position Z0 by +2Δz, a position Z0 - 2Δz separated from the position Z0 by -2Δz...

[0009] As shown in Figures 1 and 2, the first microscope unit 10 includes a first light source 11, a first illumination optical system 20, a first detection optical system 30, and a first detector 40. The first light source 11 is configured using a white light source such as a halogen lamp or an LED (Light Emitting Diode). Alternatively, the first light source 11 may be configured using a near-infrared light source such as a halogen lamp or an LED. The first light source 11 generates illumination light (hereinafter referred to as the first illumination light) in a predetermined wavelength band.

[0010] The first illumination optical system 20 irradiates the sample SA with first illumination light L1 emitted from the first light source 11 in the -z direction (first direction). As shown in Figure 2, the first illumination optical system 20 includes, in order from the first light source 11 side, a collector lens 21, a field diaphragm 23, a relay lens 24, a first modulation element 25, an aperture diaphragm 26, and a condenser lens 27. When a white light source is used as the first light source 11, it is preferable to provide an element that narrows the wavelength band of the first illumination light. For example, the wavelength band of the first illumination light can be narrowed by inserting a bandpass filter 22 having predetermined spectral transmittance characteristics into the optical path between the collector lens 21 and the field diaphragm 23 in the first illumination optical system 20. By narrowing the wavelength band of the first illumination light, the accuracy of calculated values ​​such as POTF, which will be described in detail later, can be increased. The spectral transmittance characteristics of the bandpass filter 22 are set based on the wavelength band of the illumination light according to the observation application, such as bright-field observation. Alternatively, a bandpass filter 22 may be inserted in the optical path between the field aperture 23 and the relay lens 24 in the first illumination optical system 20.

[0011] The first modulation element 25 and the aperture diaphragm 26 are positioned on a plane perpendicular to the optical axis AX1 of the first microscope unit 10 (first illumination optical system 20) at position P1 of the pupil (hereinafter sometimes referred to as the illumination pupil) between the relay lens 24 and the condenser lens 27 in the first illumination optical system 20. The first modulation element 25 is positioned adjacent to the aperture diaphragm 26 (for example, above the aperture diaphragm 26 as shown in Figure 2). The plane perpendicular to the optical axis AX1 of the first microscope unit 10 at position P1 of the illumination pupil is referred to as the illumination pupil plane. The first modulation element 25 is, for example, a light-transmitting flat plate, and is a flat plate whose light transmittance changes within its plane. This flat plate is formed, for example, by depositing a light-shielding film that can reduce light transmittance onto a parallel plate such as a glass substrate. For example, a metal film is deposited. For example, by changing the film thickness according to the area of ​​the parallel plate on which the film is deposited, the light transmittance can be changed according to the area of ​​the parallel plate (the thicker the film thickness, the lower the transmittance). By placing this first modulation element 25 on the surface of the illumination pupil, the light transmittance can be changed within the surface of the illumination pupil. Therefore, it can be said that the light transmittance of the first modulation element 25 changes within the surface of the illumination pupil. The light transmittance of the first modulation element 25 changes continuously (or discretely) within the surface of the illumination pupil.

[0012] Furthermore, the distribution of light transmittance of the first modulation element 25 (in other words, the distribution of light transmittance on the surface of the illumination pupil) is determined by the change in light transmittance depending on the location of the first modulation element 25. As the first modulation element 25, it is possible to select one of several first modulation elements 25 with different changes in light transmittance, i.e., different distributions of light transmittance, and place it at the position P1 of the illumination pupil. Details of the light transmittance of the first modulation element 25 will be described later. Note that the position in which the first modulation element 25 is placed is not limited to the position P1 of the illumination pupil. For example, the first modulation element 25 may be placed on a plane perpendicular to the optical axis AX1 at a position conjugate to the illumination pupil (in other words, on a plane conjugate to the illumination pupil). Also, the first light source 11 is placed at a position conjugate to the illumination pupil.

[0013] The condenser lens 27 is positioned opposite to the stage 2 above it. It is possible to select one of several condenser lenses 27 with different optical characteristics and place it above the stage 2.

[0014] The first detection optical system 30 receives light from the sample SA in response to the irradiation of the first illumination light L1 from the side opposite the first illumination optical system 20, with the sample SA in between. As shown in Figure 2, the first detection optical system 30 has, in order from the sample SA side, an objective lens unit 31, an imaging lens 36, and a mirror 37. The objective lens unit 31 has a plurality of first objective lenses 32, a lens holder 33, and a unit drive unit 34. The first objective lenses 32 are positioned opposite each other below the stage 2. The lens holder 33 holds a plurality of first objective lenses 32 with different focal lengths. The lens holder 33 is constructed using, for example, a revolving nosepiece or a turret. The unit drive unit 34 drives the lens holder 33 and can select one of the plurality of first objective lenses 32 to be positioned below the stage 2. The unit drive unit 34 may also move the lens holder 33 along the z-axis. In this case, the aforementioned stage drive unit may be used in combination, or the stage drive may not be used at all.

[0015] Light from the sample SA, corresponding to the irradiation of the first illumination light, is incident on the first objective lens 32 located below stage 2. The light that passes through the first objective lens 32 is incident on the imaging lens 36. The light that passes through the imaging lens 36 is reflected by the mirror 37 and forms an image on a predetermined image plane I. Here, the position of the predetermined image plane I is a position conjugate to the focal position of the first objective lens 32 on the sample SA. Alternatively, a half-mirror may be provided instead of the mirror 37, and an observation optical system (not shown) having an eyepiece lens (not shown) may be provided in the optical path of the light passing through the half-mirror. This allows the observer to observe the image of the sample SA using the eyepiece lens.

[0016] A first detector 40 is positioned on the image plane I of the first detection optical system 30. The first detector 40 is constructed using an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). The first detector 40 detects light from the sample SA via the first detection optical system 30.

[0017] As shown in Figures 1 and 2, the second microscope unit 50 includes a light source unit 51, a second illumination optical system 70, a second detection optical system 75, and a second detector 80. The light source unit 51 is also called a beam steering unit. As shown in Figure 2, the light source unit 51 includes a second light source 52, a first lens 53, a first galvanometer mirror 54, a second lens 55, a third lens 56, a second galvanometer mirror 57, a liquid lens 59, a fourth lens 60, and a fifth lens 61. The light source unit 51 also has a cylindrical lens (not shown) that can be inserted into or removed from the optical path between the first lens 53 and the first galvanometer mirror 54.

[0018] The second light source 52 is configured using a laser light source. The second light source 52 emits illumination light (hereinafter referred to as the second illumination light) in a predetermined wavelength band. The first lens 53 aligns the second illumination light emitted from the second light source 52. The first galvanometer mirror 54 reflects the second illumination light from the first lens 53 toward the second lens 55. The first galvanometer mirror 54 can change the direction of propagation of the second illumination light by changing the orientation of its reflective surface. By changing the direction of propagation of the second illumination light, the first galvanometer mirror 54 changes the focal position of the second illumination light in the sample SA in the y direction. The second lens 55 and the third lens 56 cause the second illumination light reflected by the first galvanometer mirror 54 to be incident on the second galvanometer mirror 57. Note that if the first galvanometer mirror 54 and the second galvanometer mirror 57 are positioned conjugate to the pupil, the second lens 55 and the third lens 56 may not be provided.

[0019] The second galvano mirror 57 reflects the second illumination light from the third lens 56 toward the liquid lens 59. The second galvano mirror 57 can change the direction of propagation of the second illumination light by changing the orientation of its reflective surface. By changing the direction of propagation of the second illumination light, the second galvano mirror 57 changes the focal position of the second illumination light in the sample SA in the z direction.

[0020] The liquid lens 59 can change its focal length by changing the radius of curvature of its lens surface. By changing the focal length of the liquid lens 59, the focusing position of the second illumination light on the sample SA is changed in the x direction. The fourth lens 60 and the fifth lens 61 cause the second illumination light that has passed through the liquid lens 59 to enter the second illumination optical system 70. The first galvanometer mirror 54, the second galvanometer mirror 57, and the liquid lens 59 make it possible to change the focusing position of the second illumination light on the sample SA in three dimensions (x, y, and z directions), thereby scanning the sample SA in three dimensions.

[0021] Furthermore, from the viewpoint of deconvolution, it is preferable to pre-adjust the irradiation range of the first illumination light from the first illumination optical system 20 (the three-dimensional observation range on the sample SA) and the irradiation range of the second illumination light from the second illumination optical system 70 (the three-dimensional observation range on the sample SA) to coincide. Adjusting the irradiation range (observation range) of the first and second illumination lights to coincide can be achieved, for example, by controlling the swing angle of at least one of the first galvanometer mirror 54 and the second galvanometer mirror 57 in the light source unit 51. Alternatively, instead of adjusting the irradiation range (observation range) of the first and second illumination lights to coincide, the common irradiation range (observation range) of the first and second illumination lights may be identified to perform image construction by three-dimensional scanning of the sample SA.

[0022] The second illumination optical system 70 focuses the second illumination light L2 emitted from the light source unit 51 (second light source 52) in the +x direction (a second direction perpendicular to the first direction) and irradiates the sample SA. In Figures 2, 15, 16, 18, 19, and 21, the second illumination light L2 is shown with a dashed line to make it easier to distinguish from the first illumination light L1. As shown in Figures 2 and 3, the second illumination optical system 70 has a second illumination objective lens 71. The second illumination objective lens 71 is positioned opposite to the left of the sample SA. The second illumination objective lens 71 focuses the second illumination light L2 emitted from the light source unit 51 (second light source 52) onto the sample SA. The optical axis AX2 between the second illumination optical system 70 and the second detection optical system 75 in the second microscope unit 50 is orthogonal to the optical axis AX1 between the first illumination optical system 20 and the first detection optical system 30 in the first microscope unit 10.

[0023] The second detection optical system 75 receives light from the sample SA in response to the irradiation of the second illumination light from the side opposite the second illumination optical system 70, with the sample SA in between. As shown in Figures 2 and 3, the second detection optical system 75 has, in order from the sample SA side, a second detection objective lens 76 and a second modulation element 77. The second detection objective lens 76 is provided on the side opposite the second illumination objective lens 71, with the sample SA in between. Light from the sample SA in response to the irradiation of the second illumination light enters the second detection objective lens 76. Note that the second illumination objective lens 71 and the second detection objective lens 76 may be the same and have a high numerical aperture (NA).

[0024] The second modulation element 77 is positioned at a position P2 conjugate to the pupil of the second detection objective lens 76 in the second detection optical system 75, on a plane perpendicular to the optical axis AX2 of the second microscope unit 50 (second detection optical system 75). The plane perpendicular to the optical axis AX2 of the second microscope unit 50 at position P2 conjugate to the detection pupil is referred to as the plane conjugate to the detection pupil. The second modulation element 77 is formed, for example, by depositing a film capable of reducing light transmittance onto a parallel plate such as a glass substrate, similar to the first modulation element 25. By positioning this second modulation element 77 on the plane conjugate to the detection pupil, the light transmittance can be changed within the plane conjugate to the detection pupil. Therefore, it can be said that the light transmittance of the second modulation element 77 changes within the plane conjugate to the detection pupil. The light transmittance of the second modulation element 77 changes continuously (or discretely) within the plane conjugate to the detection pupil. As the second modulation element 77, it is possible to select one of several second modulation elements 77 with different light transmittance distributions and place it at a position P2 conjugate to the detection pupil. Details of the light transmittance of the second modulation element 77 will be described later. Note that the position in which the second modulation element 77 is placed is not limited to the position P2 conjugate to the detection pupil. For example, the second modulation element 77 may be placed on a plane perpendicular to the optical axis AX2 at the position of the detection pupil (in other words, the plane of the detection pupil). In this case, for example, the second modulation element 77 may be built into the second detection objective lens 76.

[0025] The second detector 80 is positioned adjacent to the second modulation element 77 at a position P2 conjugate to the detection pupil in the second detection optical system 75. The second detector 80 is constructed using a PMT (Photomultiplier tube), NDD (Non-Descanned Detection), or the like. The second detector 80 detects light from the sample SA via the second detection optical system 75.

[0026] An example of the change in light transmittance within the pupil plane (in other words, the distribution of light transmittance within the pupil plane) of the first modulation element 25 and the second modulation element 77 is described below. Figure 4 is a graph showing an example of the distribution of light transmittance of the first modulation element 25 and the second modulation element 77. In Figure 4, X is the x-coordinate with the origin at the coordinate position through which the optical axis (optical axis AX1 of the first microscope unit 10 or optical axis AX2 of the second microscope unit 50) passes, and Y is the y-coordinate with the origin at the coordinate position through which the optical axis passes.

[0027] In the example shown in Figure 4, the light transmittance of the first modulation element 25 or the second modulation element 77 changes along one direction according to a continuous function. Specifically, the light transmittance of the first modulation element 25 or the second modulation element 77 decreases monotonically in the X direction (e.g., the -X direction) according to a sine function (the portion of light transmittance that is equal is distributed in a straight line extending in the Y direction). In other words, the light transmittance of the first modulation element 25 or the second modulation element 77 changes according to a sine function. It should also be said that the light transmittance of the first modulation element 25 or the second modulation element 77 increases monotonically in the X direction (e.g., the +X direction) according to a sine function within the plane of the pupil, or it can be said that it decreases or increases monotonically in the X direction.

[0028] Furthermore, the light transmittance of the first modulation element 25 or the second modulation element 77 may decrease or increase monotonically in the Y direction according to a sine function (the portion with equal light transmittance may be distributed in a straight line extending in the X direction). Also, the light transmittance of the first modulation element 25 or the second modulation element 77 may decrease or increase monotonically in any direction in the XY coordinate system, not limited to the X or Y direction (the portion with equal light transmittance may be distributed in a straight line extending in a direction perpendicular to any direction in the XY coordinate system).

[0029] In the example shown in Figure 4, the light transmittance of the first modulation element 25 or the second modulation element 77 may decrease monotonically in the X direction (e.g., the -X direction) according to a linear function. In other words, the light transmittance of the first modulation element 25 or the second modulation element 77 is not limited to a sine function, but may change according to a linear function.

[0030] In the example shown in Figure 4, the continuous function can be any of the following: a linear function, a quadratic function, a Gaussian function, a sine function, or a cosine function. Note that the continuous function is not limited to linear, quadratic, Gaussian, sine, or cosine functions; it may also be a cubic function or other function. Furthermore, the range in which the light transmittance changes in the first modulation element 25 and the second modulation element 77 should be set to match the size (diameter) of the pupil (illumination pupil, detection pupil). For example, in the case shown in Figure 4, the first modulation element 25 and the second modulation element 77 are formed so that the region where the light transmittance is 0 coincides with the outer circumference of the pupil (illumination pupil, detection pupil).

[0031] Furthermore, when the light transmittance of the first modulation element 25 or the second modulation element 77 changes according to a cosine or sine function, it is desirable that the change follows a cosine or sine function within a range smaller than one period in the plane of the pupil (illumination pupil, detection pupil). This is because if the range exceeds one period, the POTF value also exhibits periodic behavior, which is undesirable from the standpoint of deconvolution. In this case, there will be multiple frequencies at which the POTF value is 0, increasing the noise generated during the deconvolution process and reducing the accuracy of the refractive index distribution of the resulting sample SA. In addition, when the light transmittance of the first modulation element 25 or the second modulation element 77 changes according to a cosine function, measures are taken to ensure that the light transmittance becomes 0 (the cosine function value becomes 0) at the outer edge of the pupil (illumination pupil, detection pupil). This is because discontinuities in light transmittance at the outer edge of the pupil cause artifacts such as ringing in the image.

[0032] In the example shown in Figure 4, the light transmittance of the first modulation element 25 or the second modulation element 77 changes along one direction within the pupil plane or the plane conjugate to the pupil according to a continuous function, and becomes 0 (zero) in a part of the outer periphery within the pupil plane or the plane conjugate to the pupil, but is not limited to this. For example, the light transmittance of the first modulation element 25 or the second modulation element 77 may change as it moves away from the optical axis within the pupil plane or the plane conjugate to the pupil according to a continuous function, and may become 0 (zero) around the entire outer periphery within the pupil plane or the plane conjugate to the pupil.

[0033] In this embodiment, when performing bright-field observation of the sample SA, the first illumination light L1 emitted from the first light source 11 of the first microscope unit 10 is incident on the collector lens 21 of the first illumination optical system 20. The first illumination light L1 that has passed through the collector lens 21 becomes parallel light, passes through the field diaphragm 23 (or the bandpass filter 22 and field diaphragm 23 when a white light source is used as the first light source 11), and is incident on the relay lens 24. The first illumination light L1 that has passed through the relay lens 24 passes through the first modulation element 25 and the aperture diaphragm 26 and is incident on the condenser lens 27. The first illumination light L1 that has passed through the condenser lens 27 becomes parallel light and irradiates the sample SA on the stage 2. As a result, the first illumination optical system 20 irradiates the sample SA with the first illumination light L1 emitted from the first light source 11 in the -z direction (first direction).

[0034] Light transmitted through the sample SA from the first illumination optical system 20 (hereinafter sometimes referred to as the first detection light) is incident on the first objective lens 32 of the first detection optical system 30. The first detection light that has passed through the first objective lens 32 is incident on the imaging lens 36. The first detection light that has passed through the imaging lens 36 is reflected by the mirror 37 and forms an image on a predetermined image plane I where the first detector 40 is located. The first detector 40 detects the light from the sample SA (first detection light) via the first detection optical system 30 and outputs a detection signal for the light. The detection signal of the light (first detection light) output from the first detector 40 is transmitted to the image processing unit 91 via the control unit 90. In other words, the first detector 40 captures an image of the sample SA via the detection optical system 40. Here, the detection signal is a signal indicating the signal intensity detected by the first detector 40 or the second detector 80 according to the intensity of the light (detection light). For example, if the first detector 40 is configured using a CCD, the signal is the signal at each pixel of the CCD. The detection signal of the first detector 40 can be rephrased as a signal indicating the signal intensity detected by the first detector 40 according to the intensity of the image of the sample SA.

[0035] When performing bright-field observation of the sample SA, the cylindrical lens (not shown) of the light source unit 51 of the second microscope unit 50 is retracted from the optical path between the first lens 53 and the first galvanometer mirror 54 in the light source unit 51. The second illumination light emitted from the second light source 52 of the light source unit 51 is incident on the first lens 53. The second illumination light that has passed through the first lens 53 becomes parallel light and is reflected by the first galvanometer mirror 54. The second illumination light reflected by the first galvanometer mirror 54 passes through the second lens 55 and the third lens 56 and is reflected by the second galvanometer mirror 57. The second illumination light reflected by the second galvanometer mirror 57 is incident on the liquid lens 59. The second illumination light that has passed through the liquid lens 59 passes through the fourth lens 60 and the fifth lens 61 and is emitted outside the light source unit 51. As a result, the light source unit 51 emits the second illumination light.

[0036] The second illumination light L2 emitted from the light source unit 51 (second light source 52) is incident on the second illumination objective lens 71 of the second illumination optical system 70. The second illumination light L2 that has passed through the second illumination objective lens 71 is focused and irradiates the sample SA on the stage 2. As a result, the second illumination optical system 70 focuses the second illumination light L2 emitted from the light source unit 51 (second light source 52) in the +x direction (a second direction perpendicular to the first direction) and irradiates the sample SA.

[0037] Light transmitted through the sample SA from the second illumination optical system 70 (hereinafter sometimes referred to as the second detection light) is incident on the second detection objective lens 76 of the second detection optical system 75. The second detection light that has passed through the second detection objective lens 76 passes through the second modulation element 77 and is incident on the second detector 80. The second detector 80 detects the light from the sample SA (second detection light) via the second detection optical system 75 and outputs a detection signal for the light. The detection signal of the light (second detection light) output from the second detector 80 is transmitted to the image processing unit 91 via the control unit 90.

[0038] Furthermore, it is possible to use the microscope device 1 as a selective planar illumination microscope (SPIM) to perform fluorescence observation of the sample SA. When performing fluorescence observation of the sample SA, the cylindrical lens (not shown) of the light source unit 51 of the second microscope unit 50 is inserted into the optical path between the first lens 53 and the first galvanometer mirror 54 in the light source unit 51.

[0039] When performing fluorescence observation of sample SA, the excitation light emitted from the second light source 52 of the light source unit 51 enters the first lens 53. The excitation light that has passed through the first lens 53 passes through a cylindrical lens (not shown) and is reflected by the first galvanometer mirror 54. The excitation light reflected by the first galvanometer mirror 54 passes through the second lens 55 and the third lens 56 and is reflected by the second galvanometer mirror 57. The excitation light reflected by the second galvanometer mirror 57 enters the liquid lens 59. The excitation light that has passed through the liquid lens 59 passes through the fourth lens 60 and the fifth lens 61 and is emitted outside the light source unit 51. As a result, the light source unit 51 emits excitation light, which is sheet light.

[0040] The excitation light emitted from the light source unit 51 (second light source 52) is incident on the second illumination objective lens 71 of the second illumination optical system 70. The excitation light that has passed through the second illumination objective lens 71 is focused and irradiates the sample SA on the stage 2. In this way, the second illumination optical system 70 irradiates the sample SA with excitation light, which is sheet light emitted from the light source unit 51 (second light source 52).

[0041] When excitation light is irradiated, the fluorescent substance contained in the sample SA is excited and emits fluorescence. The fluorescence from the sample SA is incident on the first objective lens 32 of the first detection optical system 30 of the first microscope unit 10. The fluorescence that has passed through the first objective lens 32 is incident on the imaging lens 36. The fluorescence that has passed through the imaging lens 36 is reflected by the mirror 37 and forms an image on a predetermined image plane I where the first detector 40 is located. The first detector 40 detects the fluorescence from the sample SA via the first detection optical system 30 and outputs a detection signal for the fluorescence. The fluorescence detection signal output from the first detector 40 is transmitted to the image processing unit 91 via the control unit 90.

[0042] The control unit 90 performs overall control of the microscope apparatus 1. The control unit 90 is electrically connected to the stage drive unit (not shown), the unit drive unit 34, the first detector 40, the light source unit 51, the second detector 80, the image processing unit 91, the operation input unit (not shown), the image display unit (not shown), and the like.

[0043] When performing bright-field observation of the sample SA, the image processing unit 91 generates refractive index data for the sample SA based on the detection signal of light output from the first detector 40 (first detection light) and the detection signal of light output from the second detector 80 (second detection light). Here, the refractive index data for the sample SA is data representing the refractive index of the sample SA, for example, the refractive index data at each position in the sample SA, i.e., data showing the refractive index distribution in the sample SA. The refractive index data for the sample SA is stored in a storage unit (not shown) as a lookup table, for example. The image processing unit 91 also generates image data (hereinafter sometimes referred to as image data of the refractive index distribution of the sample SA) in which the brightness value of each pixel is set according to the refractive index value at each position in the refractive index distribution of the sample SA. Furthermore, the image processing unit 91 generates image data (hereinafter sometimes referred to as bright-field observation image data of the sample SA) in which the brightness value of each pixel is set according to the signal intensity value of the detection signal at each position in the sample SA (each pixel of the first detector 40) based on the detection signal of the light output from the first detector 40 (first detection light) and the detection signal of the light output from the second detector 80 (second detection light).

[0044] When performing fluorescence observation of the sample SA, the image processing unit 91 generates image data (hereinafter sometimes referred to as fluorescence observation image data of the sample SA) in which the brightness value of each pixel is set according to the signal intensity value of the detection signal at each position in the sample SA, based on the fluorescence detection signal output from the first detector 40.

[0045] This makes it possible to display an image of the refractive index distribution of the sample SA on the image display unit (not shown) based on the refractive index distribution image data of the sample SA generated by the image processing unit 91. Furthermore, it is possible to display an image of the sample SA obtained by bright-field observation on the image display unit based on the bright-field observation image data of the sample SA generated by the image processing unit 91. It is also possible to display an image of the sample SA obtained by fluorescence observation based on the fluorescence observation image data of the sample SA generated by the image processing unit 91.

[0046] Next, a known method for determining the three-dimensional refractive index distribution in the sample SA as refractive index data using the image processing unit 91 will be described. A typical example of determining the three-dimensional refractive index distribution in the sample SA is a method that uses a theory called PC-ODT (Partially Coherent-Optical Diffraction Tomography). The theory of PC-ODT will be briefly described below. From the equation for partially coherent imaging, the intensity I(x,y,z) of the image of a three-dimensional object can be expressed as shown in equation (1) below.

[0047]

number

[0048] In equation (1), o represents the complex amplitude transmittance of the object. TCC represents the transmission cross coefficient. (ξ,η,ζ) represents the direction cosines of the diffracted light (or direct light). In this case, the image is the image of the sample SA obtained by imaging the light (detection light) that has passed through at least a part of the sample SA due to illumination. Therefore, the intensity I(x,y,z) of the image of the three-dimensional object, i.e., the image of the three-dimensional sample SA, can be replaced in image processing by the signal intensity of the detection signal output from the first detector 40, etc. (for example, the signal intensity at each pixel of the first detector 40 when the sample SA is imaged by the first detector 40). As shown in Figure 2, the coordinate axis extending in the optical axis direction (vertical direction) of the first microscope unit 10 is the z axis, and the coordinate axes perpendicular to the z axis are the x axis and y axis. The transmission cross coefficient TCC can be expressed as shown in equation (2) below.

[0049]

number

[0050] In equation (2), S represents the illuminating pupil, and G represents the detection pupil. The cross-transmission coefficient TCC is Hermitian conjugated and therefore has the properties shown in equation (3) below.

[0051]

number

[0052] For thin samples such as cells, the effect of scattering is small, so the first-order Born approximation (low-contrast approximation) holds. In this case, we only need to consider the interference between the direct light transmitted through the sample (zero-order diffracted light) and the diffracted light diffracted by the sample (first-order diffracted light). Therefore, using the first-order Born approximation, equation (4) below can be obtained from equations (1) to (3) above.

[0053]

number

[0054] Furthermore, the complex amplitude transmittance o of an object can be approximated by the following equation (5).

[0055]

number

[0056] In equation (5), P represents the real part of the scattering potential, and Φ represents the imaginary part of the scattering potential. Equation (4) above can be expressed as equation (6) below using equation (5).

[0057]

number

[0058] Here, we will change TCC to WOTF (Weak Object Transfer Function). WOTF is defined by the following equation (7).

[0059]

number

[0060] From equations (6) and (7) above, the intensity I(x,y,z) of the image of a three-dimensional object obtained by a transmitted light microscope can be expressed as shown in equation (8) below.

[0061]

number

[0062] Here, we assume that the amplitude change of the sample is small and negligible. That is, we set P=0. In this case, when we express the above equation (8) in real space, we obtain the following equation (9).

[0063]

number

[0064] In equation (9), EPSF represents the Effective Point Spread Function. EPSF is equivalent to the inverse Fourier transform of WOTF. EPSF is generally a complex function. The first term of equation (9) represents the background intensity. The second term of equation (9) shows that the imaginary part Φ of the scattering potential of the sample is multiplied by the imaginary part Im[EPSF] of EPSF. Using this equation (9), the imaginary part Φ of the scattering potential of the sample can be determined.

[0065] One method for determining Φ(x,y,z) is to perform deconvolution directly using Im[EPSF]. Figure 5 schematically shows the process of moving stage 2 in the z direction (i.e., in the optical axis direction) to obtain the intensity of images (signal intensity of the detection signal output from the first detector 40, etc.) of multiple cross-sections (xy cross-sections) where the z-direction position (i.e., the optical axis direction) of the sample SA is different, and then performing deconvolution. Note that the images of multiple cross-sections where the z-direction position (i.e., the optical axis direction) of the sample SA are sometimes collectively referred to as the z-stacked image of the sample SA. The first term of equation (9) is a constant term representing the background intensity. First, both sides of equation (9) are divided by this constant term to normalize it, and then the first term of the normalized equation (9) is removed in real space (or frequency space). Then, by performing deconvolution using Im[EPSF], the following equation (10) is obtained.

[0066]

number

[0067] In equation (10), the Phase Optical Transfer Function (POTF) is obtained by the 3D Fourier transform of Im[EPSF]. Since Im[EPSF] can take on values ​​from positive to negative, the value of POTF can also take on values ​​from positive to negative. Here, POTF is an index that represents the contrast and resolution of the image of the sample SA obtained by bright-field observation. Specifically, the absolute value of POTF represents the contrast of the image, and the higher the absolute value of POTF, the higher the contrast of the image of the sample SA obtained by bright-field observation. Also, the wider the region in the frequency domain where the value of POTF is not zero, the higher the resolution of the image of the sample SA obtained by bright-field observation. Furthermore, I' is the normalized value of the intensity I of the image of each cross section of the sample SA in the z-stack image of the sample SA (for example, I1 to I6 in Figure 5) using the constant term in equation (9). γ can take on any small value.

[0068] A second method for determining Φ(x,y,z) involves calculating the difference in intensity between images of two cross-sections of the sample SA at different positions in the z-direction (i.e., positions along the optical axis). After removing the constant term from equation (9), deconvolution is performed using the calculated difference in intensity, Im[EPSF]. This method is also described in International Publication No. 2021 / 064807, and therefore will not be explained here.

[0069] The scattering potential Φ is defined by equation (11) below when P=0.

[0070]

number

[0071] In equation (11), n(x,y,z) represents the three-dimensional refractive index distribution in sample SA, k0 represents the wave number in vacuum, and n mθ represents the refractive index of the medium. Using equation (11), the scattering potential Φ obtained by the method described above can be converted into a three-dimensional refractive index distribution. The image processing unit 91 uses equations (10) and (11) above to calculate the three-dimensional refractive index distribution n(x,y,z) in the sample SA from the signal intensity of the detection signal output from the first detector 40, etc., i.e., the intensity of the three-dimensional image of the sample SA, I(x,y,z). As an example, the image processing unit 91 generates image data in which the brightness value of each pixel is set according to the refractive index value at each position (coordinate) of the calculated three-dimensional refractive index distribution of the sample SA, i.e., image data of the three-dimensional refractive index distribution of the sample SA. The intensity of the image of the three-dimensional sample SA can be expressed as the intensity of the image of each cross-section of the sample SA in the z-stack image of the sample SA. In other words, the intensity of the image of the three-dimensional sample SA can also be said to be the intensity of multiple images of the sample SA that are at different positions in the z direction (i.e., positions in the optical axis direction).

[0072] When detecting light from a sample SA using only the first microscope unit 10, as in the conventional method, the POTF has a region where information is lost in the z direction (hereinafter referred to as the missing cone region), resulting in errors in the change of refractive index in the z direction. Therefore, it is difficult to generate images of the three-dimensional refractive index distribution of the sample SA or images of the sample SA obtained by bright-field observation. Figure 6 shows an example of an image of a sample SA (xz cross section) obtained by bright-field observation using the conventional method. Figure 7 shows an example of an image of the three-dimensional refractive index distribution (xz cross section) of a sample SA generated by the conventional method. As the sample SA shown in Figures 6 and 7, a roughly spherical pseudo-cell with known refractive index is used. In the examples shown in Figures 6 and 7, the refractive index of the sample SA (pseudo-cell) is set to ~1.35, and the refractive index of the medium is set to 1.33. In addition, the illumination-side NA (numerical aperture) and detection-side NA of the microscope used in the examples shown in Figures 6 and 7 are set to 0.95.

[0073] Figure 8 shows the conventional POTF distribution. In Figure 8, white (background) indicates a POTF value of 0, and black indicates a POTF value that is positive or negative. Therefore, in Figure 8, the darker the black, the larger the absolute value of POTF. As shown in Figure 8, a large missing cone region exists on the central side of the conventional POTF distribution. As a result, the image of the sample SA obtained by bright-field observation deviates from its original approximately spherical shape and is stretched in the z direction, as shown in Figure 6, for example.

[0074] Furthermore, because POTF has a missing cone region, refractive index correction is necessary when determining the three-dimensional refractive index distribution using equations (10) and (11) above. Therefore, refractive index correction is performed using missing cone estimation methods such as the Gercberg-Papoulis method, Edge-Preserving Regularization method, and Total Variation Regularization method. Specifically, the missing cone region is estimated by setting constraints using a missing cone estimation algorithm so that the minimum refractive index value becomes a predetermined refractive index value (for example, the refractive index value of the medium in a known sample SA).

[0075] Figure 9 shows the spectral distribution obtained by estimating and interpolating the missing cone region of the POTF in the conventional method. As shown in Figure 9, even when the missing cone region of the POTF is interpolated, the three-dimensional refractive index distribution image of the sample SA deviates from its original approximately spherical shape and is extended in the z direction, as shown in Figure 7, for example.

[0076] In this embodiment, the image processing unit 91 generates image data of the three-dimensional refractive index distribution of the sample SA based on the detection signal of light output from the first detector 40 when the first illumination optical system 20 irradiates the sample SA with first illumination light L1 in the -z direction (first direction), and the detection signal of light output from the second detector 80 when the second illumination optical system 70 focuses and irradiates the sample SA with second illumination light L2 in the +x direction (second direction orthogonal to the first direction). As a result, according to this embodiment, it is possible to reduce the missing cone region in the POTF based on the detection signal of light output from the second detector 80, making it possible to generate images of the three-dimensional refractive index distribution of the sample SA and images of the sample SA by bright-field observation more accurately.

[0077] Figure 10 shows an example of an image of a sample SA (xz section) obtained by bright-field observation using the method according to this embodiment. Figure 11 shows an example of an image of the three-dimensional refractive index distribution (xz section) of the sample SA generated by the method according to this embodiment. As the sample SA shown in Figures 10 and 11, a roughly spherical pseudo-cell with a known refractive index is used. In the examples shown in Figures 10 and 11, the refractive index of the sample SA (pseudo-cell) is set to ~1.35, and the refractive index of the medium is set to 1.33. In addition, the illumination-side NA (numerical aperture) and detection-side NA of the first microscope unit 10 are set to 0.95, and the illumination-side NA and detection-side NA of the second microscope unit 50 are set to 0.5.

[0078] Figure 12 shows the distribution of POTF in this embodiment. In Figure 12, as in Figure 8, the darker the black, the larger the absolute value of POTF. As shown in Figure 12, in this embodiment, a POTF based on the detection signal of light output from the second detector 80 exists on the central side of the POTF distribution, filling the missing cone region. Therefore, the image of the sample SA obtained by bright-field observation becomes closer to the original approximately spherical shape, as shown in Figure 10, for example. Figure 13 shows the spectral distribution in this embodiment where the missing cone region of the POTF is estimated and interpolated. As shown in Figure 13, the missing cone region of the POTF can be interpolated over a wider range than conventionally, and the image of the three-dimensional refractive index distribution of the sample SA becomes closer to the original approximately spherical shape, as shown in Figure 11, for example.

[0079] Furthermore, let f be the spatial frequency, Pill(f) be the pupil function of the lens (objective lens / condenser lens) that determines the illumination-side NA (numerical aperture), and Pcol(f) be the pupil function of the lens (objective lens / condenser lens) that determines the detection-side NA. POTF is obtained as the convolution of the illumination system's effective pupil function and the detection system's effective pupil function. In the first microscope section 10, which is a Köhler illumination microscope, the illumination system's effective pupil function is |Pill(f)| 2 P * It is known that the effective pupil function of the detection system is |col(f)| and the effective pupil function of the detection system is |Pcol(f)|. On the other hand, in the second microscope section 50, which is a non-confocal laser microscope, the effective pupil function of the illumination system is Pill(f)| and the effective pupil function of the detection system is |Pcol(f)|. 2 Pill * (f) is known to be the case. Note that the convolution does not change even if the order is changed, so the same POTF can be obtained with the Köhler illumination microscope (first microscope section 10) and the non-confocal laser microscope (second microscope section 50) if the conditions such as NA are the same.

[0080] When a linear intensity transmission mask (for example, a first modulation element 25 whose light transmittance monotonically increases or decreases in one direction within the pupil plane according to a linear function) is placed at the position of the illumination pupil in a Köhler illumination microscope (first microscope section 10), |Pill(f)| 2 Since it is a linear function, the effective pupil function of the illumination system is a linear function. Therefore, in the example shown in Figure 12, the distribution of POTF increases symmetrically, and the resolution of the sample SA in the direction perpendicular to the optical axis is improved. On the other hand, when a linear function intensity transmission mask (for example, a second modulation element 77 whose light transmittance increases or decreases monotonically along one direction in a plane conjugate to the pupil according to a linear function) is placed at a position conjugate to the detection pupil in a non-confocal laser microscope (second microscope unit 50), |Pcol(f)| 2 Since this is a linear function, the effective pupil function of the detection system is also a linear function. As a result, the distribution of POTF increases symmetrically in the example shown in Figure 12, and the resolution of the sample SA in the direction perpendicular to the optical axis is improved. By arranging the Köhler illumination microscope and the non-confocal laser microscope orthogonally, the resolution in two directions (x and z directions in the example shown in Figure 12) can be improved, and an accurate three-dimensional image of the transparent phase object sample SA can be formed. Therefore, by calculations based on the light detection signal from the Köhler illumination microscope (first microscope unit 10) and the light detection signal from the non-confocal laser microscope (second microscope unit 50), it becomes possible to more accurately determine the three-dimensional refractive index distribution of the sample SA.

[0081] Furthermore, the Köhler illumination microscope (first microscope section 10) and the non-confocal laser microscope (second microscope section 50) may satisfy the following equation (12) if the illumination-side NA and the detection-side NA are the same.

[0082]

number

[0083] Here, NA_1 is the numerical aperture (NA) of the Köhler illumination microscope (first microscope section 10). NA_2 is the numerical aperture (NA) of the non-confocal laser microscope (second microscope section 50). n_1 is the refractive index of the immersion liquid in the sample SA using the Köhler illumination microscope (first microscope section 10). n_2 is the refractive index of the immersion liquid in the sample SA using the non-confocal laser microscope (second microscope section 50). By satisfying equation (12), the missing cone region in the POTF is completely filled, making it possible to more accurately determine the three-dimensional refractive index distribution in the sample SA.

[0084] Furthermore, while it is ideal to arrange the first microscope unit 10 and the second microscope unit 50 orthogonally, this is not the only option. Even if the first microscope unit 10 and the second microscope unit 50 are arranged at different angles, it is possible to determine the three-dimensional refractive index distribution in the sample SA more accurately compared to using only the first microscope unit 10.

[0085] By arranging the first microscope unit 10, which is a Köhler illumination microscope, vertically and the second microscope unit 50, which is a non-confocal laser microscope, horizontally, the configuration of a conventional Köhler illumination microscope can be utilized, thereby increasing the practicality of the microscope device. However, the combination of the first and second microscope units is not limited to a combination of a Köhler illumination microscope and a non-confocal laser microscope. For example, even with a combination of two Köhler illumination microscopes or two non-confocal laser microscopes, it is possible to more accurately determine the three-dimensional refractive index distribution in the sample SA in the same manner.

[0086] Ideally, the light transmittances of the first modulation element 25 and the second modulation element 77 should increase or decrease monotonically in one direction within the pupil plane or the plane conjugate to the pupil according to a linear function, but this is not limited to this. The light transmittance of one of the first modulation element 25 and the second modulation element 77 may increase or decrease monotonically in one direction within the pupil plane or the plane conjugate to the pupil according to a linear function. As mentioned above, the light transmittance of at least one of the first modulation element 25 and the second modulation element 77 may increase or decrease monotonically in one direction within the pupil plane or the plane conjugate to the pupil according to a continuous function, and the continuous function may be any of the following functions: sine function, cosine function, quadratic function, or Gaussian function. Even with such a configuration, it is possible to more accurately determine the three-dimensional refractive index distribution in the sample SA.

[0087] As mentioned above, it is possible to select one of a plurality of first modulation elements 25, each having a different light transmittance distribution, and place it at the position P1 of the illuminating pupil. In this case, a turret (not shown) holding a plurality of first modulation elements 25 may be provided, and the first modulation element 25 to be placed at the position P1 of the illuminating pupil may be selected by rotating the turret. Note that the element selection unit, which is capable of selecting one of the plurality of first modulation elements 25 and placing it at the position P1 of the illuminating pupil, is not limited to a turret, but may also use an existing mechanism such as a slider. As a result, the control unit 90 controls the element selection unit to switch to one of the plurality of first modulation elements 25 and place it at the position P1 of the illuminating pupil, thereby changing the distribution of light transmittance within the plane of the illuminating pupil.

[0088] Furthermore, as the second modulation element 77, it is possible to select one of a plurality of second modulation elements 77 having different light transmittance distributions and place it at a position P2 conjugate to the detection pupil. In this case, a turret (not shown) holding a plurality of second modulation elements 77 may be provided, and the second modulation element 77 to be placed at the position P2 conjugate to the detection pupil may be selected by rotating the turret. Note that the means for selecting one of the plurality of second modulation elements 77 and placing it at the position P2 conjugate to the detection pupil can be the same means (element selection unit) as the means for selecting one of the plurality of first modulation elements 25 and placing it at the position P1 of the illumination pupil. As a result, the control unit 90 controls the element selection unit to switch to one of the plurality of second modulation elements 77 and place it at the position P2 conjugate to the detection pupil, thereby changing the light transmittance distribution in the plane conjugate to the detection pupil.

[0089] Next, a method for generating refractive index data in the microscope apparatus 1 according to the first embodiment will be described. Figure 14 is a flowchart of the data generation method according to the first embodiment. The sample SA is assumed to be placed on the stage 2 in advance. The control unit 90 includes, for example, a computer system. The control unit 90 reads a control program stored in the memory unit and executes various processes according to this control program.

[0090] First, the first illumination optical system 20 of the first microscope unit 10 irradiates the sample SA with first illumination light directed in a first direction (step ST1). Next, the first detection optical system 30 receives light from the sample SA corresponding to the irradiation with first illumination light (step ST2). Next, the first detector 40 detects the light from the sample SA via the first detection optical system 30 and outputs a light detection signal (step ST3). Next, the second illumination optical system 70 of the second microscope unit 50 irradiates the sample SA with second illumination light directed in a second direction perpendicular to the first direction (step ST4). Next, the second detection optical system 75 receives light from the sample SA corresponding to the irradiation with second illumination light (step ST5). Next, the second detector 80 detects the light from the sample SA via the second detection optical system 85 and outputs a light detection signal (step ST6). Furthermore, the processing in steps ST1 to ST6 is repeated to detect images of multiple cross-sections of the sample SA with different positions in the z-direction (position in the optical axis direction), i.e., light from each cross-section of the sample SA corresponding to the z-stacked image of the sample SA. For example, in steps ST1 to ST3, the stage 2 (sample SA) may be moved in the z-direction by a stage drive unit (not shown), thereby allowing the first microscope unit 10 (first detector 40) to detect light from each cross-section of the sample SA. In steps ST4 to ST6, the sample SA may be scanned three-dimensionally by a first galvanometer mirror 54, a second galvanometer mirror 57, and a liquid lens 59, thereby allowing the second microscope unit 50 (second detector 80) to detect light from each cross-section of the sample SA. Then, the image processing unit 91 generates a three-dimensional refractive index distribution in the sample SA (for example, image data of the three-dimensional refractive index distribution of the sample SA) based on the light detection signal output from the first detector 40 and the light detection signal output from the second detector 80 (step ST7). In this case, for example, the image processing unit 91 may calculate the three-dimensional refractive index distribution of the sample SA using equations (10) and (11) above, based on the intensity of the image of the sample SA obtained by merging the intensity of the image of the sample SA corresponding to the signal intensity of the detection signal output from the first detector 40 and the intensity of the image of the sample SA corresponding to the signal intensity of the detection signal output from the second detector 80. This makes it possible to determine the three-dimensional refractive index distribution of the sample SA more accurately.

[0091] In the flow chart shown in Figure 14, the first microscope unit 10 irradiates the sample SA with first illumination light directed in a first direction and detects the light from the sample SA corresponding to the irradiation of the first illumination light (steps ST1 to ST3), and the second microscope unit 50 irradiates the sample SA with second illumination light directed in a second direction perpendicular to the first direction and detects the light from the sample SA corresponding to the irradiation of the second illumination light (steps ST4 to ST6). These steps are performed in this order, but are not limited to this. For example, each step by the first microscope unit 10 (steps ST1 to ST3) and each step by the second microscope unit 50 (steps ST4 to ST6) may be performed simultaneously. This makes it possible to determine the three-dimensional refractive index distribution of the sample SA in a short time.

[0092] In the first embodiment described above, the image processing unit 91 performs three-dimensional deconvolution for each cross-section of the sample SA corresponding to the z-stack image of the sample SA, based on the light detection signal detected by the first detector 40 and the light detection signal detected by the second detector 80, to obtain the three-dimensional refractive index distribution of the sample SA (for example, image data of the three-dimensional refractive index distribution of the sample SA). However, it is not limited to this. For example, the image processing unit 91 may perform two-dimensional deconvolution for each cross-section of the sample SA corresponding to the z-stack image of the sample SA, based on the light detection signal detected by the first detector 40 and the light detection signal detected by the second detector 80 (for example, two-dimensional deconvolution based on POTF at the cross-section where Fz(Fx)=0 in the three-dimensional POTF distribution illustrated in Figure 12), to obtain the refractive index distribution at each cross-section of the sample SA, thereby obtaining the three-dimensional refractive index distribution of the sample SA.

[0093] In the first embodiment described above, the light source unit 51 of the second microscope unit 50 has a first galvanometer mirror 54 and a second galvanometer mirror 57, but is not limited to this. For example, as in the microscope apparatus 1a shown in Figure 15, the light source unit 51a of the second microscope unit 50a may have a configuration having only one galvanometer mirror 64. In this case, the mirror 67 is placed in place of the second galvanometer mirror 57 in the light source unit 51a. Instead of the second galvanometer mirror 57, the stage 2 is moved in the z direction by a stage drive unit (not shown), thereby changing the focusing position of the second illumination light on the sample SA in the z direction. The galvanometer mirror 64 has the same configuration as the first galvanometer mirror 54, and changes the direction of propagation of the second illumination light, thereby changing the focusing position of the second illumination light on the sample SA in the y direction. The galvanometer mirror 64, the liquid lens 59, and the stage drive unit allow the focusing position of the second illumination light on the sample SA to be changed in three dimensions, thereby enabling three-dimensional scanning of the sample SA. In this case, the stage drive unit moves the stage 2 (sample SA) in the z direction, and the galvanometer mirror 64 and liquid lens 59 scan the sample SA in conjunction with the detection of light from each cross-section of the sample SA by the first microscope unit 10 (first detector 40), so that the second microscope unit 50a (second detector 80) can detect light from each cross-section of the sample SA. This makes it possible to perform each step (steps ST1 to ST3) by the first microscope unit 10 and each step (steps ST4 to ST6) by the second microscope unit 50a simultaneously.

[0094] <Second Embodiment> Next, the microscope apparatus 101 according to the second embodiment will be described with reference to Figure 16. The microscope apparatus 101 according to the second embodiment has a configuration that is common to the microscope apparatus 1 according to the first embodiment, except for the second microscope section. Therefore, the same reference numerals as in the first embodiment are used for components similar to those in the first embodiment, and detailed descriptions are omitted. The microscope apparatus 101 according to the second embodiment has a first microscope section 110 and a second microscope section 150. Furthermore, the microscope apparatus 101 according to the second embodiment has a stage 2, a control unit 90, and an image processing unit 91.

[0095] The first microscope unit 110 includes a first light source 11, a first illumination optical system 20, a first detection optical system 130, and a first detector 40. The first light source 11, the first illumination optical system 20, and the first detector 40 are configured in the same way as in the first embodiment. The first detection optical system 130 includes, in the same way as in the first embodiment, an objective lens unit 31, an imaging lens 36, and a mirror 37, in that order from the sample SA side. Furthermore, the first detection optical system 130 includes a half mirror 172 of the second microscope unit 150. The objective lens unit 31, the imaging lens 36, and the mirror 37 are configured in the same way as in the first embodiment.

[0096] The second microscope unit 150 includes a light source unit 51, a second illumination optical system 170, a second detection optical system 175, and a second detector 80. The light source unit 51 and the second detector 80 are configured in the same manner as in the first embodiment.

[0097] The second illumination optical system 170 focuses the second illumination light L2 emitted from the light source unit 51 (second light source 52) onto the sample SA in the -x direction (a second direction perpendicular to the first direction) and irradiates it. As shown in Figures 16 and 17, the second illumination optical system 170 includes a half mirror 172 and an illumination mirror 174. Furthermore, the second illumination optical system 170 includes the objective lens unit 31 (first objective lens 32) of the first microscope unit 110 (first detection optical system 130).

[0098] The half mirror 172 is positioned in the optical path between the first objective lens 32 and the imaging lens 36 in the first microscope unit 110 (first detection optical system 130). The ratio of transmittance to reflectance of the half mirror 172 is set to, for example, 1:1. The half mirror 172 reflects a portion of the second illumination light L2 emitted from the light source unit 51 (second light source 52) toward the detection pupil (rear focal plane) of the first objective lens 32. The first objective lens 32 focuses the second illumination light L2 reflected by the half mirror 172. Since the second illumination light L2 reflected by the half mirror 172 passes through the detection pupil (rear focal plane) of the first objective lens 32 from a direction inclined with respect to the central axis of the first objective lens 32, after passing through the first objective lens 32, it travels in the +z direction offset with respect to the central axis of the first objective lens 32. The illumination mirror 174 is positioned above the first objective lens 32 (and stage 2) and opposite the sample SA to the right. The illumination mirror 174 reflects the second illumination light L2, which passes through the first objective lens 32 and travels in the +z direction offset from the central axis of the first objective lens 32, toward the -x direction (second direction). The optical axis AX2 between the second illumination optical system 170 and the second detection optical system 175 in the second microscope unit 150 is perpendicular to the optical axis AX1 between the first illumination optical system 20 and the first detection optical system 130 in the first microscope unit 110.

[0099] The second detection optical system 175 receives light from the sample SA in response to the irradiation of the second illumination light from the side opposite the second illumination optical system 170, with the sample SA in between. As shown in Figures 16 and 17, the second detection optical system 175 has, in order from the sample SA side, a second detection objective lens 176 and a second modulation element 177. The second detection objective lens 176 is provided on the side opposite the illumination mirror 174, with the sample SA in between. Light from the sample SA in response to the irradiation of the second illumination light enters the second detection objective lens 176. The second modulation element 177 is positioned on a plane conjugate to the detection pupil of the second detection objective lens 176 in the second detection optical system 175 (a plane perpendicular to the optical axis AX2 of the second microscope unit 150 (second detection optical system 175) at a position P2 conjugate to the detection pupil). The second modulation element 177 is configured similarly to the second modulation element 77 in the first embodiment.

[0100] In the second embodiment, when performing bright-field observation of the sample SA, the first illumination optical system 20 of the first microscope unit 110 irradiates the sample SA with the first illumination light L1 emitted from the first light source 11 in the -z direction (first direction), similar to the first embodiment.

[0101] The first detection light transmitted through the sample SA from the first illumination optical system 20 enters the first objective lens 32 of the first detection optical system 130. The first detection light transmitted through the first objective lens 32 enters the half mirror 172. A portion of the first detection light that enters the half mirror 172 enters the imaging lens 36 after passing through the half mirror 172. The first detection light transmitted through the imaging lens 36 is reflected by the mirror 37 and forms an image on a predetermined image plane I where the first detector 40 is located. The first detector 40 detects the light (first detection light) from the sample SA via the first detection optical system 130 and outputs a detection signal for the light. The detection signal of the light (first detection light) output from the first detector 40 is transmitted to the image processing unit 91 via the control unit 90.

[0102] The second illumination light L2 emitted from the light source unit 51 (second light source 52) of the second microscope section 150 is incident on the half mirror 172 of the second illumination optical system 170. A portion of the second illumination light L2 incident on the half mirror 172 is reflected by the half mirror 172 and incident on the first objective lens 32, passing through the detection pupil of the first objective lens 32 from a direction inclined with respect to the central axis of the first objective lens 32. The second illumination light L2 that has passed through the first objective lens 32 is reflected by the illumination mirror 174, focused, and irradiated onto the sample SA on the stage 2. As a result, the second illumination optical system 170 focuses the second illumination light L2 emitted from the light source unit 51 (second light source 52) in the -x direction (a second direction perpendicular to the first direction) and irradiates the sample SA.

[0103] The second detection light, transmitted through the sample SA from the second illumination optical system 170, enters the second detection objective lens 176 of the second detection optical system 175. The second detection light, having passed through the second detection objective lens 176, enters the second detector 80 via the second modulation element 177. The second detector 80 detects the light from the sample SA (second detection light) via the second detection optical system 175 and outputs a detection signal for the light. The detection signal of the light (second detection light) output from the second detector 80 is transmitted to the image processing unit 91 via the control unit 90.

[0104] Furthermore, when performing fluorescence observation of the sample SA using the second microscope unit 150, the cylindrical lens (not shown) of the light source unit 51 of the second microscope unit 150 is inserted into the optical path between the first lens 53 and the first galvanometer mirror 54 in the light source unit 51. In addition, a fluorescence filter cube 173 is inserted in place of the half mirror 172 in the optical path between the first objective lens 32 and the imaging lens 36. The fluorescence filter cube 173 has a dichroic mirror 173a, an excitation filter 173b, and an absorption filter 173c.

[0105] Excitation light emitted from the light source unit 51 (second light source 52) passes through the excitation filter 173b of the fluorescence filter cube 173 and enters the dichroic mirror 173a. The excitation light that enters the dichroic mirror 173a of the fluorescence filter cube 173 is reflected by the dichroic mirror 173a and enters the first objective lens 32, passing through the detection pupil of the first objective lens 32 from a direction inclined with respect to the central axis of the first objective lens 32. The excitation light that has passed through the first objective lens 32 is reflected by the illumination mirror 174, focused, and irradiated onto the sample SA on the stage 2. As a result, the second illumination optical system 170 irradiates the sample SA with excitation light, which is sheet light emitted from the light source unit 51 (second light source 52).

[0106] When excitation light is irradiated, the fluorescent substance contained in the sample SA is excited and emits fluorescence. The fluorescence from the sample SA is incident on the first objective lens 32 of the first detection optical system 130. The fluorescence that has passed through the first objective lens 32 is incident on the dichroic mirror 173a of the fluorescence filter cube 173. The fluorescence that has been incident on the dichroic mirror 173a of the fluorescence filter cube 173 passes through the dichroic mirror 173a, through the absorption filter 173c, and is incident on the imaging lens 36. The fluorescence that has passed through the imaging lens 36 is reflected by the mirror 37 and is imaged on a predetermined image plane I where the first detector 40 is located. The first detector 40 detects the fluorescence from the sample SA via the first detection optical system 130 and outputs a fluorescence detection signal. The fluorescence detection signal output from the first detector 40 is transmitted to the image processing unit 91 via the control unit 90.

[0107] In the second embodiment, the three-dimensional refractive index distribution in the sample SA (for example, image data of the three-dimensional refractive index distribution of the sample SA) can be generated in the same manner as the refractive index data generation method according to the first embodiment. Therefore, the same effects as in the first embodiment can be obtained according to the second embodiment.

[0108] In the second embodiment described above, the light source unit 51 of the second microscope unit 150 has a first galvanometer mirror 54 and a second galvanometer mirror 57, but is not limited to this. For example, as in the microscope device 101a shown in Figure 18, the light source unit 51a of the second microscope unit 150a may have a configuration having only one galvanometer mirror 64. In this case, similar to the microscope device 1a according to the modification of the first embodiment, the mirror 67 is arranged in place of the second galvanometer mirror 57 in the light source unit 51a.

[0109] <Third Embodiment> Next, the microscope apparatus 201 according to the third embodiment will be described with reference to Figure 19. The microscope apparatus 201 according to the third embodiment has a configuration that is common to the microscope apparatus 1 according to the first embodiment, except for the second microscope section. Therefore, the same reference numerals as in the first embodiment are used for components similar to those in the first embodiment, and detailed descriptions are omitted. The microscope apparatus 201 according to the third embodiment has a first microscope section 210 and a second microscope section 250. Furthermore, the microscope apparatus 201 according to the third embodiment has a stage 2, a control unit 90, and an image processing unit 91.

[0110] The first microscope unit 210 includes a first light source 11, a first illumination optical system 20, a first detection optical system 230, and a first detector 40. The first light source 11, the first illumination optical system 20, and the first detector 40 are configured in the same way as in the first embodiment. The first detection optical system 230 includes, in order from the sample SA side, an objective lens unit 31, an imaging lens 36, and a polarizing beam splitter 237. Furthermore, the first detection optical system 230 includes a half mirror 272 of the second microscope unit 250. The objective lens unit 31 and the imaging lens 36 are configured in the same way as in the first embodiment. The polarizing beam splitter 237 reflects light from the sample SA (s-polarized) corresponding to the irradiation of the first illumination light and transmits light from the sample SA (p-polarized) corresponding to the irradiation of the second illumination light.

[0111] The second microscope unit 250 includes a light source unit 51, a second illumination optical system 270, a second detection optical system 275, and a second detector 80. The light source unit 51 and the second detector 80 are configured in the same manner as in the first embodiment.

[0112] The second illumination optical system 270 focuses the second illumination light L2 emitted from the light source unit 51 (second light source 52) onto the sample SA in the -x direction (a second direction perpendicular to the first direction) and irradiates it. As shown in Figures 19 and 20, the second illumination optical system 270 includes a half mirror 272 and an illumination mirror 274. Furthermore, the second illumination optical system 270 includes the objective lens unit 31 (first objective lens 32) of the first microscope unit 210 (first detection optical system 230).

[0113] The half mirror 272 is positioned in the optical path between the first objective lens 32 and the imaging lens 36 in the first microscope unit 210 (first detection optical system 230). The ratio of transmittance to reflectance of the half mirror 272 is set to, for example, 1:1. The half mirror 272 reflects a portion of the second illumination light L2 emitted from the light source unit 51 (second light source 52) toward the detection pupil (rear focal plane) of the first objective lens 32. The first objective lens 32 focuses the second illumination light L2 reflected by the half mirror 272. Since the second illumination light L2 reflected by the half mirror 272 passes through the detection pupil (rear focal plane) of the first objective lens 32 from a direction inclined with respect to the central axis of the first objective lens 32, after passing through the first objective lens 32, it travels in the +z direction offset with respect to the central axis of the first objective lens 32. The illumination mirror 274 is positioned above the first objective lens 32 (and stage 2) and opposite the sample SA to the right. The illumination mirror 274 reflects the second illumination light L2, which passes through the first objective lens 32 and travels in the +z direction offset from the central axis of the first objective lens 32, toward the -x direction (second direction). The optical axis AX2 between the second illumination optical system 270 and the second detection optical system 275 in the second microscope unit 250 is perpendicular to the optical axis AX1 between the first illumination optical system 20 and the first detection optical system 230 in the first microscope unit 210.

[0114] The second detection optical system 275 receives light from the sample SA in response to the irradiation of the second illumination light from the side opposite the second illumination optical system 270, with the sample SA in between. As shown in Figures 19 and 20, the second detection optical system 275 includes a detection mirror 276, a relay lens 278, and a second modulation element 277. Furthermore, the second detection optical system 275 includes the objective lens unit 31 (first objective lens 32) of the first microscope unit 210 (first detection optical system 230), an imaging lens 36, a polarizing beam splitter 237, and a half mirror 272 of the second illumination optical system 270.

[0115] The detection mirror 276 is positioned above the first objective lens 32 (and stage 2), on the side opposite the illumination mirror 274, with the sample SA in between. The illumination mirror 274 reflects light from the sample SA in response to the irradiation of the second illumination light toward the -z direction (i.e., the first objective lens 32) offset with respect to the central axis of the first objective lens 32. Light from the sample SA in response to the irradiation of the second illumination light enters the first objective lens 32 via the detection mirror 276. The relay lens 278 causes the light (p-polarized) from the sample SA that has passed through the polarizing beam splitter 237 to enter the second modulation element 277. The second modulation element 277 is positioned in the second detection optical system 275 on a plane conjugate to the detection pupil of the first objective lens 32 (a plane perpendicular to the optical axis of the second microscope section 250 (second detection optical system 275) at a position P2 conjugate to the detection pupil). The second modulation element 277 is configured in the same manner as the second modulation element 77 according to the first embodiment.

[0116] In the third embodiment, when performing bright-field observation of the sample SA, the first illumination optical system 20 of the first microscope unit 110 irradiates the sample SA with the first illumination light L1 emitted from the first light source 11 in the -z direction (first direction), similar to the first embodiment. In the third embodiment, a polarizing plate (not shown) is placed in the optical path of the first illumination optical system 20 (for example, the optical path between the collector lens 21 and the field diaphragm 23) so that s-polarized light (first detection light) is incident on the polarizing beam splitter 237.

[0117] The first detection light (s-polarized) transmitted through the sample SA from the first illumination optical system 20 enters the first objective lens 32 of the first detection optical system 230. The first detection light that has passed through the first objective lens 32 enters the half mirror 272. A portion of the first detection light that has entered the half mirror 272 enters the imaging lens 36 after passing through the half mirror 272. The first detection light (s-polarized) that has passed through the imaging lens 36 is reflected by the polarizing beam splitter 237 and forms an image on a predetermined image plane I where the first detector 40 is located. The first detector 40 detects the light (first detection light) from the sample SA via the first detection optical system 230 and outputs a detection signal for the light. The detection signal of the light (first detection light) output from the first detector 40 is transmitted to the image processing unit 91 via the control unit 90.

[0118] The second illumination light L2 emitted from the light source unit 51 (second light source 52) of the second microscope section 250 is incident on the half mirror 272 of the second illumination optical system 270. In the third embodiment, for example, the light source unit 51 (second light source 52) emits linearly polarized second illumination light L2 so that p-polarized light (second detection light) is incident on the polarizing beam splitter 237. A portion of the second illumination light L2 incident on the half mirror 272 is reflected by the half mirror 272 and incident on the first objective lens 32, passing through the detection pupil of the first objective lens 32 from a direction inclined with respect to the central axis of the first objective lens 32. The second illumination light L2 that has passed through the first objective lens 32 is reflected by the illumination mirror 274, focused, and irradiated onto the sample SA on the stage 2. As a result, the second illumination optical system 270 focuses the second illumination light L2 emitted from the light source unit 51 (second light source 52) in the -x direction (a second direction perpendicular to the first direction) and irradiates the sample SA with it.

[0119] The second detection light (p-polarized) transmitted through the sample SA from the second illumination optical system 270 is reflected by the detection mirror 276 of the second detection optical system 275 and incident on the first objective lens 32. The second detection light transmitted through the first objective lens 32 is incident on the half mirror 272. A portion of the second detection light incident on the half mirror 272 passes through the half mirror 272 and is incident on the imaging lens 36. The second detection light (p-polarized) transmitted through the imaging lens 36 passes through the polarizing beam splitter 237 and is incident on the relay lens 278. The second detection light transmitted through the relay lens 278 passes through the second modulation element 277 and is incident on the second detector 80. The second detector 80 detects the light from the sample SA (second detection light) via the second detection optical system 275 and outputs a detection signal for the light. The detection signal of the light (second detection light) output from the second detector 80 is transmitted to the image processing unit 91 via the control unit 90.

[0120] In the third embodiment, the three-dimensional refractive index distribution in the sample SA (for example, image data of the three-dimensional refractive index distribution of the sample SA) can be generated in the same manner as the refractive index data generation method according to the first embodiment. Therefore, the same effects as in the first embodiment can be obtained according to the third embodiment.

[0121] In the third embodiment described above, the light source unit 51 of the second microscope unit 250 has a first galvanometer mirror 54 and a second galvanometer mirror 57, but is not limited to this. For example, as in the microscope apparatus 201a shown in Figure 21, the light source unit 51a of the second microscope unit 250a may have a configuration having only one galvanometer mirror 64. In this case, similar to the microscope apparatus 1a according to the modification of the first embodiment, the mirror 67 is arranged in place of the second galvanometer mirror 57 in the light source unit 51a.

[0122] <Fourth Embodiment> Next, the microscope apparatus 301 according to the fourth embodiment will be described with reference to Figures 22 and 23. The microscope apparatus 301 according to the fourth embodiment has the same configuration as the microscope apparatus 1 according to the first embodiment, except for the first microscope unit and the second microscope unit. Therefore, the same reference numerals as in the first embodiment are used for components similar to those in the first embodiment, and detailed descriptions are omitted. As shown in Figures 22 and 23, the microscope apparatus 301 according to the fourth embodiment has a first microscope unit 310 and a second microscope unit 350. Furthermore, as shown in Figure 23, the microscope apparatus 301 according to the fourth embodiment has a stage 2, a control unit 90, and an image processing unit 91. In the fourth embodiment, as shown in Figure 23, the coordinate axis extending in the optical axis direction (vertical direction) of the second microscope unit 350 is defined as the z-axis, and the coordinate axes perpendicular to the z-axis are defined as the x-axis and y-axis.

[0123] As shown in Figures 22 and 23, the first microscope unit 310 includes a first light source 311, a first illumination optical system 320, a first detection optical system 330, and a first detector 340. The first light source 311 is configured similarly to the first light source 11 in the first embodiment. The first light source 311 generates first illumination light. The first light source 311 is positioned conjugate to the illumination pupil.

[0124] The first illumination optical system 320 irradiates the sample SA with the first illumination light L1 emitted from the first light source 311 in the +x direction (first direction). In Figure 23, the first illumination light L1 is shown with a dashed line for easier distinction from the second illumination light L2. As shown in Figure 23, the first illumination optical system 320 includes, in order from the first light source 311 side, a collector lens 321, a field diaphragm (not shown), a relay lens 324, a first modulation element 325, an aperture diaphragm (not shown), and a condenser lens 327. The collector lens 321, field diaphragm, relay lens 324, first modulation element 325, aperture diaphragm, and condenser lens 327 are arranged side by side in the x direction, but are otherwise configured similarly to the collector lens 21, field diaphragm 23, relay lens 24, first modulation element 25, aperture diaphragm 26, and condenser lens 27 according to the first embodiment. The condenser lens 327 is positioned opposite to the left side of the stage 2. Furthermore, if a white light source is used as the first light source 311, an element that narrows the wavelength band of the first illumination light (for example, a bandpass filter) may be provided, similar to the first embodiment.

[0125] The first detection optical system 330 receives light from the sample SA in response to the irradiation of the first illumination light L1 from the side opposite the first illumination optical system 320, with the sample SA in between. As shown in Figures 23 and 24, the first detection optical system 330 has, in order from the sample SA side, a first objective lens 332 and an imaging lens 336. The first objective lens 332 is located on the side opposite the condenser lens 327, with the sample SA in between. Light from the sample SA in response to the irradiation of the first illumination light is incident on the first objective lens 332. The light that has passed through the first objective lens 332 is incident on the imaging lens 336. The light that has passed through the imaging lens 336 is imaged on a predetermined image plane I (see Figure 22).

[0126] A first detector 340 is positioned on the image plane I of the first detection optical system 330. The first detector 340 is configured similarly to the first detector 40 in the first embodiment. The first detector 340 detects light from the sample SA via the first detection optical system 330.

[0127] As shown in Figures 22 and 23, the second microscope unit 350 includes a light source unit 351, a second illumination optical system 360, a second detection optical system 370, and a second detector 380. The light source unit 351 is also called a beam steering unit. As shown in Figure 23, the light source unit 351 includes a second light source 352, a first lens 353, a first galvanometer mirror 354, a second lens 355, a third lens 356, a second galvanometer mirror 357, and a fourth lens 358. The light source unit 351 also has a cylindrical lens (not shown) that can be inserted into or removed from the optical path between the first lens 353 and the first galvanometer mirror 354.

[0128] The second light source 352, first lens 353, first galvano mirror 354, second lens 355, third lens 356, and second galvano mirror 357 are configured in the same way as the second light source 52, first lens 53, first galvano mirror 54, second lens 55, third lens 56, and second galvano mirror 57 according to the first embodiment. The first galvano mirror 354 changes the direction of propagation of the second illumination light, thereby changing the focusing position of the second illumination light on the sample SA in the y direction. The second galvano mirror 357 changes the direction of propagation of the second illumination light, thereby changing the focusing position of the second illumination light on the sample SA in the x direction.

[0129] The fourth lens 358 focuses the second illumination light reflected by the second galvanometer mirror 357 at a predetermined intermediate image plane IM and directs it into the second illumination optical system 360. The position of the predetermined intermediate image plane IM is conjugate to the focal position of the second objective lens 366 on the sample SA. The first galvanometer mirror 354, the second galvanometer mirror 357, and the stage drive unit allow the focusing position of the second illumination light on the sample SA to be changed in three dimensions (x, y, and z directions), enabling three-dimensional scanning of the sample SA.

[0130] The second illumination optical system 360 focuses the second illumination light L2 emitted from the light source unit 351 (second light source 352) in the +z direction (a second direction perpendicular to the first direction) and irradiates the sample SA with it. As shown in Figures 22 and 23, the second illumination optical system 360 has, in order from the light source unit 351 side, a mirror 361, a collimator lens 362, and an objective lens unit 366. The mirror 361 reflects the second illumination light L2 emitted from the light source unit 351 (second light source 352) toward the collimator lens 362. The collimator lens 362 makes the second illumination light L2 reflected by the mirror 361 parallel.

[0131] The objective lens unit 366 includes a plurality of illumination second objective lenses 367, a lens holder 368, and a unit drive unit 369. The illumination second objective lenses 367 are positioned opposite each other below the stage 2. The lens holder 368 holds a plurality of illumination second objective lenses 367 with different optical properties. The lens holder 368 is constructed, for example, using a revolving nosepiece or a turret. The unit drive unit 369 drives the lens holder 368, making it possible to select one of the plurality of illumination second objective lenses 367 and position it below the stage 2. The unit drive unit 369 may also move the lens holder 368 along the z-axis. In this case, the aforementioned stage drive unit may be used in combination, or the stage drive may not be used.

[0132] The second illumination objective lens 367, positioned below stage 2, focuses the second illumination light L2, which has passed through the collimator lens 362, onto the sample SA. As shown in Figure 24, the optical axis AX2 between the second illumination optical system 360 and the second detection optical system 370 in the second microscope unit 350 is perpendicular to the optical axis AX1 between the first illumination optical system 320 and the first detection optical system 330 in the first microscope unit 310.

[0133] The second detection optical system 370 receives light from the sample SA in response to the irradiation of the second illumination light from the side opposite the second illumination optical system 360, with the sample SA in between. As shown in Figures 22 and 23, the second detection optical system 370 has, in order from the sample SA side, a second detection objective lens 371, an aperture diaphragm 372, a second modulation element 373, a focusing lens 374, a mirror 376, and a relay lens 377. The second detection objective lens 371 is provided on the side opposite the second illumination objective lens 367, with the sample SA in between. It is possible to select one of several second detection objective lenses 371 with different optical characteristics and place it above the stage 2. Light from the sample SA in response to the irradiation of the second illumination light is incident on the second detection objective lens 371.

[0134] The aperture diaphragm 372 and the second modulation element 373 are positioned on a plane conjugate to the detection pupil of the second objective lens 371 in the second detection optical system 370 (a plane perpendicular to the optical axis of the second microscope unit 350 (second detection optical system 370) at position P2A conjugate to the detection pupil). The second modulation element 373 is positioned adjacent to the aperture diaphragm 372 (for example, above the aperture diaphragm 372 as shown in Figure 23). The second modulation element 373 is configured similarly to the second modulation element 77 in the first embodiment. The condensing lens 374 focuses the light that has passed through the aperture diaphragm 372 and the second modulation element 373. The mirror 376 reflects the light that has passed through the condensing lens 374 toward the relay lens 377. The relay lens 377 directs the light reflected by the mirror 376 toward the second detector 380.

[0135] The second detector 380 is positioned at a position P2B conjugate to the detection pupil in the second detection optical system 370. The second detector 380 is configured similarly to the second detector 80 in the first embodiment. The second detector 380 detects light from the sample SA via the second detection optical system 370.

[0136] In the fourth embodiment, when performing bright-field observation of the sample SA, the first illumination light L1 emitted from the first light source 311 of the first microscope unit 310 is incident on the collector lens 321 of the first illumination optical system 320. The first illumination light L1 that has passed through the collector lens 321 becomes parallel light, passes through the field diaphragm (not shown), and is incident on the relay lens 324. The first illumination light L1 that has passed through the relay lens 324 passes through the first modulation element 325 and the aperture diaphragm (not shown) and is incident on the condenser lens 327. The first illumination light L1 that has passed through the condenser lens 327 becomes parallel light and irradiates the sample SA on the stage 2. As a result, the first illumination optical system 320 irradiates the sample SA with the first illumination light L1 emitted from the first light source 311 in the +x direction (first direction).

[0137] The first detection light transmitted through the sample SA from the first illumination optical system 320 enters the first objective lens 332 of the first detection optical system 330. The first detection light transmitted through the first objective lens 332 enters the imaging lens 336. The first detection light transmitted through the imaging lens 336 forms an image on a predetermined image plane I where the first detector 340 is located. The first detector 340 detects the light from the sample SA (first detection light) via the first detection optical system 330 and outputs a detection signal for the light. The detection signal of the light (first detection light) output from the first detector 340 is transmitted to the image processing unit 91 via the control unit 90.

[0138] When performing bright-field observation of the sample SA, the cylindrical lens (not shown) of the light source unit 351 of the second microscope unit 350 is retracted from the optical path between the first lens 353 and the first galvanometer mirror 354 in the light source unit 351. The second illumination light emitted from the second light source 352 of the light source unit 351 is incident on the first lens 353. The second illumination light that has passed through the first lens 353 becomes parallel light and is reflected by the first galvanometer mirror 354. The second illumination light reflected by the first galvanometer mirror 354 passes through the second lens 355 and the third lens 356 and is reflected by the second galvanometer mirror 357. The second illumination light reflected by the second galvanometer mirror 357 passes through the fourth lens 358 and is emitted outside the light source unit 351. As a result, the light source unit 351 emits the second illumination light.

[0139] The second illumination light L2 emitted from the light source unit 351 (second light source 352) is reflected by the mirror 361 of the second illumination optical system 360 and incident on the collimator lens 362. The second illumination light L2 that has passed through the collimator lens 362 becomes parallel light and incident on the second illumination objective lens 367. The second illumination light L2 that has passed through the second illumination objective lens 367 is focused and irradiates the sample SA on the stage 2. As a result, the second illumination optical system 360 focuses the second illumination light L2 emitted from the light source unit 351 (second light source 352) in the +z direction (a second direction perpendicular to the first direction) and irradiates the sample SA.

[0140] The second detection light, transmitted through the sample SA from the second illumination optical system 360, enters the second detection objective lens 371 of the second detection optical system 370. The second detection light, having passed through the second detection objective lens 371, enters the focusing lens 374 after passing through the aperture diaphragm 372 and the second modulation element 373. The second detection light, having passed through the focusing lens 374, is reflected by the mirror 376 and enters the relay lens 377. The second detection light, having passed through the relay lens 377, enters the second detector 380. The second detector 380 detects the light from the sample SA (second detection light) via the second detection optical system 370 and outputs a detection signal for the light. The detection signal of the light (second detection light) output from the second detector 380 is transmitted to the image processing unit 91 via the control unit 90.

[0141] Furthermore, when performing fluorescence observation of the sample SA using the second microscope unit 350, the cylindrical lens (not shown) of the light source unit 351 of the second microscope unit 350 is inserted into the optical path between the first lens 353 and the first galvanometer mirror 354 in the light source unit 351.

[0142] The excitation light emitted from the light source unit 351 (second light source 352) is reflected by the mirror 361 of the second illumination optical system 360 and incident on the collimator lens 362. The excitation light that has passed through the collimator lens 362 becomes parallel light and incident on the second illumination objective lens 367. The excitation light that has passed through the second illumination objective lens 367 is focused and irradiates the sample SA on the stage 2.

[0143] When excitation light is irradiated, the fluorescent substance contained in the sample SA is excited and emits fluorescence. The fluorescence from the sample SA is incident on the first objective lens 332 of the first detection optical system 330 of the first microscope unit 310. The fluorescence that has passed through the first objective lens 332 is incident on the imaging lens 336. The fluorescence that has passed through the imaging lens 336 is imaged on a predetermined image plane I where the first detector 340 is located. The first detector 340 detects the fluorescence from the sample SA via the first detection optical system 330 and outputs a detection signal for the fluorescence. The fluorescence detection signal output from the first detector 340 is transmitted to the image processing unit 91 via the control unit 90.

[0144] In the fourth embodiment, the three-dimensional refractive index distribution in the sample SA (for example, image data of the three-dimensional refractive index distribution of the sample SA) can be generated in the same manner as the refractive index data generation method according to the first embodiment. Therefore, according to the fourth embodiment, the same effects as in the first embodiment can be obtained.

[0145] <Fifth Embodiment> Next, the microscope apparatus 401 according to the fifth embodiment will be briefly described with reference to Figure 25. The microscope apparatus 401 according to the fifth embodiment has a configuration that is common to the microscope apparatus 1 according to the first embodiment, except for the second microscope section. Therefore, the same reference numerals as in the first embodiment are used for components similar to those in the first embodiment, and detailed descriptions are omitted. The microscope apparatus 401 according to the fifth embodiment has a first microscope section 410 and a second microscope section 450. Furthermore, although not shown in the figures, the microscope apparatus 401 according to the fifth embodiment has a stage, a control unit, and an image processing unit. The stage, control unit, and image processing unit are configured similarly to the stage 2, control unit 90, and image processing unit 91 according to the first embodiment.

[0146] The first microscope unit 410 includes a first light source 11, a first illumination optical system 20, a first detection optical system 30, and a first detector 40. The first light source 11, the first illumination optical system 20, the first detection optical system 30, and the first detector 40 are configured in the same way as in the first embodiment. The first modulation element 25 of the first illumination optical system 20 is positioned at the position P1A of the illumination pupil. The first detector 40 is positioned on a predetermined image plane IA.

[0147] The second microscope unit 450 includes a second light source 451, a second illumination optical system 460, a second detection optical system 470, and a second detector 480. The second light source 451 is configured in the same way as the first light source 311 in the fourth embodiment. The second illumination optical system 460 is configured in the same way as the first illumination optical system 320 in the fourth embodiment. The second modulation element 465 of the second illumination optical system 460 is configured in the same way as the first modulation element 325 in the fourth embodiment and is positioned at the position P1B of the illumination pupil. The second detection optical system 470 is configured in the same way as the first detection optical system 330 in the fourth embodiment. The second detector 480 is configured in the same way as the first detector 340 in the fourth embodiment and is positioned on a predetermined image plane IB.

[0148] In the fifth embodiment, the three-dimensional refractive index distribution in the sample SA (for example, image data of the three-dimensional refractive index distribution of the sample SA) can be generated in the same manner as the refractive index data generation method according to the first embodiment. Therefore, according to the fifth embodiment, the same effects as in the first embodiment can be obtained.

[0149] <Sixth Embodiment> Next, the microscope apparatus 501 according to the sixth embodiment will be briefly described with reference to Figure 26. The microscope apparatus 501 according to the sixth embodiment has a configuration that is common to the microscope apparatus 1 according to the first embodiment, except for the first microscope section. Therefore, the same reference numerals as in the first embodiment are used for components similar to those in the first embodiment, and detailed descriptions are omitted. The microscope apparatus 501 according to the sixth embodiment has a first microscope section 510 and a second microscope section 550. Furthermore, although not shown in the figures, the microscope apparatus 501 according to the sixth embodiment has a stage, a control unit, and an image processing unit. The stage, control unit, and image processing unit are configured in the same way as the stage 2, control unit 90, and image processing unit 91 according to the first embodiment.

[0150] The first microscope unit 510 includes a first light source unit 511 having a first light source 512, a first illumination optical system 520, a first detection optical system 530, and a first detector 540. The first light source unit 511 is configured in the same way as the second light source unit 351 according to the fourth embodiment. The first illumination optical system 520 is configured in the same way as the second illumination optical system 360 according to the fourth embodiment. The first detection optical system 530 is configured in the same way as the second detection optical system 370 according to the fourth embodiment. The first modulation element 533 of the first detection optical system 530 is configured in the same way as the second modulation element 373 according to the fourth embodiment and is positioned at a position P2A conjugate to the detection pupil. The first detector 540 is configured in the same way as the second detector 380 according to the fourth embodiment and is positioned at a position P2B conjugate to the detection pupil.

[0151] The second microscope unit 550 includes a light source unit 51, a second illumination optical system 70, a second detection optical system 75, and a second detector 80. The light source unit 51, the second illumination optical system 70, the second detection optical system 75, and the second detector 80 are configured in the same way as in the first embodiment. The second modulation element 77 of the second detection optical system 75 is positioned at a position P2C conjugate to the detection pupil. The second detector 80 is also positioned at a position P2C conjugate to the detection pupil.

[0152] In the sixth embodiment, the three-dimensional refractive index distribution in the sample SA (for example, image data of the three-dimensional refractive index distribution of the sample SA) can be generated in the same manner as the refractive index data generation method according to the first embodiment. Therefore, according to the sixth embodiment, the same effects as in the first embodiment can be obtained.

[0153] <Variation> In the first to third embodiments described above, the first detection optical systems 30, 130, and 230 of the first microscope units 10, 110, and 210 are provided separately from the first illumination optical system 20, but are not limited to this, and may include a part of the first illumination optical system. For example, as shown in Figure 27, the modified first microscope unit 610 has a first light source 11, a first illumination optical system 620, a first detection optical system 630, and a first detector 40. The first light source 11 and the first detector 40 are configured in the same way as in the first embodiment.

[0154] The first illumination optical system 620 includes, in order from the first light source 11 side, a collector lens 621, a first relay lens 622, a first modulation element 623, a second relay lens 624, a focusing lens 625, a half mirror 626, an objective lens unit 631, and an illumination mirror 628. When a white light source is used as the first light source 11, an element that narrows the wavelength band of the first illumination light (for example, a bandpass filter) may be provided, as in the first embodiment. The first modulation element 623 is positioned on a plane perpendicular to the optical axis AX1 of the first illumination optical system 620 at the illumination pupil position P1 between the first relay lens 622 and the second relay lens 624. The first modulation element 623 is configured in the same way as the first modulation element 25 in the first embodiment.

[0155] The half mirror 626 reflects a portion of the first illumination light from the first light source 11 toward the stage 2. The half mirror 626 also transmits a portion of the light (first detection light) that has passed through the sample SA on the stage 2 toward the imaging lens 636 of the detection optical system 630. The ratio of transmittance to reflectance of the half mirror 626 is set to, for example, 1:1. The objective lens unit 631 has a plurality of first objective lenses 632, a lens holder 633, and a unit drive unit 634. The first objective lenses 632 are positioned opposite each other below the stage 2. The lens holder 633 holds a plurality of first objective lenses 632 with different focal lengths. The lens holder 633 is configured, for example, using a revolving nosepiece or a turret. The unit drive unit 634 drives the lens holder 633, making it possible to select one of the plurality of first objective lenses 632 and position it below the stage 2. The illumination mirror 628 is positioned opposite each other above the stage 2.

[0156] The first detection optical system 630 includes an objective lens unit 631 and a half mirror 626. Furthermore, the first detection optical system 630 has, in order from the half mirror 626 side, an imaging lens 636 and a mirror 637. The imaging lens 636 and mirror 637 are configured in the same way as the imaging lens 36 and mirror 37 in the first embodiment.

[0157] The first illumination light emitted from the first light source 11 of the first microscope unit 610 enters the collector lens 621 of the first illumination optical system 620. The first illumination light that has passed through the collector lens 621 enters the first relay lens 622 as parallel light. The first illumination light that has passed through the first relay lens 622 enters the second relay lens 624 through the first modulation element 623. The first illumination light that has passed through the second relay lens 624 enters the half mirror 626 through the focusing lens 625. A portion of the first illumination light that has entered the half mirror 626 is reflected by the half mirror 626 and enters the first objective lens 632. The first illumination light that has passed through the first objective lens 632 enters the stage 2 and the sample SA and is reflected by the illumination mirror 628. The first illumination light reflected by the illumination mirror 628 irradiates the sample SA on the stage 2. As a result, the first illumination optical system 620 directs the first illumination light emitted from the first light source 11 towards the sample SA in the -z direction (first direction).

[0158] The first detection light, reflected by the illumination mirror 628 and transmitted through the sample SA, enters the first objective lens 632, which serves as the first detection optical system 630. The first detection light transmitted through the first objective lens 632 enters the half mirror 626. A portion of the first detection light that enters the half mirror 626 enters the imaging lens 636. The first detection light transmitted through the imaging lens 636 is reflected by the mirror 637 and forms an image on a predetermined image plane I where the first detector 40 is located. The first detector 40 detects the light from the sample SA (first detection light) via the first detection optical system 630 and outputs a detection signal for the light.

[0159] In each of the embodiments described above, the image processing unit 91 determines the three-dimensional refractive index distribution in the sample SA based on the detection signal of light detected under one detection condition with respect to light transmittance, but is not limited to this. The image processing unit 91 may determine the three-dimensional refractive index distribution in the sample SA based on the detection signals of light detected under multiple detection conditions with respect to light transmittance. For example, the image processing unit 91 calculates a linear sum or difference of POTFs based on the detection signals of light detected under two detection conditions set by the user. This makes it possible to obtain a higher absolute value of POTF over a wider frequency band than when based on the detection signal of light detected under one detection condition set by the user. Therefore, it is possible to generate an image of the three-dimensional refractive index distribution in the sample SA with high contrast and resolution. The image processing unit 91 can calculate the three-dimensional refractive index distribution n(x,y,z) in the sample SA using the above equations (10) and (11) which include POTF.

[0160] For example, in the microscope apparatus according to the first to fourth embodiments, under the first detection condition, a first modulation element (a first modulation element with unidirectional change) whose light transmittance changes according to a linear function as shown in the example in Figure 4 is placed at the position of the illumination pupil. Under the second detection condition, a first modulation element (a first modulation element with reverse direction change) whose light transmittance changes in the opposite direction to that of the example in Figure 4 (first detection condition) is placed at the position of the illumination pupil. In this modified example, the difference between the POTF based on the detection signal of light detected under the first detection condition and the POTF based on the detection signal of light detected under the second detection condition is calculated. As a result, the POTF value under the first detection condition and the POTF value under the second detection condition have opposite signs, so although the width of the frequency band in which the absolute value of POTF is not zero does not change significantly, the absolute value of POTF increases. Therefore, it is possible to generate a three-dimensional refractive index distribution image of the sample SA with excellent contrast.

[0161] In the embodiments described above, the first and second modulation elements are exemplified as elements whose light transmittance changes within the plane of a flat plate, and are formed by depositing a film capable of reducing light transmittance onto a parallel plate such as a glass substrate. However, the invention is not limited to this. For example, at least one of the first and second modulation elements may be formed by forming a minute dot pattern (light-shielding) capable of reducing light transmittance onto a parallel plate such as a glass substrate. In this case, it is possible to change the light transmittance by forming a dot pattern with different densities on a parallel plate (glass substrate) using an existing lithography process or the like (regions with a dense dot pattern have lower transmittance than regions with a sparse dot pattern). At least one of the first and second modulation elements is not limited to the optical elements described above, but may also be configured using an SLM (Spatial Light Modulator) such as a transmissive liquid crystal element, a reflective liquid crystal element, or a DMD (Digital Mirror Device). When using an SLM, the SLM is placed at the pupil (at least one of the illumination pupil and the detection pupil) or at a position conjugate to the pupil, similar to the optical elements in the embodiments described above. For example, when using a transmissive liquid crystal element as the SLM, the desired light transmittance distribution can be set by controlling the transmittance of each pixel of the element. Similarly, when using a DMD as the SLM, the desired light transmittance distribution can be set by controlling the angle of each mirror.

[0162] Furthermore, when using an optical element (i.e., a light-transmitting flat plate) as the first modulation element, the control unit 90 may change the distribution of light transmittance within the plane of the illumination pupil by controlling the element selection unit to switch to one of the multiple first modulation elements and position it at the position of the illumination pupil. When using an optical element (i.e., a light-transmitting flat plate) as the second modulation element, the control unit 90 may change the distribution of light transmittance within the plane conjugate to the detection pupil by switching to one of the multiple second modulation elements and positioning it at a position conjugate to the detection pupil. When using SLMs as the first and second modulation elements, the control unit 90 changes the distribution of light transmittance within the plane of the pupil or within the plane conjugate to the pupil by controlling the SLM. Therefore, it is not necessary to provide multiple elements or an element selection unit in order for the control unit 90 to change the distribution of light transmittance.

[0163] In each of the embodiments described above, the "lens" such as the collector lens 21 is shown as a single lens in each figure for the sake of explanation, but it is not limited to this. For example, the "lens" such as the collector lens 21 may be composed of multiple lenses, or it may be a combination of a lens and an existing optical element other than a lens. [Explanation of Symbols]

[0164] 1. Microscope apparatus (first embodiment) 2 stages 10 First Microscope Department 50 Second Microscope Department 90 Control Unit 91 Image Processing Unit (Data Processing Unit) 101 Microscope apparatus (second embodiment) 110 First Microscope Department 150 Second Microscope Department 201 Microscope apparatus (third embodiment) 210 First Microscope Section 250 Second Microscope Section 301 Microscope apparatus (fourth embodiment) 310 First Microscope Department 350 Second Microscope Department 401 Microscope apparatus (Fifth embodiment) 410 First Microscope Department 450 Second Microscope Department 501 Microscope apparatus (6th embodiment) 510 First Microscope Department 550 Second Microscope Department

Claims

1. An illumination optical system including a scanning unit, which scans and illuminates a specimen with illumination light from a light source via the scanning unit, A detection optical system that guides the light transmitted through the aforementioned sample to a detection unit, An image processing unit that generates an image based on the signal from the detection unit, It comprises a control unit and, The detection optical system has an intensity modulation element positioned at the pupil or its conjugate position, wherein the transmittance of the light changes within the plane of the pupil or the plane of the conjugate position. The control unit controls the intensity modulation element or switches the intensity modulation element with another intensity modulation element to change the state of the light transmittance, thereby causing the light that the detection optical system guides to the detection unit to be in different states. The detection unit receives the light in each of the different states, The image processing unit generates a refractive index distribution based on multiple images formed by multiple signals output from the detection unit, and is a microscope.

2. The microscope according to claim 1, wherein the transmittance of the intensity modulation element changes according to a continuous function.

3. The microscope according to claim 2, wherein the transmittance of light of the intensity modulation element changes along one direction in the plane of the pupil or the plane of the pre-conjugate position according to the continuous function.

4. The microscope according to claim 2, wherein the transmittance of light of the intensity modulation element increases or decreases monotonically along one direction in the plane of the pupil or the plane of the conjugate position according to the continuous function, and becomes zero in a part of the outer circumference of the plane of the pupil or the plane of the conjugate position.

5. The microscope according to claim 2, wherein the continuous function is any of the following functions: a linear function, a quadratic function, a Gaussian function, a sine function in a range of less than one period, or a cosine function in a range of less than one period.

6. The microscope according to claim 1, wherein the change in the state of light transmittance is a change in the transmittance from a state in which it is monotonically increasing according to a linear function in one direction to a state in which it is monotonically decreasing according to a linear function in one direction.

7. The microscope according to claim 1, wherein the image processing unit generates a difference image based on the plurality of images and generates the refractive index distribution based on the difference image.

8. The microscope according to claim 1, wherein the pre-illumination optical system and the detection optical system are arranged in a positional relationship opposite to each other via a stage on which the specimen is placed.

9. The microscope according to claim 1, wherein the illumination optical system and the detection optical system are arranged in a positional relationship opposite to each other with respect to the specimen, without passing through a stage on which the specimen is placed.

10. The microscope according to any one of claims 1 to 9, wherein the numerical aperture of the illumination optical system and the numerical aperture of the detection optical system are the same.

Images