Microscope, observation method, and program

Abstract

A microscope comprising: a light-transmitting optical system which illuminates a sample with illumination light; a light-receiving optical system which receives signal light emitted from the sample; a phase modulation element which adds a predetermined phase distribution to the illumination light or the signal light; a phase distribution measurement unit which measures, at each of a plurality of sample points, a first phase distribution corresponding to an aberration caused by the sample at the sample point in the sample; a phase distribution calculation unit which creates, on the basis of the plurality of first phase distributions, a phase data model indicating a phase change amount exhibited when the illumination light or the signal light passes through a predetermined position in the sample and which calculates, on the basis of the phase data model, a second phase distribution to be added to the illumination light or the signal light in order to detect a detection point in the sample in a state in which the aberration caused by the sample is reduced; and a phase distribution setting unit which sets the second phase distribution in the phase modulation element.

Description

Technical Field

[0001] The present invention relates to a microscope, an observation method, and a program. This application claims priority based on Japanese Patent Application No. 2021-051951 filed on March 25, 2021, the content of which is incorporated herein by reference.

Background Art

[0002] When observing the inside of a biological sample with an optical microscope, aberrations occur due to distortions in the surface shape of the sample and three-dimensional non-uniformities in the refractive index of the sample, resulting in a decrease in imaging performance. A method has been proposed to correct such aberrations caused by the observation sample using an aberration correction element (phase modulation element) such as a deformable mirror to obtain a high-quality microscope image (Patent Document 1). In addition, a method has also been proposed to calculate the spherical aberration correction amounts at different z positions by interpolation or function approximation based on the spherical aberration correction amounts calculated at a plurality of z positions (Patent Document 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

[0004] According to a first aspect, a microscope includes a light transmission optical system that irradiates a sample with illumination light from a light source, a light reception optical system that receives signal light emitted from the sample, a phase modulation element provided in at least one of the light transmission optical system or the light reception optical system that adds a predetermined phase distribution to the illumination light or the signal light, a phase distribution measurement unit that measures, at each of a plurality of sample points of the sample, a first phase distribution corresponding to an aberration caused by the sample at the sample point of the sample, and the measured The foregoingBased on the first phase distribution for each of the plurality of sample points, create a phase data model indicating the amount of phase change that the illumination light or the signal light undergoes when passing through a predetermined position within the sample. Based on the phase data model, calculate a second phase distribution that should be added to at least one of the illumination light or the signal light in order to detect at least one detection point of the sample in a state where the aberration generated by the sample is reduced. A phase distribution calculation unit that calculates the second phase distribution, and a phase distribution setting unit that sets the second phase distribution in the phase modulation element. , for each of the plurality of sample points, the first phase distribution is measured by the phase distribution measurement unit by sequentially applying a plurality of phase modulation patterns to the phase modulation element, and each time a phase modulation pattern is applied, the illumination light is irradiated by the light transmission optical system at least to the sample point that is the measurement target of the first phase distribution, and is measured based on the signal light received by the light reception optical system and. According to a second aspect, an observation method is an observation method of irradiating a sample with illumination light from a light source, detecting signal light emitted from the sample, and observing the sample. Measuring a first phase distribution corresponding to the aberration generated by the sample at a plurality of sample points of the sample, and the measured The foregoing Based on the first phase distribution for each of the plurality of sample points, create a phase data model indicating the amount of phase change that the illumination light or the signal light undergoes when passing through a predetermined position within the sample. Based on the phase data model, calculate a second phase distribution that should be added to at least one of the illumination light or the signal light in order to detect at least one detection point of the sample in a state where the aberration generated by the sample is reduced. Adding the second phase distribution to at least one of the illumination light or the signal light, and detecting the detection point. , for each of the plurality of sample points, a plurality of phase modulation patterns are sequentially applied to a phase modulation element that adds a predetermined phase distribution to the illumination light or the signal light, and each time a phase modulation pattern is applied, the illumination light is irradiated to at least the sample point that is the measurement target of the first phase distribution, and is measured based on the detected signal light and. According to a third aspect, a program is a program for controlling at least a part of a microscope system for observing a sample. Cause a processing device including a computer to execute reading of position information of a plurality of sample points of the sample and information regarding a first phase distribution corresponding to an aberration signal light generated in the illumination light or the signal light by the sample at each of the sample points. Cause the processing device to The foregoing the position information of the plurality of sample points, and The foregoingBased on the information regarding the first phase distribution of the plurality of sample points, create a phase data model indicating the amount of phase change received when the illumination light or the signal light passes through a predetermined position within the sample, and based on the phase data model, calculate a second phase distribution to be added to at least one of the illumination light or the signal light in order to detect at least one detection point of the sample in a state where the aberration generated by the sample is reduced. Program That is, for each of the plurality of sample points, a plurality of phase modulation patterns are sequentially applied to a phase modulation element that adds a predetermined phase distribution to the illumination light or the signal light, and each time a phase modulation pattern is applied, the illumination light is irradiated to at least the sample point that is the measurement target of the first phase distribution, and is measured based on the detected signal light 。

Brief Description of Drawings

[0005]

Figure 1

Figure 2A

Figure 2B

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7A

Figure 7B

Figure 8

Figure 9A

Figure 9B

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

[0006] (Microscope of the First Embodiment) FIG. 1 is a diagram schematically showing the configuration of the microscope 1a of the first embodiment. The X direction, Y direction, and Z direction indicated by arrows in FIG. 1 and each figure referred to below are orthogonal directions, and each of the X direction, Y direction, and Z direction indicates the same direction in each figure. Hereinafter, the directions indicated by the respective arrows are referred to as the +X direction, +Y direction, and +Z direction. The +Z direction is a downward direction parallel to the optical axis AX of the objective lens 21. Also, the position in the X direction is called the X position, the position in the Y direction is called the Y position, and the position in the Z direction is called the Z position.

[0007] The microscope 1a includes a light transmission optical system ILO (the region surrounded by the broken line in FIG. 1), a light reception optical system DTO (the region surrounded by the two-dot chain line in FIG. 1), a control unit 40, a stage 25 for placing the sample 24, and the like. The light transmission optical system ILO includes a collimator lens 12, a phase modulation element 14, relay lenses 15, 16, 18, a dichroic mirror 20, a swing mirror 17, a second objective lens 19, an objective lens 21, and the like, which are arranged along the optical path of the illumination light IL.

[0008] The light reception optical system DTO includes an objective lens 21, a dichroic mirror 20, relay lenses 27a, 27b, a detection filter 28, a detector 29a, and the like. Among these, the objective lens 21 and the dichroic mirror 20 are included in both the light transmission optical system ILO and the light reception optical system DTO.

[0009] The light source 10 emits illumination light IL for illuminating the sample 24. For example, a laser light source or the like is used as the light source 10. In the light transmission optical system ILO, the shutter 11 controls the passage or blocking of the illumination light IL emitted from the light source 10, and the collimator lens 12 converts the illumination light emitted from the light source 10 into substantially parallel light. The light transmission optical system ILO irradiates the sample 24 with the illumination light IL from the light source 10.

[0010] The illumination light IL that has passed through the collimator lens 12 enters the phase modulation element 14. The phase modulation element 14 is, for example, a deformable mirror whose shape in the direction perpendicular to the reflection surface 14s is variable, and modulates the phase distribution (the advance or delay of the light phase) within the cross section of the light beam reflected by the phase modulation element 14.

[0011] The phase modulation element 14 may be a reflective liquid crystal SLM (spatial light modulator), or may be a so-called MEMS-SLM in which a plurality of micro mirrors are arranged in the reflection surface 14s and the positions of the micro mirrors are made variable by MEMS. The reflection surface 14s of the phase modulation element 14 is arranged so as to generally coincide with the pupil plane IPP of the light transmission optical system ILO.

[0012] Note that the phase modulation element 14 is not limited to a reflective element. For example, it may be an element that modulates the phase distribution within the cross-section of the transmitted light beam, such as a transmissive liquid crystal SLM. In this case, the transmissive phase modulation member of the phase modulation element 14 may be arranged to generally coincide with the pupil plane IPP of the light transmission optical system ILO.

[0013] When the phase modulation element 14 has reflective surfaces 14s (or transmissive regions) partitioned into minute regions such as minute mirrors, like the above-described MEMS-SLM, reflective liquid crystal SLM, or transmissive liquid crystal SLM, each of these minute regions is also referred to as a "unit element" of the phase modulation element 14. Hereinafter, the phase difference imparted to the illumination light IL by each region (e.g., the above-described unit element) of the phase modulation element 14 is also referred to as the "phase value".

[0014] The illumination light IL reflected by the phase modulation element 14 passes through two relay lenses 15 and 16 and is reflected by the oscillating mirror 17. The oscillating mirror 17 is a reflective member held such that the azimuth angle of the reflective surface, such as a galvanometer mirror, varies. Due to the change in the azimuth angle of the reflective surface of the oscillating mirror 17, the traveling direction of the illumination light IL reflected by the oscillating mirror 17 is deflected. As the oscillating mirror 17, a resonant mirror may be used, or instead of the oscillating mirror 17, a transmissive deflector such as an acousto-optic deflector (AOD) may be used to deflect the illumination light IL.

[0015] The illumination light IL reflected by the oscillating mirror 17 passes through the relay lens 18 and the second objective lens 19, and then passes through the dichroic mirror 20. The dichroic mirror 20 transmits the illumination light IL emitted from the light source 10 and reflects light in a predetermined wavelength range such as fluorescence among the signal light DL generated by the sample 24.

[0016] The objective lens 21 is disposed near the sample 24 and faces the sample 24. When observing the sample 24, the space between the objective lens 21 and the sample 24 may be filled with an immersion liquid 23, or may be filled with a gas such as air. A cover glass (not shown) may be disposed between the sample 24 and the immersion liquid 23. The illumination light IL emitted from the light source 10 is condensed by the objective lens 21 and illuminates a specific position inside the sample 24.

[0017] The objective lens 21 is attached to the housing of the microscope 1a via an objective lens holder 22. The objective lens holder 22 includes a drive unit such as an electric motor, for example, and moves the objective lens 21 vertically in the z direction. When the objective lens 21 is moved in the z direction by the objective lens holder 22, the relative position of the objective lens 21 with respect to the sample 24 changes, and the focal position of the objective lens 21 with respect to the sample 24 changes in the z direction. By displacing the focal position of the objective lens 21 in the z direction by the objective lens holder 22, images of cross-sections at different positions in the z direction inside the sample 24 can be acquired. Hereinafter, images of a plurality of cross-sections of the sample 24 at different positions in the z direction are also referred to as "z-stack images" IZ.

[0018] FIG. 4 is a diagram showing an example of the z-stack image IZ of the sample 24. Each of the two-dimensional images I1, I2, and I3 is a two-dimensional image of an xy cross-section at a different z position of the sample 24. The z-stack image IZ includes the two-dimensional images I1, I2, and I3 at different z positions of the sample 24. Note that the number of the two-dimensional images I1 to I3 is not limited to the three shown in FIG. 4 and may be any number. Signs such as R1, F1, Px, and Rx shown in FIG. 4 will be described later.

[0019] The stage 25 supports the sample 24 to be observed directly or via a sample container (not shown) that holds the sample 24. A stage drive unit 26 is provided on the stage 25. The stage drive unit 26 includes an electric motor, a piezo element, etc., and moves the stage 25 within a plane perpendicular to the z-axis (within the xy plane). By moving the stage 25 within a plane perpendicular to the z-axis by the stage drive unit 26, a wide-range image of the sample 24 can be acquired. Also, the stage drive unit 26 may move the stage 25 in the z-axis direction as well.

[0020] As an example of the sample 24, for example, there is a biological sample. The biological sample is a sample with thickness such as cells or biological tissues. As another example of the sample 24, for example, there are beads which are minute spheres made of polystyrene with a diameter of about 0.2 μm. The sample 24 may be stained with a fluorescent dye. Also, fluorescently stained beads or metal particles may be introduced into the sample 24.

[0021] From the portion of the sample 24 where the illumination light IL is focused, the signal light DL is generated. The signal light DL is not limited to fluorescence and may be, for example, scattered light or reflected light. Also, the signal light DL may be a signal that shows a non-linear response with respect to the intensity of the illumination light IL, and may be, for example, fluorescence emitted as a result of multi-photon excitation of the sample 24, second harmonic wave, third harmonic wave, etc.

[0022] The signal light DL emitted from the sample 24 enters the objective lens 21 and then enters the dichroic mirror 20. Since the signal light DL has a different wavelength from the illumination light IL, it is reflected by the dichroic mirror 20, passes through two relay lenses 27a, 27b, and further passes through the detection filter 28. The detection filter 28 transmits light of a predetermined wavelength band (for example, fluorescence) among the light emitted from the sample 24. The detection filter 28 blocks at least a part of the light such as the illumination light IL reflected by the sample 24, external light, and stray light.

[0023] The signal light DL that has passed through the detection filter 28 is detected by the detector 29a, converted into an electrical signal, and sent to the control unit 40 as the signal S1. The detector 29a includes, for example, a photomultiplier tube, a photodiode, an avalanche photodiode, and the like. The collimator lens 12, each relay lens 15, 16, 18, 27a, 27b, and the second objective lens 19, and the objective lens 21 may all be composed of a plurality of lenses and may include a reflecting mirror.

[0024] The microscope 1a is not limited to the upright microscope shown in FIG. 1 and may be an inverted microscope. In the case of an inverted microscope, the sample 24 may be supported by a stage 25 having an opening in the center and may face the objective lens 21 through the opening of the stage 25.

[0025] The control unit 40 includes an arithmetic unit 41, a storage unit 42, an optical drive 43, an input unit 44, a display unit 45, a phase distribution measurement unit 46, a phase distribution calculation unit 47, an image generation unit 48, a phase distribution setting unit 49, and an interface unit IF.

[0026] The arithmetic unit 41 includes a CPU and controls the microscope 1a including the control unit 40 based on a program stored in the storage unit 42. The storage unit 42 includes a storage medium such as a memory element or a hard disk and temporarily stores data such as signals detected by the detector 29a in addition to the above-described program. The interface unit IF communicates data with a server or the like disposed outside the microscope 1a via the network line NW.

[0027] The control unit 40 sends the signal S3 to the oscillating mirror 17 to control the azimuth angle of the reflecting surface of the oscillating mirror 17, sends the signal S5 to the stage drive unit 26 to control the position of the stage drive unit 26, and sends the signal S5 to the objective lens holding unit 22 to control the position of the objective lens 21. The control unit 40 further sends the signal S6 to the shutter 11 to control the opening and closing of the shutter 11.

[0028] The image generation unit 48 included in the control unit 40 generates an image of the sample 24 based on the signal S1 sent from the detector 29a. The phase distribution setting unit 49 sends the signal S2 to the phase modulation element 14 and controls the modulation of the phase of the illumination light IL by the phase modulation element 14.

[0029] The input unit 44 is an input interface operable by the user and includes at least one of, for example, a mouse, a keyboard, a touch pad, a trackball, etc. The input unit 44 detects an operation by the user and outputs the detection result to the arithmetic unit 41 as input data input by the user. The display unit 45 is, for example, a liquid crystal display or the like. The arithmetic unit 41 causes the display unit 45 to display a GUI (Graphical User Interface) necessary for operating the microscope 1a and an image of the sample 24 generated by the image generation unit 48.

[0030] The phase distribution measurement unit 46 measures the phase distribution corresponding to the aberration caused by the sample 24 based on the signal S1 sent from the detector 29a. Although details will be described later, in this specification, the phase distribution corresponding to the aberration caused by the sample 24 in the illumination light IL or the signal light DL is also referred to as the "first phase distribution".

[0031] Based on the first phase distribution measured by the phase distribution measurement unit 46, the phase distribution calculation unit 47 calculates the phase distribution that the phase modulation element 14 should add to the illumination light IL in order to detect the sample 24 in a state where the aberration caused by the sample 24 itself is reduced. In this specification, the phase distribution that the phase modulation element 14 should add to the illumination light IL in order to detect the sample 24 in a state where the aberration caused by the sample 24 itself is reduced is also referred to as the "second phase distribution".

[0032] As shown in various forms described later, the phase distribution that the phase modulation element 14 should add to the signal light DL in order to detect the sample 24 in a state where the aberration caused by the sample 24 itself is reduced is also referred to as the "second phase distribution". Details of the phase distribution measurement unit 46 and the phase distribution calculation unit 47 will be described later.

[0033] Here, the aberration caused by the sample 24 described above will be described with reference to FIGS. 2A and 2B. FIGS. 2A and 2B are diagrams for explaining the aberration caused by the sample 24, and are enlarged xz cross-sectional views showing the sample 24 and the objective lens 21. The inside of the broken-line circle C1 shown in FIG. 2A is an enlarged view of the vicinity of the focal point FP, and the same applies to the inside of the broken-line circle C2 shown in FIG. 2B.

[0034] When the sample 24 is, for example, a biological sample, the refractive index may vary depending on each part within the sample 24. Therefore, as shown in FIG. 2A, the illumination light IL that has traveled through the sample 24 and reached the focal point FP generates a zero-order phase distribution WF0, which is an aberration caused by the non-uniformity of the refractive index of the sample 24. This makes it difficult to focus the illumination light IL to the focal point FP with a size on the order of the theoretical resolution limit.

[0035] In the microscope 1a of the first embodiment, as shown in FIG. 2B, a first phase distribution WF1 for canceling the zero-order phase distribution WF0 caused by the sample 24 and reducing the aberration is measured. Ideally, the first phase distribution WF1 is a distribution with the sign (+-) inverted with respect to the zero-order phase distribution WF0. Since the first phase distribution WF1 is an aberration that cancels the zero-order phase distribution WF0 caused by the sample 24, it can be said that it corresponds to the aberration caused by the sample 24.

[0036] Then, when observing the sample 24, this first phase distribution WF1 or a second phase distribution WF2 calculated based on the first phase distribution WF1 is added to the illumination light IL by the phase modulation element 14. As a result, the illumination light IL can be focused on the focal point FP in a state (phase distribution WF3) where the aberration caused by the sample 24 is reduced, and the resolution of the microscope 1a can be improved.

[0037] However, when measuring the first phase distribution WF1 at each of the detection points U (U1, U2, U3, U4, ···, Uo, etc., where the subscript o is a symbol indicating the number of the detection point U) shown by white circles in FIG. 4 of the sample 24, it takes a long time to observe the sample 24. In addition, there is also a risk that the illumination light IL irradiated on the sample 24 during the measurement of the first phase distribution WF1 may cause phototoxicity to the sample 24.

[0038] Therefore, in the observation of the sample 24 using the microscope 1a of the first embodiment, as shown in FIG. 4, for example, a plurality of predetermined sample points F (F1, F2, ···, Fj, etc.) shown by black circles are selected from the inside of the sample 24 based on the z-stack image IZ of the sample 24. Here, j is a subscript representing the number of the sample point F. Then, the first phase distribution WF1 corresponding to the aberration generated by the sample 24 is measured at each of these sample points F. Note that at least one of the plurality of sample points F may be selected from the detection points U. That is, some of the plurality of sample points F may be both a sample point F and a detection point U.

[0039] Then, based on the first phase distribution WF1 measured at each sample point F, a second phase distribution WF2 to be added to the illumination light IL is calculated in order to detect an arbitrary detection point Uo in the sample 24 in a state where the aberration generated by the sample 24 is reduced. When detecting the detection point Uo, the calculated second phase distribution WF2 suitable for the detection point Uo is added to the illumination light IL to illuminate the detection point Uo.

[0040] Thereby, the observation time of the sample 24 can be shortened, the phototoxicity to the sample 24 can be reduced, and high-resolution observation can be performed. The measurement of the first phase distribution WF1 is performed by the phase distribution measurement unit 46 based on the signal S1 sent from the detector 29a. The calculation of the second phase distribution WF2 is performed by the phase distribution calculation unit 47 based on the measured first phase distribution WF1. The phase distribution setting unit 49 sends the signal S2 to the phase modulation element 14 and controls the phase modulation element 14 to modulate the phase of the illumination light IL according to the calculated second phase distribution WF2.

[0041] Note that FIG. 4 shows the z-stack image IZ of the sample 24 and does not show the sample 24 itself. However, since the z-stack image IZ corresponds one-to-one with the sample 24 itself, in this specification, by designating each point (F1, F2, ···, Fj, U1, U2, ···, Uo, etc.) shown in the z-stack image IZ of FIG. 4, each corresponding point in the sample 24 is indicated.

[0042] As the sample 24, for example, pre-fluorescently stained cells or the like are used, but it is not necessarily limited to substances that emit fluorescence. Also, when using a substance that emits fluorescence as the sample 24, as the wavelength of the light source 10, a wavelength that excites the fluorescent substance contained in the sample 24 may be selected. As the wavelength of the light source 10, a wavelength that multi-photon excites the substance contained in the sample 24 may be selected. Note that the light source 10 may be provided so as to be replaceable (attachable and detachable) to the microscope 1a, or may be externally attached to the microscope 1a during observation with the microscope 1a.

[0043] Note that by the oscillation of the oscillating mirror 17 or the movement of the stage 25 by the stage driving unit 26, the position of the condensing point FP is scanned relatively to the sample 24. Therefore, at least one of the oscillating mirror 17 and the stage 25 can also be called a "scanning unit".

[0044] (Observation method of the first embodiment) Hereinafter, the flow of the observation method using the microscope 1a of the first embodiment described above, the method of measuring the first phase distribution WF1 described above, and the method of calculating the second phase distribution WF2 described above will be described. Note that the following description is both an explanation of the observation method and an explanation of the microscope 1a of the first embodiment.

[0045] FIG. 3 shows a flowchart when the arithmetic unit 41 of the control unit 40 executes a program stored in the storage unit 42 to control the control unit 40 and the microscope 1a to execute the observation method of the first embodiment. However, in the description of each of the following steps, the description of the program and the arithmetic unit 41 which is the execution entity of the program will be omitted as appropriate.

[0046] In step S100, the arithmetic unit 41 of the control unit 40 causes the display unit 45 to display a screen such as a GUI that prompts the user to set the range Rx in the x direction, the range Ry in the y direction, and the range Rz in the z direction (all refer to FIG. 4) of the imaging range IR which is the range for the user to capture an image. When the user inputs a provisional imaging range to the input unit 44, the control unit 40 sends a signal S6 to the shutter 11 to open the shutter 11, and according to the input provisional imaging range, sends a signal S4 to the stage drive unit 26 to move the stage 25 to a predetermined position.

[0047] Then, the control unit 40 swings the swing mirror 17 to scan the condensing point FP of the illumination light IL (refer to FIG. 6(b)) in the x direction and the y direction within the sample 24, and holds the signal S1 detected by the detector 29a in the storage unit 42 as the scan progresses. The arithmetic unit 41 performs a process of aligning the detection signals held in the storage unit 42 in synchronization with the two-dimensional scan by the swing mirror, and generates, as an image of the sample 24, for example, the z-stack image IZ shown in FIG. 4 and displays it on the display unit 45.

[0048] While viewing the image of the sample 24 displayed on the display unit 45, the user uses the GUI to input an instruction to move the x position, the y position of the stage 25, and the position of the objective lens 21 or the stage 25 to the input unit 44, and sets the desired imaging range IR.

[0049] For example, the ranges in the x direction and the y direction of the imaging range IR may be set by the number of pixels of the image, and the ranges Rx in the x direction and Ry in the y direction may be specified by designating the scan pitch (Px and Py in FIG. 4) which is the pitch of the pixels of the image. Also, for the z direction, for example, the number of acquired images I1 to I3 of a plurality of xy cross-sections having different positions in the z direction that constitute the z-stack image IZ to be acquired, and the distance Pz in the z direction between the images I1 to I3 of the plurality of cross-sections are input, whereby the range Rz in the z direction can be specified. Note that the method for specifying the imaging range IR is not limited to the above. For example, it may be specified by the lengths in the x, y, and z directions of the rectangular parallelepiped region for acquiring an image.

[0050] In step S110, NF sample points F for measuring the first phase distribution WF1 (see FIG. 2A) corresponding to the aberration generated by the sample 24 are determined from among the samples 24. For example, the arithmetic unit 41 may display the entire image of the imaging range IR of the sample 24 on the display unit 45, and the user may specify the sample points F with respect to the input unit 44.

[0051] Specifically, when a start command for determining the sample points F from the user is input to the input unit 44, the control unit 40 swings the swing mirror 17 according to the numerical value input as the imaging range IR at the predetermined position of the stage 25 and the predetermined position of the objective lens 21. Then, an image of the sample 24 within the imaging range IR is acquired and displayed on the display unit 45. The image displayed on the display unit 45 may be, as an example, the z-stack image IZ shown in FIG. 4.

[0052] While viewing the image displayed on the display unit 45, the user specifies the sample points F using the GUI. For example, the user may move the mouse cursor to a location on the image displayed on the display unit 45 that is to be set as the sample points F (F1, F2, Fj, etc.) indicated by, for example, black circles, and click (input) the mouse to specify.

[0053] The control unit 40 causes the storage unit 42 to store the coordinates of the sample points F input to the input unit by clicking, that is, the x position, y position, and z position. By repeating the above steps, the user specifies a plurality of sample points F and their coordinates, and the storage unit 42 stores those coordinates. Let the total number of sample points F specified by the user be NF. The x position, y position, and z position of the sample point Fj held in the storage unit 42, that is, the xyz coordinates, with respect to an arbitrary reference point within the sample 24 are denoted as (xFj, yFj, zFj). Do.

[0054] In addition to being specified by the user, the NF sample points F may also be automatically determined. For example, when the user makes a predetermined input to the input unit 44, the control unit 40 may acquire a z-stack image IZ according to the imaging range IR input to the input unit 44 in step S100. Then, in the acquired z-stack image IZ, the arithmetic device 41 may select, for example, the top NF points in descending order of luminance value as the sample points F, and store the coordinates of these sample points Fj in the storage unit 42.

[0055] Note that, in order to shorten the operation time of the microscope 1a and reduce the phototoxicity to the sample 24, the range in the z-direction of the z-stack image acquired at this time may be made narrower than the range Rz in the z-direction of the imaging range IR, or the scan pitch (Px, Py) may be made larger than the set value.

[0056] In step S120, a loop for each sample point F for measuring the first phase distribution WF1 is started. Then, regarding the processing up to step S160 described later, a loop of FN times from j = 1 to j = NF is repeated for the subscript j of the sample point Fj.

[0057] In step S130, the control unit 40 operates the microscope 1a so that the illumination light IL is focused on the sample point Fj. Specifically, the stage 25 may be moved using the stage drive unit 26 to move the sample point Fj to the focus point FP of the illumination light IL (see FIGS. 2A and 2B), or the azimuth angle of the reflecting surface of the swing mirror 17 may be changed so that light is focused on the sample point Fj. Further, by moving the z position of the stage 25 or the objective lens 21, the focus point FP is moved to be located at the z coordinate zFj of the sample point Fj.

[0058] In step S140, the phase distribution measurement unit 46 measures a first phase distribution WF1 (see FIG. 2A) corresponding to the aberration generated by the sample 24 at the sample point Fj based on the signal S1 sent from the detector 29a. Details of the measurement of the first phase distribution WF1 will be described later. In step S160, the loop for each sample point F ends.

[0059] In step S170, as shown in FIG. 8, the imaging range IR, which is the rectangular parallelepiped region set in step S100, is divided into a plurality of parts, and small regions R (R1, R2, ···, Rj’, ···, RNR) are set. Here, j’ is a subscript representing the number of the small region R. Also, the maximum value of j’, that is, the total number of the small regions R is NR. One or more detection points U are included in each small region Rj’. Let the coordinates of the three-dimensional center position of each small region Rj’ be (xRj’, yRj’, zRj’).

[0060] The size of each small region R may be set so that it becomes a region where the aberration of the illumination light IL is sufficiently small by adding the second phase distribution WF2 calculated in the small region R (for example, at the center position of the small region R). Since the size of the small region R varies according to the sample 24, values may be set in advance for each sample 24.

[0061] In step S180, the phase distribution calculation unit 47 calculates the second phase distribution WF2 for the NR small regions R based on each first phase distribution WF1 measured at the NF sample points. The specific calculation method of the second phase distribution WF2 will be described later.

[0062] In step S190, the phase distribution setting unit 49 sends the signal S2 to the phase modulation element 14, and acquires an image of the sample 24 in the imaging range IR with the calculated second phase distribution WF2 set for the illumination light IL. Details of the image acquisition will be described later. Through the above steps, the acquisition of the image of the sample 24 is completed.

[0063] (Measurement of the First Phase Distribution) Hereinafter, with reference to FIGS. 5, 7A, and 7B, the measurement of the first phase distribution WF1 in step S140 described above will be explained. FIG. 5 is a diagram showing the flow of measurement of the first phase distribution WF1 in step S140. FIG. 7A is a diagram showing the reflecting surface 14s of the phase modulation element 14. FIG. 7A also shows the illumination light IL (inside the broken-line circle) irradiated on the reflecting surface 14s.

[0064] In S141, as shown in FIG. 7A, the reflecting surface 14s of the phase modulation element 14 is virtually divided into a plurality of segments SGa. Note that even when the phase modulation element 14 has unit elements such as a plurality of micro mirrors separated from each other, one segment SGa does not necessarily correspond to one unit element, and one segment SGa may include a plurality of unit elements.

[0065] The center point of each segment SGa is hereinafter referred to as a phase sampling point P (P1, P2, ···, Pi, ···, PNP). Here, i is a subscript representing the number of the phase sampling point P. Also, the maximum value of i, that is, the total number of the phase sampling points P is NP.

[0066] As described above, the reflecting surface 14s is arranged substantially coincident with the pupil plane IPP of the light transmitting optical system ILO. Therefore, let the x, y coordinates of the phase sampling point Pi on the pupil plane IPP be (xPi, yPi). Note that the shape of each segment SGa is not limited to a square, and may be any shape. Also, the arrangement position of the phase sampling point Pi is not limited to the points arranged in a lattice pattern in the xy plane, and may be any arrangement position.

[0067] In step S142, a loop of the process for measuring the first phase distribution WF1 is started. Then, for the process up to step S149 described later, a loop of NI times from itr = 1 to itr = NI is repeated with respect to the subscript itr of the number of repetitions.

[0068] In step S143, the NP segments SGa are randomly classified into M groups (M is a natural number of 2 or more). In step S144, loops for each classified group m (m = 1, 2, 3, ···, M) are started.

[0069] In step S145, for each segment SGa belonging to group m which is the m-th group, the phase value added to the illumination light IL is changed to a plurality of different predetermined values over a plurality of times. On the other hand, for segments SGa belonging to groups other than group m, the phase value added to the illumination light IL is not changed. The phase values added by each segment SGa belonging to the above group m to the illumination light IL over a plurality of times may be set such that, for example, the phase value changes at different frequencies over time for each segment SGa.

[0070] For each of the plurality of set phase values, the control unit 40 opens the shutter 11 to irradiate the sample 24 with the illumination light and detects the signal emitted from the sample point Fj. At this time, the control unit 40 may scan the oscillating mirror 17 and acquire an image of the vicinity region of the sample point Fj based on the signal S1 from the detector 29a. Assuming the number of segments SGa belonging to group m is Ns, the phase modulation pattern may be changed to 2×Ns or more patterns, and the signal from the sample point Fj may be acquired 2×Ns or more times, or an image of the vicinity region of the sample point Fj may be acquired.

[0071] In step S146, the phase distribution measurement unit 46 determines the phase value to be set for reducing the aberration caused by the sample 24 for each segment SGa of group m, for example, based on the 2×Ns phase modulation patterns set in the phase modulation element 14 in step S145 and, among the above 2×Ns patterns, the light amount of the signal light DL from the sample point Fj or the evaluation value calculated from the image of the vicinity of the sample point Fj. As the evaluation value calculated from the image, for example, at least one of the contrast of the image, the maximum luminance value, the standard deviation of the luminance value, or the integrated value of the power spectrum in a predetermined frequency region when the image is Fourier-transformed may be used. Also, the correlation value between a plurality of images acquired by scanning the vicinity of the sample point Fj a plurality of times may be used.

[0072] In the subsequent step S147, the phase distribution setting unit 49 sends the signal S2 to the phase modulation element 14, and sets the phase value to be set for the illumination light IL for the portion corresponding to each segment SGa of group m among the reflecting surfaces 14s of the phase modulation element 14 to the determined phase value described above. In step S148, the loop of group m ends.

[0073] Through the above steps, the first determination and setting of the phase values of each segment SGa belonging to each group (m = 1 to M) are completed. However, if the so-called optimization described above is only performed once, the phase values of each segment SGa are not necessarily determined and set to the optimal values. Therefore, if necessary, the processing from step S142 to step S149 is repeated the necessary number of times (NI times) to make the phase values of each segment SGa closer to the optimal values.

[0074] When the loop of the measurement process ends in step S149, the process proceeds to step S150. In step S150, the control unit 40 causes the storage unit to store, in correspondence with the coordinates of the sample point Fj in the sample 24, the phase value to be set for each segment SGa, that is, the first phase distribution WF1, measured by the phase distribution measurement unit 46.

[0075] Hereinafter, at the sample point Fj, the first phase distribution WF measured (determined) for each segment SGa is expressed as Φ(xFj, yFj, zFj; xPi, yPi) using the coordinates of the sample point Fj and the coordinates of the center point Pi of each segment SGa. Through the above steps, the measurement of the first phase distribution WF1 for one sample point Fj is completed.

[0076] Note that the first phase distribution WF1 does not necessarily have to be measured continuously over the entire pupil plane IPP. For example, as described in the observation method of the second embodiment to be described later, the first phase distribution WF1 may be measured only for several discrete points or regions within the pupil plane IPP. That is, the first phase distribution WF1 may be a distribution of phase values at discrete points or regions within the pupil plane IPP.

[0077] (Calculation of the second phase distribution) Next, the method for calculating the second phase distribution WF2 performed in step S180 described above will be explained. As described above, based on the first phase distribution WF1 measured at each sample point F of the sample 24, the phase distribution calculation unit 47 calculates the second phase distribution WF2 in each small region Rj' of the sample 24, that is, the phase value to be added to the illumination light IL in order to reduce the aberration generated by the sample 24.

[0078] FIG. 7B is a diagram showing the reflecting surface 14s of the phase modulation element 14, similar to FIG. 7A. As shown in FIG. 7B, the phase distribution calculation unit 47 virtually divides the reflecting surface 14s of the phase modulation element 14 into a plurality of segments SGb. Then, the phase distribution calculation unit 47 calculates the second phase distribution WF2 by calculating an appropriate phase value to be set for the illumination light IL for each of the plurality of segments SGb.

[0079] Note that even when the phase modulation element 14 has unit elements such as a plurality of micro mirrors separated from each other, one segment SGb does not necessarily have to correspond to one unit element, and a plurality of unit elements may be included in one segment SGb. Alternatively, one segment SGb may correspond to one unit element.

[0080] The center points of the respective segments SGb are called phase sampling points PE (PE1, PE2, ···, PEi’, ···, PENPE), in the same manner as the example shown in FIG. 7A described above. Here, i’ is a subscript representing the number of the phase sampling point PE. Also, the maximum value of i’, that is, the total number of phase sampling points PE is NPE.

[0081] As described above, the reflecting surface 14s is arranged substantially in agreement with the pupil plane IPP of the light transmitting optical system ILO. Therefore, let the x, y coordinates on the pupil plane IPP of the phase sampling point PEi’ be (xPEi’, yPEi’). Note that the shape of each segment SGb is not limited to a square, and may be any shape. Also, the arrangement positions of the phase sampling points PEi’ are not limited to the points arranged in a lattice pattern within the xy plane, and may be any arrangement positions.

[0082] Also, the number of divisions of the segment SGb and the size of each in step S180 may be the same as or different from the number of divisions of the segment SGa and the size in step S141 described above.

[0083] In the present embodiment, the second phase distribution WF2 at an arbitrary detection point U is calculated using an expression model of field-dependent aberration as an example of the phase data model. In this method, first, the model parameters of the expression model of field-dependent aberration are determined by solving an inverse problem based on the first phase distribution WF1 described above. Thereafter, the second phase distribution WF2 at an arbitrary detection point U is calculated by solving a forward problem using the determined model parameters. Note that specific examples of the expression model of field-dependent aberration will be described later.

[0084] A model parameter vector α is introduced, and the relationship between the model parameter vector α and a function Φ(xFj, yFj, zFj; xPi, yPi) representing the first phase distribution WF1 measured at the j-th sample point Fj is expressed by Equation (1) using a model G.

Equation

[0085] The model parameter vector α can be determined by solving the inverse problem represented by the following equation (2) based on the measured first-phase distribution WF1.

Number

[0086] Here, λ is a regularization parameter, and R(α) is a regularization term, which is introduced for the purpose of preventing overfitting to the first-phase distribution WF1 or enabling the solution of the inverse problem even in the case of a poor problem where the number of information included in the first-phase distribution WF1 is less than the number of model parameters. The regularization term can be set as in the following equation (3) using, for example, the L1 norm.

Number

[0087] Here, T represents an operator for converting the model parameters to another basis, and for example, an identity matrix, a gradient operator, etc. can be used. The above inverse problem can be solved using an algorithm such as the proximal gradient method or the extended Lagrange method.

[0088] Once the model parameter vector α is obtained, the second-phase distribution WF2 to be set for the illumination light IL can be obtained by equation (4) as a forward problem when detecting an arbitrary small region R(xR, yR, zR). Here, the value ΦE shown in equation (4) represents the value at the phase sampling point PEi' on the pupil plane IPP, that is, on the reflection surface 14s of the phase modulation element 14.

Number

[0089] As an example, when using a linear model, the phase value φ, which is the value of the first phase distribution WF1, can be expressed by Equation (5) using the horizontal vector GL(xFj, yFj, zFj; xPi, yPi). Also, the phase value φE, which is the value of the second phase distribution WF2, can be expressed by Equation (6).

Number

Number

[0090] The arguments of the horizontal vector GL indicate that each component depends on the coordinates of the sample point F or the detection point U in the sample 24 and the coordinates of the phase sampling point P on the pupil plane. It is possible to create a matrix by arranging the horizontal vectors GL in Equation (5) vertically for the coordinates of all sample points Fj, create a pseudo-inverse matrix using the singular value decomposition of this matrix, and determine the model parameters.

[0091] Next, as an example of an expression model for field-dependent aberration, a phase layer model will be described. As shown in FIG. 9A, the phase layer model is a model that uses a plurality of phase layers L (L1, ···, Lk, ···, LNL) that are virtually arranged between the boundary surface SF of the sample 24 and the focusing point FP. k is an index indicating the order of the phase layer L. Note that the focusing point FP coincides with the sample point Fj, which is any one of the plurality of sample points F described above.

[0092] In the phase layer model, each phase layer L represents a model of the phase value given to the illumination light IL at each position in the respective xy plane. In the phase layer model, the phase value generated in the light ray connecting the point P on the pupil plane IPP and the focusing point FP in the sample 24 is expressed as the sum of the phase values generated when passing through the plurality of phase layers L. Also, a phase layer LP may be arranged on the pupil plane IPP to represent the field-independent aberration component that commonly occurs within the imaging range IR.

[0093] Note that the phase change of the light ray passing through each xy position of each phase layer L uniquely corresponds to the refractive index value at that xy position of the phase layer L. Therefore, as a phase layer model, a model representing the refractive index distribution of the sample 24 at each position within each xy plane of each phase layer L may be used.

[0094] The phase layer model can be a linear model represented by the above-described equations (5) and (6). Based on equations (5) and (6), the setting of the model parameter vector α in the phase layer model and the specific labeling of the matrix GL(xFj, yFj, zFj; xPi, yPi) are derived. In the following derivation, the coordinates (xPi, yPi) of the phase sampling point Pi on the pupil plane TPP are represented as the coordinates in the pupil (not shown) of the objective lens 21, but appropriately, the pupil of the objective lens 21 and the pupil magnification (imaging magnification) on the phase modulation element 14 may be multiplied for scale conversion.

[0095] While referring to FIGS. 9A and 9B, the definitions of symbols are made. FIG. 9A is an x-z cross-sectional view of the sample 24, the objective lens 21, and the pupil plane IPP, and FIG. 9B is an x-z cross-sectional view of the sample 24, the objective lens 21, and the pupil plane IPP, and is an xy cross-sectional view of the sample 24.

[0096] The coordinate origin O is set at the position of the boundary surface of the sample 24 on the optical axis AX. FIG. 9A shows an example in which NL phase layers L are arranged at positions where z is positive. Let the z coordinate of the k-th phase layer be zLk. The intersections of the light ray TR included in the illumination light IL emitted from the point (phase sampling point) Pi on the pupil plane IPP with the NL phase layers L until it converges at the focus point FP (sample point Fj) are respectively denoted as Iij1, ···, Iijk, ···, IijNL. The coordinates of the intersection Iijk are denoted as (xLijk, yLijk, zLk).

[0097] Note that the number of phase layers L to be arranged, NL, may be set so that the improvement effect is maximized when corrected by the second phase distribution WF2 according to the observation conditions. Also, the z position of each phase layer L may be set, for example, to be equally spaced between the sample point Fj and the boundary surface SF of the sample 24, or if the z position where there is a structure with a large variation in refractive index distribution such as bone exists in the sample 24 in advance, it may be set to that z position.

[0098] The coordinates of the intersection point Iijk of the phase layer L and the light ray TR are geometrically uniquely determined if the coordinates (xFj, yFj, zFj) of the sample point Fj and the coordinates (xPi, yPi) of the phase sampling point Pi on the pupil plane IPP are determined. Let θ be the angle between the light ray TR converging on the sample point Fj and the optical axis, and Ψ be the angle between the projection of the light ray onto the x - y plane and the x - axis.

[0099] Since the z - coordinate of the phase layer Lk is zLk, the distance between the sample point Fj and the phase layer Lk is zFj - zLk. The distance dijk when the sample point Fj and the point ILijk are projected onto the xy - plane is expressed by Equation (7).

Equation

[0100] From the above, the x and y coordinates of the intersection point ILijk where the light ray TR converging on the sample point Fj intersects the phase layer Lk can be described as in Equations (8) and (9).

Equation

Equation

[0101] On the other hand, the coordinates of the point Pi where the light ray TR intersects the pupil plane IPP are determined by the azimuth angles (θ, Ψ) of the light ray TR. When the sine condition is satisfied for the objective lens 21, the relationship between the angle of the light ray TR and the coordinates of the point Pi can be described as in Equations (10) to (12) using the focal length f of the objective lens 21. [Mathematics] [Mathematics] [Mathematics]

[0102] By eliminating θ and Ψ in Equation (8) and Equation (9) using Equations (10) to (12), the following Equation (13) and Equation (14) are obtained. [Mathematics] [Mathematics]

[0103] According to Equation (13) and Equation (14), the coordinates ILijk of the intersection point of the light ray TR and the phase layer Lk can be expressed using both the coordinates of the sample point Fj in the sample 24, which are both known, and the coordinates of the phase sampling point Pi on the pupil plane IPP.

[0104] The phase distribution of the phase layer LP on the pupil plane IPP is denoted as φP(xPi, yPi), and the phase distribution of each of the NL phase layers L set in the sample 24 is denoted as φKL(xLijk,yLijk). At this time, the phase value Φ(xFj, yFj, zFj; xPi, yPi) generated in the light ray TR emitted from the point Pi on the pupil and focused on the sample point Fj is represented by the sum of the phase values generated in the multiple phase layers L, as shown in Equation (15). [Mathematics]

[0105] At this time, the phase distribution of each phase layer L can be represented by a linear combination of basis functions, as shown in Equation (16) and Equation (17). [Mathematics]

Number

[0106] In Equations (16) and (17), fPn(xP,yP) and fLn(xL,yL) are the n-th terms of the basis functions, and αpn is the coefficient of the n-th term of the basis function representing the phase layer LP arranged on the pupil plane IPP. Also, αLk,n is the coefficient of the n-th term of the basis function representing the k-th phase layer Lk, and MP and MLk are the numbers of terms of the basis functions used.

[0107] As the basis functions fPn(xP,yP) and fLn(xL,yL), for example, trigonometric functions, wavelet functions, Zernike polynomials, Legendre polynomials, etc. can be used. Alternatively, as the basis function, for example, the phase layer L can be segmented into a lattice pattern, and a function that takes a value of 1 inside the segment and 0 outside the segment can be used. Also, different basis functions and numbers of terms MP and MLk can be used for each of the plurality of phase layers L, or the basis functions can be selectively used according to the sample 24 of the observation target. The coefficients αPn of the basis function and the coefficients αLk,n are the model parameters in the present phase layer model.

[0108] The relationship between the model parameters in Equations (15) to (17) and the first phase distribution WF1 can be expressed using a matrix as shown below. Arrange the coefficients of the basis functions of each phase layer L on the pupil plane IPP and in the sample 24 vertically for each phase layer L, and express them as a vector as in Equation (18).

Number

[0109] Furthermore, arrange the values φFj of the first phase distribution WF1 measured at the sample points Fj vertically according to the number j of the subscript of the phase sampling points Pi on the pupil plane IPP, and express them as a vector as in Equation (19).

Number

[0110] Based on equations (15) to (19), the relationships between αP and αk and φFj can be expressed as in equation (20) using matrices.

Number

Number

[0111] Also, the matrix Aj,k in equation (20) is a matrix of NP rows × MLk columns, and its (i, m) component is given by equation (22).

Number

[0112] Furthermore, using the vector obtained by vertically arranging the vectors αP and αk, and the matrix obtained by horizontally arranging the matrices Ap and Aj,k, it is also possible to express it as in equation (23).

Number

[0113] Furthermore, using the vector obtained by vertically arranging the values φFj (φF1, φF2, ···, φFNF) of the first phase distribution WF1 at each of the NF focusing points Fj, it can be expressed as in equation (24).

Number

[0114] Since the coordinates (xFj, yFj, zFj) of the sample point Fj, the coordinates (xPi, yPi) of the phase sampling points Pi of each pupil, and the z position zLk of each previously set phase layer Lk are known, each component of the matrix A in Equation (24) can be determined. Note that the components of the matrix Ap and the matrix Aj,k are not limited to those described above. For example, a correction coefficient corresponding to the azimuth angles (θ, Ψ) of the light rays TR corresponding to the respective components may be multiplied to each component of the matrix, or the like may be used.

[0115] By the method described above, a linear relationship between the model parameters and the first phase distribution WF1 can be derived. The phase distribution calculation unit 47 creates the above-described model (phase data model) and solves the inverse problem based on the above-described Equation (2) to determine the model parameter vectors αP, αL1, ···, αLNL from the first phase distribution WF1. Then, the phase distribution calculation unit 47 calculates a second phase distribution WF2 suitable for the detection of each small region R based on the above-described Equation (6). At this time, by implementing the derivation of the above-described Equation (23) after replacing the sample point Fj with the small region Rj' and the sampling point Pi with the sampling point PEi', it is possible to derive the horizontal vector GL(xR, yD, zR,; xPEi', yPEi') of Equation (6). The arithmetic unit 41 stores the second phase distribution WF2 calculated by the phase distribution calculation unit 47 in the storage unit 42 in association with the number j' of each small region R.

[0116] Note that, as shown in FIGS. 7A and 7B, the total number NPE of the phase sampling points PEi' when calculating the second phase distribution WF2 may be set to be larger than the total number NP of the phase sampling points Pi when measuring the first phase distribution WF1. That is, as the total number of the phase sampling points Pi, a number larger than the expression freedom degree of the first phase distribution WF1 may be set. In that case, when deriving the horizontal vector GL(xR, yD, zR,; xPEi', yPEi') in Equation (6) based on Equation (23), the total number of the phase sampling points may be replaced from NP to NPE.

[0117] As described above, the second phase distribution WF2 for one detection point U is determined based on the first phase distributions WF1 for two or more sample points F. Therefore, in other words, it can also be said that the amount of information characterizing the second phase distribution WF2 for one detection point U may be set to be larger than the amount of information characterizing the first phase distribution WF1 for one sample point F.

[0118] In this way, when calculating the second phase distribution WF2, by calculating the phases at a larger number of phase sampling points P than when measuring the first phase distribution WF1, the value of the second phase distribution WF2 can be calculated for the more finely divided segments SGb of the phase modulation element 14. Also, a higher improvement effect can be obtained compared to the case where aberration correction is performed with the first phase distribution WF1 as the second phase distribution WF2 as it is.

[0119] (Method for acquiring an image) Hereinafter, with reference to FIG. 6, the method for acquiring an image in step S190 described above will be described. In step S191, a loop for a plurality of small regions R (see FIG. 8) within the imaging range IR for image acquisition is started. Then, for the processing up to step S195 described later, a loop of NE times from j' = 1 to j' = NE is repeated for the subscript j' of the small region R.

[0120] In step S192, the j'-th small region Rj' is selected. Then, in step S193, the phase distribution setting unit 49 sends a signal S2 to the phase modulation element 14 and sets the phase modulation element 14 to add the second phase distribution WF2 suitable for detecting the small region Rj' to the illumination light IL. That is, in step S193, the phase distribution setting unit 49 sets the second phase distribution WF2 for the phase modulation element 14 in synchronization with the scanning of the condensing point FP and the sample 24 by the scanning unit (oscillating mirror 17 or stage 25).

[0121] In step S194, a z-stack image within the small region Rj’ is acquired. When acquiring the image, the control unit 40 sends signal S3 to oscillate the oscillating mirror 17 within the small region Rj’. Also, the control unit 40 sends signal S3 to the stage driving unit 26 or sends signal S5 to the objective lens holding unit 22 to relatively move the light collection point FP in the z direction within the small region Rj’. Then, the image generation unit 48 generates a z-stack image based on the signal S1 acquired by the detector 29a when the light collection point FP coincides with each detection point Uo.

[0122] The pitches (Px, Py, Pz) of the detection points U in the x, y, and z directions when acquiring the z-stack image follow the values input in step S100. Note that the acquired z-stack image may include images of a plurality of xy cross-sections with different positions in the z direction within the small region Rj’, or may include only an image of one xy cross-section.

[0123] Since the second phase distribution WF2 is added to the illumination light IL in the image acquired in step S194, the aberration caused by the sample 24 is corrected, and thus the image quality such as the resolution is improved. The control unit 40 associates the captured z-stack image with the number j’ of the small region Rj’ and stores it in the storage unit 42.

[0124] In step S195, the loop for the plurality of small regions R ends. Then, in step S196, a plurality of z-stack images acquired in repeatedly executed step S194 are joined together to create one z-stack image, which is stored in the storage unit 42. The created z-stack image may be displayed on the display unit 45.

[0125] Note that the method for acquiring an image of the sample 24 is not limited to that described above. For example, after setting the average phase distribution of the second phase distribution WF2 for all the small regions R in the entire imaging range IR in the phase modulation element 14 and performing imaging, only a small number of small regions R set by the user are separately imaged after setting the second phase distribution WF2 calculated for the small region R in the phase modulation element 14.

[0126] Note that the signal light DL emitted from the sample 24 when acquiring the z-stack image in step S194 may be of a different type from the signal light detected when measuring the first phase distribution WF1 in step S145. For example, the first phase distribution WF1 may be measured using a signal other than fluorescence, such as scattered light emitted from the sample 24, and fluorescence emitted from the sample 24 may be used when acquiring the z-stack image in step S194.

[0127] When detecting a signal other than fluorescence to measure the first phase distribution WF1, the illuminance of the illumination light IL irradiating the fluorescent dye in the sample 24 during the measurement of the first phase distribution WF1 can be kept relatively low, and fading of the fluorescent dye in the sample 24 can be reduced when acquiring the first phase distribution WF1. Also, even when there is no object that emits a sufficient signal to measure the first phase distribution WF1 at the position where the signal light DL due to fluorescence in the sample 24 is to be observed, by using this embodiment, the second phase distribution WF2 at that position can be calculated to correct the aberration.

[0128] (Microscope of Modification 1 of the First Embodiment) FIG. 10 is a diagram showing a microscope 1b of Modification 1 of the first embodiment. Since many configurations of the microscope 1b of Modification 1 are common to those of the microscope 1a of the first embodiment, the same reference numerals are given to the common configurations, and the description thereof will be omitted as appropriate.

[0129] The microscope 1b of Modification 1 is different from the microscope 1a of the first embodiment in that the signal light DL generated by the sample 24 is received by the detector 29a via the relay lenses 19, 18, and the swing mirror 17. The signal light DL reflected by the swing mirror 17 is reflected by a dichroic mirror 20 disposed between the swing mirror 17 and the relay lens 16.

[0130] The signal light DL then passes through the detection filter 28, is condensed by the relay lens 27, and is condensed onto the pinhole of the pinhole plate 30 disposed in front of the detector 29a. Then, the signal light DL passes through the pinhole and is detected by the detector 29a, converted into an electrical signal, and sent to the control unit 40 as the signal S1.

[0131] As shown by being surrounded by the two-dot chain line frame, the light-receiving optical system DTO of the microscope 1b of Modification 1 includes the objective lens 21, relay lenses 19 and 18, the swing mirror 17, the dichroic mirror 20, the detection filter 28, the relay lens 27c, the pinhole plate 30, and the detector 29a, etc. Among these, the configuration from the objective lens 21 to the swing mirror 17 along the optical path of the signal light DL is included in both the light-sending optical system ILO and the light-receiving optical system DTO.

[0132] In the microscope 1b of Modification 1, since the signal light DL generated by the sample 24 is descanned by the swing mirror 17, it is condensed onto the pinhole of the pinhole plate 30 disposed in front of the detector 29a regardless of the azimuth angle of the reflection surface of the swing mirror 17. Therefore, since the microscope 1b of Modification 1 functions as a so-called confocal microscope, the resolution is further improved.

[0133] Furthermore, in the microscope 1b of Modification 1, for the signal light DL from the portion of the sample 24 that is displaced in the +z direction or -z direction from the condensing point FP of the illumination light IL (see FIGS. 2A and 2B), most of it is shielded by the pinhole plate 30 and is not received by the detector 29a. Therefore, in the microscope 1b of Modification 1, even when detecting the signal light DL that linearly responds to the intensity of the illumination light IL such as reflected light and scattered light, it is possible to remove the signal light DL generated from other than the condensing point FP, which is so-called background light.

[0134] (Microscope of Modification 2 of the First Embodiment) FIG. 11 is a diagram showing a microscope 1c according to Modification 2 of the first embodiment. Since many components of the microscope 1c of Modification 2 are common to the microscope 1a of the first embodiment and the microscope 1b of Modification 1, the same reference numerals are given to the common components, and the description thereof will be omitted as appropriate.

[0135] The microscope 1c of Modification 2 is different from the microscope 1a of the first embodiment and the microscope 1b of Modification 1 in that the signal light DL generated by the sample 24 is received by the detector 29a via the relay lenses 19, 18, the swing mirror 17, the relay lenses 16, 15, and the phase modulation element 14. The signal light DL reflected by the phase modulation element 14 is reflected by a dichroic mirror 20 disposed between the phase modulation element 14 and the collimator lens 12.

[0136] Thereafter, the signal light DL passes through the detection filter 28, is condensed by the relay lens 27, and is condensed on the pinhole of a pinhole plate 30 disposed in front of the detector 29a. Then, the signal light DL passes through the pinhole and is detected by the detector 29a, converted into an electrical signal, and sent to the control unit 40 as a signal S1.

[0137] As shown by being surrounded by a two-dot chain line frame, the light receiving optical system DTO of the microscope 1c of Modification 2 includes an objective lens 21, relay lenses 19, 18, a swing mirror 17, relay lenses 16, 15, a phase modulation element 14, a dichroic mirror 20, a detection filter 28, a relay lens 27c, a pinhole plate 30, a detector 29a, and the like. Among these, the configuration from the objective lens 21 to the dichroic mirror 20 along the optical path of the signal light DL is included in both the light transmission optical system ILO and the light receiving optical system DTO. The reflecting surface 14s of the phase modulation element 14 is disposed so as to generally coincide with a surface that is the pupil plane IPP of the light transmission optical system ILO and also the pupil plane DPP of the light receiving optical system DTO.

[0138] In the microscope 1c of Modification 2, similar to the microscope 1b of Modification 1, since the signal light DL generated by the sample 24 is descanned by the rocking mirror 17, it is focused on the pinhole of the pinhole plate 30 disposed in front of the detector 29a regardless of the azimuth angle of the reflecting surface of the rocking mirror 17. Therefore, since the microscope 1c of Modification 2 also functions as a so-called confocal microscope, the resolution is further improved. Also, similar to the microscope 1b of Modification 1, even when detecting the signal light DL that linearly responds to the intensity of the illumination light IL, the above-described background light can be removed.

[0139] In the microscope 1c of Modification 2, the phase modulation element 14 can correct not only the aberration generated in the illumination light IL by the sample 24 but also the aberration generated in the signal light DL by the sample 24. Thereby, the resolution of the microscope 1c of Modification 2 is further improved. Note that instead of the pinhole plate 30 and the detector 29a, an imaging element that acquires a two-dimensional image may be disposed as a detector at the position where the pinhole plate 30 is disposed.

[0140] (Observation Method of the Second Embodiment) Hereinafter, the observation method of the second embodiment will be described. The observation method of the second embodiment is a method of observing the sample 24 using the microscopes 1a to 1c of the first embodiment and each modification described above. The observation method of the second embodiment mostly coincides with the observation method of the first embodiment described above. Therefore, hereinafter, the differences from the observation method of the first embodiment will be described, and the common parts will be omitted as appropriate.

[0141] In the observation method of the second embodiment, when measuring the first phase distribution WF1 in step S140 in the flowchart when executing the observation method of the first embodiment shown in FIG. 3, compared with the observation method of the first embodiment described above, the total number NP of phase sampling points Pi to be measured is reduced. As a result, the number of times of acquiring signals emitted from each sample point Fj or the number of times of acquiring images of the vicinity regions of each sample point Fj for calculating the first phase distribution WF1, that is, for calculating the phase values of each phase sampling point Pi for canceling the aberration generated by the sample 24, can be reduced. Thereby, the measurement time of the first phase distribution WF1 can be shortened, and the fading and phototoxicity of the sample 24 can be reduced.

[0142] FIG. 12 is a diagram showing an example of the division of the segment SGa of the phase modulation element 14 in the observation method of the second embodiment, and is a diagram showing the reflection surface 14s of the phase modulation element 14, similar to FIG. 7A. In the observation method of the second embodiment, the measurement of the first phase distribution WF1 is performed by changing only the phase value of the segment SGa indicated by the white quadrangle in FIG. 7A. For the region with dots other than the segment SGa on the reflection surface 14s, the first phase distribution WF1 is not measured.

[0143] In the observation method of the second embodiment, the total number NP of phase sampling points Pi used for measuring the first phase distribution WF1 is reduced compared with the total number in the observation method of the first embodiment shown in FIG. 7A. And the segments SGa including the respective phase sampling points Pi are discretely arranged on the reflection surface 14s, that is, on the pupil plane IPP.

[0144] Hereinafter, with reference to FIG. 13, the flow of the measurement of the first phase distribution WF1 (step S140 in the flowchart shown in FIG. 3) in the observation method of the second embodiment will be described. First, in step S241, NP segments SGa that perform phase modulation as shown in segment SGa in FIG. 7A are determined. Then, in the loop for the subsequent Nt phase modulation patterns, Nt phase modulation patterns set in the phase modulation element 14 are set. For example, a pattern in which the relative phase difference between each segment SGa in the reflecting surface 14s and the region other than the segment SGa changes at different frequencies over time for each segment may be set, or different random phase modulation patterns may be set.

[0145] In step S241, an Nt - loop for performing phase modulation a plurality of times for each of the NP segments SGa is started. Since this loop performs phase modulation according to different phase modulation patterns for each of the NP segments SGa each time, it can be called a loop of phase modulation patterns. Let the loop counter be t.

[0146] In step S242, the phase distribution setting unit 49 sends signal S2 to the phase modulation element 14 and sets phase values according to the t - th phase modulation pattern to the NP segments SGa of the phase modulation element 14. Then, the control unit 40 opens the shutter 11 to irradiate the illumination light onto the material 24 and detects the signal emitted from the sample point Fj. At this time, the control unit 40 may scan the oscillating mirror 17 and acquire an image of the sample point Fj and its vicinity based on the signal 1 from the detector 29a.

[0147] In step S244, the loop of the phase modulation pattern ends. In step S245, the phase distribution measurement unit 46 calculates (measures) the phase values of each segment SGa and the phase sampling point Pi corresponding to the aberration caused by the sample 24, that is, the first - order phase distribution WF1, based on the Nt phase modulation patterns set in the phase modulation element 14 in step S243 and the Nt signals emitted from the sample point Fj or the Nt images of the vicinity of the sample point Fj. Since the method of calculating the phase value of each segment SGa in step S245 is the same as the method in step S146 described above, the description is omitted.

[0148] Also in the observation method of the second embodiment, for the steps other than step S140 in the flow shown in FIG. 3, that is, steps S241 to S245 described above, since they are the same as the observation method of the first embodiment described above, the description thereof is omitted.

[0149] (Observation method of the third embodiment) Hereinafter, the observation method of the third embodiment will be described. The observation method of the third embodiment is a method of observing the sample 24 using the microscopes 1a to 1c of the first embodiment and each modification described above. The observation method of the second embodiment mostly shares the same as the observation method of the second embodiment described above. Therefore, hereinafter, the differences from the observation method of the first embodiment will be described, and the common parts will be omitted as appropriate.

[0150] In the observation method of the second embodiment described above, at one sample point F in the sample 24, a plurality of signals (images) are acquired while changing the phase modulation pattern of the phase modulation element 14. Then, it moves to the next sample point F, and again, a plurality of signals (images) are acquired while changing the phase modulation pattern of the phase modulation element 14.

[0151] In contrast, in the third embodiment, after setting a predetermined one phase modulation pattern in the phase modulation element 14, signal acquisition of a predetermined region in the field of view within the sample 24 is sequentially performed. Then, it is different in that the phase modulation pattern set in the phase modulation element 14 is changed, and again, signal acquisition of a predetermined region in the field of view within the sample 24 is sequentially performed.

[0152] FIG. 14 is a diagram showing the flow of the observation method of the third embodiment. First, in step S300, the imaging range IR is set. The content executed in step S300 is the same as step S100 in the flow of the observation methods of the first and second embodiments shown in FIG. 3.

[0153] In the subsequent step S310, similar to step S241 in the flow of the observation method of the second embodiment shown in FIG. 13, a segment SGa that performs phase modulation on the phase modulation element 14 and a plurality of phase modulation patterns used in the subsequent loop are determined.

[0154] In step S320, the measurement ranges in the x, y, and z directions of the first phase distribution WF1 (see FIG. 2A) are set. For example, part or all of the imaging range IR set by the user in step S300 may be set as the measurement range of the first phase distribution WF1.

[0155] In step S330, a loop of Nt times of performing phase modulation a plurality of times for each of the NP segments SGa, that is, a loop of phase modulation patterns, is started. Let the loop counter be t.

[0156] In step S340, the phase distribution setting unit 49 sends a signal S2 to the phase modulation element 14 and sets phase values according to the t-th phase modulation pattern to the NP segments SGa of the phase modulation element 14.

[0157] Then, in step S350, the control unit 40 performs scanning of the oscillating mirror 17 and driving in the z direction of the objective lens 21 or the stage 25, and acquires a signal emitted from the sample 24 within the measurement range determined in step S320.

[0158] In step S360, the loop of the phase modulation pattern ends. In step S370, similar to step S170 in the flow of the observation methods of the first and second embodiments, the imaging range IR is divided and a plurality of small regions R are set.

[0159] In step S380, based on the signals in the measurement range obtained in each step S350 in the loop of the phase modulation pattern, a second phase distribution WF2 for each small region R is calculated. In so doing, within the measurement range in which the signals were obtained in step S350, a plurality of sample points F for calculating the first phase distribution WF1 are determined.

[0160] For example, within the measurement range in which the signals obtained in step S350 using a predetermined t-th phase modulation pattern are acquired, a plurality of NF positions (coordinates (x, y, z)) may be selected as sample points F from the top in descending order of signal intensity. By selecting points with high signal intensity as the sample points F, the phase data error due to noise can be reduced.

[0161] The phase distribution measurement unit 46 performs the same processing as in step S245 in the observation method of the second embodiment based on the Nt phase modulation patterns set in the phase modulation element 14 and the Nt detection signals of each detected sample point F. Thereby, the phase distribution measurement unit 46 calculates (measures) the phase value at the center position Pi of each segment SGa on the phase modulation element 14, that is, the first phase distribution WF1.

[0162] In step S380, further, in the same manner as in step S180 of the observation methods of the first and second embodiments, the phase distribution calculation unit 47 calculates the second phase distribution WF2 for the NR small regions R based on the respective first phase distributions WF1 measured at a plurality of sample points.

[0163] In the subsequent step S390, in the same manner as in step S190 of the observation methods of the first and second embodiments, the phase distribution setting unit 49 sends the signal S2 to the phase modulation element 14, and acquires an image of the sample 24 in the imaging range IR with the calculated second phase distribution WF2 set in the illumination light IL.

[0164] When using the phase modulation element 14 that requires a relatively long time to change the phase modulation pattern, the more times the phase modulation pattern of the phase modulation element 14 is changed during the measurement of the first phase distribution WF1, the longer the measurement time of the first phase distribution WF1 becomes. For example, when using a liquid crystal SLM for the phase modulation element 14, the response time of the liquid crystal becomes the rate-determining factor and the measurement time becomes long.

[0165] In the observation method of the third embodiment, the detection of signals from a predetermined region within the field of view is performed with a predetermined phase modulation pattern set for the phase modulation element 14, and then the process of changing the phase modulation pattern of the phase modulation element 14 is repeated. Thereby, the number of times of changing the phase modulation pattern of the phase modulation element 14 can be reduced, and the time required for measuring the first phase distribution WF1 can be shortened. It can be done.

[0166] (Observation method of the fourth embodiment) Hereinafter, the observation method of the fourth embodiment will be described. The observation method of the fourth embodiment is a method of observing the sample 24 using the microscopes 1a to 1c of the first embodiment and each modification described above. The observation method of the fourth embodiment is different from the observation methods of the first embodiment and the second embodiment in the method of measuring the first phase distribution WF1 in step S140 and the method of calculating the second phase distribution WF2 in step S180 described above. However, the other steps are the same as the observation methods of the first embodiment and the second embodiment shown in FIG. 3.

[0167] In the observation method of the fourth embodiment, in the measurement of the first phase distribution WF1 in step S140, while changing the coefficient value of each mode of the basis function, which represents the phase value set for the phase modulation element 14, each sample point F is detected (imaged), and the coefficient value of each mode of the basis function that maximizes the evaluation value calculated based on the detected signal is obtained.

[0168] Let the number of terms of the basis function be NB, the m-th term (m = 1, 2, ···, NB) of the basis function be hm(x, y), and the coefficient value of the m-th term at the sample point Fj be bj,m. At this time, the phase value Φ(xFj, yFj, zFj; xP, yP) of the point (xP, yP) on the pupil plane IPP of the first phase distribution WF1 measured at the point Fj can be expressed by Equation (25).

Number

[0169] Hereinafter, the m-th component hm(xP, yP) of the basis function is also referred to as the "mode m" of the basis function. As the basis function, for example, Zernike polynomials, Legendre polynomials, trigonometric functions, wavelet functions, etc. can be used. When steps S441 to S452 described later are executed, NB coefficient values bj,1, bj,2, ···, bj,NB are stored in the storage unit.

[0170] FIG. 15 is a diagram showing the flow of measurement of the first phase distribution WF in the observation method of the fourth embodiment. First, in step S441, the coefficient values of each mode of the basis function are set to initial values and stored in the storage unit 42. Specifically, as an example, bm’ = 0 (m’ = 1, 2, …, NB) is set. Here, m’ represents the m-th of the basis functions in the same way as m. Since m is used as a loop counter in step S443 described later, m’ is used instead of m to prevent confusion.

[0171] In step S442, a loop of the process of measuring the first phase distribution WF1 is started. Then, for the process up to step S451 described later, a loop of NI times from itr = 1 to itr = NI is repeated for the subscript itr of the number of repetitions.

[0172] In step S443, a loop for mode m is started. Then, for the process up to step S450 described later, a loop of NB times from m = 1 to m = NB is repeated for the subscript m of mode m.

[0173] In step S444, for Nt loops of the loop counter t starting from step S445, Nt coefficient values bm(t) (t = 1, 2, ···, Nt) for the mode m are set. As the coefficient value bm(t), for example, it may be set by equally dividing a predetermined numerical range. Also, the numerical range may be changed according to the loop counter itr. For example, by narrowing the numerical range each time itr increases, it is possible to accurately determine the coefficient value bm(t) of each basis function mode. The coefficient value of the current mode m is stored in the storage unit 42 as bold.

[0174] In step S445, a loop that changes the shape of the phase modulation element Nt times for the basis function of the mode m is started. As described above, this loop can be called the phase modulation pattern loop. With the loop counter as t, the loop up to step S448 is repeated Nt times from t = 1 to t = Nt.

[0175] In step S446, the coefficient value bm of the basis function of the mode m is changed to bm = bold + bm(t), and a phase modulation pattern represented by a linear sum of basis functions with coefficient values bm' (m' = 1, 2, ···, NB) for each mode is set for the phase value of the phase modulation element 14. Then, the sample 24 is irradiated with the illumination light IL, and the signal light DL emitted from the sample 24 is detected by the detector 29a. The detection may be performed by scanning a predetermined range of the sample 24 with the illumination light IL centered on the position of the focusing point FP.

[0176] The phase distribution measurement unit 46 calculates an evaluation value V(t) based on the detected signal S1. As the evaluation value V(t), for example, an integrated value of the signal S1 emitted from the sample 24 may be used. For example, the evaluation value V(t) may be calculated based on an image created by scanning the sample 24. As the evaluation value based on the image, for example, at least one of the contrast of the image, the maximum luminance value, the standard deviation of the luminance values, or the integrated value of the power spectrum in a predetermined frequency region when the image is Fourier-transformed may be used. Also, a correlation value between a plurality of images obtained by scanning the sample 24 a plurality of times may be used.

[0177] Also, the cut-off frequency of the image calculated based on Fourier ring correlation simplifies image restoration in fluorescence microscopy, nature communications 10, Article number: 3103 (2019), which is a known document, may be used. Note that a plurality of evaluation values may be combined and used. In step S448, the loop of the phase modulation pattern ends.

[0178] In step S449, an appropriate coefficient value bm for mode m is determined based on the evaluation values obtained while changing bm Nm times. * For bm * For example, for t at which V(t) takes the maximum value, bm(t) set in step S444 may be selected. For bm * For example, V(t) may be expressed as a function with bm(t) as an argument, and the coefficient value bm that maximizes the evaluation value V may be calculated by interpolating the data points of the function V(t). As a method for interpolating data points, for example, Lagrange interpolation, Newton interpolation, Hermite interpolation, spline interpolation, etc. can be used. For bm * For example, based on the function approximation of the function V(t), the coefficient value bm that maximizes the evaluation value V may be calculated. As the approximation function, for example, a quadratic function or a Gaussian function can be used.

[0179] After that, bm = bm * is set, and a phase modulation pattern represented by a linear sum of basis functions whose coefficient values for each mode are bm’ (m’ = 1, 2, ···, NB) is set in the phase modulation element 14. In step S450, the loop for mode m ends. In step S451, the loop of measurement repetition ends.

[0180] By the above steps, the first calculation and determination of the coefficient value bm for each mode of the basis function are completed. However, just performing the so-called optimization described above once does not necessarily mean that the coefficient value bm for each mode has been determined and set to the optimal value. Therefore, if necessary, the processing from step S442 to step S451 is repeated the required number of times (NI times) to bring the coefficient value bm for each mode closer to the optimal value.

[0181] In step S452, the coefficient value bm’ (m’ = 1, 2, ···, NB) for each mode of the basis function determined by the above high and low is stored in the storage unit 42 as the first phase distribution WF1 at the current sample point Fj. Thus, the measurement of the first phase distribution WF1 in the observation method of the fourth embodiment is completed.

[0182] Subsequently, the calculation method in the observation method of the fourth embodiment for calculating the second phase distribution WF2 for each small region R in step S180 of the flow shown in FIG. 3 will be described. By executing the loop from step S120 to step S160 of the flow shown in FIG. 3, NF × NB coefficient values b 1,1 , b 1,2 , ···, b NF,NB are obtained and stored in the storage unit 42. Here, after solving the inverse problem from the obtained coefficient values to determine the model parameters, the coefficient values of the basis function representing the second phase distribution WF2 in the small region Rj’ are calculated by solving the forward problem. As a specific example, the method using the phase layer model described above will be explained.

[0183] In the observation methods of the first and second embodiments described above, when the phase values at a plurality of phase sampling points Pi on the pupil plane IPP are obtained as the first phase distribution WF1, the model parameters are obtained using the model represented by Equation (24). When the first phase distribution WF1 is expressed as a sum of basis functions as in the fourth embodiment, the relationship between the model parameters and the coefficient values bj,m of the basis functions can be expressed by multiplying each submatrix of Equation (24) by a transformation matrix to the coefficient values bj,m of the basis functions.

[0184] The coefficient bj,m of the basis function measured at the sample point Fj is arranged vertically from m = 1 to m = NB, and is expressed as a vector bFj as in Equation (26).

Equation

[0185] Equation (25) can be expressed as Equation (27) using the matrix H.

Equation

[0186] If the phase value Φ Fj on the pupil plane IPP of the first phase distribution WF1 were known, the coefficient values of the basis functions could be obtained by the least squares method, and the least squares solution of bFj is represented by Equation (28).

Number

[0187] The measurement coefficient value vector bFj and the model parameter vector α P , α L1 , ···, α LNL The relationship between, can be expressed by Equation (29) by performing a basis transformation with the matrix (HTH) for Equation (24). -1 HT.

Number

[0188] By solving the inverse problem based on Equation (2) using the linear model expressed by Equation (29), the model parameter vectors α P , α L1 , ···, α LNL can be determined from the measurement coefficient value vectors bF1, bF2, …, bFNF.

[0189] Subsequently, a method for calculating the second phase distribution WF2 of the j'-th small region Rj' based on the determined model parameter vectors α P , α L1 , ···, α LNL will be explained. The second phase distribution WF2 in the small region Rj' is expressed as a linear sum of orthonormal functions as in Equation (30).

Number

[0190] Here, (xP, yP) are the coordinates on the pupil plane IPP, Φ(xRj’, yRj’, zRj’; xP, yP) is the phase value at the coordinates (xP, yP) on the pupil plane IPP, that is, the value of the second phase distribution WF2. Also, the number of terms of the basis function used for aberration correction is NB’, and the m-th (m = 1, 2, ···, NB) term of the basis function used for aberration correction is h’ m(x, y), the coefficient value of the m-th term is denoted as b’j’,m.

[0191] Note that as the basis function h’m(xP, yP) used for aberration correction, a basis function different from the basis function hm(xP, yP) used at the time of measuring the first phase distribution WF1 may be selected. Also, the number of terms NB’ used at the time of aberration estimation may be set to a value different from the number of terms NB used at the time of measuring the first phase distribution WF1. In particular, if NB’>NB is set, aberrations with a higher degree of freedom than at the time of measuring the first phase distribution WF1 can be estimated, and by using this for aberration correction, it is possible to improve the image quality improvement effect. From this, it can also be said that the amount of information characterizing the second phase distribution WF2 may be set to be larger than the amount of information characterizing the first phase distribution WF1.

[0192] Equation (30) can be described as in Equation (31) using a matrix in the same manner as Equation (27). [Number] By performing a basis transformation based on the least squares method on Equation (31) in the same manner as Equation (29), Equation (32) is obtained. [Number]

[0193] Using the coefficient value b’j’,m of the basis function of Equation (32), the second phase distribution WF2 in the j’-th small region Rj’ is calculated. Also in the observation method of the fourth embodiment, a series of processes related to the calculation of the second phase distribution WF2 described above are performed by the phase distribution calculation unit 47.

[0194] The observation method of the fourth embodiment is suitable for use in the microscope 1a using an element capable of continuously changing the phase value of the phase modulation element 14, such as a deformable mirror, an adaptive lens, or a liquid lens, as the phase modulation element 14. In these elements, usually, the phase value can be set to a desired distribution by spatially continuously changing the shape of the reflecting surface 14s or the spatial distribution of the thickness of the element in accordance with the voltage applied to a plurality of actuators.

[0195] Generally, the phase value (shape of the wavefront) modulated by these elements is often expressed in the form of a linear sum of basis functions such as Zernike polynomials. By calibrating in advance the relationship between the voltage applied to the actuator and the coefficient values of the basis functions representing the wavefront shape, the modulated wavefront shape can be controlled as a linear sum of basis functions using arbitrary coefficients. In addition, since these elements have the effect that the phase value added by the phase modulation element 14 changes continuously within the pupil plane IPP, the decrease in light utilization efficiency due to diffraction is small.

[0196] On the other hand, the observation methods from the first embodiment to the third embodiment are suitable for use in the microscope 1a using an element having unit elements capable of independently controlling the phase value, such as a liquid crystal SLM or a MEMS - SLM, as the phase modulation element 14. In addition, in the observation methods from the first embodiment to the third embodiment, a phase modulation element 14 in which the added phase value changes continuously within the pupil plane IPP may be used. Also, in the observation method of the fourth embodiment, a phase modulation element 14 having unit elements capable of independently controlling the phase value may be used.

[0197] (Microscope of the second embodiment) Hereinafter, the microscope 1d of the second embodiment will be described with reference to FIG. 16. Since the configuration of the microscope 1d of the second embodiment has many configurations in common with the microscope 1a of the first embodiment described above, in the following, the same reference numerals will be given to the common configurations and the description will be omitted as appropriate.

[0198] The microscope 1d of the second embodiment is different from the microscope 1a of the first embodiment described above in that it is a wide-field epi-fluorescence microscope. The light source 10a emits illumination light IL for illuminating the sample 24. As the light source 10a, for example, a mercury lamp, an LED, or the like is used.

[0199] The collimator lens 12a makes the illumination light IL emitted from the light source 10a into substantially parallel light. The illumination filter 13 transmits at least a part of the illumination light IL radiated from the light source 10 and blocks other light. The field stop 31 is provided at a position conjugate with the focal position of the objective lens 21 and restricts the region where the sample 24 is illuminated. The relay lens 15a guides the illumination light IL that has become substantially parallel light to the dichroic mirror 20 while focusing it. The dichroic mirror 20 has the property that the illumination light IL from the light source 10a is reflected and light in a predetermined wavelength band (for example, fluorescence) among the signal light DL generated by the sample 24 is transmitted.

[0200] The signal light DL emitted from the sample 24 passes through the dichroic mirror 20, passes through the second objective lens 19 and the relay lens 16a, and enters the phase modulation element 14. The signal light DL reflected by the phase modulation element 14 forms an image on the two-dimensional detector 29b. The two-dimensional detector 29b has a function of detecting the signal light DL generated by the sample 24, and for example, a CCD camera, a CMOS camera, or the like is used. Note that the two-dimensional detector 29b is also a detector that detects the signal light DL, and there is no change in this regard.

[0201] The two-dimensional detector 29b performs photoelectric conversion of the incident signal light DL and generates data (signal S1) corresponding to the light amount (brightness) of the light as a detection signal. The control unit 40 generates one image data based on the (signal S1) for a plurality of pixels input from the two-dimensional detector 29b and stores it in the storage unit 42.

[0202] Unlike the microscopes 1a to 1c of the first embodiment and its various modifications described above, in the microscope 1d of the second embodiment, the phase modulation element 14 is disposed not in the light transmission optical system ILO but in the light reception optical system DTO. Specifically, the phase modulation element 14 is disposed substantially in conformity with the pupil plane DPP with respect to the sample 24 in the light reception optical system DTO. Therefore, the phase modulation element 14 imparts a predetermined phase value to the signal light DL, rather than to the illumination light IL.

[0203] Therefore, in the microscope 1d of the second embodiment, the first phase distribution WF1 and the second phase distribution WF2 are not the phase distributions added to the illumination light IL to cancel the aberration caused by the sample 24, but the phase distributions added to the signal light DL to cancel the aberration caused by the sample 24. However, even in this case, the measurement of the first phase distribution WF1 and the calculation of the second phase distribution WF2 can be performed in the same manner as the method described in the observation method of the fourth embodiment described above.

[0204] The microscope 1d of the second embodiment is suitable for use in the observation method of the fourth embodiment described above. However, the acquisition of the image near the sample point F of the sample 24 and the acquisition of the image near the detection point U of the sample 24 are performed by the two-dimensional detector 29b acquiring an image, instead of scanning the condensing point FP of the illumination light IL with respect to the sample 24.

[0205] Since the microscope 1d of the second embodiment images a wide area of the sample 24 at once by the two-dimensional detector 29b, the image of the sample 24 can be acquired at high speed, that is, in a short time.

[0206] (Microscope of the Third Embodiment) Hereinafter, the microscope 1e of the third embodiment will be described with reference to FIG. 17. The configuration of the microscope 1e of the third embodiment has many configurations in common with the microscope 1c of the modification 2 of the first embodiment described above. In the following, the common configurations are denoted by the same reference numerals and the description thereof is appropriately omitted.

[0207] The microscope 1e of the third embodiment is different from the microscope 1c of the second modification of the first embodiment described above in that it has a wavefront sensor 33 for measuring the first phase distribution WF1. And the configuration other than that related to the wavefront sensor 33 is the same as that of the microscope 1c of the second modification of the first embodiment shown in FIG. 11. Since the microscope 1e of the third embodiment measures the first phase distribution WF1 using the wavefront sensor 33, the first phase distribution WF1 can be measured in a short time.

[0208] In the light-receiving optical system DTO of the microscope 1e of the third embodiment, a detachable mirror 32 that can be attached to and detached from the optical path of the signal light DL is provided in front of the detection filter 28 and the relay lens 27c after being branched from the light-transmitting optical system ILO by the dichroic mirror 20. When the signal light DL is reflected by the detachable mirror 32, it reaches the pinhole plate 30 and is detected by the detector 29a. Then, the signal light DL is photoelectrically converted by the detector 29a and sent to the control unit 40 as a signal S1a.

[0209] On the other hand, when the detachable mirror 32 is removed from the optical path of the signal light DL, the signal light DL passes through the pinhole of the second pinhole plate 30a, is made into substantially parallel light by the relay lens 27d, and reaches the wavefront sensor 33. The wavefront sensor 33 measures the wavefront information (phase distribution) of the signal light DL. Then, the wavefront information of the signal light DL is sent to the control unit 40 as a signal S1b. Note that the wavefront sensor 33 is still a detector that detects the signal light DL. Instead of the detachable mirror 32, a beam splitter may be used so that a part of the signal light DL reaches the detector 29a and the rest reaches the wavefront sensor 33.

[0210] The phase distribution measurement unit 46 of the control unit 40 measures the first phase distribution WF1 based on the signal S1b sent from the wavefront sensor 33. However, the phase distribution measurement unit 46 may also measure the first phase distribution WF1 based on the signal S1a sent from the detector 29a. The image generation unit 48 of the control unit 40 forms an image of the sample 24 based on the signal S1a sent from the detector 29a.

[0211] The second pinhole plate 30a not only reduces the incidence of background light on the wavefront sensor 33 but also has a function as a low-pass filter. By blocking the high-order aberration components of the wavefront, the measurement accuracy of the wavefront sensor 33 is improved. As an example of the wavefront sensor 33, a Shack-Hartmann wavefront sensor or the like is used.

[0212] (Observation method of the fifth embodiment) The observation method of the fifth embodiment is a method of observing the sample 24 using the microscope 1e of the third embodiment shown in FIG. 17 described above. Since most of the observation method of the fifth embodiment is the same as the observation method of the first embodiment shown in the flow of FIG. 3 described above, in the following, the same steps will be denoted by the same reference numerals, and the description will be omitted as appropriate.

[0213] FIG. 18 is a diagram showing the flow of the observation method of the fifth embodiment. However, it is the same as the flow of the observation method of the first embodiment shown in FIG. 3 except that step S115 is inserted between step S110 and step S120, and step S185 is inserted between step S180 and step S190.

[0214] That is, in the flow of the observation method of the fifth embodiment, in step S115 after the execution of step S110, the removable mirror 32 is removed from the optical path of the signal light DL. Then, from step S120 to step S160, the signal light DL is received by the wavefront sensor 33, and the first phase distribution WF1 as wavefront data is measured.

[0215] After the execution of step S180, the removable mirror 32 is loaded into the optical path of the signal light DL. Then, in step S190, the signal light DL is received by the detector 29a, and an image of the sample 24 is acquired.

[0216] Note that the calculation of the second phase distribution WF2 for each small region R in step S180 may be performed in the same manner as the observation method of the first embodiment, using the phase values of the portion corresponding to the phase sampling points P shown in FIG. 2A among the first phase distribution WF1 measured by the wavefront sensor 33. Alternatively, the distributions of the slopes in the x and y directions of the wavefront measured by the wavefront sensor 33 may be fitted with functions obtained by differentiating the basis functions in the x and y directions, and the calculation may be performed in the same manner as the observation method of the fourth embodiment based on the obtained coefficient values of the basis functions.

[0217] (Microscope of Modification 1 of the Third Embodiment) Hereinafter, the microscope 1f of Modification 1 of the third embodiment will be described with reference to FIG. 19. Many components of the configuration of the microscope 1f of Modification 1 of the third embodiment are common to the microscope 1e of the third embodiment or the microscope 1b of Modification 1 of the first embodiment described above. Hereinafter, the same reference numerals will be given to the common components and the description will be appropriately omitted.

[0218] In the microscope 1f of Modification 1 of the third embodiment, among the light receiving optical system DTO, the configuration of the portion branched from the light transmitting optical system ILO by the dichroic mirror 20 (detection filter 28, relay lens 27, removable mirror 32, detector 29a, wavefront sensor 33, etc.) is the same as the configuration in the microscope 1e of the third embodiment described above. However, the arrangement position of the dichroic mirror 20 is different from that of the microscope 1e of the third embodiment described above. The dichroic mirror 20 of the microscope 1f of Modification 1 of the third embodiment is arranged between the swing mirror 17 and the relay lens 16, similar to the arrangement position of the dichroic mirror 20 in the microscope 1b of Modification 1 of the first embodiment.

[0219] Therefore, in the microscope 1f of Modification 1 of the third embodiment, the phase modulation element 14 corrects the aberration generated in the illumination light IL by the sample 24, but does not correct the aberration generated in the signal light DL by the sample 24. However, in the microscope 1f, since the signal light DL enters the detector 29a or the wavefront sensor 33 without passing through the phase modulation element 14, there is an advantage that the loss of the signal light DL due to the diffraction or reflectance characteristics of the phase modulation element 14 is small.

[0220] (Microscope of Modification 2 of the Third Embodiment) Hereinafter, the microscope 1g of Modification 2 of the Third Embodiment will be described with reference to FIG. 20. Many components of the configuration of the microscope 1g of Modification 2 of the Third Embodiment are common to the microscope 1f of Modification 1 of the Third Embodiment or the microscope 1a of the First Embodiment described above. In the following description, the same reference numerals will be given to the common components and the description will be omitted as appropriate.

[0221] The microscope 1g of Modification 1 of the Third Embodiment is different from the microscope 1f of Modification 1 of the Third Embodiment in that, as its light-receiving optical system DTO, in addition to the configuration of the light-receiving optical system DTO of the microscope 1f of Modification 1 of the Third Embodiment described above, it also has the configuration of the light-receiving optical system DTO of the microscope 1a of the First Embodiment described above.

[0222] Among the light-receiving optical system DTO of the microscope 1g, the configuration of the light-receiving optical system DTO of the microscope 1f of Modification 1 of the Third Embodiment is a detector 29a or the like that receives the signal light DL reflected by the dichroic mirror 20 disposed between the swing mirror 17 and the relay lens 16. On the other hand, the configuration of the light-receiving optical system DTO of the microscope 1a of the First Embodiment is a detector 29c or the like that receives the signal light DL reflected by the detachable dichroic mirror 20a disposed between the objective lens 21 and the second objective lens 19. Here, the detector 29c is the same detector as the detector 29a in the microscope 1a of the First Embodiment shown in FIG. 3, but is designated as the detector 29c to avoid confusion with the other detector 29a in FIG. 20.

[0223] When the detachable dichroic mirror 20a is inserted between the objective lens 21 and the second objective lens 19, the signal light DL emitted from the sample 24 is reflected by the detachable dichroic mirror 20a and detected by the detector 29c. Then, the signal light DL is photoelectrically converted by the detector 29c and sent to the control unit 40 as the signal S1a2.

[0224] When the removable dichroic mirror 20a is removed from the optical path of the signal light DL, the signal light DL emitted from the sample 24 is reflected by the oscillating mirror 17, reflected by the dichroic mirror 20, and reaches the wavefront sensor 33 or the detector 29a. When the signal light DL is detected by the wavefront sensor 33, the wavefront sensor 33 transmits the wavefront information (phase distribution) of the signal light DL to the control unit 40 as a signal S1b. When the signal light DL is detected by the detector 29a, the detector 29a photoelectrically converts the signal light DL and transmits it to the control unit 40 as a signal S1a1.

[0225] The phase distribution measurement unit 46 of the control unit 40 measures the first phase distribution WF1 based on the signal S1b sent from the wavefront sensor 33. However, the phase distribution measurement unit 46 may measure the first phase distribution WF1 based on the signal S1a sent from the detector 29a or the signal S1a2 sent from the detector 29c. The image generation unit 48 of the control unit 40 forms an image of the sample 24 based on the signal S1a sent from the detector 29a or the signal S1a2 sent from the detector 29c.

[0226] (Observation method of the sixth embodiment) In the observation method of the sixth embodiment, when the above-described field-dependent aberration is expressed using the phase layer model, the phase distribution of the phase layer L is sequentially determined from a shallow position (a position close to the objective lens 21) to a deep position in the z position.

[0227] The observation method of the sixth embodiment can be implemented using any of the microscopes 1a to 1g in the above-described embodiments and each modification. Each step described later in the observation method of the sixth embodiment may be the same as any of the steps in the observation methods of the above-described embodiments. Also, depending on the device configuration of the microscopes 1a to 1g used, a part of the flow may change slightly.

[0228] FIG. 21 is a diagram showing the flow of the observation method according to the sixth embodiment. The process of step S600 is the same as step S100 in the observation method of the first embodiment shown in FIG. 3. In subsequent step S610, a plurality of phase layers L are set. The number NL of the phase layers L to be arranged may be set so that the improvement effect is maximized when corrected by the second phase distribution WF2 according to the observation conditions. The z position of each phase layer L is set to be a z position between the deepest z (the z coordinate having the maximum negative value) in the imaging range IR set in step S160 and z = 0. For example, the z positions may be arranged at equal intervals, or if it is known in advance that there is a z position where the sample 24 has a structure with a large unevenness in the refractive index distribution such as bone, the phase layer L may be set at that z position. The set phase layers L are divided into, for example, NG (a plurality of) groups according to their z positions.

[0229] FIG. 22 is a diagram showing an example of a state in which the LNL phase layers L (L1 to LLN) shown in FIG. 9A are divided into NG groups G (G1 to GNG). In FIG. 22, one group G surrounded by a two-dot chain line is assumed to include two or three phase layers L, but the number of phase layers L included in each group G may be any number. Also, the number of phase layers L included in each group G may be different. Hereinafter, the number of the group G is denoted by the symbol ig, and the ig-th group G is denoted as Gig with the subscript ig attached. Note that a plurality of phase layers L may be set at the same z position and each may belong to a different group Gig.

[0230] A loop for the group Gig of the phase layer L is started in step S620. The loop counter is ig, and the loop up to step S670 is repeated NG times from ig = 1 to ig = NG. In step S630, the positions and sizes of Na phase data measurement regions PA (PA1 to PANa) shown in FIG. 22 are set. The size of one phase data measurement region may be set in advance, for example, or may be changed according to the current ig. The number Na of measurement regions may be set according to the size of the imaging range IR set in step S100, for example. It is desirable that the central z coordinates of the plurality of phase data measurement regions PA be set at positions deeper than the z position of the phase layer Lk+1 at the deepest position belonging to the group Gig. For example, it may be set between the z position of Lk+1 and the z position of the phase layer Lk+2 at the shallowest position belonging to the ig+1-th group Gig+1.

[0231] Alternatively, the central z coordinates of the plurality of phase data measurement regions PA may always be set at the same position during the loop for the group Gig. For example, it may be set at a z position deeper than the z position of the phase layer LNL at the deepest position belonging to the NG-th group GNG shown as the phase data measurement region PB (PB1 to PBNa) in FIG. 22.

[0232] In step S640, the second phase distribution WF2 at the central coordinates of the plurality of phase data measurement regions PA and PB set in step S630 is calculated. Using the phase distributions of the phase layers L for which the phase values have been determined and that belong to the groups from G1 to Gig-1 as model parameters, it is possible to calculate by performing the same processing as in step S180 or step S380 according to the apparatus configurations of the microscopes 1a to 1g in each embodiment and each modification.

[0233] In step S650, the first phase distribution WF1 is measured at a plurality of (NF) sample points F. In step S650, when performing the same observation as the observation methods of the first embodiment, the second embodiment, the fourth embodiment, or the fifth embodiment described above, the sub-flow shown in FIG. 23 is implemented. On the other hand, when performing the same observation as the observation method of the third embodiment described above, the sub-flow shown in FIG. 24 is implemented.

[0234] In step S660, based on the first phase distribution WF1 measured in step S650, model parameters for expressing the phase distribution of the phase layer L are determined. Then, the phase value of the phase layer L belonging to the ig-th group Gig is calculated.

[0235] In the case of the configurations of the microscopes 1a to 1e other than the microscope 1f of the first modification example and the microscope 1g of the second modification example of the third embodiment shown in FIGS. 19 and 20, since aberration calculated from the phase layers L belonging to the groups from group G1 to group Gig-1 is corrected and then phase data is acquired, model parameters related to the phase layer L belonging to group Gig are determined. Specifically, according to the configurations of the microscopes 1a to 1e, the same processing as that for determining the model parameters performed in step S180 or S380 in the observation methods of the first to fifth embodiments described above may be performed.

[0236] On the other hand, in the case of the configurations of the microscope 1f of the first modification example and the microscope 1g of the second modification example of the third embodiment shown in FIGS. 19 and 20, the signal light DL detected by the wavefront sensor 33 is not corrected (phase addition) by the phase modulation element 14 for the aberration generated in the signal light DL by the sample 24. Therefore, when using these microscopes 1f and 1g, the model parameters related to the phase layers L belonging to all of the groups from group G1 to group Gig are determined by performing the same calculation as that for determining the model parameters in step S180 in the observation method of the fifth embodiment described above.

[0237] At this time, the model parameters of the phase layers L belonging to the groups from group G1 to group Gig-1 may be fixed at the already determined values, and only the phase distribution of the phase layer L belonging to group Gig may be obtained. In step S670, the loop regarding the group G of the phase layer L (the loop of ig) ends.

[0238] In step S6680, each small region Rj' within the imaging range IR is set in the same manner as in step S170 of the observation method of the first embodiment described above. Then, in step S690, using the determined phase distribution of the phase layer L belonging to groups G1 to GNG, the second phase distribution WF2 at the three-dimensional central position coordinates (xRj’, yRj’, zRj’) of each small region Rj’ is calculated. The calculation of the second phase distribution WF2 may be performed in the same manner as either step S180 or step S380 described above, depending on the configuration of the microscopes 1a to 1g to be used.

[0239] Subsequently, in step S700, an image of the sample 24 within the imaging range IR is acquired in the same manner as step S190 in the observation method of the first embodiment or step S390 in the observation method of the third embodiment described above.

[0240] (An example of the measurement of the first phase distribution in step S650 of the observation method of the sixth embodiment) Hereinafter, with reference to FIG. 23, an example of the flow of the measurement of the first phase distribution WF1 of a plurality of sample points F in step S650 of the observation method of the sixth embodiment will be described.

[0241] First, in step S641, sample points F are set inside each of the plurality of phase data measurement regions PA set in step S630. A plurality of sample points F may be set inside one phase data measurement region PA, or one sample point F may be set, or no sample point F may be set. The sample points F may be set, for example, by scanning within the imaging range IR of the sample 24 to acquire the signal light DL emitted from the sample 24, and setting a plurality of points where the intensity of the signal light DL is strong.

[0242] In step S642, a loop regarding the sample points F is started. Then, for the processing up to step S646 described later, with respect to the subscript j of the sample point Fj, a loop of FN times from j = 1 to j = NF is repeated. In step S643, the control unit 40 operates the microscope 1a so that the illumination light IL is focused on the sample point Fj. The specific operation method is the same as the operation in step S130 in the first observation method described above.

[0243] In step S644, the second phase distribution WF2 calculated in step S640 is set in the phase modulation element 14 for the phase data measurement region PA to which the sample point Fj belongs.

[0244] In step S645, according to the device configurations of the microscopes 1a to 1g to be used, the first phase distribution WF1 is measured by performing the process of step S140 described above. The measured first phase distribution WF1 is held in the storage unit 42 in association with the coordinates of the sample point F. In step S646, the loop regarding the sample point F ends.

[0245] (Another example of the measurement of the first phase distribution in step S650 of the observation method of the sixth embodiment) Hereinafter, with reference to FIG. 24, another example of the flow of the measurement of the first phase distribution WF1 of a plurality of sample points F in step S650 of the observation method of the sixth embodiment will be described.

[0246] First, in step S641a, a loop for the plurality of phase data measurement regions PAa set in step S630 is started. Then, regarding the subscript a of the phase data measurement region PAa, the loop of Na times from a = 1 to a = Na is repeated for the processes up to step S648a described later.

[0247] In step S642a, the a-th phase data measurement region PAa is set. That is, the position of the focusing point FP is set to the position of the a-th phase data measurement region PAa in the sample 24. In the subsequent step S643a, the second phase distribution WF2 calculated in step S640 is set in the phase modulation element 14 for the a-th phase data measurement region PAa. In subsequent step S644a, a loop for the phase modulation pattern added by the phase modulation element 14 to the illumination light IL or the signal light DL is started. Let the loop counter be t, and the loop until step S646a is repeated Nt times from t = 1 to t = Nt.

[0248] In step S645a, a phase modulation pattern that is the sum of the t-th phase modulation pattern similar to step S340 in the observation method of the third embodiment and the phase modulation pattern of the second phase distribution WF2 set in step S643a is set as the phase value of the phase modulation element 14. Then, in step S646a, the a-th phase data measurement region PAa is scanned to acquire a signal. That is, the control unit 40 relatively moves the condensing point FP inside the a-th phase data measurement region PAa of the sample 24 to acquire the signal S1 obtained by photoelectrically converting the signal light DL, and stores it in the storage unit 42. Note that the phase modulation pattern of the phase modulation element 14 may be changed in synchronization with the scanning of the sample to a phase modulation pattern that is the sum of a plurality of previously calculated second phase distributions and the t-th phase modulation pattern for each scan position. In step S647a, the loop for the phase modulation pattern ends.

[0249] In step S648a, a plurality of sample points F for calculating the first phase distribution WF1 are determined in the same manner as step S350 described above, and the first phase distribution WF1 at each sample point F is calculated based on the t-th phase modulation pattern set in step S646a at each t from t = 1 to Nt, and the Nt signals obtained in step S646a. Then, the control unit 40 stores the calculated first phase distribution WF1 at each sample point F in the storage unit 42 in correspondence with the coordinates of each sample point F.

[0250] In step S649a, the loop for the phase data measurement region PA ends. Let the total number of determined sample points F be NF, and proceed to the next process of the main flow, that is, step S660 of the flow in FIG. 21.

[0251] Generally, in the deep part of the sample 24, the deterioration of the imaging performance due to the aberration caused by the sample 24 is large. Therefore, the detailed structure of the sample 24 cannot be observed, and it may not be possible to select an appropriate sample point F for measuring the first phase distribution WF1, or the measurement error of the first phase distribution WF1 may become large.

[0252] In the observation method of the sixth embodiment, for each loop of the group ig of the phase layers L, the imaging performance when the second phase distribution WF2 is set is improved, and the measurement accuracy of the first phase distribution WF1 can be improved. Further, the first phase distribution WF1 is measured at the sample point F in the shallow region of the sample 24, and the phase distribution of the phase layer L virtually arranged at the shallow z position of the sample 24 is determined. Then, when measuring the first phase distribution WF1 of the sample point F at the deep position of the sample 24, the imaging performance is improved by correcting the aberration caused by the sample 24 up to the intermediate depth of the sample 24, an appropriate sample point F can be selected, and the measurement accuracy of the first phase distribution WF1 can also be improved.

[0253] In the microscopes 1a to 1g of the above-described embodiments and each modification, the measurement method of the first phase distribution WF1 is not limited to the above-described method. For example, the phase distribution at the pupil planes IPP and DPP of the illumination light IL or the signal light DL measured by the methods disclosed in the following documents can also be used as the first phase distribution WF1.

[0254] As an example of such a document, there is the WO2011 / 006106 publication (corresponding to Japanese Patent Application Laid-Open No. 2015-092253). As another example, there is "Adaptive wavefront correction in two-photon microscopy using coherence-gated wavefront sensing PNAS", 103 (46) 17137-17142 (2006). As still another example, there is "Scattering compensation by focus scanning holographic aberration probing (F-SHARP)", Nature Photonics, 11, 116-123 (2017).

[0255] As yet another example of such a document, there is "Adaptive optics via self-interference digital holography for non-scanning three dimensional imaging in biological samples", Biomedical Optics EXPRESS, Vol. 9, No. 6 (2018).

[0256] Note that, as the light source 10 used in the microscopes 1a to 1g of each embodiment and modification, a plurality of light sources may be used, and illumination light emitted from different light sources may be used at the time of measuring the first phase distribution WF1 and at the time of acquiring (imaging) an image of the sample 24.

[0257] In the microscopes 1a to 1g of each embodiment and modification, the signal light DL emitted from the sample 24 does not necessarily have to be detected after passing through the same objective lens 21 as the illumination light IL. For example, the signal light DL may be detected through another detection objective lens rotated at an arbitrary angle with respect to the illumination objective lens that irradiates the illumination light IL.

[0258] Alternatively, as the signal S1, the intensity of the probe light irradiated onto the sample 24 separately from the illumination light IL passing through the phase modulation element 14 may be detected. In this case, the first phase distribution WF1 can be measured based on the change in the detected intensity of the probe light. Also, the detected signal is not limited to the signal light DL, and may be, for example, an acoustic signal generated by a photoacoustic effect when the illumination light IL is irradiated onto the sample 24. In this case, the signal S1 is, for example, a signal obtained by converting the acoustic wave detected by an acoustic wave transducer into an electrical signal.

[0259] The microscopes 1a to 1g of each embodiment and modification example and the field-dependent aberration representation model in the observation method of each embodiment are not limited to the above-described phase layer model. For example, a field-dependent aberration may be represented using a basis function system having as arguments the coordinates of both the pupil plane and the image plane, as described in the document "Orthogonal Aberration Functions for Microlithographic Optics", (OPTICAL REVIEW Vol. 11, No. 4 (2004) 199-207). Further, a model considering the diffraction of light using the refractive index distribution of the sample as a model parameter may be used.

[0260] The second phase distribution WF2 calculated in each of the above-described embodiments may be used for aberration correction of one or both of the illumination light IL to the sample 24 and the signal light DL from the sample 24 in known super-resolution microscopy methods such as localization microscopy (PALM, STORM), structured illumination microscopy (SIM), Image scanning microscopy (ISM), stimulated emission depletion microscopy (STED), saturated excitation microscopy (SAX), and known fluorescence lifetime imaging microscopy (FLIM).

[0261] The second phase distribution WF2 calculated in each of the above-described embodiments may be used for purposes other than correcting the aberration caused by the sample 24 during imaging of the sample 24. For example, it may be used for correcting the aberration with respect to the stimulation light when performing optical stimulation inside the sample 24. Further, the PSF in a plurality of field regions may be calculated using the second phase distribution WF2, and this may be used for deconvolution processing of an image obtained without correcting the aberration caused by the sample 24. Further, among the second phase distribution WF2, only some aberration components may be corrected by the phase modulation element 14 to obtain an image, and deconvolution processing using the PSF calculated based on the remaining aberration components may be performed on the obtained image.

[0262] (Effects of the microscopes of each embodiment and each modification) (1) The microscopes 1a to 1g of each of the embodiments and each of the modifications described above include a light transmission optical system ILO that irradiates the sample 24 with illumination light IL from the light source 10, a light reception optical system DTO that receives the signal light DL emitted from the sample 24, and a phase modulation element 14 that is provided in at least one of the light transmission optical system ILO or the light reception optical system DTO and adds a predetermined phase distribution to the illumination light IL or the signal light DL. And it includes a phase distribution measurement unit 46 that measures the first phase distribution WF1 corresponding to the aberration caused by the sample 24 at each sample point F of the sample 24 at a plurality of sample points F. Further, based on the first phase distribution WF1 for each of the plurality of measured sample points F, a phase data model indicating the amount of phase change received when the illumination light IL or the signal light DL passes through a predetermined position in the sample 24 is created, and based on the phase data model, at least one detection point U of the sample 24 is detected in a state where the aberration caused by the sample 24 is reduced. It includes a phase distribution calculation unit 47 that calculates the second phase distribution WF2 that the phase modulation element 14 should add to at least one of the illumination light IL or the signal light DL. And it includes a phase distribution setting unit 49 that sets the second phase distribution WF2 in the phase modulation element 14. With this configuration, it is possible to shorten the time required for measuring the first phase distribution WF1 for correcting the aberration caused by the sample 24 and calculating the second phase distribution WF2, and it is possible to shorten the observation time of the sample 24. In addition, since it is possible to shorten the measurement time of the first phase distribution WF1 corresponding to the aberration caused by the sample 24, it is possible to reduce the fading of the sample 24 and reduce the phototoxicity to the sample 24. As a result, the sample 24 can be observed more accurately.

[0263] (Effects of the observation methods of each embodiment and each modification) (2) The observation methods of each of the embodiments and each of the modifications described above are observation methods for irradiating the sample 24 with the illumination light IL from the light source 10, detecting the signal light DL emitted from the sample 24, and observing the sample 24, and measuring the first phase distribution WF1 corresponding to the aberration caused by the sample 24 at a plurality of sample points F of the sample 24, respectively. Then, based on the first phase distribution WF1 for each of the plurality of measured sample points F, a phase data model indicating the amount of phase change received when the illumination light IL or the signal light DL passes through a predetermined position in the sample 24 is created, and based on the phase data model, at least one detection point U of the sample 24 is calculated for at least one of the illumination light IL or the signal light DL in order to detect the detection point U of the sample 24 in a state where the aberration caused by the sample 24 is reduced. calculating the second phase distribution WF2 to be added; Further, it includes adding the second phase distribution WF2 to at least one of the illumination light IL or the signal light DL and detecting the detection point U. With this configuration, it is possible to shorten the time required for measuring the first phase distribution WF1 for correcting the aberration caused by the sample 24 and calculating the second phase distribution WF2, and it is possible to shorten the observation time of the sample 24. In addition, since it is possible to shorten the measurement time of the first phase distribution WF1 corresponding to the aberration caused by the sample 24, it is possible to reduce the fading of the sample 24 and reduce the phototoxicity to the sample 24. As a result, the sample 24 can be observed more accurately.

[0264] (Embodiment of the program) As described in the explanations of microscopes 1a to 1g in each embodiment and each modification, the program of the embodiment is stored in the storage unit 42 of the control unit 40 and executed by the arithmetic unit 41, thereby controlling microscopes 1a to 1g via each part of the control unit 40. The program causes the microscopes 1a to 1g to perform predetermined operations via the arithmetic unit 41 according to each flow described in the observation method of each of the above-described embodiments.

[0265] As an example, the program causes the phase distribution measurement unit 46 to measure the first phase distribution WF1 via the arithmetic unit 41, and causes the phase distribution calculation unit 47 to calculate the second phase distribution WF2. Then, the program causes the phase modulation element 14 to set the second phase distribution WF2 via the arithmetic unit 41 and the phase distribution setting unit 49.

[0266] A program for realizing the above functions may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read from an optical drive 43 or the like into the control unit 40 as a computer system and executed.

[0267] Here, the "computer system" is assumed to include an OS (Operating System) and the hardware of peripheral devices. Further, the "computer-readable recording medium" refers to a portable recording medium such as a flexible disk, a magneto-optical disk, an optical disk, a memory card, or a storage device such as a hard disk built in a computer system.

[0268] Furthermore, the "computer-readable recording medium" may include those that dynamically hold a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, and those that hold a program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or a client in that case. Also, the above program may be for realizing a part of each of the aforementioned functions, and furthermore, it may be for realizing each of the aforementioned functions in combination with a program already recorded in a computer system.

[0269] Also, the above program can be provided through a data signal such as the Internet. For example, the control unit 40 including the arithmetic unit 41 and the storage unit 42 shown in each figure such as Figure 1 has a connection function (interface unit IF) with the network line NW. An external server (not shown) connected to the network functions as a server computer that provides the above program and transfers the program to a recording medium such as the storage unit 42. That is, the program is carried by a carrier wave as a data signal and transmitted via the network cable NW. In this way, the program can be supplied as a computer-readable computer program product in various forms such as a recording medium and a carrier wave.

[0270] In the above, various embodiments and modifications have been described, but the present invention is not limited to these contents. Also, each embodiment and modification may be applied alone or in combination. Other aspects conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention. Also, to the extent permitted by law, the disclosures of various technical documents and patent documents cited in the specification are incorporated by reference as part of the description of the present text.

Explanation of Reference Numerals

[0271] 1a~1g: Microscope, ILO: Light delivery optical system, DTO: Light receiving optical system, 10, 10a: Light source, 11: Shutter, 12: Collimator lens, 13: Illumination filter, 14: Phase modulation element, 15, 16, 18: Relay lens, 17: Swing mirror, 19: Second objective lens, 20: Dichroic mirror, 21: Objective lens, 22: Objective lens holder, 23: Immersion liquid, 24: Specimen, 25: Stage, 26: Stage drive unit, 27a, 27b: Relay lens, 28: Detection filter, 29a, 29b, 29c: Detector, 40: Control unit, 41: Arithmetic unit, 42: Memory unit, 43: Optical drive, 44: Input unit, 45: Display unit, 46: Phase distribution measurement unit, 47: Phase distribution calculation unit, 48: Image generation unit, 49: Phase distribution setting unit

Claims

1. A light transmission optical system that irradiates a sample with illumination light from a light source, A light reception optical system that receives signal light emitted from the sample, A phase modulation element provided in at least one of the light transmission optical system or the light reception optical system, which adds a predetermined phase distribution to the illumination light or the signal light, A phase distribution measurement unit that measures, at each of a plurality of sample points of the sample, a first phase distribution corresponding to an aberration generated by the sample, Based on the first phase distribution for each of the plurality of measured sample points, a phase data model indicating the amount of phase change received when the illumination light or the signal light passes through a predetermined position in the sample is created, and based on the phase data model, at least one detection point of the sample is detected in a state where the aberration generated by the sample is reduced. A phase distribution calculation unit that calculates a second phase distribution to be added to at least one of the illumination light or the signal light by the phase modulation element, A phase distribution setting unit that sets the second phase distribution in the phase modulation element, and For each of the first phase distributions of the plurality of sample points, the phase distribution measurement unit sequentially applies a plurality of phase modulation patterns to the phase modulation element, and each time the phase modulation pattern is applied, the illumination light is irradiated by the light transmission optical system at least to the sample point that is the measurement target of the first phase distribution, and is measured based on the signal light received by the light reception optical system. A microscope.

2. In the microscope according to Claim 1, A microscope in which the amount of information characterizing the second phase distribution for one of the detection points is larger than the amount of information characterizing the first phase distribution for one of the sample points.

3. In the microscope according to Claim 1 or Claim 2, The phase distribution measurement unit measures the first phase distribution for each of a plurality of sample points separated in the in-plane direction of a first plane that intersects at least a first direction in which the illumination light is irradiated to the sample. A microscope.

4. In the microscope according to Claim 1, The phase modulation element does not change the phase of a partial region of either the illumination light or the signal light during the measurement of the first phase distribution. A microscope.

5. In the microscope according to any one of Claims 1 to 4, The phase modulation element is provided in the light transmission optical system, The light transmission optical system condenses the illumination light and irradiates the sample, and A microscope comprising a scanning unit that relatively scans a condensing point where the illumination light is condensed and the sample.

6. In the microscope according to claim 5, During the measurement of the first phase distribution, the phase distribution added to the illumination light or the signal light by the phase modulation element and the scanning of a predetermined region of the sample are alternately performed, A microscope that selects the sample point from among the predetermined regions.

7. In the microscope according to claim 5 or claim 6, The phase distribution setting unit sets the second phase distribution to the phase modulation element in synchronization with the scanning. A microscope.

8. In the microscope according to any one of claims 1 to 7, The second phase distribution is set to the phase modulation element, and the first phase distribution is measured at a plurality of the sample points. A microscope.

9. An observation method for irradiating a sample with illumination light from a light source, detecting signal light emitted from the sample, and observing the sample, Measuring a first phase distribution corresponding to an aberration generated by the sample at a plurality of sample points of the sample, respectively, Based on the first phase distribution for each of the plurality of measured sample points, a phase data model indicating a phase change amount received when the illumination light or the signal light passes through a predetermined position in the sample is created, and based on the phase data model, at least one detection point of the sample is detected in a state where the aberration generated by the sample is reduced. Calculating a second phase distribution to be added to at least one of the illumination light or the signal light, Adding the second phase distribution to at least one of the illumination light or the signal light and detecting the detection point, Comprising, For each of the plurality of sample points, a plurality of phase modulation patterns are sequentially applied to a phase modulation element that adds a predetermined phase distribution to the illumination light or the signal light. Each time the phase modulation pattern is applied, at least the sample point that is the measurement target of the first phase distribution is irradiated with the illumination light and measured based on the detected signal light. Observation method.

10. In the observation method according to claim 9, The amount of information characterizing the second phase distribution for one of the detection points is greater than the amount of information characterizing the first phase distribution for one of the sample points. Observation method.

11. In the observation method according to claim 9 or claim 10, An observation method for measuring the first phase distribution, wherein for each of a plurality of sample points spaced apart in the in-plane direction of a first plane that intersects at least the first direction in which the illumination light is irradiated onto the sample, the first phase distribution is measured.

12. In the observation method according to claim 11, An observation method in which during the measurement of the first phase distribution, a phase is not changed for a partial region of either the illumination light or the signal light.

13. In the observation method according to any one of claims 9 to 12, Both the first phase distribution and the second phase distribution are added to the illumination light, and The illumination light is condensed and irradiated onto the sample, and the condensing point where the illumination light is condensed and the sample are relatively scanned. An observation method.

14. In the observation method according to any one of claims 9 to 13, The second phase distribution is added to at least one of the illumination light or the signal light, and the first phase distribution is measured at a plurality of the sample points. An observation method.

15. In a program for controlling at least a part of a microscope system for observing a sample, A processing device including a computer is caused to read position information of a plurality of sample points of the sample and information regarding a first phase distribution corresponding to an aberration signal light generated in the illumination light or the signal light by the sample at each of the sample points, Based on the position information of the plurality of sample points and the information regarding the first phase distribution of the plurality of sample points, the processing device creates a phase data model indicating a phase change amount received when the illumination light or the signal light passes through a predetermined position in the sample, and based on the phase data model, calculates a second phase distribution to be added to at least one of the illumination light or the signal light in order to detect at least one detection point of the sample in a state where aberration generated by the sample is reduced. A program, For the first phase distribution of each of the plurality of sample points, a plurality of phase modulation patterns are sequentially applied to a phase modulation element that adds a predetermined phase distribution to the illumination light or the signal light. Each time the phase modulation pattern is applied, the illumination light is irradiated onto at least the sample points that are the measurement targets of the first phase distribution, and the measurement is performed based on the detected signal light.

Images