Scanning observation device

Abstract

To provide a scanning observation device that facilitates the matching of image quality between phase-contrast images and Raman images. [Solution] The scanning observation device comprises a first emission optical system 10 that emits a first light containing a spatially modulated component, a second emission optical system 20 that emits a coherent and temporally modulated second light, a multiplexing unit 15 that combines the first light and the second light, a scanning unit 30 that synchronously scans the combined first light and the second light, a first objective lens 40 that focuses the scanned first light and the second light, a placement unit 50 that places the sample at the focusing position of the first objective lens, a second objective lens 45 that collects secondary light from the sample, and a detection unit 70 that detects the secondary light. The signal from the detection unit 70 is separated into a first signal corresponding to the nonlinear photothermal effect generated in the sample by irradiation with the second light and a second signal that does not include the nonlinear photothermal effect, and an image is formed based on at least one of the signals.

Description

Technical Field

[0001] The present invention relates to a scanning observation device.

Background Art

[0002] In recent years, in regenerative medicine, intraoperative rapid diagnosis, bioproduction, etc., there has been an increasing expectation for methods of observing or evaluating living cells without staining them. As methods for imaging living cells without staining and without contact, there are a method of detecting the phase difference of light transmitted through cells and a method of detecting Raman scattered light generated due to the vibration of molecules constituting cells.

[0003] In Patent Document 1, a device capable of observing living cells using both a phase contrast imaging method for detecting a phase difference as the intensity of light and a CARS imaging method for detecting anti-Stokes Raman scattered light generated by a non-linear optical effect is disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In Patent Document 1, phase-contrast images are obtained by illuminating the entire observation area at once and capturing the image with a CCD camera, while CARS images are obtained by focusing the illumination light on a single point in the observation area and scanning it. In other words, different imaging methods and different image-forming means were used. As a result, the primary images obtained by each imaging method had inconsistent image quality, such as different observation areas and different pixel counts. Therefore, in order to unify the image quality, operators had to perform cumbersome pre-calibration of the equipment and post-processing of the images. Thus, there was a need to reduce the burden on operators who wanted to observe living cells using multiple methods.

[0006] The object of the present invention is to provide a scanning observation device that can easily perform matching of phase contrast images and Raman images. [Means for solving the problem]

[0007] A scanning observation apparatus according to an embodiment of the present invention includes: a first emission optical system that emits a first light containing a spatially modulated component; a second emission optical system that emits a coherent and temporally modulated second light; a multiplexing unit that combines the first light and the second light; a scanning unit that synchronously scans the combined first light and the second light; a focusing lens that focuses the scanned first light and the second light, respectively; a placement unit on which a sample is placed at the focusing position of the focusing lens; and the above-described placement unit. The device is characterized by comprising: a light-collecting lens located on the opposite side of the focusing lens and collecting secondary light from the sample; a detection unit that detects the secondary light collected by the light-collecting lens; a separation unit that temporally demodulates the signal from the detection unit and separates it into a first signal corresponding to the nonlinear photothermal effect generated in the sample by the irradiation of the second light and a second signal that does not include the nonlinear photothermal effect; and an image forming unit that forms an image based on scanning information related to the scanning unit and at least one of the first signal and the second signal. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide a scanning observation device that can easily perform matching of phase contrast images and Raman images. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram illustrating the schematic connection relationships of the elements constituting the scanning observation device according to the first embodiment of the present invention. [Figure 2] This is a schematic diagram illustrating the configuration of a scanning observation device according to the first embodiment. [Figure 3] This is a schematic diagram illustrating the phase difference observation system of a scanning observation apparatus according to the first embodiment. [Figure 4] This is a schematic diagram illustrating the SRP induction system of a scanning observation device according to the first embodiment. [Figure 5] This is a schematic diagram illustrating the light irradiation timing of the scanning observation apparatus according to the first embodiment, for multiple measurement cycles (A) and one measurement cycle (B)(C). [Figure 6] This diagram illustrates the method for generating SRP images and phase-contrast images. [Figure 7] This figure illustrates the observation flow (A), the relationship between the region of interest and the observation field (B), and the schematic configuration of the scanning condition determination unit (C) using a scanning observation device according to the first embodiment. [Figure 8] This is a schematic diagram illustrating the configuration of a scanning observation device according to the second embodiment. [Figure 9] This is a schematic diagram illustrating the relationship between the phase plate and the transmitted light beam in the phase difference detection unit according to the second embodiment. [Figure 10] This is a schematic diagram illustrating the configuration of a scanning observation device according to the third embodiment. [Figure 11] This is a schematic diagram illustrating the configuration of a scanning observation device according to the fourth embodiment. [Figure 12] This figure illustrates an observation flow using a scanning observation device according to the fourth embodiment. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the drawings.

[0011] <First Embodiment> The configuration of the scanning observation device according to the first embodiment will be explained using Figures 1 to 4.

[0012] The scanning observation device 1 according to this embodiment acquires a Raman signal by detecting the stimulated Raman photothermal effect (SRP) using phase difference. This Raman signal is sometimes referred to as the SRP signal. A plot showing the wavenumber dependence of the SRP signal intensity (sometimes simply called the SRP intensity) is called the SRP spectrum, and a mapping of the two- or three-dimensional spatial distribution of the SRP signal intensity is called the SRP image. The SRP spectrum and SRP image are sometimes referred to as the Raman spectrum and Raman image, respectively. The stimulated Raman photothermal effect is a phenomenon in which the refractive index of a medium changes due to the heat generated by the relaxation of molecular vibrations associated with stimulated Raman scattering (SRS). (Yifan Zhu et al., Stimulated Raman photothermal microscopy toward ultrasensitive chemical imaging. Sci. Adv.9, eadi2181 (2023).) Figure 1 is a block diagram illustrating the schematic connection relationships of the elements constituting the scanning observation device 1. In Figure 1, solid lines connecting blocks (rectangles) represent the optical connections of corresponding components, and dashed lines connecting blocks (rectangles) represent connections that allow for the transmission of measurement or control signals between corresponding components.

[0013] The scanning observation device 1 includes an optically connected first emission optical system 10 (phase difference light source unit), a second emission optical system 20 (SRS light source unit), a multiplexer 15, a scanning unit 30, a first objective lens 40 (condensing lens), and a placement unit 50. The scanning observation device 1 further includes a second objective lens 45 (light collection lens), a relay unit 60, a demultiplexer 75, and a first detection unit 70 (phase difference detection unit). In addition, the scanning observation device 1 includes a control unit 90. The control unit 90 is communicably or controllably connected to the first emission optical system 10 (phase difference light source unit), the second emission optical system 20 (SRS light source unit), the scanning unit 30, the placement unit 50, the relay unit 60, and the first detection unit 70 (phase difference detection unit).

[0014] In the scanning observation device 1, the first emission optical system 10 (phase difference light source unit) forms the first emission optical system, and the second emission optical system 20 (SRS light source unit) forms the second emission optical system. Also, the first detection unit 70 (phase difference detection unit) forms the first detection unit. Note that the scanning observation device may be equivalently referred to as an observation device, a scanner, a microscope, or a microscopic observation device, and the scanning type may be equivalently referred to as a point scanning type or a spot scanning type.

[0015] FIG. 2 is a schematic diagram for explaining the configuration of the scanning observation device 1.

[0016] (Placement unit) The placement unit 50 includes a stage 511 and a stage scanner 512. The stage 511 supports a sample 501, which is an observation target of the scanning observation device 1, such that at least a part of the sample 501 overlaps with the condensing position of the first objective lens 40 described later. The placement unit 50 may be equivalently referred to as a support unit 50 for the sample 501 or a sample stage 50. Also, the condensing position of the first objective lens 40 may be equivalently referred to as the focal point of the first objective lens 40, the condensing point of the first objective lens, etc.

[0017] The stage 511 has a portion that is optically opened so that the incidence of primary light and the emission of secondary light with respect to the sample 501 are possible. The aperture can adopt either a form that is closed in the circumferential direction or a form that is not closed.

[0018] This configuration ensures that light from the first objective lens 40 illuminates the sample 501 placed on the stage 511. The stage scanner 512 is coupled to the stage 511. The stage scanner 512 moves the stage 511 parallel to the surface on which the sample 501 is placed. This allows the operator to easily move the part of the sample 501 they wish to observe to the observation area of ​​the scanning observation device 1 after placing the sample 501. The stage scanner 512 also moves the stage 511 perpendicular to the surface on which the sample 501 is placed. This allows the operator to easily focus on the sample 501 and also observe the sample 501 in three dimensions.

[0019] (First objective lens and second objective lens) The first objective lens 40 and the second objective lens 45 are each equipped with objective lenses 401 and 451. The objective lenses 401 and 451 are positioned on opposite sides of the stage 511. The objective lenses 401 and 451 are positioned to share a focal plane P501. This focal plane P501 may also be referred to as the sample plane P501. Furthermore, the objective lenses 401 and 451 may be said to have a region in the optical axis direction where their depths of focus overlap. The focal plane P501 may also be said to be included in this region where the depths of focus overlap. In this case, the pupil planes P401 and P451 of objective lenses 401 and 405 are in a conjugate positional relationship. This makes it possible to suppress the superposition of artifacts on the acquired phase difference signal, SRP image, and SRP spectrum, as well as a decrease in detection sensitivity. It is desirable that the objective lenses 401 and 451 have the same on-axis and off-axis focal points at all wavelengths from the visible to the near-infrared region. Therefore, it is desirable that on-axis and lateral chromatic aberrations are sufficiently corrected. This suppresses a decrease in the detection sensitivity of the SRP signal. Objective lenses 401 and 451 may be referred to as the condensing lens 401 and the light-gathering lens 451.

[0020] (Phase difference light source) The first emission optical system 10 (phase difference light source unit) includes an LED 101, a collimator 102, a ring slit 103, relay lenses 104 and 106, and a pinhole 105. The LED 101 emits incoherent light of a predetermined wavelength. The LED 101 emits light of a predetermined wavelength in the visible light range. It is desirable that the LED 101 can be considered as a point light source. The LED 101 may, for example, be optically coupled to one end of a multimode fiber and emit visible light from the other end of the multimode fiber. The LED 101 may generate light pulses on the order of picoseconds or nanoseconds, or it may generate continuous light. However, when the LED 101 emits pulsed light, its emission timing is synchronized with the pulsed light emission timing of the pulsed lasers 201 and 211 of the second emission optical system, which will be described later. In addition, LDs and other various lasers can be used instead of the LED 101. In this case, in order to suppress noise components in the acquired image resulting from high coherence, such as speckle noise, an optical element that brings the light as close as possible to incoherent light may be inserted into the light path after emission. The ring slit 103 has a transmissive portion through which light passes as an annular light beam, and a light-shielding portion that shields the periphery of the transmissive portion and the center of the ring. The ring slit 103 is positioned at a position P103 conjugate to the pupil plane P401 of the objective lens 401. The pinhole 105 is positioned at a position P105 conjugate to the focal plane P501 of the objective lens, which will be described later. The ring slit 103 may be referred to as a ring aperture 103, annular slit 103, or circular annular slit 103. The ring slit 103 can be described as a modulation optical element having an annular portion that modulates the intensity and phase components of an incoherent first primary light. Furthermore, the first emission optical system 10 (phase difference light source section) can be rephrased as comprising a ring slit 103 as a modulation optical element having an annular portion that modulates the intensity and phase components of an incoherent first primary light.

[0021] The first emission optical system 10 (phase difference light source) can be rephrased as an illumination optical system configured to emit first primary light containing spatially modulated components.

[0022] (Phase difference detection unit) The first detection unit 70 (phase difference detection unit) comprises relay lenses 702, 704, a pinhole 703, a phase plate 705, a tube lens 706, and a photodetector 701. The pinhole 703 is positioned at a position P703 conjugate to the focal plane P501 of the light-gathering lens 451. The phase plate 705 consists of an annular portion corresponding to the ring slit 103 and the remaining portion. The annular portion has a waveplate that shifts the phase of light by 1 / 4 relative to the remaining portion and an ND filter that attenuates light. The phase plate 705 is positioned at a position P705 conjugate to the pupil plane P451 of the light-gathering lens 451. In other words, the phase plate 705 is a demodulating optical element that demodulates the intensity component of the first secondary light from the sample 501 in an annular manner. Furthermore, the first detection unit 70 (phase difference detection unit) is said to include a phase plate 705 as a demodulating optical element that demodulates the intensity component of the first secondary light from the sample 501 in an annular shape. The light-receiving surface of the photodetector 701 is positioned on the focal plane P701 of the tube lens 706 and is conjugate to the focal plane P501 of the light-gathering lens 451. The focal plane P701 may be referred to as the light-receiving surface P701. The photodetector 701 may include, for example, a photodiode or a photomultiplier tube. The intensity of the visible light received by the photodetector 701 is acquired by the control unit 90 as a phase difference signal. It is desirable that the sampling rate of the signal output by the photodetector 701 is higher than the pixel rate, and that the response characteristics of the photodetector 701 be fast relative to the sampling rate. The annular portion of the phase plate 705 in this embodiment may be referred to as the circular portion. The first detection unit 70 (phase difference detection unit) is provided with a pinhole 105 that blocks a portion of the secondary light, and a pair of relay lenses 104 and 106 positioned on either side of the pinhole 105, in the optical path between the reverse scanning unit 60 and the first photodetector 701.

[0023] (Second ejection optical system) The scanning observation apparatus 1 according to this embodiment includes pulsed lasers 201 and 211 as pulsed light sources that are optically coupled to a second emission optical system and emit two pulsed lights (two types of pulsed lights) with different and synchronized oscillation wavelengths. These two pulsed lights include Stokes light and pump light, which exhibit a nonlinear optical effect on the sample 501.

[0024] The second emission optical system 20 (SRS light source unit) includes pulse lasers 201 and 211 as a pair of pulse light sources that emit a pair of pulse light trains with different and synchronized oscillation wavelengths. Corresponding to each of these pair of pulse light sources are relay lenses 202, 203, mirror 204, and relay lenses 212, 213. The pair of pulse light trains correspond to a Stokes light train and a pump light train that exhibit a nonlinear optical effect on the sample. The Stokes light train and pump light train may be referred to as a Stokes light pulse train and a pump light pulse train. The second emission optical system 20 further includes a dichroic mirror 214 that combines the Stokes light train and the pump light train. Furthermore, the second emission optical system 20 (SRS light source unit) includes photoacoustic modulators 205 and 215 that periodically modulate the intensity of the pulse light emitted from pulse lasers 201 and 211, respectively. In addition, the second emission optical system 20 (SRS light source unit) includes a laser synchronization detection unit 221 that detects the synchronization of the emission timing of pulsed light from pulsed lasers 201 and 211. Therefore, the second emission optical system 20 (SRS light source unit) also includes beam splitters 222 and 223 that reflect a portion of the light emitted from pulsed lasers 201 and 211, respectively, and guide it to the pulse synchronization detection unit 221. The second emission optical system 20 (SRS light source unit) can be replaced with a configuration that includes at least one of photoacoustic modulators 205 and 215 to periodically modulate the intensity of at least one of the pulsed light trains emitted from pulsed lasers 201 and 211. The photoacoustic modulators 205 and 215 may be referred to as modulation units 205 and 215.

[0025] As pulse lasers 201 and 211, for example, mode-locked picosecond titanium-sapphire lasers, mode-locked picosecond neodymium lasers, mode-locked picosecond ytterbium lasers, etc., can be used. The pulse duration of the pulses output by pulse lasers 201 and 211 may be femtoseconds. One of the pulse lasers 201 and 211 may be replaced with an optical parametric oscillator that converts the wavelength of the pulse laser generated by the other pulse laser.

[0026] Of the pulsed light emitted from pulsed lasers 201 and 211, the light with the shorter wavelength is used as a pump light for SRP induction, and the light with the longer wavelength is used as Stokes light for SRP induction.

[0027] The pulsed lasers 201 and 211 are adjusted so that the repetition frequencies of pulsed light emission are the same. The synchronization of the pump light and Stokes light is detected by the pulse synchronization detection unit 221. Based on the synchronization signal detected by the pulse synchronization detection unit 221, the control unit 90 controls, for example, the resonator length of one or both of the pulsed lasers 201 and 211 to maintain the synchronized state of the pump light and Stokes light for a sufficiently long time relative to the observation time.

[0028] The photoacoustic modulators 205 and 215 operate as optical switches that turn the transmitted pump light and Stokes light on and off at a predetermined repetition frequency and duty cycle. That is, when SRP is induced (SRP on), the photoacoustic modulators 205 and 215 transmit both the pump light and Stokes light, while when SRP is not induced (SRP off), they block either the pump light or the Stokes light, or both. If only one of the pump light or the Stokes light is blocked, the photoacoustic modulator can be omitted from the optical path that always transmits light. Note that the photoacoustic modulators may be replaced with other optical elements that periodically transmit and block light, such as optical choppers.

[0029] The dichroic mirror 214 has wavelength characteristics that allow it to transmit pulsed light emitted from the pulsed laser 201 while reflecting pulsed light emitted from the pulsed laser 211. The dichroic mirror 214 is arranged so that these reflected and transmitted light overlap coaxially.

[0030] The pulse-synchronized detection unit 221 comprises a mirror 224, a dichroic mirror 225, a lens 226, and a two-photon detector 227. The dichroic mirror 225 has wavelength characteristics that transmit pulsed light emitted from the pulsed laser 201 while reflecting pulsed light emitted from the pulsed laser 211. The dichroic mirror 225 is arranged so that the transmitted and reflected light overlap coaxially. The two-photon detector 227 detects two-photon absorption that occurs when both the pump light and the Stokes light pulses arrive simultaneously.

[0031] Furthermore, the second emission optical system 20 (SRS light source unit) may be equipped with a delayed optical path (not shown) in, for example, the pulse lasers 201, 211 or the subsequent optical path, or the pulse synchronization detection unit 221, in order to adjust the timing of each pulse of light.

[0032] (Scanning Department) The scanning unit 30 comprises a two-axis scanner 301 and relay lenses 302 and 303. The two-axis scanner 301 has two mirrors that oscillate around two mutually orthogonal axes, and emits incident light with a two-dimensional displacement relative to the optical axis. This displacement is controlled by the angles of the two mirrors, thereby controlling the scanning point and scanning range on the sample surface P501. By making the oscillation frequencies of the two mirrors different, the sample surface P501 is scanned in two dimensions. It is desirable that the midpoint between the two mirrors is conjugate to the pupil plane P401 of the objective lens 401 via the relay lenses 302 and 303. The two-axis scanner 301 can use a two-axis galvanometer scanner. It is also possible to configure the two-axis scanner 301 by combining a one-axis resonant scanner and a galvanometer scanner. The relay lenses 302 and 303 ensure that light is incident on the pupil of the objective lens 401 with an appropriate beam diameter and maximum angle.

[0033] (Relay team) The relay unit 60 comprises relay lenses 601 and 602 and a two-axis scanner 603. Light is incident on the two-axis scanner 603 by the relay lenses 601 and 602 at an appropriate beam diameter and maximum angle. The two-axis scanner 603 has two mirrors that oscillate around two mutually orthogonal axes. The angles of these mirrors are controlled to cancel out the angular displacement of the incident light beam relative to the optical axis. That is, one mirror of the two-axis scanner 603 oscillates with respect to one mirror of the two-axis scanner 301, and the other mirror of the two-axis scanner 603 oscillates with respect to the other mirror of the two-axis scanner 301, each at the same frequency and in phase or out of phase (depending on the installation method). Therefore, the light beam emitted from the two-axis scanner 603 travels parallel to the optical axis, or ideally, on the optical axis. This operation of the two-axis scanner is sometimes called reverse scanning (descan). It is desirable that the intermediate position between the two mirrors of the two-axis scanner be conjugate to the pupil plane P451 of the light-gathering lens 451 via relay lenses 601 and 602.

[0034] In particular, when the light-gathering lens 451 and the objective lens 401 are the same, the relay lenses 601 and 602 can be the same as the relay lenses 303 and 302, respectively, and the two-axis scanner 603 can be the same as the two-axis scanner 301. In this case, the relay lenses 601 and 602 and the two-axis scanner 603 can be arranged symmetrically with respect to the sample surface P501 with respect to the relay lenses 303 and 302 and the two-axis scanner 301.

[0035] In the first embodiment, which has a two-axis scanner 603, the relay unit 60 may be referred to as the reverse scanning unit 60 based on its function. The reverse scanning unit 60 is configured to reverse scan in synchronization with the scanning unit 30 on the optical path between the demultiplexer 75 and the second objective lens 45.

[0036] (Wave combining section and wave splitting section) The multiplexing section 15 and the demultiplexing section 75 are each equipped with dichroic mirrors 151 and 751, respectively. The dichroic mirror 151 has wavelength characteristics that transmit light emitted from the LED 101 and reflect light emitted from the pulsed lasers 201 and 211. The dichroic mirror 151 is positioned so that the reflected and transmitted light overlap coaxially. The dichroic mirror 751 has wavelength characteristics that transmit light emitted from the sample 501 after it has been emitted from the LED 101, while reflecting light emitted from the sample 501 after it has been emitted from the pulsed lasers 201 and 211. The light reflected by the dichroic mirror 751 is blocked by a beam block 806 installed outside the demultiplexing section 75. The dichroic mirror 751 may be replaced with a short-pass filter, long-pass filter, or band-pass filter having similar wavelength-selective characteristics. The wave splitter 75 guides a portion of the secondary light collected by the light-gathering lens 451 to the first detection unit 70, while separating the other portion of the secondary light so as not to be guided to the first detection unit 70.

[0037] (Control Unit) The control unit 90 comprises a control device 901, a keyboard 911, a mouse 912, and a display 921. The control device 901 can be formed by implementing a program that executes the control flow on a computer. The control device 901 may also include waveform or signal generators, measuring instruments, FPGAs, microcontrollers, electrical circuits, and servers that execute part of the control flow, and part of the program may be implemented on these. For example, a waveform generator that controls the output of LED 101 can be included in the control device 901. Also, for example, an electrical circuit that controls one or both of the pulse lasers 201 and 211 based on the synchronization signal of the pulse synchronization detection unit 211 can be included in the control device 901. Also, for example, a waveform generator that generates control signals for photoacoustic modulators 205 and 215 can be included in the control device 901. Also, for example, an electrical circuit that generates drive signals for two-axis scanners 301 and 603 can be included in the control device 901. Furthermore, the control device 901 may also include, for example, a digitizer that converts the analog signal output from the photodetector 701 into digital data, and an analyzer that analyzes the signal waveform. Additionally, the control device 901 may include, for example, an electrical circuit that acquires desired data at a predetermined pixel rate. Furthermore, the control device 901 may also include a server that stores the generated phase-contrast images and SRP images, and performs image processing and analysis, as well as network equipment required for communication with this server.

[0038] The control device 901 generates a phase contrast image and an SRP image based on data acquired by analyzing the signal output by the photodetector 701. During image generation, the control device 901 identifies the position of the focal point (measurement point) on the sample surface P501 from the control signal of the two-axis scanner 301 and generates data for each pixel based on that information. The control device 901 identifies the Raman shift corresponding to the generated SRP image from the difference in wavelengths between the pulsed lasers 201 and 211. The control device 901 generates an SRP image while sequentially changing the wavelengths of one or both of the pulsed lasers 201 and 211, and generates an SRP spectrum by, for example, plotting the average SRP intensity of the same pixel region against the Raman shift. The control device 901 may execute a program to extract feature quantities from the generated phase contrast image and SRP image or SRP spectrum, or to discover a region of interest on the sample 501 from the feature quantities. The control device 901 may also execute preprocessing programs and analysis programs for the images and spectra necessary for this purpose. The generated phase-contrast image, SRP image, SRP spectrum, feature quantities, and information regarding the region of interest are stored in the memory device (not shown) of the control device 901 or output to the display 921. The memory device is, for example, a solid-state drive or a hard disk drive. The memory device may also be provided by a server.

[0039] The keyboard 911 and mouse 912 are connected to the control unit 901, and the operator inputs instructions to the control unit 901 by operating the keyboard 911 and mouse 912. The control unit 901 is connected to the LED 101, pulse laser 201, pulse laser 211, 2-axis scanners 301 and 603, stage scanner 512, and photodetector 701, and controls the operation of these devices according to instructions from the operator.

[0040] The display 921 provides visual feedback to the operator's actions and also displays images and text output by the control device 901.

[0041] (Phase contrast observation system) Figure 3 is a schematic diagram illustrating the phase difference observation system using the scanning observation device 1. Phase difference observation in the scanning observation device 1 uses a first emission optical system 10 (phase difference light source unit), a multiplexing unit 15, a scanning unit 30, a first objective lens 40, a mounting unit 50, a second objective lens 45, a relay unit 60, a demultiplexing unit 75, a first detection unit 70 (phase difference detection unit), and a control unit 90.

[0042] As shown by the dotted line in Figure 3, the light emitted by the LED 101 of the first emission optical system 10 (phase difference light source) becomes a parallel beam of light in the collimator 102. The annular beam of light that has passed through the ring slit 103 is focused to a single point toward the pinhole 105 by the relay lens 104. At this time, the first-order and higher diffracted light generated at the edge of the ring slit 103 broadens the focal point. The pinhole 105 blocks the light from the periphery of this focal point while allowing light closer to the center to pass through, thereby increasing the proportion of light that has traveled straight through the ring slit 103 (in other words, zero-order diffracted light). This suppresses the deformation of the annular beam of light that has been re-formed by the relay lens 106 as it propagates along the subsequent optical path. Therefore, artifacts in the acquired phase difference signal can be suppressed.

[0043] The parallel light beam, whose annular diameter is expanded or contracted by the relay lens 106, passes through the dichroic mirror 151 of the multiplexer 15 and is incident on the two-axis scanner 301 of the scanning unit 30. The annular light beam is reflected by each of the two mirrors of the two-axis scanner 301. The annular light beam emitted from the two-axis scanner 301 parallel to the optical axis, or tilted relative to the optical axis, has its annular diameter expanded or contracted by the relay lenses 302 and 303 and is incident on the objective lens 401 of the first objective lens 40. The objective lens 401 focuses the light beam to a single point toward the sample 501 on the mounting unit 50. The position of this focusing point within the sample surface P501 corresponds to the angle of the light beam emitted from the two-axis scanner 301 with respect to the optical axis. At this time, in addition to the straight-traveling light (in other words, zero-order diffracted light) that travels in a straight line through the sample 501, first-order or higher diffracted light is generated according to the spatial distribution of the refractive index of the sample 501. Therefore, the refractive index of sample 501 changes due to the stimulated Raman photothermal effect (SRP), which also changes the diffracted light.

[0044] The straight-traveling and diffracted light emitted from the sample 501 is collected by the objective lens 451 (light-gathering lens 451) and emitted as a parallel beam. At this time, the straight-traveling light from the sample 501 becomes an annular shape, and the inside and outside of the annulus become the diffracted light generated at the sample 501. The emission angle of this parallel beam corresponds to the position of the focal point within the sample surface P501, that is, the angle of the beam emitted from the two-axis scanner 301 with respect to the optical axis. The beam beam emitted from the objective lens 451 is expanded or contracted in diameter by the relay lenses 601 and 602 of the relay unit 60 and incident on the two-axis scanner 603. This beam is reflected by each of the two mirrors of the two-axis scanner 603. Regardless of the angle of incidence, the light beam emitted from the two-axis scanner 603 parallel to or aligned with the optical axis passes through the dichroic mirror 751 of the demultiplexer 75 and enters the relay lens 702 of the first detection unit 70 (phase difference detection unit).

[0045] This light beam is focused to a single point towards the pinhole 703 by the relay lens 702. The pinhole 703 blocks the light from the periphery of this focal point, thereby blocking diffracted and scattered light generated outside the focal point on the sample surface P501. This suppresses artifacts in the acquired phase difference signal.

[0046] The light beam that passes through the pinhole 703 is re-converted to a parallel beam by the relay lens 704, with the beam diameter being expanded or reduced, and then transmitted through the phase plate 705. At this time, the straight-traveling light from the sample 501 passes through the annular portion and undergoes phase modulation and attenuation, while the diffracted light from the sample 501 passes through the portion other than the annular. The light beam emitted from the phase plate 705 is focused to a single point toward the light-receiving surface P701 of the photodetector 701 by the tube lens 706. At the light-receiving surface P701, the straight-traveling light and the diffracted light from the sample 501 interfere, and the photodetector 701 detects the resulting light intensity. Therefore, the change in diffracted light due to the change in refractive index of the sample 501 caused by SRP is expressed as a change in light intensity detected by the photodetector 701. The light that reaches the detection surface of the photodetector 701 is sometimes called phase difference detected light.

[0047] (SRP-induced system) Figure 4 is a schematic diagram illustrating the SRP induction system using the scanning observation device 1. In SRP induction using the scanning observation device 1, a second emission optical system 20 (SRS light source unit), a multiplexing unit 15, a scanning unit 30, a first objective lens 40, a mounting unit 50, a second objective lens 45, a relay unit 60, a demultiplexing unit 75, and a control unit 90 are used.

[0048] As shown by the dotted line in Figure 4, the parallel beam of light emitted by the pulsed laser 201 of the SRS light source is partially reflected by the beam splitter 222 and incident on the pulse synchronization detection unit 221, while the majority is transmitted and incident on the photoacoustic modulator 205. The beam of light emitted from the photoacoustic modulator 205 is then incident on the relay lens 202. The beam of light, whose beam diameter has been expanded or reduced by the relay lenses 202 and 203, is reflected by the mirror 204 and then transmitted through the dichroic mirror 214.

[0049] The parallel beam of light emitted by the pulsed laser 211 is partially reflected by the beam splitter 223 and incident on the pulse synchronization detection unit 221, while the majority is transmitted and incident on the photoacoustic modulator 215. The beam of light emitted from the photoacoustic modulator 215 is then incident on the relay lens 212. The beam of light, whose beam diameter has been expanded or reduced by the relay lenses 212 and 213, is reflected by the dichroic mirror 214. At this time, it is superimposed coaxially with the beam of light that has passed through the aforementioned dichroic mirror 214.

[0050] The beam of light from the pulsed laser 201, reflected by the beam splitter 222, is reflected by the mirror 224 of the pulse-synchronous detection unit 221 and passes through the dichroic mirror 225. The beam of light from the pulsed laser 211, reflected by the beam splitter 223, is reflected by the dichroic mirror 225. At this time, it is superimposed coaxially with the beam of light that has passed through the dichroic mirror 225. This coaxially superimposed beam of light (in other words, both the pump beam and the Stokes beam) is focused toward the two-photon detector 227 by the lens 226.

[0051] The light beams (in other words, both the pump light and the Stokes light beams) superimposed coaxially by the dichroic mirror 214 are reflected by the dichroic mirror 151 of the multiplexing section 15. At this time, they are superimposed coaxially with the annular light beam emitted from the first emission optical system 10 (phase difference light source section) described above and transmitted through the dichroic mirror 151.

[0052] The light beam reflected by the dichroic mirror 151 enters the two-axis scanner of the scanning unit 30 and is reflected by each of the two mirrors. The light beam emitted from the two-axis scanner 301 has its beam diameter expanded or reduced by the relay lenses 302 and 303 and enters the objective lens 401 of the first objective lens 40. The objective lens 401 focuses the light beam to a single point toward the sample 501 on the mounting unit 50. The position of the focal point within the sample surface P501 corresponds to the angle with respect to the optical axis when the light beam is emitted from the two-axis scanner 301. At this time, an stimulated Raman process occurs at the focal point, inducing SRP. That is, stimulated Raman loss occurs for the pump light and stimulated Raman gain occurs for the Stokes light, corresponding to the vibrational levels of the molecules present at the focal point, and a change in refractive index occurs due to the heat associated with the relaxation of molecular vibrations. Furthermore, since the light emitted from the first emission optical system 10 (phase difference light source) is also focused coaxially with the pump light and Stokes light, the focal points of these three beams have a spatial overlap. Therefore, the change in refractive index due to SRP is detected as a change in diffracted light in the phase difference.

[0053] The pump light and Stokes light emitted from the sample 501 are collected by the light-collecting lens 451 and emitted as parallel beams. At this time, the emission angle with respect to the optical axis corresponds to the position of the focal point within the sample surface P501, in other words, the angle of the beam emitted from the two-axis scanner 301 with respect to the optical axis. The beam of light emitted from the light-collecting lens 451 is expanded or reduced in diameter by the relay lenses 601 and 602 of the relay unit 60 and incident on the two-axis scanner 603. The beam is reflected by each of the two mirrors of the two-dimensional scanner 603. Regardless of the incident angle, the pump light and Stokes light beams emitted from the two-axis scanner 603 parallel to or coincide with the optical axis are reflected by the dichroic mirror 751 of the demultiplexer 75 and blocked by the beam block 806.

[0054] (Light irradiation timing) Figure 5 is a schematic diagram illustrating the timing of the light irradiated onto the sample 501 in the scanning observation device 1. Figure 5(A) illustrates the time progression of the intensities of the pump light, Stokes light, and phase difference illumination light at a series of measurement points when the energy difference between the pump light and Stokes light approximately matches the energy difference of the molecular vibrational levels. Here, one measurement point can be rephrased as one measurement cycle. Furthermore, a series of measurement points can be considered as a repetition of the measurement cycle, and can be rephrased as multiple measurement cycles. In addition, Figure 5(A) also illustrates the time progression of the magnitude of the SRP signal, in other words, the magnitude of the change in phase difference detection light intensity |ΔI| correlated with the refractive index change Δn due to SRP. Furthermore, Figure 5(A) also illustrates the pixel clock signal (voltage V). Here, the rising edge of the pixel clock signal is set to the timing of the start of measurement for each pixel, and one pixel corresponds to one measurement point. Note that one pixel may be associated with two or more measurement points, and data for one pixel may be generated using signals and data acquired at multiple measurement points. Figures 5(B) and (C) illustrate enlarged time transitions of the pump light, Stokes light, and phase difference illumination light at any measurement point in Figure 5(A). Figure 5(B) shows the case where the phase difference illumination light is pulsed light, and Figure 5(C) shows the case where the phase difference illumination light is continuous light.

[0055] Within one measurement cycle, there are periods when both the pump light and Stokes light irradiate the sample 501 (SRP-on) and periods when they do not (SRP-off) (Figure 5(A)). On the other hand, the phase-contrast illumination light continuously irradiates the sample 501 regardless of whether it is SRP-on or SRP-off. During the SRP-on period, the pump light and Stokes light irradiate the sample 501 in overlapping time (Figures 5(B) and (C)). Both pulsed lights repeatedly irradiate the sample 501 as long as the SRP-on period continues. On the other hand, if the phase-contrast illumination light is pulsed, it irradiates the sample 501 in overlapping time with the pump light and Stokes light, or slightly delayed (Figure 5(B)). This allows for efficient detection of the SRP before the heat caused by the stimulated Raman process resulting from a single pulse of the pump light and Stokes light dissipates. If the phase-contrast illumination light is continuous light, it continues to irradiate the sample 501 at a constant intensity regardless of the pulse irradiation timing of the pump light and pulsed light (Figure 5(C)).

[0056] During the SRP-on period, the repeatedly occurring stimulated Raman process causes heat to accumulate at the measurement point faster than it dissipates, increasing the change in refractive index. Consequently, the SRP signal (intensity change of phase difference detection light) also increases (Figure 5(A)). It should be noted that if the SRP-on time is too long, the generation and dissipation of heat at the measurement point balance out, causing the SRP signal to saturate.

[0057] During the SRP-off period, heat locally accumulated at the measurement point dissipates. As heat dissipates, the refractive index asymptotically approaches its original value before the SRP was turned on. Consequently, the phase difference detection light also asymptotically approaches its original intensity before the SRP was turned on (Figure 5(A)). It is desirable to wait until the time variation of the phase difference detection light intensity becomes sufficiently small (ensure sufficient SRP-off time), and this is especially important when one pixel corresponds to multiple measurement points (multiple measurement cycles). Therefore, it is desirable to set the ratio of the SRP-on time in one cycle time (duty cycle) to less than 50%.

[0058] (Generation of Raman and phase-contrast images) The intensity of the phase difference detection light acquired in the latter half of the SRP-on period, immediately after the timing when the SRP signal is strongest within one measurement cycle as shown in Figure 5, is called the SRP-on intensity. The spatial distribution of the SRP-on intensity on the sample surface P501 is called the SRP-on image. On the other hand, the intensity of the phase difference detection light acquired in the latter half of the SRP-off period, when the SRP signal becomes sufficiently small within one measurement cycle, is called the SRP-off intensity. The spatial distribution of the SRP-off intensity on the sample surface P501 is called the SRP-off image. If the measurement start timing for each pixel (here, the rising edge of the pixel clock signal) is before the SRP-on start timing, the intensity of the phase difference detection light immediately before the SRP-on starts can also be used as the SRP-off intensity.

[0059] Figure 6 illustrates the method for generating SRP images and phase-contrast images. The SRP image, in other words, the Raman image, is obtained by subtracting the SRP-off image from the SRP-on image. On the other hand, the phase-contrast image can be obtained by reusing the SRP-off image. This is because the effect of SRP is negligible in the SRP-off image. Since the SRP image and the phase-contrast image are generated based on signals acquired at the same measurement point and time, each pixel in the SRP image and the phase-contrast image correspond one-to-one both spatially and temporally. In the SRP image generation flow, the SRP image may be calculated after generating the SRP-on and SRP-off images, or the SRP intensity may be calculated for each pixel from the SRP-on and SRP-off intensities, and then the SRP image may be generated.

[0060] (Observation Flow) Figure 7 illustrates the observation flow of sample 501 using the scanning observation device 1. Figure 7(A) is a flowchart explaining the observation flow, and Figure 7(B) is a diagram explaining the low-magnification field of view R502, high-magnification field of view R503, and region of interest R504 in the flowchart. Figure 7(C) is a diagram showing the schematic configuration of the scanning condition determination unit.

[0061] As shown in Figure 7(C), the control unit 90 includes an image generation unit 92 that generates a first image captured by a first objective lens 40 and a second objective lens 45 at a predetermined magnification, and displays the first image on a display unit 921 (display). The control unit 90 further includes an input unit 910 that receives input from the operator regarding a region of interest based on the displayed first image, and a scanning condition acquisition unit 94 that acquires scanning conditions SC0 for the scanning unit 30 based on the received information regarding the region of interest. The control unit 90 further includes a scanning condition determination unit 96 that determines scanning conditions SC1 for imaging for the scanning unit 30 based on the acquired scanning conditions SC0. The control unit 90 further includes an update unit 98 that displays the scanning conditions SC1 acquired by the scanning condition acquisition unit 96 on the display unit and accepts updates to the scanning conditions SC1.

[0062] (separation part) The control unit 90 includes a separation unit 91 that temporally demodulates the signal from the first detection unit 70 and separates it into a first signal S1 corresponding to the nonlinear photothermal effect PTE generated in the sample 501 by irradiation with the second light, and a second signal S2 that does not include the nonlinear photothermal effect. The separation unit 91 includes a configuration in which it temporally demodulates the signal from the first detection unit 70 by attenuating the first signal S1 corresponding to the nonlinear photothermal effect PTE, and separates and acquires the second signal S2 that substantially does not include the nonlinear photothermal effect. The separation unit 91 also includes a configuration in which it temporally demodulates the signal from the first detection unit 70 by attenuating the second signal S2 that substantially does not include the nonlinear photothermal effect, and separates and acquires the first signal S1 corresponding to the nonlinear photothermal effect PTE.

[0063] (Image generation unit) The control unit 90 includes an image generation unit 92 that generates an image based on scanning information 30i related to the scanning unit 30 and at least one of the first signal S1 and the second signal S2 separated by the separation unit 91. That is, the image generation unit 92 generates at least one of a stimulated Raman photothermal effect image (SRP image) which is an image of the first signal S1, or a phase contrast image which is an image of the second signal S2. The scanning information 30i includes the focal position (X coordinate, Y coordinate) of the first light (second light), time t, etc.

[0064] First, the sample 501 is held on the stage 511. From this state, the observation flow begins. First, the sample 501 is observed using phase contrast observation with a wide field of view. The conditions for low-magnification phase contrast observation are input to the control device 901, and low-magnification phase contrast observation is started (steps S10, S20).

[0065] By driving the stage scanner 512 to change the position of the stage 511, the low-magnification field of view R502 shown in Figure 7(B) is moved. If the region of interest R504 of the sample 501 is not present within the low-magnification field of view R502, the low-magnification field of view R502 is moved and low-magnification phase contrast observation is continued. If the region of interest R504 is found, the low-magnification phase contrast observation is completed (step S30). In this way, by first performing real-time phase contrast observation of the sample 501 in a wide field of view, the time required to determine the position of the sample 501 relative to the sample surface P501 and to find the region of interest R504 can be shortened. This helps to suppress damage to the sample 501.

[0066] Next, the details of the phase contrast image of region of interest R504 are checked from the low-magnification phase contrast image, or a decision is made (step S40). If details are to be checked, the field of view is narrowed and real-time phase contrast observation is performed. In that case, the high-magnification phase contrast observation conditions are input to the control device 901 and high-magnification phase contrast observation is started (steps S50, S60). If necessary, the high-magnification field of view R503 shown in Figure 7(B) is moved so that the region of interest R504 is included, and the necessity of SRP observation is considered from the high-magnification phase contrast image. On the other hand, if the details of the phase contrast image of region of interest R504 are not to be checked, the necessity of SRP observation is considered from the low-magnification phase contrast image. If it is determined from these phase contrast images that SRP observation is unnecessary, the process returns to low-magnification phase contrast observation and another region of interest R504 of sample 501 is searched for. On the other hand, if it is determined that SRP observation is necessary, the process proceeds to SRP observation (step S70). In this way, by smoothly magnifying the region of interest R504 of sample 501 and performing phase contrast observation, the necessity of SRP observation can be quickly determined. Therefore, unnecessary SRP observation can be reduced, improving measurement efficiency and minimizing damage to sample 501. Both the pump light and Stokes light can be kept off until the above phase contrast observation is complete. In other words, the phase contrast image can be acquired with only the phase contrast illumination light on.

[0067] Next, the SRP observation conditions (in other words, the SRP image acquisition conditions) are input to the control device 901, and SRP observation is started (step S90). At this stage, the pump light and Stokes light may be turned on for the first time. Once simultaneous observation of SRP observation and phase contrast observation (in other words, observation of SRP image and SRP-off image) is completed, the series of observation flows is terminated.

[0068] The low-magnification and high-magnification phase-contrast observation conditions include, for example, the light output intensity, pulse duration, number of integrations, and field of view of LED101. The SRP observation conditions include, for example, the light output intensity, center wavelength, pulse wavelength width, pulse duration, and number of integrations of pulsed lasers 201 and 211, as well as the duty cycle of the transmission / blocking repetition frequency of photoacoustic modulators 205 and 215, and the field of view.

[0069] In the observation flow shown in Figure 7(A), if the region of interest R504 is discovered in step S30, the operator inputs the observation conditions in the subsequent steps S50 and S90. To improve convenience for the operator, a step S35 may be added between steps S30 and S40, in which the operator inputs the location information of the region of interest R504. The location information can be input, for example, by the operator using the mouse 912 to trace the outline of the region of interest R504 or draw a mark on the low-magnification phase-contrast image displayed on the display 921 of the control unit 90. Based on the above location information, the control device 901 may predict appropriate observation conditions and automatically input them in steps S50 and S90, or present them to the operator. By supporting the operator's input of each observation condition in this way, the burden on the operator can be reduced.

[0070] If step S35 is added, the control device 901 may determine appropriate observation conditions based on the position information and automatically input each observation condition in steps S50 and S90. This eliminates the need for the operator to manually input each observation condition, further reducing their workload.

[0071] In the observation flow shown in Figure 7(A), the control unit 90 may automatically perform the detection of the region of interest R504 and the identification of its location information. This reduces the burden on the operator during observation.

[0072] In the scanning observation device 1 shown in Figure 2, the ring slit 103 is placed in the first ejection optical system 10 (phase difference light source unit) and the phase plate 705 is placed in the first detection unit 70 (phase difference detection unit). However, even if the two are swapped in their positions, a phase difference signal can still be acquired.

[0073] As described above, the scanning observation device according to the first embodiment of the present invention generates an SRP image (in other words, a Raman image) and a phase difference image based on SRP on / off phase difference signals acquired at the same measurement point both spatially and temporally, so that the pixels of both images correspond one-to-one. That is, it is possible to acquire primary images of a phase difference image and an SRP image (Raman image) with matched image quality. This makes it possible to reduce the burden on the operator regarding image quality matching.

[0074] <Second Embodiment> The configuration of the scanning observation device according to the second embodiment will be described with reference to Figure 8. Figure 8 is a schematic diagram illustrating the configuration of the scanning observation device 2 of this embodiment. The scanning observation device 2 has the same configuration as the scanning observation device 1 of the first embodiment, except for the parts described below. Therefore, the same reference numerals are used for common elements, and redundant explanations are omitted.

[0075] In the scanning observation device 2 shown in Figure 8, the two-axis scanner 603 of the relay unit 60 of the scanning observation device 1 is removed, and the dichroic mirror 751 of the demultiplexer 75 is installed in its place. In addition, the pinhole 703 of the first detection unit 70 (phase difference detection unit) of the scanning observation device 1 is removed. The relay unit 60 of the second embodiment differs from that of the first embodiment in that it is not electrically connected to the control unit 90.

[0076] (Relay team) The scanning observation device 2 does not have a two-axis scanner to cancel out the angular displacement of the incident light beam of the relay unit 60 with respect to the optical axis. Therefore, the angle of the light beam emitted from the relay unit 60 of the scanning observation device 2 with respect to the optical axis corresponds to the position of the focal point on the sample surface P501, in other words, the angle of the light beam emitted from the two-axis scanner 301 of the scanning unit 30 with respect to the optical axis.

[0077] (Demultiplexer) The dichroic mirror 751 of the demultiplexer 75 of the scanning observation device 2 has the same wavelength characteristics as the scanning observation device 1. The dichroic mirror 751 of the scanning observation device 2 is installed near a position conjugate to the pupil plane P451 of the light-gathering lens 451 of the second objective lens 45, via the relay lenses 601 and 602 of the relay unit 60.

[0078] (Phase difference detection unit) The first detection unit 70 (phase difference detection unit) of the scanning observation device 2 does not have a pinhole 703 at a position conjugate to the focal plane P501 of the light-gathering lens 451 (in other words, an intermediate imaging plane) P703. At the intermediate imaging plane P703, a point-like image moves in accordance with the focal point scanning the sample surface P501. Similarly, a point-like image also moves at the focal plane P701 of the tube lens 706, which is located at a position conjugate to the focal plane P501. Therefore, the photodetector 701 of the scanning observation device 2 requires a larger light-receiving area than that of the scanning observation device 1. For this reason, the photodetector 701 of the scanning observation device 2 may utilize a photomultiplier tube or photodiode with a large light-receiving area, or a photodiode array in which multiple photodiodes are arranged in two dimensions.

[0079] The phase plate 705 of the first detection unit 70 (phase difference detection unit) of the scanning observation device 2 is the same as that of the scanning observation device 1 in that it is positioned conjugate to the pupil surface P451 of the light-gathering lens 451. However, it differs from the scanning observation device 1 in that the angle of the light beam transmitted through the phase plate 705 with respect to the optical axis changes in accordance with the position of the focal point on the sample surface P501, in other words, the angle of the light beam emitted by the two-axis scanner 301 with respect to the optical axis. Taking this into consideration, it is necessary to design an annular portion that modulates the phase of the straight-traveling light (zero-order diffracted light) at the sample 501 and attenuates the light.

[0080] Figure 9 is a schematic diagram illustrating the relationship between the phase plate 705 of the scanning observation device 2 and the light beam transmitted through it. The phase plate 705 has two concentric cylinders 705_1 and 705_2 around a central axis 705_0 that coincides with the optical axis. A quarter-wave plate and an ND filter are formed in the region between the inner cylinder 705_1 and the outer cylinder 705_2 (shaded area in Figure 9). An annular parallel light beam created by the straight-traveling light from the sample 501 is transmitted through this region. Of this annular light beam, the innermost light beam L103_1 and the outermost light beam L103_2 trace concentric cylinders around the principal rays of the parallel light beam (including both the straight-traveling and diffracted light from the sample 501) transmitted through the phase plate 705. The diameters of the cylinders of these luminous beams L103_1 and L103_2 correspond to the inner and outer diameters of the ring slit 103, which is located at a position conjugate to the plane P705 on which the center of the thickness of the phase plate 705 is positioned. Specifically, by multiplying the inner radius I and outer radius E of the ring slit 103 by the imaging magnification m, the radius of luminous beam L103_1 becomes mI and the radius of luminous beam L103_2 becomes mE.

[0081] Figure 9(A) shows the point when the angle of the parallel luminous beams passing through the phase plate 705 with respect to the optical axis is at its maximum. In Figure 9(A), the principal ray L103_0 makes the maximum angle θ0 with respect to the optical axis 705_0 and intersects on the plane P705 which coincides with the center of the thickness of the phase plate 705. At this time, all of the luminous beams between L103_1 and L103_2 pass through the region between the cylinders 705_1 and 705_2 of the phase plate 705, so the radii of cylinders 705_1 and 705_2 satisfy the following equations 1 and 2.

number

number

[0082] Figure 9(B) shows the case when the parallel luminous beams passing through the phase plate 705 are parallel to the optical axis. When equations 1 and 2 are satisfied, the entire luminous beam between luminous beams L103_1 and L103_2 passes through the region between the cylinders 705_1 and 705_2 of the phase plate 705.

[0083] Thus, by designing the system so that all straight-traveling light from the sample 501 undergoes phase modulation and attenuation at the phase plate 705, regardless of the angle with respect to the optical axis when passing through the phase plate 705, artifacts in the acquired phase difference image can be suppressed.

[0084] As described above, the scanning observation device according to the second embodiment of the present invention does not have a two-axis scanner in the relay unit 60, which simplifies control and simplifies optical path adjustment. This improves the stability of the scanning observation device.

[0085] <Third Embodiment> The configuration of the scanning observation device according to the third embodiment will be described with reference to Figure 10. Figure 10 is a schematic diagram illustrating the configuration of the scanning observation device 3 of this embodiment. The scanning observation device 3 has the same configuration as the scanning observation device 2 of the second embodiment, except for the parts described below. Therefore, the same reference numerals are used for common elements, and redundant explanations are omitted.

[0086] In the scanning observation device 3 of Figure 10, the phase plate 705, which was located in the first detection unit 70 (phase difference detection unit) in the scanning observation device 2, has been moved into the objective lens 45 (light-gathering lens). That is, the phase plate 705, which was located on surface P705 in the scanning observation device 2, is incorporated into the light-gathering lens 451 so as to include the pupil plane 451. Accordingly, the relay unit 60 and the relay lenses 702 and 704 of the first detection unit 70 (phase difference detection unit) are removed. The annular portion of the phase plate 705 incorporated into the light-gathering lens 451 of the scanning observation device 3 (transmission of straight-line light in the sample 501) substantially coincides with the image of the annular portion of the ring slit 103 of the first emission optical system 10 (phase difference light source unit) (transmission of emitted light from the LED 101). That is, the annular portion of the phase plate 705 is designed to encompass the image of the annular portion of the ring slit 103 at the pupil plane 451. Therefore, a phase-contrast microscope objective lens can be used for the light-gathering lens 451 of the scanning observation device 3. However, the output of the pulsed lasers 201 and 211 is adjusted so that the phase plate 705 is not damaged by absorbing both the pump light and the Stokes light pulses.

[0087] As described above, the scanning observation device according to the third embodiment of the present invention can use a phase-contrast microscope objective lens for the light-gathering lens 451, thereby reducing the optical system from there to the photodetector 701. This simplifies optical path adjustment and improves the stability of the scanning observation device.

[0088] <Fourth Embodiment> The configuration of the scanning observation device according to the third embodiment will be described with reference to Figure 11. Figure 11 is a schematic diagram illustrating the configuration of the scanning observation device 4 of this embodiment. The scanning observation device 4 has the same configuration as the scanning observation device 2 of the second embodiment, except for the parts described below. Therefore, the same reference numerals are used for common elements, and redundant explanations are omitted.

[0089] The scanning observation device 4 in Figure 11 includes a second detection unit 80 (SRS detection unit) instead of the beam block 806. That is, in addition to the function of indirectly detecting the stimulated Raman process by SRP, the scanning observation device 4 has the function of directly detecting SRS. In other words, the scanning observation device 4 can provide the Raman signal or Raman image not only as an SRP signal or SRP image, but also as an SRS signal or SRS image. Furthermore, the scanning observation device 4 can simultaneously acquire a phase-contrast image and an SRS image of the same field of view. The control unit 90 of the scanning observation device 4 is also electrically connected to the second detection unit 80 (SRS detection unit). The second detection unit 80 receives the other component of the secondary light guided from the demultiplexer 75 and detects a third signal corresponding to the nonlinear optical effect generated in the sample 501 by the irradiation of the second light.

[0090] (Objective lens) It is desirable that the objective lenses 401 and 451 of the scanning observation device 4 have equal numerical apertures. This makes it possible to suppress artifacts superimposed on the acquired SRS image and SRS spectrum.

[0091] (Second ejection optical system) When detecting SRS, the photoacoustic modulators 205 and 215 of the scanning observation device 4 modulate the intensity of one of the pulsed light emitted by the pulsed lasers 201 and 211 at a specific frequency, while simply transmitting the other. Note that in the scanning observation device 4, the photoacoustic modulator that does not modulate the intensity of the pulsed light during SRS detection may be omitted. In this configuration, SRP on / off during SRP detection is performed by turning the photoacoustic modulator that was not omitted on / off.

[0092] (Second detection unit) The second detection unit 80 (SRS detection unit) comprises relay lenses 804 and 805, a bandpass filter 802, and a photodetector 801. The relay lenses 804 and 805 enlarge or reduce the beam diameter of the incident light beam to suit the size of the light-receiving surface of the photodetector 801. Between the relay lenses 804 and 805, there exists a plane P804 conjugate to the focal plane P501 of the light-gathering lens 451 of the second objective lens 45. The light-receiving surface of the photodetector 801 is positioned conjugate to the pupil plane P451 of the light-gathering lens 451. The bandpass filter 802 has wavelength characteristics that transmit only the pulsed light that is not intensity-modulated by the photoacoustic modulator 205 or 215, among the pump light and Stokes light, and is placed between the relay lens 805 and the photodetector 801. The photodetector 801 has, for example, a photodiode. The control unit 90 performs lock-in detection on the modulation component of the intensity of the pulsed light received by the photodetector 801, thereby acquiring the SRS signal. The image generation unit 92 generates a nonlinear optical effect image (SRS image) by imaging based on the scanning information 30i related to the scanning unit 30 and the third signal (SRS signal) from the photodetector 801 of the second detection unit 80.

[0093] (Control Unit) The control unit 90 of the scanning observation device 4 also includes a lock-in amplifier (not shown) that detects the modulation component of the light intensity detected by the photodetector 801. The control unit 901 of the scanning observation device 4 is electrically connected to the photodetector 801. The control unit 901 calculates the SRS signal intensity (also simply called SRS intensity) based on the signal output by the photodetector 801, generates data for each pixel based on the control signal of the 2-axis scanner 301, and generates an SRS image. In addition, it is also possible to generate a phase difference image and an SRS image, respectively, based on signals output by the photodetector 701 and the photodetector 801 during the same time period. At this time, the phase difference signal and the SRS signal are read out at the same pixel rate and converted to luminance, respectively, to generate an image with the same number of pixels. The generated phase difference image and SRS image are stored in the memory device (not shown) of the control unit 901 or output to the display 921.

[0094] (Observation Flow) Comparing phase contrast observation, SRS observation, and SRP observation, phase contrast observation allows for rapid acquisition of a morphological image of sample 501 using weak illumination light. On the other hand, SRS observation and SRP observation require strong pulsed light irradiation but allow for the acquisition of Raman images that provide molecular information. Comparing SRS observation and SRP observation, SRS images can be acquired faster, while SRP images take longer to acquire but have higher detection sensitivity (a higher signal-to-noise ratio is obtained compared to averaging SRS signals over the same time).

[0095] Figure 12 illustrates an observation flow using the scanning observation device 4. Based on the characteristics of each observation method described above, observation flows such as (A) to (C) are possible. (A) Discover the region of interest R504 by low-magnification phase-contrast observation. (B) Quickly compare and confirm the phase-contrast image and Raman image by simultaneous observation of phase-contrast and high-speed SRS at medium magnification. (C) If necessary, perform simultaneous observation of phase-contrast and high-sensitivity SRP at high magnification based on the spatial distribution of SRS intensity and signal-to-noise ratio.

[0096] As described above, the scanning observation apparatus according to the fourth embodiment of the present invention can generate not only SRP images but also SRS images as primary images of Raman images with matching phase contrast and image quality. Since high-speed SRS observation and high-sensitivity SRP observation can be selected according to the region of interest R504 of the sample 501, observation efficiency can be improved.

[0097] In the scanning observation devices 1 to 4 of the above embodiments, a separation unit can be formed by the control unit 90, or by the demultiplexing unit 75 and the control unit 90. The control unit 90 can also form an image generation unit and a display unit. Furthermore, a detection unit can be formed by the first detection unit 70 (phase difference detection unit), or by the first detection unit 70 and the second detection unit (SRS detection unit). The phase difference illumination light may be called the first light, and the pump light and Stokes light may be called the second light. The phase difference illumination light emitted from the sample 501 (straight light and diffracted light), or the phase difference illumination light emitted from the sample 501, the pump light and Stokes light may be called secondary light. The signal that gives the SRP intensity may be called the first signal, the signal that gives the SRP off intensity may be called the second signal, and the signal that gives the SRS intensity may be called the third signal.

[0098] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. It will be clear from the claims that such modified or improved forms may also be included in the technical scope of the present invention. [Explanation of Symbols]

[0099] 1, 2, 3, 4 Scanning observation device 10. First emission optical system (phase difference light source section) 15 Wave section 20. Second Ejection Optical System (SRS Light Source Unit) 30 Scanning Unit 40. First objective lens (focusing lens) 45. Second objective lens (light-gathering lens) 50 Mounting section 60 Relay section (reverse scanning section) 70 First detection unit (phase difference detection unit) 75 Branch 80 Second detection unit (SRS detection unit) 90 Control Unit 91 Separation part 92 Image generation unit 806 Beam Block

Claims

1. A first emission optical system that emits first light containing spatially modulated components, A second emission optical system that emits a coherent, time-modulated second beam of light, A wave-combining unit that combines the first light and the second light, A scanning unit that scans the combined first light and the second light in a synchronized manner, A focusing lens that focuses the scanned first light and the second light, A mounting section on which a sample is placed at the focusing position of the aforementioned focusing lens, A light-collecting lens is located on the opposite side of the mounting portion from the light-collecting lens and collects secondary light from the sample, A detection unit for detecting the secondary light collected from the light-collecting lens, A separation unit demodulates the signal from the detection unit in time and separates it into a first signal corresponding to the nonlinear photothermal effect generated in the sample by the irradiation of the second light, and a second signal that does not include the nonlinear photothermal effect. An image generation unit that generates an image based on scanning information relating to the scanning unit and at least one of the first signal and the second signal, A scanning observation device equipped with the following features.

2. The scanning observation apparatus according to claim 1, wherein the wave-combining unit coaxially combines the first light and the second light.

3. The scanning unit scans the first light and the second light coaxially, according to claim 1 or 2.

4. The scanning observation apparatus according to claim 1, wherein the focusing lens focuses the first light and the second light coaxially.

5. The scanning observation apparatus according to claim 1 or 2, wherein the condensing lens and the light-gathering lens have a portion in which their depths of focus overlap in the optical axis direction.

6. The scanning observation apparatus according to claim 1 or 2, wherein the second emission optical system comprises a pair of pulse light sources that emit a pair of pulse light trains having different and synchronized oscillation wavelengths.

7. The scanning observation apparatus according to claim 6, wherein the pulse light source comprises a modulation unit that periodically modulates the intensity of at least one of the pair of pulse light trains.

8. The scanning observation apparatus according to claim 6, wherein the pair of pulsed light trains include a Stokes light train and a pump light train that exhibit a nonlinear optical effect on the sample.

9. The scanning observation apparatus according to claim 1 or 2, wherein the first emission optical system comprises a first light source that emits incoherent light of a predetermined wavelength.

10. The scanning observation apparatus according to claim 1 or 2, wherein the first emission optical system comprises a modulation optical element that modulates the intensity component of the first primary light in an annular manner, and the first detection unit comprises a demodulation optical element having an annular portion that demodulates the intensity component and phase component of the first secondary light.

11. The scanning observation apparatus according to claim 1 or 2, wherein the first emission optical system comprises a pinhole that blocks a portion of the focused first primary light.

12. The scanning observation apparatus according to claim 1 or 2, wherein the first emission optical system comprises a modulation optical element having an annular portion that modulates the intensity component and phase component of the first primary light, and the first detection unit comprises a demodulation optical element that demodulates the intensity component of the first secondary light in an annular manner.

13. The scanning observation apparatus according to claim 1 or 2, further comprising a wave splitter that guides a portion of the secondary light collected from the light-collecting lens to the detection unit and separates the other portion of the secondary light so as not to guide it to the detection unit.

14. The scanning observation apparatus according to claim 13, further comprising a second detection unit that receives the other component of the secondary light guided from the wave decompressor and detects a third signal corresponding to a nonlinear optical effect generated in the sample by irradiation with the second light.

15. The scanning observation apparatus according to claim 14, wherein the image generation unit generates an image based on the scanning information relating to the scanning unit and the third signal, and generates a nonlinear optical effect image.

16. The scanning observation apparatus according to claim 13, further comprising a reverse scanning unit that scans in reverse in synchronization with the scanning unit, on the optical path between the wave demultiplexing unit and the second objective lens.

17. The scanning observation apparatus according to claim 16, further comprising a pinhole that blocks a portion of the secondary light and a pair of relay lenses arranged on either side of the pinhole.

18. The scanning observation apparatus according to claim 1 or 2, further comprising a display unit that displays an image generated by the image generation unit.

19. The scanning observation apparatus according to claim 18, further comprising: an input unit that receives input from the operator regarding a region of interest based on the first image displayed on the display unit; and a scanning condition acquisition unit that acquires scanning conditions for the scanning unit based on the received information regarding the region of interest.

20. The scanning observation apparatus according to claim 19, further comprising a scanning condition determination unit that determines the scanning conditions for the scanning unit during imaging based on the acquired scanning conditions.

21. The scanning observation apparatus according to claim 20, further comprising an update unit that displays the scanning conditions acquired by the scanning condition acquisition unit on the display unit and accepts updates to the scanning conditions.

Images