Sample observation device and sample observation method

Abstract

A sample observation device 1 comprises a radiation optical system 3 that radiates planar light L2A, L2B of different wavelengths along respective cross-sections that are at different positions on a sample S, a scanning unit 4 that scans the sample S relative to irradiation planes Va, Vb, an image formation optical system 5 that has an observation axis P2 that is inclined relative to the irradiation planes Va, Vb and forms respective optical images of observation light L3A, L3B, an image acquisition unit 6 that acquires each of first image data and second image data while the scanning unit 4 scans the sample S, and an image generation unit 8 that generates observation image data 24A, 24B for the sample S on the basis of the first image data and the second image data.

Description

[Technical Field]

[0001] The present disclosure relates to a sample observation device and a sample observation method. [Background technology]

[0002] Selective Plane Illumination Microscopy (SPIM) is known as one of the techniques for observing the interior of a sample with a three-dimensional structure, such as a cell. For example, the tomographic image observation device described in Patent Document 1 discloses the basic principle of SPIM. In this conventional device, planar light is irradiated onto the sample, and fluorescence or scattered light generated inside the sample is imaged on an imaging plane to obtain observation image data of the interior of the sample.

[0003] Another example of a sample observation device using planar light is the sample observation device described in Patent Document 2. This sample observation device includes an illumination optical system that illuminates the sample with planar light and a scanning unit that scans the sample across the illumination surface of the planar light. The device also includes an imaging optical system that has an observation axis tilted with respect to the illumination surface and that forms an image of the observation light generated on the sample by the illumination of the planar light, an image acquisition unit that acquires multiple pieces of partial image data corresponding to parts of the light image formed by the observation light by the imaging optical system, and an image generation unit that generates observation image data of the sample based on the multiple pieces of partial image data generated by the image acquisition unit. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 180241 / 1983 [Patent Document 2] Japanese Patent Application Publication No. 2018-063292 Summary of the Invention [Problem to be solved by the invention]

[0005] In the sample observation device described above, depending on the type and state of the sample, it is expected that it may be more effective to perform observation by irradiating the sample with planar light of different wavelengths than when using planar light of a single wavelength. However, in an embodiment in which the sample is scanned multiple times with planar light of different wavelengths, there is an issue in that it takes a long time to obtain observation image data.

[0006] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a sample observation device and a sample observation method that improve the throughput until observation image data is obtained, even when planar light of different wavelengths is irradiated onto the sample. [Means for solving the problem]

[0007] A sample observation device according to one aspect of the present disclosure includes an illumination optical system that illuminates a first planar light and a second planar light having different wavelengths, respectively, so that the first and second planar lights pass through a cross section of the sample at different positions; a scanning unit that scans the sample against a first illumination surface by the first planar light and a second illumination surface by the second planar light; an imaging optical system that has an observation axis inclined with respect to the first illumination surface and the second illumination surface, and that forms an optical image of the first observation light generated in the sample by illumination with the first planar light and forms an optical image of the second observation light generated in the sample by illumination with the second planar light; an image acquisition unit that acquires, while the scanning unit is scanning the sample, first image data corresponding to the optical image of the first observation light and second image data corresponding to the optical image of the second observation light formed by the imaging optical system, respectively.

[0008] In this sample observation device, first and second planar lights having different wavelengths are irradiated onto the cross section of the sample so as to pass through different positions on the cross section. Then, observation image data of the sample is generated based on first image data and second image data corresponding to the optical images of the first and second observation lights generated on the sample by the irradiation of these planar lights. Therefore, in this sample observation device, even when planar lights of different wavelengths are irradiated onto the sample, observation image data of the sample can be obtained with a single scan, thereby improving the throughput until the observation image data is obtained.

[0009] The illumination optical system may illuminate the sample with the first planar light and the second planar light so that the first illumination surface and the second illumination surface are positioned offset from each other in the scanning direction of the sample or in a direction equivalent to the scanning direction of the sample. With this configuration, when the sample is scanned, the first illumination surface and the second illumination surface pass through the same position on the cross section of the sample, so that first image data and second image data for the cross section at the same position on the sample can be acquired. Therefore, observation results for the cross section at the same position on the sample can be obtained for each illumination with the first planar light and the second planar light.

[0010] If the width direction of the planar light is the X axis, the scanning direction of the sample is the Y axis, and the optical axis of the planar light is the Z axis, the image acquisition unit acquires a plurality of first XZ image data corresponding to the first image data and a plurality of second XZ image data corresponding to the second image data, and the image generation unit generates a plurality of first X image data by integrating the plurality of first XZ image data in the Z axis direction and a plurality of second X image data by integrating the plurality of second XZ image data in the Z axis direction, and may generate observation image data based on the first XY image data obtained by combining the plurality of first X image data in the Y axis direction and the second XY image data obtained by combining the plurality of second X image data in the Y axis direction. This configuration facilitates continuous scanning with planar light. Furthermore, since observation image data with reduced background can be generated, sample observation can be performed with high accuracy.

[0011] The image acquisition unit may have an imaging surface divided into a first imaging area for acquiring the first image data and a second imaging area for acquiring the second image data, thereby enabling the first image data and the second image data to be acquired on a single imaging surface, thereby enabling the device to be made more compact.

[0012] The first and second imaging areas may be continuous on the imaging surface, which allows for efficient use of the imaging surface.

[0013] The image generating unit may generate the observation image data in consideration of the number of frames of delay between the acquisition of the first image data and the acquisition of the second image data by the image acquiring unit, thereby enabling to obtain accurate observation results of the cross section of the sample at the same position in response to irradiation with the first planar light and the second planar light.

[0014] The image generating unit may generate regional image data relating to a region in the image where the sample exists using one of the first image data and the second image data, and generate observation image data based on the regional image data. In this case, for example, by identifying the region in which the sample exists using the image data from which it is easier to obtain an optical image of the observation light, information relating to the region in which the sample exists can be applied to analysis of the sample using the image data from which it is more difficult to obtain an optical image of the observation light. Therefore, sample observation can be performed with higher accuracy.

[0015] A sample observation method according to one aspect of the present disclosure includes an illumination step of illuminating a sample with first and second planar light having different wavelengths, respectively, so that the light passes through a cross section of the sample at different positions; a scanning step of scanning the sample against a first illumination surface by the first planar light and a second illumination surface by the second planar light; an imaging step of using an imaging optical system having an observation axis inclined with respect to the first illumination surface and the second illumination surface to form a light image of the first observation light generated in the sample by illumination with the first planar light and to form a second light image of the second observation light generated in the sample by illumination with the second planar light; an image acquisition step of acquiring, during scanning of the sample, first image data corresponding to the light image of the first observation light and second image data corresponding to the light image of the second observation light formed by the imaging optical system; and an image generation step of generating observation image data of the sample based on the first image data and the second image data.

[0016] In this sample observation method, first and second planar lights having different wavelengths are irradiated onto a cross section of the sample so as to pass through different positions on the cross section. Then, observation image data of the sample is generated based on first image data and second image data corresponding to the optical images of the first and second observation lights generated on the sample by the irradiation of these planar lights. Therefore, in this sample observation method, even when planar lights of different wavelengths are irradiated onto the sample, observation image data of the sample can be obtained with a single scan, thereby improving the throughput until the observation image data is obtained.

[0017] In the irradiating step, the first planar light and the second planar light may be irradiated onto the sample such that the first irradiated surface and the second irradiated surface are positioned offset in the scanning direction of the sample or in a direction equivalent to the scanning direction of the sample. With this configuration, when the sample is scanned, the first irradiated surface and the second irradiated surface pass through the same position on the cross section of the sample, so that first image data and second image data for the cross section at the same position on the sample can be acquired. Therefore, observation results of the cross section at the same position on the sample can be obtained for each irradiation with the first planar light and the second planar light.

[0018] If the width direction of the planar light is the X axis, the scanning direction of the sample is the Y axis, and the optical axis of the planar light is the Z axis, the image acquisition step acquires a plurality of first XZ image data corresponding to the first image data and a plurality of second XZ image data corresponding to the second image data, and the image generation step generates a plurality of first X image data by integrating the plurality of first XZ image data in the Z axis direction and a plurality of second X image data by integrating the plurality of second XZ image data in the Z axis direction, and the observation image data may be generated based on the first XY image data obtained by combining the plurality of first X image data in the Y axis direction and the second XY image data obtained by combining the plurality of second X image data in the Y axis direction. This configuration makes it easy to continuously perform scanning with planar light. Furthermore, since observation image data with reduced background can be generated, the sample can be observed with high accuracy.

[0019] In the image acquiring step, an imaging surface divided into a first imaging area for acquiring the first image data and a second imaging area for acquiring the second image data may be used, which makes it possible to acquire the first image data and the second image data on a single imaging surface, thereby achieving miniaturization of the device.

[0020] The first and second imaging areas may be continuous on the imaging surface, which allows for efficient use of the imaging surface.

[0021] In the image generating step, the observation image data may be generated taking into consideration the number of frames of delay between the acquisition of the first image data and the acquisition of the second image data, thereby enabling accurate observation results of the cross section of the sample at the same position upon irradiation with the first planar light and the second planar light.

[0022] In the image generating step, regional image data relating to a region in the image where the sample exists may be generated using one of the first image data and the second image data, and observation image data may be generated based on the regional image data. In this case, for example, by identifying the region in which the sample exists using the image data from which it is easier to obtain an optical image of the observation light, information relating to the region in which the sample exists can be applied to analysis of the sample using the image data from which it is more difficult to obtain an optical image of the observation light. Therefore, sample observation can be performed with higher accuracy. [Effects of the Invention]

[0023] According to the present disclosure, even when planar light of different wavelengths is irradiated onto a sample, the throughput until observation image data is obtained is improved. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a schematic configuration diagram showing an embodiment of a sample observation device according to the present disclosure. [Figure 2] FIG. 2 is an enlarged view of a main part showing the vicinity of a sample. [Figure 3] 2 is a flowchart showing an example of a sample observation method using the sample observation device shown in FIG. [Figure 4] 1A is a schematic diagram showing XZ image data corresponding to an optical image of a first observation light, and FIG. 1B is a schematic diagram showing XZ image data corresponding to an optical image of a second observation light. [Figure 5] (A) is a schematic diagram showing the positional relationship between the first observation light and the second observation light directed toward the imaging surface, and (B) is a schematic diagram showing the first imaging area and the second imaging area of ​​the imaging surface. [Figure 6] (A) is a schematic diagram showing how XY image data (observation image data) is generated from XZ image data corresponding to the optical image of the first observation light, and (B) is a schematic diagram showing how XY image data (observation image data) is generated from XZ image data corresponding to the optical image of the second observation light. [Figure 7] FIG. 10 is a schematic diagram illustrating parameters used to calculate the number of delay frames. [Figure 8] 10(A) and 10(B) are schematic diagrams showing modified examples of the imaging optical system. [Figure 9] 10 is a schematic diagram showing an example of generation of observation image data based on XZ image data. FIG. [Figure 10] 10 is a schematic diagram showing another example of generation of observed image data based on XZ image data. FIG. [Figure 11] 10 is a schematic diagram showing yet another example of generation of observation image data based on XZ image data. FIG. [Figure 12] 10 is a schematic diagram showing yet another example of generation of observation image data based on XZ image data. FIG. [Figure 13] FIG. 10 is a schematic diagram showing a modified example of the scanning unit. DETAILED DESCRIPTION OF THE INVENTION

[0025] Hereinafter, preferred embodiments of a sample observation device and a sample observation method according to one aspect of the present disclosure will be described in detail with reference to the drawings.

[0026] 1 is a schematic diagram showing the configuration of one embodiment of a sample observation device according to the present disclosure. This sample observation device 1 is configured as a device that irradiates a sample S with planar light (first planar light) L2A and planar light (second planar light) L2B having different wavelengths, and acquires observation image data of the inside of the sample S by forming an image of observation light (first observation light) L3A and observation light (second observation light) L3B generated inside the sample S on an imaging plane.

[0027] Examples of this type of sample observation device 1 include a slide scanner that acquires and displays an image of the sample S held on a glass slide, or a plate reader that acquires and analyzes image data of the sample S held on a microplate. Examples of the sample S to be observed include human or animal cells, tissues, organs, animals or plants themselves, plant cells, tissues, etc. Furthermore, the sample S may be contained in a solution, a gel, or a substance with a refractive index different from that of the sample S.

[0028] The light source 2 is a light source that outputs light to be irradiated onto the sample S. Examples of the light source 2 include laser light sources such as laser diodes and solid-state laser light sources. The light source 2 may also be a light-emitting diode, a superluminescent diode, or a lamp-based light source. The light source 2 outputs light L1A, which is the source of generating the planar light L2A, and light L1B, which is the source of generating the planar light L2B. The light L1A and the light L1B have different wavelengths. The light source 2 may have a function to switch the wavelength and intensity of each light when outputting the light L1A and the light L1B, or may be a combination of separate light sources that can output light of different wavelengths. The light L1A and L1B output from the light source 2 are guided to the irradiation optical system 3.

[0029] The irradiation optical system 3 is an optical system that shapes the light L1A and light L1B output from the light source 2 into planar light L2A and L2B, respectively, and irradiates the sample S with the shaped planar light L2A and L2B along an optical axis P1. In the following description, the optical axis P1 of the irradiation optical system 3 may also be referred to as the optical axis of the planar light L2A and L2B. The irradiation optical system 3 is configured to include a light shaping element such as a cylindrical lens, an axicon lens, or a spatial light modulator, and is optically coupled to the light source 2. The irradiation optical system 3 may be configured to include an objective lens.

[0030] The planar light beams L2A and L2B formed by the illumination optical system 3 are irradiated onto the sample S in the sample container 11. In the sample S irradiated with the planar light beams L2A and L2B, observation light beams L3A and L3B are generated at a surface (first illumination surface) Va irradiated with the planar light beam L2A and a surface (second illumination surface) Vb irradiated with the planar light beam L2B. Examples of the observation light beams L3A and L3B include fluorescence excited by the planar light beams L2A and L2B, scattered light of the planar light beams L2A and L2B, and diffusely reflected light of the planar light beams L2A and L2B. The sample observation device 1 of this embodiment employs, for example, a multi-excitation-single-fluorescence observation method. In this case, the observation light beams L3A and L3B are fluorescence beams (autofluorescence, etc.) of the same wavelength generated in the sample S by irradiation with the planar light beams L2A and L2B.

[0031] When observing the sample S in the thickness direction, it is preferable that the planar lights L2A and L2B be thin planar lights with a thickness of 2 mm or less, taking resolution into consideration. Furthermore, when the thickness of the sample S is very small, that is, when observing a sample S with a thickness equal to or less than the resolution in the Z-axis direction, the thickness of the planar lights L2A and L2B does not affect the resolution. Therefore, planar lights L2A and L2B with a thickness exceeding 2 mm may be used.

[0032] The scanning unit 4 is a mechanism for scanning the sample S across the irradiation surface V of the planar light L2. In this embodiment, the scanning unit 4 is configured with a moving stage 12 that moves a sample container 11 that holds the sample S. The sample container 11 is, for example, a microplate, a slide glass, a petri dish, etc. In this embodiment, a microplate is exemplified. As shown in FIG. 2, the sample container 11 has a plate-shaped main body 14 in which a plurality of wells 13 in which the samples S are placed are arranged in a straight line (or in a matrix), and a plate-shaped transparent member 15 provided on one side of the main body 14 so as to cover one end of the wells 13.

[0033] When placing the sample S in the well 13, the well 13 may be filled with a medium such as water. The transparent member 15 has an input surface 15a for the planar light L2A, L2B onto the sample S placed in the well 13. The material of the transparent member 15 is not particularly limited as long as it is a member that is transparent to the planar light L2, and may be, for example, glass, quartz, or synthetic resin. The sample container 11 is placed relative to the moving stage 12 so that the input surface 15a is perpendicular to the optical axis P1 of the planar light L2. The other end of the well 13 is open to the outside. The sample container 11 may be fixed to the moving stage 12.

[0034] As shown in FIG. 1, the moving stage 12 scans the sample container 11 in a predetermined direction at a constant speed in accordance with a control signal from the computer 7. In this embodiment, the moving stage 12 scans the sample container 11 in one direction within a plane perpendicular to the optical axis P1 of the planar light L2A and L2B. In the following description, the width direction of the planar light L2A and L2B is referred to as the X-axis, the scanning direction of the sample S by the scanning unit 4 is referred to as the Y-axis, and the direction of the optical axis P1 of the planar light L2A and L2B is referred to as the Z-axis. In this embodiment, the planar light L2A and L2B are incident on the sample S from the input surface 15a of the well 13 at positions offset from each other in the Y-axis direction. Therefore, the illumination surface Va of the planar light L2A and the illumination surface Vb of the planar light L2B are offset from each other in the scanning direction of the sample S. The illumination surface Va of the planar light L2A and the illumination surface Vb of the planar light L2B relative to the sample S are both within the XZ plane.

[0035] The imaging optical system 5 is an optical system that forms images of the observation light beams L3A and L3B generated in the sample S by irradiation with the planar light beams L2A and L2B. The imaging optical system 5 is configured to include, for example, an objective lens 16, a filter 17, and a relay lens 18. In this embodiment, the observation light beams L3A and L3B are fluorescence beams of the same wavelength generated in the sample S, and therefore, the filter 17 can be, for example, a bandpass filter that has a transmission band that includes the wavelength of the fluorescence.

[0036] The optical axis of the imaging optical system 5 is the optical axis of the observation lights L3A and L3B (hereinafter referred to as the "observation axis P2"). The observation axis P2 is inclined at an inclination angle θ with respect to each of the irradiation surfaces Va and Vb of the planar light L2 on the sample S. The inclination angle θ also coincides with the angle formed by the optical axis P1 of the planar light L2A and L2B directed toward the sample S and the observation axis P2. The inclination angle θ is, for example, 10° to 80°. From the viewpoint of improving the resolution of the observed image, the inclination angle θ is preferably 20° to 70°. Furthermore, from the viewpoint of improving the resolution of the observed image and stabilizing the field of view, the inclination angle θ is more preferably 30° to 65°.

[0037] The image acquisition unit 6 is a device that acquires first image data corresponding to the optical image of the observation light L3A and second image data corresponding to the optical image of the observation light L3B formed by the imaging optical system 5 during scanning of the sample S. The image acquisition unit 6 is configured to include, for example, an imaging device that captures the optical images of the observation light L3A and L3B. Examples of the imaging device include area image sensors such as CMOS image sensors and CCD image sensors. These area image sensors are placed on the imaging plane formed by the imaging optical system 5 and output two-dimensional image data to a computer 7. The readout method of the imaging device may be a global shutter method in which the exposure periods of each pixel row are the same, or a rolling shutter method in which the exposure periods of each pixel row are shifted by a predetermined time.

[0038] The image acquisition unit 6 has an imaging surface 31 that captures optical images by the observation lights L3A and L3B (see FIGS. 5(A) and 5(B)). The imaging surface 31 is divided into two: a first imaging area 31A that acquires first image data, and a second imaging area 31B that acquires second image data. In this embodiment, the first imaging area 31A and the second imaging area 31B are both rectangular regions and are continuous on the imaging surface 31.

[0039] The computer 7 is physically configured to include memories such as RAM and ROM, a processor (arithmetic circuit) such as a CPU, a communication interface, a storage unit such as a hard disk, and a display unit such as a display. Examples of such a computer 7 include a personal computer, a cloud server, and a smart device (a smartphone, a tablet terminal, etc.). By executing a program stored in the memory on the CPU of the computer system, the computer 7 functions as a controller that controls the operation of the light source 2 and the moving stage 12, an image generation unit 8 that generates observation image data of the sample S, and an analysis unit 10 that analyzes the observation image data.

[0040] The computer 7 as a controller receives a user's operation to start measurement and synchronously drives the light source 2, the scanning unit 4, and the image acquisition unit 6. In this case, the computer 7 may control the light source 2 so that the light source 2 continuously outputs the lights L1A and L1B while the sample S is being moved by the moving stage 12, or may control the ON / OFF of the output of the lights L1A and L1B from the light source 2 in accordance with the image capture by the image acquisition unit 6. Furthermore, if the irradiation optical system 3 is equipped with an optical shutter (not shown), the computer 7 may control the optical shutter to turn ON / OFF the irradiation of the planar lights L2A and L2B onto the sample S.

[0041] The computer 7 serving as the image generation unit 8 generates observation image data of the sample S based on the image data generated by the image acquisition unit 6. The image generation unit 8 generates observation image data of the sample S, for example, in a plane (XY plane) perpendicular to the optical axis P1 of the planar lights L2A and L2B, based on the image data output from the image acquisition unit 6. The image generation unit 8 stores the generated observation image data, displays it on a monitor, etc., in accordance with a predetermined operation by the user, for example.

[0042] 3 is a flowchart showing an example of a sample observation method using a sample observation device. As shown in the figure, this sample observation method includes an irradiation step (step S01), a scanning step (step S02), an imaging step (step S03), an image acquisition step (step S04), an image generation step (step S05), and an analysis step (step S06).

[0043] In the irradiation step S01, planar light L2A and L2B are irradiated onto the sample S so that the irradiation surface Va of the planar light L2A and the irradiation surface Vb of the planar light L2B on the sample S are shifted from each other in the scanning direction of the sample S. When a user inputs an operation to start measurement, the light source 2 is driven based on a control signal from the computer 7, and light L1A and L1B are output from the light source 2. The light L1A and L1B output from the light source 2 are shaped by the irradiation optical system 3 to become planar light L2A and L2B, and are irradiated onto the sample S so as to pass through cross sections of the sample S at different positions.

[0044] In the scanning step S02, the sample S is scanned with the irradiation surface Va of the planar light L2A and the irradiation surface Vb of the planar light L2B. When the user inputs an operation to start measurement, the moving stage 12 is driven in synchronization with the driving of the light source 2 based on a control signal from the computer 7. As a result, the sample container 11 is driven linearly in the Y-axis direction at a constant speed, and the sample S in the well 13 is scanned with the irradiation surfaces Va and Vb of the planar light L2A and L2B. Here, the irradiation surface Va is located on the front side in the scanning direction, and the irradiation surface Vb is located on the back side in the scanning direction. Therefore, the scanning of a cross section at a certain position of the sample S with the planar light L2A precedes in time the scanning of the same cross section of the sample S with the planar light L2B.

[0045] In the imaging step S03, an imaging optical system 5 having an observation axis P2 inclined with respect to the irradiation plane V is used to image the observation light beams L3A and L3B generated in the sample S by irradiation with the planar light beams L2A and L2B onto the imaging plane of the image acquisition unit 6. In the image acquisition step S04, a plurality of first image data corresponding to the optical image of the observation light beam L3A formed by the imaging optical system 5 during scanning of the sample S is acquired in the Y-axis direction. In addition, a plurality of second image data corresponding to the optical image of the observation light beam L3B formed by the imaging optical system 5 during scanning of the sample S is acquired in the Y-axis direction.

[0046] More specifically, in the image acquisition step S04, as shown in Fig. 4(A), a plurality of XZ image data (first image data) 21A corresponding to the optical image of the observation light L3A is acquired in the Y-axis direction. Also, as shown in Fig. 4(B), a plurality of XZ image data (second image data) 21B corresponding to the optical image of the observation light L3B is acquired in the Y-axis direction. The acquired plurality of XZ image data 21A and plurality of XZ image data 21B are sequentially output from the image acquisition unit 6 to the image generation unit 8.

[0047] As described above, in the sample observation device 1, the sample S is irradiated with the planar light L2A and L2B so that the irradiation surfaces Va and Vb are positioned offset in the scanning direction of the sample S. In addition, the imaging optical system 5 has an observation axis P2 that is inclined with respect to the irradiation surfaces Va and Vb. Therefore, as shown in FIG. 5(A), the observation light L3A generated at the irradiation surface Va of the planar light L2A and the observation light L3B generated at the irradiation surface Vb of the planar light L2B are guided to the image acquisition unit 6 in a state where they are offset from each other in a direction perpendicular to the observation axis P2.

[0048] 5(B), on the imaging surface 31 of the image acquisition unit 6, XZ image data 21A corresponding to the optical image of the observation light L3A is acquired in the first imaging area 31A, and XZ image data 21B corresponding to the optical image of the observation light L3B is acquired in the second imaging area 31B. In the example of FIG. 5(B), the optical image acquired in the first imaging area 31A and the optical image acquired in the second imaging area 31B are aligned in a direction corresponding to the Z-axis direction of the cross section of the sample S.

[0049] In the image generation step S05, observation image data of the sample S is generated based on the plurality of XZ image data 21. Here, as shown in Fig. 6(A), the brightness values ​​of each pixel included in the plurality of XZ image data 21A obtained in the image acquisition step S04 are integrated in the Z-axis direction to generate a plurality of X image data (first X image data) 22A. Then, the plurality of X image data 22A are combined in the Y-axis direction to generate XY image data (first XY image data) 23A.

[0050] 6(B), the brightness values ​​of each pixel included in the plurality of XZ image data 21B obtained in the image acquisition step S04 are integrated in the Z-axis direction to generate a plurality of X image data (second X image data) 22B. Then, the plurality of X image data 22B are combined in the Y-axis direction to generate XY image data (second XY image data) 23B. These XY image data 23A and 23B become observation image data 24A and 24B, respectively, which show XY cross-sectional images of the sample S at any position in the Z-axis direction and having any thickness.

[0051] In generating the X image data 22A, 22B, the luminance values ​​of each pixel in an arbitrary range in the Z-axis direction in the plurality of XZ image data 21A, 21B may be integrated in the Z-axis direction. In generating the XY image data 23A, 23B from the plurality of X image data 22A, 22B, the XY image data 23A, 23B may be directly generated by reconstructing the plurality of X image data 22A, 22B.

[0052] As described above, in the sample observation device 1, the irradiation surface Va is located on the front side in the scanning direction, and the irradiation surface Vb is located on the back side in the scanning direction, and the scanning of the sample S with the planar light L2A precedes in time the scanning of the sample S with the planar light L2B. Therefore, in the image acquisition unit 6, the timing of acquiring XZ image data 21B at any position on the sample S is delayed by a predetermined number of frames from the timing of acquiring XZ image data 21A at the same position on the sample S.

[0053] 7(A) and 7(B) are diagrams showing examples of calculating the number of delay frames. These calculation examples are all based on the premise that the thickness T of the sample S in the Z-axis direction is smaller than the observation depth D of the planar light L2A and L2B. The observation depth D corresponds to the length in the Z-axis direction of the planar light L2A and L2B that can be observed in the first imaging area 31A and the second imaging area 31B of the imaging surface 31. FIG. 7(A) illustrates a case where the refractive index n1 of the medium A in which the imaging optical system 5 and the image acquisition unit 6 are located is equal to the refractive index n2 of the medium B in which the irradiation surface Va of the planar light L2A and the irradiation surface Vb of the planar light L2B are located. In this case, the distance in the Y-axis direction between the planar light L2A and L2B is L, the inclination angle θ of the observation axis P2 with respect to the irradiation surfaces Va and Vb is θ, and the resolution in the scanning direction is Y. resо Then, the number of delay frames Δf is Δf=L / Y resо L can be calculated by L≧D×tanθ.

[0054] 7B illustrates a case where the refractive index n1 of the medium A where the imaging optical system 5 and the image acquisition unit 6 are located is greater than the refractive index n2 of the medium B where the irradiation surface Va of the planar light L2A and the irradiation surface Vb of the planar light L2B are located. In this case as well, the number of delay frames Δf is given by Δf=L / Y resо L is the distance at the boundary between media A and B. Tilt angle of observation axis P2 on medium B side When θ' is used, it can be calculated by L≧D×tanθ' and sinθ'=(n1 / n2)sinθ.

[0055] In both cases of FIG. 7(A) and FIG. 7(B), the timing of acquiring the XZ image data 21B for the same position on the sample S is determined by the distance L between the planar lights L2A and L2B and the resolution in the scanning direction Y. resо Therefore, if the frame in which the n-th XZ image data 21A is acquired is denoted by f(n), by associating the n-th XZ image data 21A with the f(n+Δf)-th XZ image data 21B, it is possible to obtain accurate observation results of the cross section of the sample S at the same position in response to the irradiation of each of the planar lights L2A and L2B.

[0056] As described above, the sample observation device 1 irradiates the sample S with planar light L2A and L2B having different wavelengths so as to pass through cross sections at different positions on the sample S. Then, observation image data 24A and 24B of the sample S are generated based on first image data corresponding to the light image of observation light L3A and second image data corresponding to the light image of observation light L3B generated on the sample S by irradiating the sample S with these planar light L2A and L2B. Therefore, in the sample observation device 1, even when the sample S is irradiated with planar light L2A and L2B of different wavelengths, the observation image data 24A and 24B of the sample S can be acquired with a single scan, thereby improving the throughput until the observation image data 24A and 24B are obtained.

[0057] In the sample observation device 1, the sample S is irradiated with planar light L2A and L2B so that the irradiation surfaces Va and Vb are positioned offset in the scanning direction of the sample S. As a result, when the sample S is scanned, the irradiation surfaces Va and Vb pass through the same position on the cross section of the sample S, so that first image data and second image data for the cross section at the same position on the sample S can be acquired. Therefore, observation results for the cross section at the same position on the sample S can be obtained for each irradiation with the planar light L2A and L2B.

[0058] The sample observation device 1 acquires a plurality of XZ image data 21A corresponding to the first image data and a plurality of XZ image data 21B corresponding to the second image data, and generates a plurality of X image data 22A obtained by integrating the plurality of XZ image data 21A in the Z-axis direction and a plurality of X image data 22B obtained by integrating the plurality of XZ image data 21B in the Z-axis direction. Then, observation image data 24A and 24B are generated based on XY image data 23A obtained by combining the plurality of X image data 22A in the Y-axis direction and XY image data 23B obtained by combining the plurality of X image data 22B in the Y-axis direction. This configuration makes it easy to continuously scan the planar light L2A and L2B. Furthermore, since observation image data 24A and 24B with reduced background can be generated, the sample S can be observed with high accuracy.

[0059] In the sample observation device 1, the image acquisition unit 6 has an imaging surface 31 divided into a first imaging area 31A for acquiring first image data and a second imaging area 31B for acquiring second image data. This makes it possible to acquire the first image data and the second image data on a single imaging surface 31, thereby achieving miniaturization of the device. In this embodiment, the first imaging area 31A and the second imaging area 31B are continuous on the imaging surface 31. This allows efficient use of the imaging surface 31.

[0060] The first imaging area 31A and the second imaging area 31B do not necessarily have to be continuous on the imaging surface 31. That is, one or more pixel rows that do not contribute to capturing an optical image may exist between the first imaging area 31A and the second imaging area 31B. Also, the first imaging area 31A and the second imaging area 31B may partially overlap on the imaging surface 31.

[0061] The present disclosure is not limited to the above-described embodiment. For example, in the above-described embodiment, the observation method of the sample observation device 1 is multi-excitation single fluorescence, and fluorescence of the same wavelength generated in the sample S is used as the observation light L3A, L3B. However, the observation method of the sample observation device 1 is not limited to this, and may be, for example, multi-excitation multi-fluorescence. When the observation method is multi-excitation multi-fluorescence, the filter 17 of the imaging optical system 5 may be configured by a multi-bandpass filter having multiple transmission bands instead of a bandpass filter.

[0062] Furthermore, when the observation method is multi-excitation multi-fluorescence, an imaging optical system 5 may be employed in which a dichroic mirror 35 splits the observation light beams L3A and L3B, as shown in FIG. 8A. In the example of FIG. 8A, image acquisition units 6A and 6B are disposed corresponding to the split observation light beams L3A and L3B. In this case, the observation light beams L3A and L3B passing through the objective lens 16 are split by the dichroic mirror 35. One of the split observation light beams L3A and L3B passes through a band-pass filter 36 and a relay lens 18 and enters the image acquisition unit 6A. The other split observation light beam L3A and L3B is reflected by a mirror 37 and passes through a band-pass filter 38 and a relay lens 18 and enters the image acquisition unit 6B. This configuration allows the band-pass filters 36 and 38 to select fluorescent images of different wavelengths, enabling more detailed observation of the sample S. Note that one or both of the band-pass filters 36 and 38 may be multi-band-pass filters.

[0063] 8(B), a single image acquisition unit 6 may be disposed in an imaging optical system 5 in which a dichroic mirror 35 splits the observation light beams L3A and L3B. One of the observation light beams L3A and L3B split by the dichroic mirror 35 passes through a bandpass filter 36 and a relay lens 18 and enters the image acquisition unit 6. The other of the observation light beams L3A and L3B split by the dichroic mirror 35 is reflected by a mirror 37 and passes through a bandpass filter 38. The other of the observation light beams L3A and L3B is then reflected by a mirror 39 and combined with the one of the observation light beams L3A and L3B by a dichroic mirror 40 downstream of the bandpass filter 36, and then enters the image acquisition unit 6 via the relay lens 18. As in the case of FIG. 8(A), one or both of the bandpass filters 36 and 38 may be multi-bandpass filters.

[0064] Even with this configuration, the bandpass filters 36, 38 can be used to select fluorescent images of different wavelengths, allowing for more detailed observation of the sample S. When the configuration of FIG. 8(B) is employed, the imaging surface 31 of the image acquisition unit 6 may be provided with a first imaging area 31A and a second imaging area 31B for acquiring optical images of one of the branched observation lights L3A, L3B, and a first imaging area 31A and a second imaging area 31B for acquiring optical images of the other of the branched observation lights L3A, L3B. This makes it possible to acquire first image data and second image data of fluorescent images of different wavelengths on a single imaging surface 31.

[0065] Furthermore, the sample observation device 1 may generate regional image data relating to the region in the image where the sample S is present using one of the first image data and the second image data, and then generate observation image data based on the regional image data. For example, when observing multiple fluorescent images of the sample S, it is expected that, depending on the type of fluorescent substance, some observation lights L3A and L3B will be easier to obtain an optical image from and others will be more difficult to obtain. Furthermore, since the intensity of fluorescence, such as autofluorescence, is generally weaker than the intensity of scattered light, if one of the observation lights L3A and L3B is scattered light and the other is fluorescent light, it is expected that it will be relatively difficult to obtain a fluorescent image. In such a case, by generating regional image data using the observation light L3A or L3B that is easier to obtain an optical image from and extracting regional information about the sample S, it becomes possible to facilitate analysis of the sample S when using the optical image that is more difficult to obtain an optical image from.

[0066] Specifically, assuming that the observation light that is easier to obtain an optical image is observation light L3A and the observation light that is harder to obtain an optical image is observation light L3B, first, observation image data 24A is generated based on multiple XZ image data 21A corresponding to observation light L3A. Next, information X (e.g., area information and brightness information) regarding the presence area of ​​sample S is extracted based on the observation image data 24A. Thereafter, observation image data 24B is generated based on multiple XZ image data 21B corresponding to observation light L3B, and the previously extracted information X is used to perform analysis of sample S based on the observation image data 24B.

[0067] In the example of FIG. 9, the sample observation device 1 acquires multiple XZ image data 21A of the observation light L3A and multiple XZ image data 21B of the observation light L3B. Next, each XZ image data 21B is binarized to generate multiple binary XZ image data 25B. Then, based on this multiple binary XZ image data 25B, regional image data 32B related to the presence region of the sample S is generated. In the example of FIG. 9, the entirety of each binary XZ image data 25B is used as regional image data 32B. Pixels corresponding to the regional image data 32B are extracted from each XZ image data 21A of the observation light L3A to generate observation image data 24A.

[0068] In the example of Figure 10, only the bottom region of each binarized XZ image data 25B obtained from XZ image data 21B of observation light L3B is selectively set as regional image data 32B. Pixels corresponding to the bottom region are extracted from each XZ image data 21A corresponding to observation light L3A to generate observation image data 24A. In the example of Figure 11, an arbitrary region (here, the center region) based on each binarized XZ image data 25B obtained from XZ image data 21B of observation light L3B is selectively set as regional image data 32B. Pixels corresponding to the arbitrary region are extracted from each XZ image data 21A corresponding to observation light L3A to generate observation image data 24A.

[0069] In the example of FIG. 12, multiple XZ image data 21A of the observation light L3A are acquired, and multiple XZ image data 21B of the observation light L3B are acquired. Each XZ image data 21A of the observation light L3A is binarized to generate multiple first binarized XZ image data 25A as luminance image data. Each XZ image data 21B of the observation light L3B is also binarized to generate multiple second binarized XZ image data 25B. Then, based on the multiple second binarized XZ image data 25B, regional image data 32B related to the presence region of the sample S is generated. In the example of FIG. 12, an arbitrary region (here, a center region) based on each second binarized XZ image data 25B is selectively set as regional image data 32B. Then, pixels corresponding to the regional image data 32B are extracted from each first binarized XZ image data 25A corresponding to the observation light L3A to generate observation image data 24A.

[0070] 9, 10, 11, and 12, a plurality of observation image data 24A, which are XZ images, may be integrated in the Z-axis direction to generate a plurality of X-image data 24B. Then, new observation image data 24D may be generated based on XY image data 24C obtained by combining the plurality of X-image data 24B in the Y-axis direction.

[0071] 9, 10, 11, and 12, the observation light L3A and the observation light L3B may be any of fluorescence excited by the planar light L2A and L2B, scattered light of the planar light L2A and L2B, and diffusely reflected light of the planar light L2A and L2B. The observation light L3A and the observation light L3B may be formed by combining different types of light. For example, the observation light L3A may be fluorescence excited by the planar light L2A, and the observation light L3B may be scattered light of the planar light L2B.

[0072] Furthermore, for example, in the above embodiment, the scanning unit 4 scans the sample container 11 in the Y-axis direction using the moving stage 12, but the manner of scanning by the scanning unit 4 is not limited to this. For example, as in the sample observation device 1 shown in Fig. 13, the scanning unit 4 may have, instead of the moving stage 12, a rotation axis 51 that rotates the sample container 11 around its central axis.

[0073] In this case, the irradiation surfaces Va and Vb may be positioned offset in a direction equivalent to the tangential direction of the rotation of the rotation axis (R-axis direction), for example. In the example of FIG. 13, the irradiation surfaces Va and Vb are offset from each other in the Y-axis direction, similar to the example of FIG. 1, which is a direction equivalent to the R-axis direction. In addition, the image generation unit 8 generates multiple X-image data by integrating multiple XZ-image data acquired by the image acquisition unit 6 in the Z-axis direction, and generates observation image data 24 using XY-image data obtained by reconstructing these multiple X-image data in the R-axis direction. This configuration enables efficient scanning of the planar light beams L2A and L2B on the sample S held in the circular sample container 11. [Explanation of symbols]

[0074] 1...sample observation device, 3...illumination optical system, 4...scanning unit, 5...imaging optical system, 6...image acquisition unit, 8...image generation unit, 21A, 21B...XZ image data (first image data, second image data), 22A, 22B...X image data (first X image data, second X image data), 23A, 23B...XY image data (first XY image data, second XY image data), 24A, 24B...observation image data, 31...imaging surface, 31A...first imaging area, 31B...second imaging area, 32B...region image data, L2A, L2B...planar light (first planar light, second planar light), L3A, L3B...observation light (first observation light, second observation light), P2...observation axis, S...sample, Va, Vb...irradiation surface (first irradiation surface, second irradiation surface).

Claims

1. an illumination optical system that illuminates a first planar light beam and a second planar light beam having different wavelengths so as to pass through a cross section of the sample at different positions; a scanning unit that scans the sample with respect to a first irradiation surface by the first planar light and a second irradiation surface by the second planar light; an imaging optical system having an observation axis inclined with respect to the first irradiation surface and the second irradiation surface, which forms an optical image of the first observation light generated on the sample by irradiation with the first planar light and an optical image of the second observation light generated on the sample by irradiation with the second planar light; an image acquisition unit that acquires, during scanning of the sample by the scanning unit, first image data corresponding to the optical image of the first observation light and second image data corresponding to the optical image of the second observation light formed by the imaging optical system; an image generating unit that generates observation image data of the sample based on the first image data and the second image data, a sample observation device in which a relationship of L≧D×tan θ′ is established, where an X-axis represents a width direction of the planar light, a Y-axis represents a scanning direction of the sample, a Z-axis represents an optical axis of the planar light, L represents a distance between a first planar light and a second planar light in the Y-axis direction, D represents an observation depth of the planar light in the Z-axis direction, and θ′ represents an inclination angle of the observation axis on the medium B side at a boundary between a medium A in which the imaging optical system and the image acquisition unit are located and a medium B in which a surface irradiated with the first planar light and a surface irradiated with the second planar light are located.

2. 2. The sample observation device according to claim 1, wherein the illumination optical system illuminates the sample with the first planar light and the second planar light so that the first illumination surface and the second illumination surface are positioned to be shifted in a scanning direction of the sample or a direction equivalent to the scanning direction of the sample.

3. When the width direction of the planar light is the X axis, the scanning direction of the sample is the Y axis, and the optical axis of the planar light is the Z axis, the image acquisition unit acquires a plurality of first XZ image data corresponding to the first image data and a plurality of second XZ image data corresponding to the second image data; 3. The sample observation device according to claim 1, wherein the image generation unit generates a plurality of first X image data pieces obtained by integrating the plurality of first XZ image data pieces in the Z-axis direction and a plurality of second X image data pieces obtained by integrating the plurality of second XZ image data pieces in the Z-axis direction, and generates the observation image data based on first XY image data obtained by combining the plurality of first X image data pieces in the Y-axis direction and second XY image data obtained by combining the plurality of second X image data pieces in the Y-axis direction.

4. 4. A sample observation device according to claim 1, wherein the image acquisition unit has an imaging surface divided into a first imaging area for acquiring the first image data and a second imaging area for acquiring the second image data.

5. 5. A sample observation device according to claim 4, wherein the first imaging area and the second imaging area are continuous on the imaging surface.

6. 6. A sample observation device according to claim 1, wherein the image generation unit generates the observation image data by taking into account the number of delay frames between the acquisition of the first image data and the acquisition of the second image data in the image acquisition unit.

7. 7. A sample observation device according to claim 1, wherein the image generation unit generates regional image data relating to a region in the image where the sample is present, using one of the first image data and the second image data, and generates the observation image data based on the regional image data.

8. an irradiation step of irradiating a first planar light beam and a second planar light beam having different wavelengths onto a cross section of the sample at different positions; a scanning step of scanning the sample with respect to a first surface irradiated by the first planar light and a second surface irradiated by the second planar light; an imaging step of using an imaging optical system having an observation axis inclined with respect to the first irradiation surface and the second irradiation surface to form an optical image of the first observation light generated on the sample by irradiation with the first planar light and an optical image of the second observation light generated on the sample by irradiation with the second planar light; an image acquiring step of acquiring, by an image acquiring unit, first image data corresponding to the optical image of the first observation light and second image data corresponding to the optical image of the second observation light formed by the imaging optical system during scanning of the sample; an image generating step of generating observation image data of the sample based on the first image data and the second image data, a width direction of the planar light is an X-axis, a scanning direction of the sample is a Y-axis, an optical axis of the planar light is a Z-axis, a distance between the first planar light and the second planar light in the Y-axis direction is L, an observation depth of the planar light in the Z-axis direction is D, and an inclination angle of the observation axis on the medium B side at a boundary surface between a medium A in which the imaging optical system and the image acquisition unit are located and a medium B in which a surface irradiated with the first planar light and a surface irradiated with the second planar light are located is θ', the relationship L≧D×tan θ' is established.

9. 9. A sample observation method according to claim 8, wherein in the irradiating step, the first planar light and the second planar light are irradiated onto the sample such that the first irradiation surface and the second irradiation surface are positioned offset from each other in a scanning direction of the sample or a direction equivalent to the scanning direction of the sample.

10. When the width direction of the planar light is the X axis, the scanning direction of the sample is the Y axis, and the optical axis of the planar light is the Z axis, In the image acquisition step, a plurality of first XZ image data corresponding to the first image data and a plurality of second XZ image data corresponding to the second image data are acquired, 10. A sample observation method according to claim 8 or 9, wherein the image generating step generates a plurality of first X image data by integrating the plurality of first XZ image data in the Z-axis direction and a plurality of second X image data by integrating the plurality of second XZ image data in the Z-axis direction, and generates the observation image data based on first XY image data obtained by combining the plurality of first X image data in the Y-axis direction and second XY image data obtained by combining the plurality of second X image data in the Y-axis direction.

11. A sample observation method according to any one of claims 8 to 10, wherein the image acquisition step uses an imaging surface divided into a first imaging area for acquiring the first image data and a second imaging area for acquiring the second image data.

12. 12. A sample observation method according to claim 11, wherein the first imaging area and the second imaging area are continuous on the imaging surface.

13. A sample observation method according to any one of claims 8 to 12, wherein the image generation step generates the observation image data taking into account the number of frames of delay between the acquisition of the first image data and the acquisition of the second image data.

14. A sample observation method according to any one of claims 8 to 13, wherein in the image generation step, regional image data relating to a region in the image where the sample is present is generated using one of the first image data and the second image data, and the observation image data is generated based on the regional image data.

Images