Sample observation method, sample observation device, and microscope

Abstract

To provide a sample observation method, a sample observation device and a microscope that make it possible to measure scattering properties of a spatially inhomogeneous object.SOLUTION: A sample observation method and a sample observation device according to the present invention can control a range of scattering angles using the frequency (that is, the size of the bright and dark areas) of the binary illumination pattern, which is composed of light-transmitting (bright) and light-blocking (dark) parts like a checkered pattern. In other words, the present sample observation method measures scattering properties of a spatially inhomogeneous object by taking advantage of that an optical sensor receives light rays with a wide scattering angle when the size of a light area is large (low frequency) and light rays with a narrow scattering angle when the size of the light area is narrow (high frequency).SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to a sample observation method and device for measuring the scattering characteristics of a sample, and a microscope equipped with the same. [Background technology]

[0002] In addition to direct reflection and absorption, scattering is one of the essential optical properties. In recent years, scattering properties, including the angular distribution of scattering, have been used in optical imaging and diagnostic applications, such as cell detection, analysis of cancer processes, and diagnosis of dysplasia and cancer (see, for example, Non-Patent Documents 1 to 3). For this reason, visualization of scattering properties, which was not possible with conventional microscopy, is expected to be applied to biomedical image analysis, etc. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] LDrezek, R., et al.: Light scattering from cervical cells throughout neoplastic progression:influence of nuclear morphology, DNA content, and chromatin texture. J.Biomed. Opt. 8(1), 7-16 (2003). [Non-Patent Document 2] Lin, X., Wan, N., Weng, L., Zhou, Y.: Light scattering from normal and cervical cancer cells. Appl. Opt. 56, 3608 (2017). [Non-Patent Document 3] Lovat, L., et al.: 4919 a novel optical biopsy technique using elastic scattering spectroscopy for dysplasia and cancer in barrett's esophagus. Gastrointest. Endosc.51, AN227 (2000). [Non-Patent Document 4] Narasimhan, S., Gupta, M., Donner, C., Ramamoorthi, R., Nayar, S., Jensen, HW: Acquiring scattering properties of participating media by dilution. ACM Trans. Graph. 25(3), 1003-1012 (2006) [Non-Patent Document 5] Nayar, SK, Krishnan, G., Grossberg, MD, Raskar, R.: Fast Separation of Direct and Global Components of a Scene Using High Frequency Illumination. ACM Transactions on Graphics 25(3), pp. 935-944 (2006) [Non-Patent Document 6] Tanaka, K., Mukaigawa, Y., Kubo, H., Matsushita, Y., Yagi, Y.: Descattering of Transmissive Observation using Parallel High-frequency Illumination. In: IEEE Con-ference on Computational Photography, (2013) Summary of the Invention [Problem to be solved by the invention]

[0004] FIG. 1 is an example of an intensity distribution that explains scattering characteristics. As shown in FIG. 1, scattering is a phenomenon in which particles 31 in tissue receive incident light 32 and re-emit light 33 in different directions with different intensities. In this specification, the intensity distribution of scattered light rays at different angles (scattering angles) is called "scattering characteristics." The distribution in FIG. 1(b) is an example of forward scattering, in which the light intensity is high at small scattering angles (e.g., θ1), and the light intensity drops sharply as the scattering angle increases (e.g., θ2, θ3). In contrast, FIG. 1(a) is an example of isotropic scattering, in which the light intensity is equal at all scattering angles.

[0005] In order to measure such scattering characteristics, it is necessary to measure the intensity distribution of light rays at multiple scattering angles. For example, Non-Patent Document 4 discloses a method for measuring scattering characteristics by observing the angular distribution of light scattered in a homogeneous liquid.

[0006] However, the method of Non-Patent Document 4 is a method for measuring the scattering characteristics of a liquid, and cannot measure the scattering characteristics of a spatially inhomogeneous object. Therefore, there is currently a problem that it is difficult to detect cells, analyze the cancer process, and diagnose dysplasia or cancer, which require measuring the scattering characteristics of a spatially inhomogeneous object, as described above.

[0007] Therefore, in order to solve the above problems, an object of the present invention is to provide a sample observation method, a sample observation device, and a microscope that make it possible to measure the scattering characteristics of a spatially inhomogeneous object. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the sample observation method and sample observation device of the present invention decompose the captured light into direct scattered light for each angle by changing the spatial frequency of the illumination pattern, which can control the range of scattering angles.

[0009] Specifically, the sample observation method according to the present invention comprises: a focusing step of aligning the focal point of the optical sensor and the focal point of the illumination pattern, which combines light areas that transmit light and dark areas that block light at an arbitrary ratio, on an XY plane including an observation point in the sample in the Z direction; a light intensity acquisition step of acquiring, by the light sensor, a first light intensity when the light transmitted through the bright portion of the illumination pattern is irradiated onto the observation point and a second light intensity when the light is irradiated onto a point other than the observation point, after the focusing step; a separation step of separating the direct component and the global component from the first light intensity by using a direct component due to single scattered light at the observation point and a global component other than the direct component, which are included in the first light intensity, and the global component included in the second light intensity; a repeating step of repeatedly performing the light intensity acquisition step and the separation step by changing a period between the bright and dark areas of the illumination pattern; a calculation step of calculating a light intensity for each scattering angle at the observation point by comparing the direct components acquired for each period of the illumination pattern; Do the following.

[0010] Furthermore, the sample observation device according to the present invention comprises: A lighting pattern in which light transmitting bright areas and light blocking dark areas are combined in any ratio; a focusing mechanism for adjusting the focus of the optical sensor and the illumination pattern in the Z direction to an XY plane including an observation point within the sample; a light intensity acquiring means for acquiring, with the light sensor, a first light intensity when the light of the illumination pattern is irradiated onto the observation point and a second light intensity when the light of the illumination pattern is irradiated onto a location other than the observation point, while the focus mechanism is set to focus both the light sensor and the illumination pattern on the observation point; a calculator that uses a direct component due to single scattered light at the observation point and a global component other than the direct component, which is included in the first light intensity, and the global component included in the second light intensity, and separates the direct component and the global component from the first light intensity by changing a period of the bright part and the dark part of the illumination pattern, and calculates a light intensity for each scattering angle at the observation point by comparing the direct component obtained for each period of the illumination pattern; Equipped with.

[0011] In order to visualize the observed sample, the computing unit estimating parameters of a scattering model at the observation point from the light intensity for each scattering angle; and Arranging the parameters for each of the observation points according to the coordinates of the observation points on the XY plane to generate a scattering characteristic map. Further steps will be taken.

[0012] Preferably, the scattering property map is visualized based on the parameters.

[0013] Here, it is preferable that the illumination pattern is a checkerboard pattern.

[0014] Furthermore, the sample observation device can be mounted on a light transmission type microscope.

[0015] The present invention is a method for extracting distribution information of the scattering spread angle by high-frequency illumination projection under a microscope. The relationship between the size of a spatially high-frequency illumination pattern and the scattering angle is derived, and based on this, the distribution of the scattering spread angle at each position in the sample is estimated by projecting the size of multiple high-frequency illumination patterns.

[0016] Therefore, the present invention can provide a sample observation method, a sample observation device, and a microscope that make it possible to measure the scattering characteristics of a spatially inhomogeneous object. Effect of the Invention

[0017] The present invention can provide a sample observation method, a sample observation device, and a microscope that make it possible to measure the scattering characteristics of a spatially inhomogeneous object. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is a diagram illustrating scattering characteristics. [Diagram 2] FIG. 2 is a diagram illustrating transmitted light and scattered light. [Diagram 3] 1A to 1C are diagrams illustrating an observation method related to the present invention. [Figure 4] 1A to 1C are diagrams illustrating an observation method related to the present invention. [Diagram 5] FIG. 1 is a diagram illustrating a microscope according to the present invention. [Figure 6] FIG. 1 is a diagram illustrating a microscope according to the present invention. [Figure 7] 1A to 1C are diagrams illustrating the measurement principle of a sample observation device according to the present invention. [Figure 8] 1A to 1C are diagrams illustrating the measurement principle of a sample observation device according to the present invention. [Figure 9] 1A to 1C are diagrams illustrating a sample observation method according to the present invention. [Figure 10] 1A to 1C are diagrams illustrating the measurement principle of a sample observation device according to the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] The embodiments of the present invention will be described with reference to the accompanying drawings. The embodiments described below are examples of the present invention, and the present invention is not limited to the following embodiments. Note that components with the same reference numerals in this specification and drawings indicate the same components.

[0020] (Embodiment 1) Transmitted light microscopes are widely used in biomedical, food safety, and many other applications. When observing a specimen under a microscope, light strikes small particles in the specimen and changes direction before reaching the image sensor. As a result, the image sensor captures the sum of scattered light rays that travel different paths. This scattering of light obscures the observed image.

[0021] Image blurring due to scattering is a major challenge in biomedical imaging. For example, in spatial spectral analysis of biological tissue, the absorption coefficient must be accurately measured at a specific point. However, the measured signal contains scattered light from other points, which contains information about different points in the tissue. To accurately measure at a specific point in biological tissue, it is essential to separate the direct light from the specific point and the scattered light from other points.

[0022] However, in the case of a microscope equipped with an adjustable focus optical system, information about the thickness direction of the sample is added. Therefore, in the reflective method as in Non-Patent Document 5 or the transmissive method using high-frequency illumination of parallel light as in Non-Patent Document 6, it is not possible to separate the light reflected by the sample or the light transmitted through the sample into direct and global components, making it difficult to sharpen the image or perform spatial spectrum analysis.

[0023] Therefore, in this embodiment, in order to solve the above problem, a sample observation method is described that enables sharpening of the sample image and spatial spectrum analysis even in the case of a focus-adjustable optical system (not parallel light).

[0024] [1] Superposition of transmitted light and scattered light As mentioned above, one of the reasons for unclear images observed with a transmitted light microscope is that the measured light is a mixture of light passing through the desired observation point and scattered light from different points, which is then absorbed at different points within the sample. In this section, we will examine these lights passing through a sample with a transmitted light microscope and explain how the overlapping lights reach the camera (image sensor).

[0025] To explain all the overlapping light, consider the case where the light intensity at the observation point is measured with a transmission microscope. There are three types of light that reach the optical sensor: observation point single scattered light (direct transmitted light) that is scattered only at the observation point, other point single scattered light that is scattered only once at other points at a depth different from the observation point, and multiple scattered light. Figure 2 is an explanatory diagram of the path of each light, with the observation points being white circles. The black solid arrow in Figure 2(b) is the observation point single scattered light (direct transmitted light) that is scattered only at the observation point and then reaches the optical sensor without being scattered. This light is the direct component of the object of observation. The dashed arrow in Figure 2(c) is the other point single scattered light that is scattered only once at a point other than the observation point that is located at a depth different from the observation point in the sample, and the multiple scattered light that is scattered multiple times within the sample, which are the global component. The scattered light of the global component overlaps with the observation point single scattered light (direct transmitted light) and reaches the same optical sensor (Figure 2(a)). The global component reduces the ratio of the direct component at the optical sensor, making the image unclear. Therefore, it is necessary to separate these overlapping lights reaching the light sensor into direct and global components.

[0026] [2] Separation of direct and global components Fig. 3(a) is a diagram for explaining the transmission microscope of this embodiment, which separates the direct component and the global component. The transmission microscope of this embodiment is a microscope equipped with an illumination 13, an optical sensor 15, an optical sensor-side lens 12, and an illumination-side lens 14, and is characterized in that the incorporation of a transmission-type sample observation device 100 makes it possible to observe single-scattered light scattered only at an observation point 31 in a sample 50. Fig. 5 is a diagram for explaining the X, Y, and Z directions, the observation point 31 in the sample 50, and the XY plane (slice) including the observation point 31, in explaining the transmission microscope of this embodiment.

[0027] Here, the transmission type sample observation device 100 is a focusing mechanism 18 that performs at least one of the steps of: a step of adjusting the focus of an optical sensor side lens 12, which has a focal depth smaller than the thickness of the sample 50 and is located between the optical sensor 15 and the sample 50, to an observation point 31 within the sample 50; and a step of adjusting the focus of an illumination side lens 14, which has a focal depth smaller than the thickness of the sample 50 and is located between the illumination 13 on the opposite side of the optical sensor 15 with respect to the sample 50 and the sample 50, to the observation point 31; a light intensity acquisition means for acquiring, by an optical sensor, a first light intensity when the light of the illumination is irradiated onto an observation point and a second light intensity when the light is irradiated onto a point other than the observation point; a calculator 17 for determining a light intensity obtained by subtracting the second light intensity from the first light intensity as a component of the transmitted light at the observation point 31; Equipped with.

[0028] The calculator 17 controls the focus mechanism 18 and the light intensity acquisition means 19 so as to obtain a transmission image (an image obtained by the optical sensor 15) in each of the states shown in Figs. 3(b) and 3(c) which will be described later. The focus mechanism 18 adjusts the focus based on the calculations of the calculator 17, and can be realized by using, for example, a focus adjustment mechanism using a stepping motor. Alternatively, the operator may adjust the focus by manually operating a predetermined operation unit such as a focus adjustment knob. The focus mechanism 18 can also move the focal points of both the sensor-side lens 12 and the illumination-side lens 14 in the optical axis direction (Z direction). This function allows the slice to be moved in the Z direction, that is, the focus can be adjusted to an observation point 31 at any depth in the sample 50.

[0029] The drive mechanism of the light intensity acquiring means 19 is for moving the light intensity acquiring means 19 so as to switch between the states shown in Fig. 3(b) and (c) described below based on the calculation of the calculator 17, and may be, for example, a drive mechanism for a moving table or a photomask driven by a stepping motor. Also, the operator may switch between the states shown in Fig. 3(b) and (c) by operating an operation unit for moving the sample 50 or an operation unit for moving the photomask.

[0030] In this embodiment, the illumination 13 is a high-frequency illumination formed by a photomask in which light areas that transmit light and dark areas that block light are combined in an arbitrary ratio. The light intensity acquisition means is a drive mechanism that moves the sample 50 in the X direction or Y direction arbitrarily while maintaining the focal points of the optical sensor 15 and the illumination 13, or moves the photomask arbitrarily (moving the light and dark areas arbitrarily in the X direction or Y direction).

[0031] The transmission microscope of this embodiment uses the bright and dark areas of the high-frequency illumination (image 35 of the high-frequency illumination) and further uses the difference in optical information between the focused and unfocused states of the optical sensor 15 and the high-frequency illumination 13 to separate the direct component from the global component. Here, a surface (XY plane) in the sample parallel to the surface on which the image 35 of the high-frequency illumination is formed is called a "slice" (see FIG. 8). For example, the thickness of the sample 50 is 10 μm, and the thickness of the in-focus slice is 3 μm (the numerical value varies depending on the sample and lens performance). FIG. 4 is a diagram explaining in-focus slices and out-of-focus slices. The slices on which both the optical sensor and the high-frequency illumination are out of focus are α and γ, and the slice on which the focus is is β. When the optical sensor is in focus, the image is as shown in FIG. 4(a), and when the optical sensor is out of focus, the image is as shown in FIG. 4(b). On the other hand, when the high-frequency illumination is in focus, the image is as shown in Fig. 4(c), and when it is not in focus, the image is as shown in Fig. 4(d). In other words, when either the light sensor or the high-frequency illumination is not in focus, the image has a blurred pattern (Fig. 4(b)(d)). In such a slice with a blurred pattern, the light intensity is assumed to be uniform. In other words, when both the light sensor side and the high-frequency illumination side are in focus, it is considered to contain a direct component, and when either the light sensor side or the high-frequency illumination side is not in focus, it is considered to contain only a global component.

[0032] First, consider the case where there is no high-frequency illumination pattern, both the camera and the illumination are in focus, and the case where either the camera or the illumination is out of focus and the light intensity is uniform. The light intensity L[p] captured at pixel p of the photosensor 15 corresponding to the observation point 31 is the sum of a direct component D[p] and a global component G[p]. (Formula 1) L[p]=D[p]+G[p]

[0033] 3(b) and 3(c) are schematic diagrams showing a state when a sample 50 is irradiated with high-frequency illumination 13 via a photomask having a checkered pattern of light transmitting bright areas and light blocking dark areas. For clarity of explanation, the transmission type sample observation device 100 is omitted in FIGS. 3(b) and 3(c).

[0034] [Explanation of separation operation] Here, the contents of the calculation performed by the calculator 17 will be described. The optical sensor 15 and the high-frequency illumination 13 are focused on the target slice including the observation point 31. FIG. 3(b) is a schematic diagram when the light intensity acquisition means 19 moves the photomask or sample 50 to irradiate the observation point X with light (positioning the bright part of the photomask). When the observation point X is irradiated with light, the optical sensor 15 acquires information including both the direct light from the observation point X (direct component (A)) and the light from other parts (global component (B) which is a component of the single scattered light from other points other than the observation point in a defocused state, and global component (C) which is a component of the multiple scattered light). On the other hand, FIG. 3(c) is a schematic diagram when the photomask or sample 50 is moved so that the observation point X is not irradiated with light (positioning the dark part of the photomask). When the observation point X is not irradiated with light, there is no direct light from the observation point X (direct component (A)), and the optical sensor 15 acquires information only about the light from other parts (global components (B) and (C)). In addition, since the “component (D) due to single scattered light from points other than the observation point at focus” is imaged at a different pixel in the camera optical system, the component (D) does not need to be considered for the pixel under consideration.

[0035] Assuming that the illumination light is uniform in the out-of-focus slices (Fig. 4(d)) as described above, when the ratio of light to dark areas of the photomask is 1:1, the light intensity in the out-of-focus slices (α, γ) will be half the light intensity of the light area of ​​the in-focus slice β. In other words, the light intensity of the multiple scattered light or single scattered light from areas other than slice β will be half the intensity of the light reflected or transmitted at observation point 31.

[0036] Using this, the luminance L at pixel p of the optical sensor corresponding to the observation point 31 when light is irradiated to the observation point 31 is calculated. + [p] and the luminance L when no light is irradiated at observation point 31 - [p] can be calculated as follows:

number

number

[0037] The focal depth d has a range such that pixel p corresponds to a set of points x. Let Ω(p,d) be the region corresponding to pixel p and focal depth d, and Ψ(p) denote all points x in the sample 50 that correspond to pixel p.

[0038] As shown in Fig. 3(b), the first term in equation (2) is the direct transmitted light illuminating the observation point 31 in the target slice (β; z=d). The second term is the single scattered light in different slices (α, γ) out of focus, and the third term is the multiple scattered light in the global component. Therefore, the light D[p] from the target slice and the light G[p] from other slices are expressed as follows:

number

[0039] To obtain the direct and global components at every pixel of the light sensor 15, the high frequency illumination pattern is moved to obtain the L max and the minimum L min Get L + =L max And L - =L min Therefore, by calculating D[p] and G[p] for each pixel using equation (4), the transmitted light and scattered light of the sample 50 can be separated. The high-frequency illumination pattern may be moved along a preset path that includes both the bright and dark areas of the high-frequency illumination. The high-frequency pattern filter may be moved randomly in the X and Y directions on an XYZ-axis motorized stage. In this method, it is necessary to acquire a sufficient number of observation images of the target slice in order to capture all points in the bright areas. The calculations of the calculator 17 have been described above.

[0040] It is necessary to pay attention to the size of the light and dark areas of the photomask that forms the high-frequency illumination. If the light areas of the photomask are large, the direct transmitted light scattered at the observation point will be scattered several times within the same illumination area. In the transmission electron microscope of this embodiment, such light that has been scattered several times within the same illumination area will be determined to be a direct component. The number of scatterings within the same illumination area depends on the characteristics of the sample and the frequency of the high-frequency illumination, so the frequency of the high-frequency illumination must be appropriately adjusted according to the characteristics of the sample.

[0041] For example, in the above embodiment, the high frequency illumination pattern has been described as a checkerboard pattern with a 1:1 ratio of bright and dark areas, but depending on the characteristics of the sample, high frequency illumination with a pattern other than a 1:1 ratio of bright and dark areas may be used, or illumination with an irregular pattern in which bright and dark areas are not regularly arranged may be used. As the illumination pattern, in addition to the checkered pattern already described, the following illumination patterns are also possible. (a) A lighting pattern consisting of areas with multiple transmittances, rather than a checkerboard pattern with bright and dark areas each having a transmittance of 100% and 0%, respectively. (b) A lighting pattern that consists of stripes of light and dark areas rather than a checkerboard pattern. (c) A lighting pattern in which the transmittance changes smoothly (e.g., has a sine curve value) rather than a light and dark stripe pattern. (d) A lighting pattern in which the stripes, the boundaries between light and dark areas, and the boundaries between areas of different transmittance are curved rather than straight.

[0042] In this case, the coefficients of the second and third terms in formula (2) and the coefficients of each term in formula (3) are changed according to the ratio of light and dark areas. Specifically, if the high frequency lighting pattern is a checkerboard pattern with a 1:1 ratio of light and dark areas, the coefficients of formulas (2) and (3) are "1 / 2", but the coefficients are changed according to the ratio of light and dark areas (however, the sum of the coefficients is 1). For example, Ratio of bright areas = a, Ratio of dark areas = 1-a, b: The transmittance of the dark area when the transmittance of the bright area is 1 (0.0≦b<1) Then, the above-mentioned formulas (2), (3), and (4) become formulas (2a), (3a), and (4a), respectively.

number

number

number

[0043] In the above embodiment, the high frequency illumination is configured with a photomask, but the high frequency illumination may be configured by arranging a plurality of minute light sources such as LEDs without using a photomask. That is, the illumination is an illumination pattern that combines bright parts that cause the light source to emit light and dark parts that cause the light source to turn off at an arbitrary ratio, and the light intensity acquisition means is a light source control unit that arbitrarily moves the bright parts and the dark parts while maintaining the ratio. Of course, LEDs can provide not only high frequency illumination but also irregular pattern illumination.

[0044] (Specific operation method) This transmission microscope system consists of a microscope, a light source, a high-frequency pattern filter (photomask), a camera (light sensor), and an XYZ-axis motorized stage. As shown in Figures 3 and 4, various checkered pattern (high-frequency illumination) photomasks with a 1:1 ratio of illuminated areas (bright areas) to non-illuminated areas (dark areas) were prepared as high-frequency pattern filters. Specifically, pattern sizes ranging from 1x1μm to 16x16μm were prepared. The size of the pattern is determined according to the amount of scattering material contained in the specimen, for example, depending on the part of the body. The illumination lens is installed on top of the filter, and the illumination focus position can be changed.

[0045] First, the optical sensor 15 and the high-frequency illumination 13 are focused on a target slice at a certain depth in the depth direction (Z direction) of the sample. Either focus can be adjusted first, but both must be adjusted. For example, the microscope sample stage is moved to focus the optical sensor 15 on the desired target slice so that the sample 50 can be observed. Then, the high-frequency illumination 13 is focused on the target slice. To adjust the focus, the illumination side lens 14 is moved in the Z direction, and the high-frequency illumination pattern filter 13 is moved and adjusted in the Z direction by the XYZ axis motorized stage.

[0046] Then, in order to capture all points in the bright area, the high frequency pattern filter 13 is moved on the XYZ axis motorized stage, and the observation image of the target slice is obtained. The method of moving the high frequency pattern filter is as described above. Then, the observation image is processed as described in the above "Description of Separation Operation" to obtain the direct component D[p] and the global component G[p] in each optical sensor (pixel) of the camera. With the above operations, the transmitted light and the scattered light can be separated for the current target slice.

[0047] Next, the high frequency pattern filter 13 is moved in the Z-axis direction on the XYZ-axis motorized stage to focus the optical sensor 15 and the high frequency illumination 13 on a target slice at a different depth from the above. Then, the high frequency pattern filter 13 is moved in the same manner on the XYZ-axis motorized stage at the target slice to obtain an observation image. As described above, the observation image is processed as in "Description of separation operation," and the transmitted light and scattered light for the target slice can be separated.

[0048] (Effects of this embodiment) In the methods described in Non-Patent Documents 5 and 6, all depth information, including overlapping on the light path, is observed by superimposing, and it is difficult to obtain information on the transmitted light and scattered light at a specific depth (target slice) or information on the transmitted light and scattered light in the depth direction (Z direction) from them. On the other hand, this transmission microscope system can obtain information on the transmitted light and scattered light at a specific depth (target slice), and further, by sequentially moving the target slice in the Z direction and repeating the above process, it is possible to separate the transmitted light and scattered light for target slices at multiple depths and obtain information. This transmission microscope system continuously moves in the Z direction at a certain step width and can continuously obtain information on the transmitted light and scattered light in the target slices for each depth, so that it is possible to observe three-dimensional information or three-dimensional image information of the separated transmitted light and scattered light. The three-dimensional information includes three-dimensional wavelength information of direct components and global components, various information extracted by calculation therefrom or images created therefrom, and results analyzed therefrom.

[0049] (Embodiment 2) In this embodiment, a sample observation method for measuring the scattering characteristics of a sample, an apparatus therefor, and a microscope equipped with the same will be described. The sample observation method etc. of this embodiment is characterized in that, in addition to the sample observation method etc. described in embodiment 1, a plurality of illumination patterns with different spatial frequencies (the period between the above-mentioned bright and dark areas) are used. In this embodiment, a case will be described in which the area ratio a of the bright and dark areas of the illumination pattern is 1 / 2, but the area ratio a is not limited to 1 / 2.

[0050] 6 is a diagram illustrating the microscope of this embodiment. This microscope includes an optical sensor 15, an optical sensor side lens 12, an illumination side lens 14, and a sample observation device 101.

[0051] The sample observation device 101 includes: an illumination pattern 13 in which light areas that transmit light and dark areas that block light are combined in any ratio; a focusing mechanism 18 for adjusting the focal point of the optical sensor 15 and the illumination pattern 13 in the Z direction to an XY plane including an observation point 31 in the sample 50; a light intensity acquiring means for acquiring, with the light sensor, a first light intensity when the light of the illumination pattern is irradiated onto the observation point and a second light intensity when the light of the illumination pattern is irradiated onto a point other than the observation point, while the focus of both the light sensor and the illumination pattern is adjusted to the observation point by a focusing mechanism; a calculator 17 that uses a direct component (A) due to single scattered light at an observation point 31, a global component (B) due to single scattered light other than the observation point, and a global component (C) due to multiple scattered light within a sample 50, which are included in the first light intensity, and the global components (B) and (C) included in the second light intensity, and separates the direct component (A), the global components (B), and the global components (C) from the first light intensity by changing a period between the bright and dark parts of an illumination pattern 13, and calculates a light intensity for each scattering angle at the observation point by comparing the direct component (A) obtained for each period of the illumination pattern 13; Equipped with. In addition, in the sample observation device 101 of this embodiment, it is not necessary to distinguish between the global component (B) and the global component (C).

[0052] In this embodiment, a case where the illumination pattern 13 is a checkerboard pattern (the ratio of bright and dark areas is 1:1) will be described. The sample observation device 101 has, for example, four types of illumination patterns (13-1 to 13-4). Each illumination pattern has a different spatial frequency (the period between the bright and dark areas described above). As described in the first embodiment, the illumination pattern 13 is moved so that the bright and dark areas of the high-frequency illumination image 35 are at the observation point 31 (FIGS. 6(b) and (c)). This is performed for each illumination pattern (13-1 to 13-4).

[0053] Fig. 7 is a diagram explaining Fig. 6(b) and (c) in more detail. Fig. 7(a) shows the case where observation point 31 is in a bright part of high-frequency illumination image 35, Fig. 7(b) shows the case where observation point 31 is in a dark part of high-frequency illumination image 35, and Fig. 7(c) shows the difference between (a) and (b). Each light ray that reaches pixel p of optical sensor 15 can be classified into four types as shown in Fig. 7. (Double arrow) Direct transmitted light without scattering T(p); (Solid arrow) Single scattered light S scattered only at observation point 31 R (p,K), (Rectangle with dotted arrow) Single scattered light M scattered only by particle K1 at a depth different from observation point 31 SR (p,K), (Dotted circle arrow) Multiple scattered light M that is scattered multiple times by other particles and ultimately overlaps with the three types of light mentioned above. MR (p, K). In addition, single scattered light S R (p,K) is the direct component (A), and the single scattered light M SR (p,K) is the global component (B), and the multiple scattered light M MR (p, K) is the global component (C).

[0054] When the observation point 31 in FIG. 7(a) is in a bright area of ​​the image 35 of the high-frequency illumination, the image intensity L captured at a pixel p of the optical sensor 15 is + (p) is defined as the sum of these four types of rays as follows:

number

[0055] When the observation point 31 in FIG. 7(b) is in the dark part of the high-frequency illumination image 35, the image intensity L captured at a pixel p of the optical sensor 15 is - (p) is the single scattered light M SR (p,K) and multiple scattered light M MR It is defined as the sum of rays (p, K) as follows:

number

[0056] And L + (p) and L ー The difference between the direct transmitted light T(p) and the single scattered light S(p) is shown in Fig. 7(c). RThe total light intensity of (p, K) is (in this embodiment, this will be referred to as "desired light intensity.") Up to this point, the explanation is the same as in the first embodiment.

[0057] In this embodiment, the spatial frequency of the illumination pattern is further changed (by switching between illumination patterns 13-1 to 13-4 in FIG. 6) to obtain the desired light intensity. FIG. 8 is a diagram for explaining the principle of the sample observation device according to the present invention. FIGS. 8(a) and 8(b) explain how scattered light changes when the spatial frequency of the illumination pattern is changed. In FIG. 8, the perpendicular direction (Z direction, the vertical direction in the figure) to the focus plane is defined as 0°, and the parallel direction (XY plane) is defined as 90°.

[0058] Figure 8(a) is an image diagram of observing the desired light intensity described in Figure 7 using an illumination pattern 13-1 with a high spatial frequency (small checkerboard pattern). Directly transmitted light T passing through observation point 31 and single scattered light S1 scattered only at observation point 31 are emitted in the range of Ar1. The scattering angle θ1 is half the apex angle when the range Ar1 is considered as a cone with the observation point 31 as its apex.

[0059] Figure 8(b) is an image diagram of observing the desired light intensity described in Figure 7 using illumination pattern 13-2 with a low spatial frequency (large checkerboard pattern). Directly transmitted light T passing through observation point 31 and single scattered light S1 and S2 scattered only at observation point 31 are emitted in the ranges Ar1 and Ar2. The scattering angle θ2 is half the apex angle when the range including ranges Ar1 and Ar2 is considered as a cone with observation point 31 as its apex.

[0060] The light intensity of the single scattered light S2, which is within the range Ar2, is obtained by comparing the desired light intensities obtained using illumination patterns with different spatial frequencies (see FIG. 8(c)).

[0061] In other words, the sample observation method of the sample observation device 101 is a new method for acquiring a light intensity distribution for each scattering angle of scattered light using a plurality of illumination patterns with different spatial frequencies (the period between the bright and dark areas described above). FIG. 9 is a flowchart for explaining this sample observation method. This sample observation method includes: A focusing step (step M01) for adjusting the focal point of the optical sensor and the focal point of the illumination pattern, in which light areas that transmit light and dark areas that block light are combined at an arbitrary ratio, to the XY plane including the observation point in the sample in the Z direction; a light intensity acquisition step (step M02) of acquiring, by the light sensor, a first light intensity when the light transmitted through the bright portion of the illumination pattern is irradiated onto the observation point and a second light intensity when the light is irradiated onto a point other than the observation point after the focusing step; a separation step (step M03) of separating the direct component (A), the global component (B), and the global component (C) from the first light intensity, the direct component (A) being caused by single scattered light at the observation point, the global component (B) being caused by single scattered light at points other than the observation point, and the global component (C) being caused by multiple scattered light within the sample, the direct component (A), the global component (B), and the global component (C) being contained in the second light intensity; If the number of repetitions has not reached the predetermined number ("No" in step M04), a repetition step (step M05) of repeatedly performing the light intensity acquisition step and the separation step by changing the period between the bright and dark areas of the illumination pattern; When the number of repetitions reaches a predetermined number ("Yes" in step M04), a calculation procedure (step M06) of calculating the light intensity for each scattering angle by comparing the direct component (A) acquired for each period of the illumination pattern; Do the following.

[0062] This sample observation method is characterized by its ability to control the range of scattering angles by the frequency (i.e., the size of the bright and dark areas) of a binary illumination pattern consisting of light-transmitting areas (bright areas) and light-blocking areas (dark areas) like a checkerboard pattern. In other words, this sample observation method measures the scattering characteristics of a spatially inhomogeneous object by utilizing the fact that the optical sensor receives light rays with a wide scattering angle when the size of the bright areas is large (low frequency) and light rays with a narrow scattering angle when the size of the bright areas is small (high frequency).

[0063] The calculator 17 utilizes this phenomenon to estimate images for each scattering angle range. For example, the calculator 17 (α) When the highest frequency illumination pattern is used, the light intensity (direct component) S1 of the light beam falling within the scattering angle range Ar1 of 0 to θ1 in Figure 1 is obtained (see also Figure 8(a)). (β) Next, when a high-frequency illumination pattern is used, the light intensity (direct component) S2 of the light rays included in the scattering angle range Ar1 from 0 to θ2 in Figure 1 and the range Ar2 is obtained (see also Figure 8(b)). (γ) When the lowest frequency illumination pattern is used, the light intensity (direct component) S3 of the light beam included in the scattering angle range Ar1 to Ar3 of 0 to θ3 in FIG. 1 is obtained. The calculator 17 compares these light intensities and estimates the image of each range (Ar1, Ar2, Ar3) from the difference. For example, the light intensity S3 of the range Ar3 is obtained by subtracting the light intensity (β) from the light intensity (γ).

[0064] The above is an explanation for making it easier to understand the principle of calculation in the calculator 17. In reality, a matrix of light intensities (a set of equations (5) and (6)) for each of a plurality of illumination patterns is obtained, and the scattering angle is calculated by the following equation.

number

[0065] The ratio of the bright and dark areas in the region (e.g., the regions S1 and S2) of the i-th scattering angle (e.g., θ2) of the illumination pattern k is a ki and a' ki (a ki +a' ki = 1). These ratios can be calculated based on the microscope specifications and illumination pattern. Matrix A is a matrix consisting of these known information.

[0066] In addition, the kth scattering angle (the scattering angle θ k and the scattering angle θ of the illumination pattern k-1 k-1 The range between Ar k ) is the light intensity at S k {S k Let |k=1, ,K}. The matrix X(p) is the row vector of single scattered light with the kth scattering angle. G(p) is the global component. T(p) is included in the first element of X(p) as T(p)+S1(p) since its direction is parallel to the incident illumination.

[0067] Matrix L(p) is a vector consisting of the light intensities of the bright (L+) and dark (L-) areas of each of the K illumination patterns. Calculator 17 calculates matrix X(p) using matrix L(p) obtained by measurement and matrix A, which is known data.

[0068] In the actual measurement, the kth illumination pattern is shifted to obtain the maximum and minimum light intensities of each point, L k max (p) and L k min (p) and obtain a set of light intensities at each pixel. Here, when the light intensity of a pixel is at its maximum / minimum value, the pixel can be considered to be in the center of the bright / dark area. L k + (p)=L k max (p), L k - (p)=L k min (p) Given the matrix L(p) and the matrix A, the mean square error

number

[0069] The set of light intensities (i.e., images) obtained in this way represents the scattering characteristics (distribution of light intensity at different angles) of the particle 31 in Fig. 1. This can also be used to generate and visualize a scattering characteristic map (g map) of the asymmetry parameter (scattering parameter) g, which represents the angular distribution using a common scattering model (e.g., the Henyey-Greenstein phase function in Equation (9)).

number

[0070] The procedure for generating the g-map is explained below. As mentioned above, the scattering angle θ for each pixel p (observation point 31) k Light intensity per unit time S k Then, the scattering angle θ k Light intensity per unit time S k A procedure for estimating the scattering parameter g that satisfies the Henyey-Greenstein phase function at the observation point 31 from the above. a map generation step of arranging the scattering parameter g for each observation point 31 according to the coordinates of the observation point 31 on the XY plane to generate a scattering characteristic map; Further steps will be taken.

[0071] The phase function of equation (9) becomes 1 when integrated in the θ direction. Scattering information for each scattering angle θ, for example, g, which satisfies equation (9), is found by an optimization method or the like. Here, the light intensity is normalized so that it becomes 1 when integrated over that distribution. Also, finding g by an optimization method means finding g that is optimal when the light intensity distribution is expressed by equation (9).

[0072] For example, for a certain pixel (observation point 31), the normalized light intensities S1, S2, S3, . . . , S k , S K These are substituted into P(θ) in equation (9) to estimate the scattering parameter g at the observation point 31. In practice, as in the above embodiment, the light intensity S k The scattering angle is θ k ~θ k-1 When the light intensity is within a certain range, such as above, information may be obtained by tracking the light beam for each scattering angle, or the light beam conditions for each scattering angle may be obtained by simulation, and the optimal parameter g that matches the phase function is estimated according to the conditions in each case.

[0073] The parameter g is estimated in a similar manner for other pixels (observation points 31). These parameters g are arranged two-dimensionally according to the pixels, and a map of the parameters g (scattering characteristic map) is generated. Here, it is preferable to visualize the value of the parameter g by displaying it in color or brightness (representing the magnitude of g by color or brightness). In this embodiment, the Henyey-Greenstein phase function model has been described as an example of a scattering model, but the parameters of each scattering model can be estimated in the same manner for an isotropic scattering model or other scattering models (phase functions).

[0074] As shown in FIG. 6(a), by mounting the sample observation device 101 on a typical transmission microscope, the light captured by the microscope can be decomposed into direct scattered light for each angle, and the spatial distribution of the scattering characteristics can be visualized. [Industrial Applicability]

[0075] The sample observation method, sample observation device, and microscope according to the present invention can also be applied to pathological diagnosis such as cancer testing, biosciences such as cell observation, the medical field, inspection of the internal structure of objects such as defect inspection, and food inspection such as cavity detection. [Explanation of symbols]

[0076] 12: Optical sensor lens 13, 13-1 to 13-4: High frequency lighting (lighting pattern) 14: Lighting lens 15: Light sensor 17: Arithmetic unit 18: Focus mechanism 19: Light intensity acquisition means 21: Integrating sphere 22: Spectrophotometer 23: Sample 24: light 35: High frequency lighting image 50: Sample 100, 101: Transmission type sample observation device

Claims

1. a focusing step of adjusting the focal point of the optical sensor and the focal point of the illumination pattern, in which light areas that transmit light and dark areas that block light are combined at an arbitrary ratio, on an XY plane including an observation point in the sample in the Z direction; a light intensity acquisition step of acquiring, by the light sensor, a first light intensity when the light transmitted through the bright portion of the illumination pattern is irradiated onto the observation point and a second light intensity when the light is irradiated onto a point other than the observation point, after the focusing step; a separation step of separating the direct component and the global component from the first light intensity by using a direct component due to single scattered light at the observation point and a global component other than the direct component, which are included in the first light intensity, and the global component included in the second light intensity; a repeating step of repeatedly performing the light intensity acquisition step and the separation step by changing a period between the bright and dark areas of the illumination pattern; a calculation step of calculating a light intensity for each scattering angle at the observation point by comparing the direct components acquired for each period of the illumination pattern; A sample observation method.

2. an estimation step of estimating parameters of a scattering model at the observation point from the light intensity for each scattering angle; a map generation step of arranging the parameters for each of the observation points according to coordinates of the observation points on the XY plane to generate a scattering characteristic map; The sample observation method according to claim 1 , further comprising the steps of:

3. 3. A sample observation method according to claim 2, wherein the scattering property map is visualized based on the parameters.

4. 4. A sample observation method according to claim 1, wherein the illumination pattern is a checkered pattern.

5. A lighting pattern in which light transmitting bright areas and light blocking dark areas are combined in any ratio; a focusing mechanism for adjusting the focus of the optical sensor and the illumination pattern in the Z direction to an XY plane including an observation point within the sample; a light intensity acquiring means for acquiring, with the light sensor, a first light intensity when the light of the illumination pattern is irradiated onto the observation point and a second light intensity when the light of the illumination pattern is irradiated onto a location other than the observation point, while the focus mechanism is set to focus both the light sensor and the illumination pattern on the observation point; a calculator that uses a direct component due to single scattered light at the observation point and a global component other than the direct component, which are included in the first light intensity, and the global component included in the second light intensity, and separates the direct component and the global component from the first light intensity by changing a period of the bright part and the dark part of the illumination pattern, and calculates a light intensity for each scattering angle at the observation point by comparing the direct component obtained for each period of the illumination pattern; A sample observation device comprising:

6. The computing unit includes: estimating parameters of a scattering model at the observation point from the light intensity for each scattering angle; and arranging the parameters for each of the observation points according to the coordinates of the observation points on the XY plane to generate a scattering characteristic map; The sample observation device according to claim 5, further comprising:

7. 7. The sample observation device according to claim 6, wherein the computing unit visualizes the scattering property map based on the parameters.

8. 8. A sample observation device according to claim 5, wherein the illumination pattern is a checkered pattern.

9. A microscope comprising the sample observation device according to any one of claims 5 to 8.

Images