Method, device, and program for determining or evaluating capillary bile duct region

Abstract

Disclosed are: a method for specifying a capillary bile duct region contained in an object to be observed using refractive index distribution data of the object to be observed containing hepatocytes; and a method for evaluating the capillary bile duct region on the basis of capillary bile duct parameters obtained from the refractive index distribution data of the object to be observed containing hepatocytes.

Description

Technical Field

[0001] The present disclosure relates to a method, device and procedure for determining or evaluating bile duct area. Background Art

[0002] Bile canaliculi (also called bile canaliculi, bile canaliculi or BC) are canaliculi formed between adjacent hepatocytes. The lumen of the bile canaliculi is sealed by the close contact between hepatocytes, and substances are transferred through transporters on the cell membrane of the hepatocytes. Bile acids synthesized in hepatocytes are secreted in the bile canaliculi, and drugs metabolized in hepatocytes are excreted. On the surface of the hepatocytes facing the bile canaliculi, there are many fine protrusions called microvilli (also called microvilli or MV) with a length ranging from submicrometer to several micrometers.

[0003] Drug-induced liver injury (DILI) is a general term for liver diseases caused by drugs and / or their metabolites administered to patients, and is one of the main reasons for the market withdrawal and use restrictions of approved drugs and the non-approval of new drugs due to safety (Non-Patent Document 1).

[0004] Regarding bile ductules and drugs, in Patent Document 1 on a method for culturing hepatocytes, the area and number of bile ductules are used as indicators for evaluating cultured hepatocytes. Patent Document 2 discloses an evaluation device for candidate compounds with excretion sensitivity in bile or blood of hepatocytes and liver metabolites of the candidate compounds, characterized in that the accumulation and excretion of liver metabolites into bile ductule-like structures are promoted. Patent Document 3 discloses an in vitro method for screening candidate compounds that induce bile ductule dysfunction, wherein the morphological changes in the bile duct space and the maximum amount of small molecules or macromolecules that can be accumulated in the bile duct and the bile duct lumen are used as evaluation indicators.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: International Publication No. 2011 / 024592

[0008] Patent document 2: Japanese Patent Application Publication No. 2020-028305.

[0009] Patent Document 3: Japanese Patent No. 6644786

[0010] Non-patent literature

[0011] Non-patent literature 1: Watkins, "Drug safety sciences and the bottleneck in drug development.", Clin. Pharmacol. Ther., 89, 788-790 (2011).

[0012] Non-patent document 2: Gissen et al., "Structural and functional hepatocyte polarity and liver disease", Journal of Hepatology, 63, 1023-1037 (2015). Summary of the invention

[0013] Problems to be solved by the invention

[0014] Bile ductules are useful as the evaluation index of hepatocyte mass and liver tissue and the analysis object of drug metabolism. However, it is reported that it is difficult to determine the bile ductules region (non-patent literature 2) by hematoxylin-eosin staining (HE staining) as a representative tissue staining method. In addition, the determination method and evaluation method of the bile ductules region currently used are electron microscopy and fluorescence microscopy, but electron microscopy is difficult to carry out living cell observation and observation through time, and is therefore not suitable for use with drug evaluation, and fluorescence microscopy must carry out fluorescent labeling.

[0015] Therefore, one aspect of the present disclosure is to provide a method, device, and procedure for non-invasively determining or evaluating bile duct area.

[0016] Methods used to solve problems

[0017] The inventors have found that, by using the refractive index distribution data of the observation object, it is possible to identify the bile duct region included in the observation object and to evaluate the identified bile duct region.

[0018] That is, a first aspect of the present disclosure is a method of specifying a bile duct region included in an observation object using refractive index distribution data of the observation object including hepatocytes.

[0019] A second aspect of the present disclosure is a method of evaluating a bile duct region based on bile duct parameters obtained from refractive index distribution data of an observation object including hepatocytes.

[0020] The third aspect of the present disclosure is a method for evaluating an observation object containing hepatocytes based on the number of capillary bile duct areas determined by the method of the first aspect of the present disclosure (a method for determining the capillary bile duct area contained in the observation object using refractive index distribution data of the observation object containing hepatocytes) and / or an evaluation of the capillary bile duct area obtained by the method of the second aspect of the present disclosure (a method for evaluating the capillary bile duct area based on parameters of the capillary bile duct obtained from the refractive index distribution data of the observation object containing hepatocytes).

[0021] The fourth aspect of the present disclosure is a method for evaluating a method for culturing hepatocytes, comprising: a step of culturing hepatocytes; and a step of evaluating an observation object obtained by culture using the method of the third aspect of the present disclosure (a method for evaluating an observation object containing hepatocytes based on the number of capillary bile duct areas determined by the method of the first aspect of the present disclosure (a method for determining the capillary bile duct area contained in the observation object using refractive index distribution data) and / or an evaluation of the capillary bile duct area obtained by the method of the second aspect of the present disclosure (a method for evaluating the capillary bile duct area by parameters of the capillary bile duct obtained from refractive index distribution data)).

[0022] The fifth aspect of the present disclosure is a drug screening method, which includes: a step of adding a drug to an observation object containing hepatocytes; and a step of evaluating the observation object to which the drug has been added using the method of the third aspect of the present disclosure (a method of evaluating an observation object containing hepatocytes based on the number of capillary bile duct areas determined by the method of the first aspect of the present disclosure (a method of determining the capillary bile duct area contained in the observation object using refractive index distribution data) and / or an evaluation of the capillary bile duct area obtained by the method of the second aspect of the present disclosure (a method of evaluating the capillary bile duct area by parameters of the capillary bile duct obtained from refractive index distribution data)).

[0023] On the other hand, a bile duct region determination device is provided, comprising: a data acquisition unit that acquires refractive index distribution data of an observation object including hepatocytes; and a determination unit that uses the refractive index distribution data of the observation object to determine the bile duct region included in the observation object.

[0024] On the other hand, a device for evaluating a bile duct region is provided, comprising: a data acquisition unit, which acquires refractive index distribution data of an observation object containing hepatocytes; a determination unit, which uses the refractive index distribution data of the observation object to determine the bile duct region contained in the observation object; and an evaluation unit, which evaluates the bile duct region based on parameters of the bile duct obtained from the refractive index distribution data.

[0025] On the other hand, a program causes a computer to execute the following steps: a data acquisition step of acquiring refractive index distribution data of an observation object including hepatocytes; and a determination step of determining a bile duct capillary region included in the observation object using the refractive index distribution data of the observation object.

[0026] On the other hand, a program causes a computer to execute the following steps: a data acquisition step of acquiring refractive index distribution data of an observation object containing hepatocytes; a determination step of using the refractive index distribution data of the observation object to determine the capillary bile duct area contained in the observation object; and an evaluation step of evaluating the capillary bile duct area based on parameters of the capillary bile duct obtained from the refractive index distribution data.

[0027] More specifically, the present disclosure relates to the following [1] to

[20] .

[0028] [1] A method in which refractive index distribution data of an observation object including hepatocytes is used to identify a bile duct region included in the observation object.

[0029] [2] The method according to [1], wherein the bile duct region is determined based on the characteristic that the region in the observation object has bile ducts.

[0030] [3] The method according to [2], wherein the characteristics of the bile duct capillaries include: one or more characteristics selected from the group consisting of a characteristic of a roughly circular or roughly tubular region existing between hepatocytes, a characteristic that the refractive index of the region is lower than the refractive index of hepatocytes, and a characteristic that the refractive index of the peripheral portion of the region is higher than the refractive index of hepatocytes.

[0031] [4] The method according to [2], wherein the characteristics of the bile duct capillaries include: a characteristic that a roughly circular or roughly tubular region exists between hepatocytes, a characteristic that the refractive index of the region is lower than the refractive index of hepatocytes, and a characteristic that the refractive index of the peripheral portion of the region is higher than the refractive index of hepatocytes.

[0032] [5] The method according to [3] or [4], wherein the characteristics of the bile duct capillaries further include: having a region derived from microvilli and having a refractive index higher than the refractive index of the region as a whole.

[0033] [6] The method according to any one of [1] to [5], wherein the refractive index distribution data is refractive index tomographic data in a predetermined direction.

[0034] [7] A method of evaluating a bile duct area based on bile duct parameters obtained from refractive index distribution data of an observation object including hepatocytes.

[0035] [8] The method according to [7], wherein the parameters of the bile duct capillaries include: one or more parameters selected from the area of ​​the cross section perpendicular to the tube flow axis or the cross section observed on the tomographic plane, the refractive index, the number of microvilli, and the average length of the microvilli.

[0036] [9] The method according to [7] or [8], which includes a step of determining a capillary bile duct area contained in the observation object by any of the methods described in [1] to [6], and evaluating the capillary bile duct area based on parameters of the capillary bile duct in the determined capillary bile duct area.

[0037]

[10] A method comprising evaluating an observation object containing hepatocytes based on the number of bile duct areas determined by the method according to any one of claims [1] to [6] and / or the evaluation of bile duct areas obtained by the method according to any one of claims [7] to [9].

[0038]

[11] A method for evaluating a method for culturing hepatocytes, comprising: a step of culturing the hepatocytes; and a step of evaluating an observation object obtained by the culturing using the method described in

[10] .

[0039]

[12] A drug screening method, comprising: a step of adding a drug to an observation object containing hepatocytes; and a step of evaluating the observation object to which the drug has been added using the method described in

[10] .

[0040]

[13] The method according to any one of [1] to

[12] , wherein the observation object containing hepatocytes is selected from a two-dimensional liver culture, a hepatocyte mass, and a liver tissue sample.

[0041]

[14] The method according to any one of [1] to

[12] , wherein the hepatocyte-containing observation object is selected from a hepatocyte mass and a liver tissue sample.

[0042]

[15] The method according to any one of [1] to

[12] , wherein the observation object containing hepatocytes is a hepatocyte mass.

[0043]

[16] A method for diagnosing liver disease, a method for assisting in the diagnosis of liver disease, or a method for collecting data for the diagnosis of liver disease, comprising: a step of obtaining refractive index distribution data of a clinical specimen of the liver; and a step of evaluating a bile duct region contained in the clinical specimen or the clinical specimen using any one of the methods [7] to

[10] .

[0044]

[17] A device for determining a bile duct region, comprising: a data acquisition unit that acquires refractive index distribution data of an observation object containing hepatocytes; and a determination unit that determines the bile duct region contained in the observation object by using any one of the methods described in [1] to [6].

[0045]

[18] A device for evaluating a bile duct region, comprising: a data acquisition unit that acquires refractive index distribution data of an observation object containing hepatocytes; a determination unit that determines the bile duct region contained in the observation object by using the method described in any one of [1] to [6]; and an evaluation unit that evaluates the bile duct region by using the method described in any one of [7] to [9].

[0046]

[19] A program that causes a computer to execute: a data acquisition step for acquiring refractive index distribution data of an observation object containing hepatocytes; and a determination step for determining a bile duct capillary region contained in the observation object by using any one of the methods described in [1] to [6].

[0047]

[20] A program that causes a computer to execute the following steps: a data acquisition step, which acquires refractive index distribution data of an observation object containing hepatocytes; a determination step, which determines the capillary bile duct area contained in the observation object by the method described in any one of [1] to [6]; and an evaluation step, which evaluates the capillary bile duct area by the method described in any one of [7] to [9].

[0048] Effects of the Invention

[0049] According to the method, device or program disclosed herein, the bile duct region in the observed object can be non-invasively determined and evaluated using the refractive index distribution data. In addition, according to the method disclosed herein, the refractive index distribution data can be used to evaluate the culture method of hepatocyte mass or screen drugs. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 (hereinafter, also referred to as "Figure A01"). This is a diagram showing the structure of the observation device 1A.

[0051] Figure 2 (Hereinafter, also referred to as "Figure A02"). This is a diagram showing the structure of the observation device 1B.

[0052] Figure 3 (Hereinafter, also referred to as "FIG. A03") is a diagram showing the structure of the observation device 1C.

[0053] Figure 4 (Hereinafter, also referred to as "Figure A04.") This is a flow chart of refractive index distribution measurement method A.

[0054] Figure 5 (hereinafter also referred to as “FIG. A05”), (a), (b), and (c) are diagrams showing examples of scanning in the light irradiation direction toward the observation object S in the interference intensity image acquisition step S1.

[0055] Figure 6 (Hereinafter, also referred to as "FIG. A06") is a diagram for explaining the kernel function g.

[0056] Figure 7 (hereinafter also referred to as “FIG. A07”), (a) and (b) are diagrams showing examples of scanning in the light irradiation direction toward the observation object S in the interference intensity image acquisition step S1.

[0057] Figure 8 (hereinafter also referred to as “FIG. A08”), (a), (b), and (c) are diagrams showing examples of scanning in the light irradiation direction toward the observation object S in the interference intensity image acquisition step S1.

[0058] Fig. 9 (Hereinafter, also referred to as “FIG. A09”) is a flowchart of the two-dimensional phase image generating step S4 in the refractive index distribution measuring method A1.

[0059] Fig.10 (hereinafter also referred to as “FIG. A10”) is a flowchart of the two-dimensional phase image generating step S4 in the refractive index distribution measuring method A2.

[0060] Fig.11 (hereinafter also referred to as "FIG. A11") This is a diagram for explaining the kernel function.

[0061] Fig.12 (hereinafter also referred to as “FIG. A12”) is a flowchart of the two-dimensional phase image generating step S4 in the refractive index distribution measuring method A3.

[0062] Fig.13 (hereinafter also referred to as "FIG. B01") This is a diagram showing the structure of an observation apparatus 1D.

[0063] Fig.14 (Hereinafter, also referred to as "Figure B02"). This is a diagram showing the structure of the observation device 1E.

[0064] Fig.15 (Hereinafter, also referred to as "FIG. B03") is a diagram showing the structure of the observation device 1F.

[0065] Fig.16 (Hereinafter, also referred to as "Figure B04"). This is a flow chart of refractive index distribution measurement method B.

[0066] Fig.17(hereinafter also referred to as “FIG. B05”) is a diagram for explaining the order and images of each process of the second complex amplitude image generating step S63 and the two-dimensional phase image generating step S65.

[0067] Fig.18 (hereinafter also referred to as “FIG. B06”) is a diagram for explaining the order and images of each process of the second complex amplitude image generating step S63, the phase conjugate calculation step S64, and the two-dimensional phase image generating step S65.

[0068] Fig.19 (hereinafter also referred to as “FIG. B07”) is a diagram for explaining the order and images of each process of the second complex amplitude image generating step S63, the phase conjugate calculation step S64, and the two-dimensional phase image generating step S65.

[0069] Fig. 20 (hereinafter also referred to as “FIG. B08”) is a diagram for explaining the order and images of each process of the second complex amplitude image generating step S63, the phase conjugate calculation step S64, and the two-dimensional phase image generating step S65.

[0070] Fig.21 (hereinafter also referred to as “FIG. B09”) is a diagram for explaining the order and images of each process of the three-dimensional phase image generation step S66 and the refractive index distribution calculation step S67.

[0071] Fig. 22 (hereinafter also referred to as “FIG. B10”) is a diagram for explaining the outline of the phase conjugate operation, and is a diagram showing input light and output light when the interference intensity image is captured by the imaging unit.

[0072] Fig.23 (hereinafter also referred to as “FIG. B11”) is a diagram for explaining the outline of the phase conjugate operation, and is a diagram showing input light and output light when the relationship between light irradiation and imaging is reversed.

[0073] Fig.24 (hereinafter also referred to as "FIG. B12") This is a diagram for explaining the image segmentation, phase conjugate operation and image combination in the phase conjugate operation step S64.

[0074] Fig.25 (hereinafter also referred to as "Figure C01") is a diagram showing the structure of an observation device 1G.

[0075] Fig.26 (Hereinafter, also referred to as "Figure C02"). This is a diagram showing the structure of the observation device 1H.

[0076] Fig. 27 (Hereinafter, also referred to as "Figure C03"). This is a diagram showing the structure of the observation device 1I.

[0077] Fig.28 (Hereinafter, also referred to as "Figure C04.") is a flow chart of refractive index distribution measurement method C.

[0078] Fig.29 (Hereinafter, also referred to as "Figure C05.") This is a flow chart of refractive index distribution measurement method C.

[0079] Fig.30 (hereinafter also referred to as “Figure C06”) is a diagram for explaining the relationship between the region including the observation object and the 1st to Jth blocks.

[0080] Fig.31 (Hereinafter, also referred to as "Figure C07") This is a figure explaining the order of processing in the 1st to Jth blocks.

[0081] Fig.32 (Hereinafter, also referred to as "Figure C08") This is a figure for explaining the processing contents of BPM.

[0082] Fig.33 (Hereinafter, also referred to as “FIG. C09”) is a flowchart of the third complex amplitude image generating step S77.

[0083] Fig.34 (Hereinafter, also referred to as "FIG. C10") is a diagram showing the structure of the observation device 1J.

[0084] Fig.35 This is a diagram showing an example of a hepatic cell mass evaluated by the method for evaluating a hepatic cell mass according to the third aspect of the present disclosure.

[0085] Fig.36 This is a diagram showing an example of a case where the evaluation method of the hepatocyte mass culture method according to the fourth aspect of the present disclosure is performed based on the number of bile ductule regions corresponding to the number of days of culture of the hepatocyte mass.

[0086] Fig.37 This is a diagram showing an example of the method for screening a drug according to the fifth aspect of the present disclosure, which screens a drug that may cause cholestatic DILI based on the cross-sectional area of ​​the bile ductules in a hepatocyte mass.

[0087] Fig.38 This is a diagram showing an example of a method for screening drugs according to the fifth aspect of the present disclosure, which screens drugs that may cause cholestatic DILI based on the length and / or number of microvilli in the bile canaliculi region in a hepatocyte mass.

[0088] Fig.39This is a diagram showing an example of a method for screening drugs according to the fifth aspect of the present disclosure, which screens drugs that may cause cholestatic DILI based on changes in microvilli length over time after drug administration.

[0089] Fig.40 This is a diagram showing an example of evaluating a bile duct network formed by adding taurocholic acid to a hepatocyte mass according to the method for determining the bile duct region according to the first aspect of the present disclosure.

[0090] Fig.41 This is a diagram showing an example in which the method for identifying the bile duct region according to the first aspect of the present disclosure is used as a guide to insert a drainage tube and collect the contents of the bile duct in a hepatocyte mass.

[0091] Fig.42 This is a diagram showing an example of assisting the diagnosis of liver diseases in clinical liver samples using the number and / or length of microvilli as an index according to the method for evaluating the bile duct region of the second aspect of the present disclosure;

[0092] Fig.43 This is a diagram showing the results of identification of the bile duct region of a two-dimensional liver culture body using phase contrast and fluorescence microscopy in Example 1.

[0093] Fig.44 This is a diagram schematically showing the evaluation results of the bile duct region of a two-dimensional liver culture body using optical diffraction tomography (ODT) in Example 2.

[0094] Fig.45 This is a diagram showing the detailed evaluation results of the bile duct region of a two-dimensional liver culture body using optical diffraction tomography (ODT) in Example 2.

[0095] Fig.46 This is a diagram schematically showing the evaluation results of the bile duct region of the hepatocyte mass 1 using optical diffraction tomography (ODT) in Example 3.

[0096] Fig.47 This is a diagram showing the detailed evaluation results of the bile duct region of the hepatocyte mass 1 using optical diffraction tomography (ODT) in Example 3.

[0097] Fig.48 This is a diagram showing the evaluation results of the bile duct region of the hepatocyte mass 2 using optical diffraction tomography (ODT) in Example 3. DETAILED DESCRIPTION

[0098] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and repeated descriptions are omitted. The actual embodiments are not limited to these examples.

[0099] The object of observation of one embodiment contains hepatocytes. Hepatocytes are derived from the liver and refer to cells that can form bile ductules by close contact between adjacent hepatocytes. The object of observation of one embodiment may contain both hepatocytes that have formed bile ductules and hepatocytes that have not formed bile ductules. Hepatocytes may also be cultured cells derived from the liver (including cell lines and primary cultured cells), or hepatocytes contained in isolated liver tissue, and the animal species that is the source of hepatocytes is not particularly limited. In the case where the object of observation is a culture of cells, the culture may also be composed of one type of hepatocytes, or may include more than two types of cells containing at least one type of hepatocytes.

[0100] The object of observation of one embodiment is an object composed mainly of cells, but in addition to cells, it may also include: components sometimes contained in tissue samples, such as extracellular matrix and neutral fat. From the viewpoint that liver cells form a close bond with each other, the object of observation of one embodiment may also be a two-dimensional liver culture, a liver cell mass and a liver tissue sample. In addition, from the same viewpoint and the viewpoint of ensuring the amount and quality of the capillary bile ducts that can be the object of determination and evaluation, it may also be a two-dimensional liver culture, a liver cell mass and a liver tissue sample. Furthermore, from the viewpoint of taking into account an environment close to the physiological environment, high reproducibility and ease of sample preparation, the object of observation of one embodiment may also be a liver cell mass. A two-dimensional liver culture refers to a monolayer colony formed by two-dimensionally cultured liver cells on a flat dish. A liver cell mass refers to a cell mass formed by three-dimensionally cultured liver cells. Liver tissue samples refer to biological samples prepared from liver tissue isolated from animals including humans, and examples thereof include tissue slides, tissue blocks, tissue panels, and tissue arrays. When the object to be observed is a hepatocyte mass, its maximum diameter may be 100 to 200 μm.

[0101] The refractive index distribution data refers to data indicating the three-dimensional distribution of the refractive index of each voxel in the space including the observation object, or data indicating the two-dimensional distribution of the refractive index of each pixel in the tomographic plane in a predetermined direction of the space including the observation object. In addition, the refractive index tomographic data refers to data indicating the two-dimensional distribution of the refractive index of each pixel in the tomographic plane in a predetermined direction of the space including the observation object among the refractive index distribution data.

[0102] The method for obtaining refractive index distribution data of one embodiment (hereinafter also referred to as "refractive index distribution measurement method") is not particularly limited, and the method is described in detail as an example. As a method for measuring the refractive index distribution of an observation object in a non-staining and non-invasive manner, optical diffraction tomography (ODT) is known. ODT develops quantitative phase imaging (QPI) into a technology capable of three-dimensional imaging, which can realize three-dimensional refractive index tomography of the observation object.

[0103] Then, the refractive index distribution measurement methods A to C described below are used. The refractive index distribution measurement method A has the forms of refractive index distribution measurement methods A1 to A3. The refractive index distribution measurement methods A1 to A3 are collectively referred to as the refractive index distribution measurement method A. Even when the observation object is a multiple scatterer, these refractive index distribution measurement methods A to C can realize three-dimensional refractive index tomography with reduced influence of multiple scattered light.

[0104] In addition, as another staining and non-invasive imaging technology, optical coherence tomography (OCT) is also known. However, the resolution of OCT is about 10 μm, while the resolution of ODT and refractive index distribution measurement methods A to C is about 1 μm. In addition, OCT does not obtain the refractive index distribution, making it difficult to perform biological interpretation of the signal obtained by imaging. In these points, ODT and refractive index distribution measurement methods A to C are superior to OCT.

[0105] First, the refractive index distribution measuring method A (A1 to A3) will be described. FIG. A01 to FIG. A03 are diagrams showing the respective configurations of observation apparatuses 1A to 1C that can be used when measuring the refractive index distribution according to the refractive index distribution measuring method A. FIG.

[0106] Fig. A01 is a diagram showing the structure of an observation device 1A, which includes a light source 11, a lens 12, a lens 21, a reflector 22, a lens 23, a condenser lens 24, an objective lens 25, a beam splitter 41, a lens 42, an imaging unit 43, and an analyzing unit 50.

[0107] The light source 11 outputs spatially and temporally coherent light, such as a laser light source. The lens 12 is optically connected to the light source 11, and focuses the light output from the light source 11 on the light incident end 13 of the optical fiber 14, and causes the light to be incident on the light incident end 13. The optical fiber 14 guides the light incident on the light incident end 13 through the lens 12 to the optical fiber coupler 15. The optical fiber coupler 15 couples light between the optical fiber 14 and the optical fibers 16 and 17, and branches the light guided by the optical fiber 14 into two parts, one branch of light is guided by the optical fiber 16, and the other branch of light is guided by the optical fiber 17. The light guided by the optical fiber 16 is emitted from the light output end 18 as divergent light. The light guided by the optical fiber 17 is emitted from the light output end 19 as divergent light.

[0108] The lens 21 is optically connected to the light output end 18, and collimates the light outputted from the light output end 18 as divergent light. The reflector 22 is optically connected to the lens 21, and reflects the light arriving from the lens 21 toward the lens 23. The orientation of the reflection surface of the reflector 22 is variable. The lens 23 is optically connected to the reflector 22. The condenser lens 24 is optically connected to the lens 23. The lens 23 and the condenser lens 24 constitute, for example, a 4f optical system. The lens 23 and the condenser lens 24 irradiate light to the observation object S from the light irradiation direction corresponding to the orientation of the reflection surface of the reflector 22. The objective lens 25 is optically connected to the condenser lens 24. The observation object S is arranged between the objective lens 25 and the condenser lens 24. The objective lens 25 inputs the light (object light) outputted from the condenser lens 24 and passing through the observation object S, and outputs the light to the beam splitter 41.

[0109] The beam splitter 41 is optically connected to the objective lens 25, and is also optically connected to the light output end 19. The beam splitter 41 combines the light (object light) output and arriving from the objective lens 25 and the light (reference light) output and arriving from the light output end 19, and outputs the two lights to the lens 42. The lens 42 is optically connected to the beam splitter 41, and collimates the object light and the reference light arriving from the beam splitter 41 respectively and outputs them to the shooting unit 43. The shooting unit 43 is optically connected to the lens 42, and shoots the interference fringe image (interference intensity image) formed by the interference of the object light and the reference light arriving from the lens 42. The incident direction of the reference light is inclined relative to the incident direction of the object light to the shooting surface of the shooting unit 43. The position where the object light and the reference light are combined by the beam splitter 41 can also be at the rear stage of the imaging lens. If the influence of aberration is considered, it is desired to be between the objective lens 25 and the lens 42 as shown in the figure.

[0110] The analyzing unit 50 is electrically connected to the imaging unit 43, and inputs the interference intensity image captured by the imaging unit 43. The analyzing unit 50 calculates the three-dimensional refractive index distribution of the observation object S by processing the input interference intensity image. The analyzing unit 50 may also be a computer. The analyzing unit 50 includes: an interference intensity image acquisition unit 51, a first complex amplitude image generation unit 52, a second complex amplitude image generation unit 53, a two-dimensional phase image generation unit 54, a three-dimensional phase image generation unit 55, a refractive index distribution calculation unit 56, a display unit 57, and a storage unit 58.

[0111] The interference intensity image acquisition unit 51 irradiates the observation object S along each of a plurality of light irradiation directions by changing the orientation of the reflection surface of the reflector 22. In addition, the interference intensity image acquisition unit 51 acquires the interference intensity image of the reference position from the shooting unit 43 for each of the plurality of light irradiation directions. The interference intensity image acquisition unit 51 includes a CPU, and has an output port for outputting a control signal for changing the orientation of the reflection surface of the reflector 22, and also has an input port for inputting the interference intensity image from the shooting unit 43. There is no need to move the objective lens 25 along the optical axis. The reference position is an image plane position that is in a conjugate relationship with the shooting surface of the shooting unit 43.

[0112] The first complex amplitude image generating unit 52, the second complex amplitude image generating unit 53, the two-dimensional phase image generating unit 54, the three-dimensional phase image generating unit 55 and the refractive index distribution calculating unit 56 perform processing based on the interference intensity image, and include a processing device such as a CPU, a GPU, a DSP or an FPGA. The display unit 57 displays the image to be processed, the image in the process of processing and the image after processing, etc., and includes, for example, a liquid crystal display. The storage unit 58 stores data of various images, and includes a hard disk drive, a flash memory, a RAM and a ROM, etc. The first complex amplitude image generating unit 52, the second complex amplitude image generating unit 53, the two-dimensional phase image generating unit 54, the three-dimensional phase image generating unit 55, the refractive index distribution calculating unit 56 and the storage unit 58 can also be configured by cloud computing.

[0113] The storage unit 58 also stores a program for causing the interference intensity image acquisition unit 51, the first complex amplitude image generation unit 52, the second complex amplitude image generation unit 53, the two-dimensional phase image generation unit 54, the three-dimensional phase image generation unit 55, and the refractive index distribution calculation unit 56 to perform each process. The program may be stored in the storage unit 58 when the observation device 1A is manufactured or shipped, or the program acquired via a communication line after shipping may be stored in the storage unit 58, or the program stored in the computer-readable storage medium 2 may be stored in the storage unit 58. The storage medium 2 is any medium such as a floppy disk, a CD-ROM, a DVD-ROM, a BD-ROM, a USB memory, or the like.

[0114] Details of the processing performed by the interference intensity image acquisition unit 51 , the first complex amplitude image generation unit 52 , the second complex amplitude image generation unit 53 , the two-dimensional phase image generation unit 54 , the three-dimensional phase image generation unit 55 , and the refractive index distribution calculation unit 56 will be described later.

[0115] Fig. A02 is a diagram showing the configuration of an observation device 1B. The observation device 1B shown in Fig. A02 includes a lens 31, a reflection mirror 32, a drive unit 33, a lens 34, and the like in addition to the configuration of the observation device 1A shown in Fig. A01.

[0116] The lens 31 is optically connected to the light output end 19, and collimates the light (reference light) output from the light output end 19 as divergent light. The reflector 32 is optically connected to the lens 31, and reflects the light arriving from the lens 31 toward the lens 34. The lens 34 is optically connected to the reflector 32, and outputs the light arriving from the reflector 32 toward the beam splitter 41. The light output from the lens 34 is temporarily focused before the beam splitter 41, and then input to the beam splitter 41 as divergent light. The beam splitter 41 combines the light (object light) output and arriving from the objective lens 25 and the light (reference light) output and arriving from the lens 34, and outputs the two lights coaxially to the lens 42. The imaging unit 43 captures the interference fringe image (interference intensity image) formed by the interference of the object light and the reference light arriving from the lens 42. The incident direction of the reference light is parallel to the incident direction of the object light to the imaging surface of the imaging unit 43.

[0117] The driving unit 33 moves the reflector 32 in a direction perpendicular to the reflective surface of the reflector 32. The driving unit 33 is, for example, a piezoelectric actuator. By the movement of the reflector 32, the difference (phase difference) in the optical path lengths of the object light and the reference light combined from the optical fiber coupler 15 to the beam splitter 41 is changed. If the optical path length difference is different, the interference intensity image captured by the imaging unit 43 is also different.

[0118] The observation device is not limited to the configuration examples of FIG. A01 and FIG. A02, and various modifications are possible. In the configurations of the observation device 1A and the observation device 1B, object light transmitted through the observation object S is observed, but object light reflected by the observation object S may also be observed as in the configuration of the observation device 1C described below.

[0119] Fig. A03 is a diagram showing the structure of the observation device 1C. The observation device 1C includes: a light source 11, a lens 12, a lens 21, a reflector 22, a lens 23, an objective lens 25, a beam splitter 41, a lens 42, a camera 43, and an analysis unit 50. The following mainly describes the differences from the observation device 1A (Fig. A01).

[0120] The lens 21 is optically connected to the light output end 18 of the optical fiber 16, and collimates the light outputted from the light output end 18 as divergent light. The reflector 22 is optically connected to the lens 21, and reflects the light arriving from the lens 21 to the lens 23. The orientation of the reflection surface of the reflector 22 is variable. The lens 23 is optically connected to the reflector 22. The objective lens 25 is optically connected to the lens 23. A beam splitter 41 is arranged between the lens 23 and the objective lens 25. The lens 23 and the objective lens 25 constitute, for example, a 4f optical system. The lens 23 and the objective lens 25 irradiate light to the observation object S from the light irradiation direction corresponding to the orientation of the reflection surface of the reflector 22. The objective lens 25 inputs the light (object light) reflected by the observation object S, and outputs the light to the beam splitter 41.

[0121] The beam splitter 41 is optically connected to the objective lens 25, and is also optically connected to the light output end 19 of the optical fiber 17. The beam splitter 41 combines the light (object light) output and arriving from the objective lens 25 and the light (reference light) output and arriving from the light output end 19, and outputs the two lights to the lens 42. The lens 42 is optically connected to the beam splitter 41, and collimates the object light and the reference light arriving from the beam splitter 41 and outputs them to the shooting unit 43. The shooting unit 43 is optically connected to the lens 42, and shoots the interference fringe image (interference intensity image) formed by the interference of the object light and the reference light arriving from the lens 42. The incident direction of the reference light is inclined relative to the incident direction of the object light to the shooting surface of the shooting unit 43. The position where the object light and the reference light are combined by the beam splitter 41 can also be at the rear stage of the imaging lens. If the influence of aberration is considered, it is desired to be between the objective lens 25 and the lens 42 as shown in the figure.

[0122] In the structure of the observation device 1C (FIG. A03), a mechanism for changing the optical path length of the reference light (lens 31, reflector 32, drive unit 33, and lens 34 in FIG. A02) may be provided similarly to the observation device 1B (FIG. A02), and the difference (phase difference) in the optical path length of the combined object light and reference light branched from the optical fiber coupler 15 to the beam splitter 41 may be changed. In this case, the incident direction of the reference light may be parallel to the incident direction of the object light to the imaging surface of the imaging unit 43.

[0123] FIG. A04 is a flow chart of a refractive index distribution measurement method A. The refractive index distribution measurement method A can be implemented when using any of the observation devices 1A to 1C. The refractive index distribution measurement method A comprises: an interference intensity image acquisition step S1, a first complex amplitude image generation step S2, a second complex amplitude image generation step S3, a two-dimensional phase image generation step S4, a three-dimensional phase image generation step S5, and a refractive index distribution calculation step S6.

[0124] The process of the interference intensity image acquisition step S1 is performed by the interference intensity image acquisition unit 51. The process of the first complex amplitude image generation step S2 is performed by the first complex amplitude image generation unit 52. The process of the second complex amplitude image generation step S3 is performed by the second complex amplitude image generation unit 53. The process of the two-dimensional phase image generation step S4 is performed by the two-dimensional phase image generation unit 54. The process of the three-dimensional phase image generation step S5 is performed by the three-dimensional phase image generation unit 55. The process of the refractive index distribution calculation step S6 is performed by the refractive index distribution calculation unit 56.

[0125] In the interference intensity image acquisition step S1, the interference intensity image acquisition unit 51 irradiates the observation object S along each of a plurality of light irradiation directions by changing the orientation of the reflection surface of the reflection mirror 22. Furthermore, the interference intensity image acquisition unit 51 acquires an interference intensity image of a reference position from the imaging unit 43 for each of the plurality of light irradiation directions.

[0126] In each of Figures A01 to A03, for the sake of convenience, an xyz rectangular coordinate system is shown. The z axis is parallel to the optical axis of the objective lens 25. The reference position is the image plane position that is in a conjugate relationship with the imaging plane of the imaging unit 43. This position is set to z = 0. The direction of light irradiation to the observation object S can be determined by the wave number vector (k x , k y , k z ) x and k y express.

[0127] Fig. A05 (a) to (c) are diagrams showing an example of scanning in the light irradiation direction toward the observation object S in the interference intensity image acquisition step S1. x , set the vertical axis to k y K x k y On the plane, the position of each circle indicates the direction of light irradiation. The scanning of the light irradiation direction can also be performed in k as shown in (a) of Figure A05. x k y The plane is arranged in a rectangular grid, or as shown in Figure A05 (b) x k y The plane is arranged on the circumference of each of the multiple concentric circles. Alternatively, as shown in (c) of FIG. A05, the k x k y The plane is arranged in a spiral shape. In any case, as long as the numerical aperture (NA: Numerical Aperture) of the focusing lens 24 allows, scanning in the light irradiation direction can be performed. It can also be any of raster scanning and random scanning. In the case of raster scanning, return scanning may or may not be performed.

[0128] In the first complex amplitude image generation step S2, the first complex amplitude image generation unit 52 generates a complex amplitude image of the reference position for each of the plurality of light irradiation directions based on the interference intensity image of the reference position acquired by the interference intensity image acquisition unit 51. In the case of the observation device 1A (FIG. A01) and the observation device 1C (FIG. A03), the first complex amplitude image generation unit 52 can generate a complex amplitude image based on one interference intensity image by the Fourier fringe analysis method. In the case of the observation device 1B (FIG. A02), the first complex amplitude image generation unit 52 can generate a complex amplitude image based on three or more interference intensity images having different optical path length differences (phase differences) between the object light and the reference light by the phase shift method.

[0129] In the second complex amplitude image generation step S3, the second complex amplitude image generation unit 53 generates, for each of the plurality of light irradiation directions, a complex amplitude image at each of the plurality of z-direction positions based on the complex amplitude image at the reference position (z=0) generated by the first complex amplitude image generation unit 52. If the two-dimensional Fourier transform of the complex amplitude image u(x, y, 0) at the reference position is represented by U(k x , k y , 0), then the complex amplitude image u(x, y, d) at position z=d and the two-dimensional Fourier transform U(k x , k y , d) is represented by the following formula. i is an imaginary unit, and k0 is the wave number of the light in the observed object.

[0130]

[0131] u(x,y,d)=∫U(k x ,k y ,d)exp(-ik x x-ik y y)dk x dk y (2)

[0132] In the two-dimensional phase image generation step S4, the two-dimensional phase image generation unit 54 generates a two-dimensional phase image for each of the plurality of positions based on the complex amplitude images in the plurality of light irradiation directions generated by the second complex amplitude image generation unit 53. The two-dimensional phase image generated here corresponds to a phase image centered on the focused z-direction position. The details of the two-dimensional phase image generation step S4 will be described later.

[0133] In addition, it is also possible that after all the complex amplitude images of the plurality of positions are generated for each of the plurality of light irradiation directions in the second complex amplitude image generation step S3, the processing after the two-dimensional phase image generation step S4 is performed. In addition, it is also possible that the complex amplitude image of a certain z-direction position is generated for each of the plurality of light irradiation directions in the second complex amplitude image generation step S3, and the processing of generating the two-dimensional phase image of the position in the two-dimensional phase image generation step S4 is used as a unit, and the unit processing is repeatedly performed while scanning the z-direction position. In the latter case, it is good in that the capacity of the image data to be stored in the storage unit 58 can be reduced.

[0134] In the three-dimensional phase image generating step S5, the three-dimensional phase image generating unit 55 generates a three-dimensional phase image based on the two-dimensional phase images at each of the plurality of positions generated by the two-dimensional phase image generating unit 54. The three-dimensional phase image generated here is an image having positions x and y in the two-dimensional phase image and position z of the two-dimensional phase image as variables.

[0135] In the refractive index distribution calculation step S6, the refractive index distribution calculation unit 56 obtains the three-dimensional refractive index distribution of the observation object by deconvolution based on the three-dimensional phase image generated by the three-dimensional phase image generation unit 55. If the refractive index distribution of the observation object is n(x, y, z), the electric susceptibility distribution is f(x, y, z), and the refractive index of the background medium is n m , then there is a relationship between the two as shown in the following formula (3). The three-dimensional phase image Φ(x, y, z) generated by the three-dimensional phase image generating unit 55 is represented by the convolution of the kernel function g(x, y, z) and the electric susceptibility distribution f(x, y, z) as shown in the following formula (4). Therefore, the three-dimensional refractive index distribution n(x, y, z) of the observed object can be obtained by deconvolution based on the three-dimensional phase image Φ(x, y, z).

[0136]

[0137] Φ(x,y,z)=∫g(xx′,yy′,zz′)f(x′,y′,z′)dx′dy′dz′ (4)

[0138] In addition, the kernel function g is based on the Green's function corresponding to the solution of the wave equation. Fig. A06 is a diagram illustrating the kernel function g. In this figure, the center position where the value of the kernel function g is the largest is the origin, the vertical direction is the z-axis, and the horizontal direction is the direction perpendicular to the z-axis.

[0139] In addition, each processing of the first complex amplitude image generating step S2, the second complex amplitude image generating step S3, the two-dimensional phase image generating step S4, the three-dimensional phase image generating step S5 and the refractive index distribution calculating step S6 can also be performed each time the interference intensity images of a specified number of light irradiation directions are acquired in the interference intensity image acquiring step S1 (Figure A07), or can be performed each time the interference intensity image of one light irradiation direction is acquired in the interference intensity image acquiring step S1 (Figure A08).

[0140] FIG. A07 and FIG. A08 are diagrams showing examples of scanning in the light irradiation direction toward the observation object S in the interference intensity image acquisition step S1. x , set the vertical axis to k y K x k y On the plane, the position of each circle represents the light irradiation direction. In the examples of scanning the light irradiation direction shown in these figures, the light irradiation direction is changed sequentially so that the light irradiation direction when the (N+n)th interference intensity image is acquired is consistent with the light irradiation direction when the nth interference intensity image is acquired. n is a positive integer, and N is an integer greater than 2.

[0141] In the example shown in FIG. A07, if the 1st to Nth interference intensity images are acquired in the interference intensity image acquisition step S1, each process of steps S2 to S6 is performed based on these 1st to Nth interference intensity images ((a) in FIG. A07). Next, if the (N+1)th to 2Nth interference intensity images are acquired in the interference intensity image acquisition step S1, each process of steps S2 to S6 is performed based on these (N+1)th to 2Nth interference intensity images ((b) in FIG. A07). Next, if the (2N+1)th to 3Nth interference intensity images are acquired in the interference intensity image acquisition step S1, each process of steps S2 to S6 is performed based on these (2N+1)th to 3Nth interference intensity images. The same is true for the following.

[0142] In the example shown in FIG. A08, if the 1st to Nth interference intensity images are acquired in the interference intensity image acquisition step S1, each process of steps S2 to S6 is performed based on these 1st to Nth interference intensity images ((a) in FIG. A08). Next, if the (N+1)th interference intensity image is acquired in the interference intensity image acquisition step S1, each process of steps S2 to S6 is performed based on the most recent N interference intensity images (the 2nd to (N+1)th interference intensity images) including the (N+1)th interference intensity image ((b) in FIG. A08). Next, if the (N+2)th interference intensity image is acquired in the interference intensity image acquisition step S1, each process of steps S2 to S6 is performed based on the most recent N interference intensity images (the 3rd to (N+2)th interference intensity images) including the (N+2)th interference intensity image ((c) in FIG. A08). Similarly, if the (N+n)th interference intensity image is obtained in the interference intensity image acquisition step S1, the processing of steps S2 to S6 is performed based on the N most recent interference intensity images (the (1+n)th to (N+n)th interference intensity images) including the (N+n)th interference intensity image.

[0143] Compared with the example shown in Figure A07, in the example shown in Figure A08, each time an interference intensity image in a light irradiation direction is acquired in the interference intensity image acquisition step S1, each process of steps S2 to S6 is performed based on the most recent multiple interference intensity images including the interference intensity image. Therefore, the number of images obtained per unit time through the each process of steps S2 to S6 is larger.

[0144] Next, the details of the two-dimensional phase image generation step S4 in the refractive index distribution measurement method A are described. In the two-dimensional phase image generation step S4, the two-dimensional phase image generation unit 54 generates a two-dimensional phase image for each of the plurality of positions based on the complex amplitude images in each of the plurality of light irradiation directions generated by the second complex amplitude image generation unit 53. The two-dimensional phase image generation step S4 differs depending on which of the refractive index distribution measurement methods A1 to A3 is used.

[0145] Fig. A09 is a flowchart of the two-dimensional phase image generation step S4 of the refractive index distribution measurement method A1. In the two-dimensional phase image generation step S4 of the refractive index distribution measurement method A1, for each of the plurality of positions, after correcting the phase of the complex amplitude images of the plurality of light irradiation directions based on the light irradiation direction in step S11, a complex amplitude sum image representing the sum of the corrected complex amplitude images is generated, and in step S12, a two-dimensional phase image is generated based on the complex amplitude sum image.

[0146] The processing of step S11 is based on CASS (Collective Accumulation of Single Scattering) (Sungsam Kang, et al, "Imaging deep within a scattering medium using collective accumulation of single-scattered waves," NATURE PHOTONICS, Vol. 9, pp. 253-258 (2015).) technology. The spatial frequency distribution of single scattered light that has only interacted with the object once in the light that irradiates the object along a certain light irradiation direction is shifted according to the light irradiation direction. In contrast, the spatial frequency distribution of multiple scattered light that has interacted with the object multiple times changes randomly according to the light irradiation direction. The CASS technology uses the difference in the dependence of the spatial frequency distribution of such single scattered light and multiple scattered light on the light irradiation direction.

[0147] That is, in step S11, by correcting the phase of each of the complex amplitude images in the plurality of light irradiation directions based on the light irradiation direction (i.e., parallel shifting the spatial frequency distribution of the complex amplitude image in the spatial frequency region according to the light irradiation direction), the spatial frequency distribution of the single scattered light component in the complex amplitude image is set to a shape and arrangement that is independent of the light irradiation direction, and on the other hand, the spatial frequency distribution of the multiple scattered light components in the complex amplitude image is set to a random shape and arrangement. Furthermore, in step S11, by generating a complex amplitude sum image representing the sum of the plurality of corrected complex amplitude images (i.e., performing synthetic aperture processing), the single scattered light components in the complex amplitude image are coherently added, and on the other hand, the multiple scattered light components in the complex amplitude image are mutually canceled.

[0148] Therefore, the complex amplitude sum image generated in step S11 reduces the influence of multiple scattered light. Moreover, the three-dimensional refractive index distribution finally obtained in the refractive index distribution calculation step S6 also reduces the influence of multiple scattered light, suppresses speckle, and improves the single-scattering to multi-scattering ratio (SMR).

[0149] Fig. A10 is a flowchart of the two-dimensional phase image generation step S4 of the refractive index distribution measurement method A2. The two-dimensional phase image generation step S4 of the refractive index distribution measurement method A2 generates, for each of the multiple positions, a complex differential interference image of each of the multiple light irradiation directions based on the complex amplitude image of each of the multiple light irradiation directions in step S21, generates a phase differential image based on the sum of the complex differential interference images of each of the multiple light irradiation directions in step S22, and generates a two-dimensional phase image based on the phase differential image in step S23.

[0150] If the complex amplitude image at the position z=d is set to u(x, y, d), the complex differential interference image q(x, y, d) generated in step S21 is expressed by the following formula (5). At least one of δx and δy is non-zero. If δx≠0, δy=0, a complex differential interference image q with the x direction as the shear direction is obtained. If δx=0, δy≠0, a complex differential interference image q with the y direction as the shear direction is obtained. If δx≠0, δy≠0, a complex differential interference image q with a direction different from both the x direction and the y direction as the shear direction is obtained. In addition, the complex differential interference image q(x, y, d) can also be obtained by formula (5) after the complex amplitude image u(x, y, d) is transformed as shown in the following formula (6).

[0151] g(x, y, d) = u * (x+δx,y+δy,d)·u(x,y,d) (5)

[0152] u(x,y,d)exp(-ik x x-ik y y) (6)

[0153] If the sum of the complex differential interference images q of the multiple light irradiation directions is set to q sum (x, y, d), the phase differential image φ(x, y, z) generated in step S22 is used as q sum The phase of (x, y, d) is expressed by the following equation (7): In step S23, by integrating or deconvolving the phase differential image φ(x, y, z), a two-dimensional phase image can be generated.

[0154] φ(x, y, d) = ∠q sum (x, y, d) (7)

[0155] In addition, in step S21, a complex differential interference image may be generated for each of the multiple different shearing directions on the complex amplitude image. In this case, in step S4, a two-dimensional phase image is generated. For each of the multiple positions, in step S21, based on the complex amplitude image of each of the multiple light irradiation directions, a complex differential interference image of each of the multiple different shearing directions on the image is generated. In step S22, for each of the multiple shearing directions, a phase differential image is generated based on the sum of the complex differential interference images of each of the multiple light irradiation directions. In step S23, a two-dimensional phase image is generated based on the phase differential images of each of the multiple shearing directions.

[0156] The phase differential image generated based on the sum of the complex differential interference images of the multiple light irradiation directions in step S22 reduces the influence of multiple scattered light. Moreover, the three-dimensional refractive index distribution finally obtained in the refractive index distribution calculation step S6 also reduces the influence of multiple scattered light and suppresses speckle. In addition, when the complex differential interference image is generated for each of the multiple shearing directions that are different from each other on the complex amplitude image in step S21, the linear noise in the two-dimensional phase image obtained in step S23 can be suppressed.

[0157] Here, the case where a two-dimensional phase image is generated by integrating or deconvolving the phase differential image in step S23 is described. However, the phase differential image can also be processed as a two-dimensional phase image. In this case, step S23 can be omitted, and a kernel (FIG. A11) including the kernel used for the deconvolution of step S23 can be used in the deconvolution of the refractive index distribution calculation step S67, thereby obtaining the three-dimensional refractive index distribution of the observation object based on the phase differential image (two-dimensional phase image) generated in step S22. The kernel shown in FIG. A11 is obtained by convolving the kernel shown in FIG. A06 and the kernel used for the deconvolution of step S23.

[0158] Fig. A12 is a flowchart of the two-dimensional phase image generation step S4 of the refractive index distribution measurement method A3. In the two-dimensional phase image generation step S4 of the refractive index distribution measurement method A3, for each of the multiple positions, in step S31, the complex amplitude images of each of the multiple light irradiation directions are divided into multiple batches, and for each of the multiple batches, after correcting the phase of the complex amplitude images included in the batch based on the light irradiation direction, a complex amplitude sum image representing the sum of these corrected complex amplitude images is generated, in step S32, complex differential interference images of each of the multiple batches are generated based on the complex amplitude sum images of each of the multiple batches, in step S33, a phase differential image is generated based on the sum of the complex differential interference images of each of the multiple batches, and in step S34, a two-dimensional phase image is generated based on the phase differential image.

[0159] The processing of step S31 of the refractive index distribution measurement method A3 is equivalent to dividing the complex amplitude images of each of the multiple light irradiation directions into multiple batches and performing the processing of step S11 of the refractive index distribution measurement method A1 on each of the multiple batches. The processing of steps S32 and S33 of the refractive index distribution measurement method A3 is equivalent to performing the processing of steps S21 and S22 of the refractive index distribution measurement method A2 on each of the multiple batches. The processing of step S34 of the refractive index distribution measurement method A3 is equivalent to performing the processing of step S23 of the refractive index distribution measurement method A2.

[0160] Furthermore, in step S32, a complex differential interference image may be generated for each of a plurality of different shearing directions on the complex amplitude image. In this case, in step S4, a two-dimensional phase image is generated. In step S32, a plurality of complex differential interference images of each batch are generated for each of a plurality of different shearing directions on the image based on the complex amplitude sum images of each batch. In step S33, a phase differential image is generated for each of the plurality of shearing directions based on the sum of the complex differential interference images of each batch. In step S34, a two-dimensional phase image is generated based on the phase differential images of each of the plurality of shearing directions.

[0161] The suppression of speckle by the refractive index profile measurement method A3 is at the same level as that of the refractive index profile measurement methods A1 and A2. The improvement of SMR by the refractive index profile measurement method A3 is at an intermediate level between those of the refractive index profile measurement methods A1 and A2.

[0162] Here, the case where a two-dimensional phase image is generated by integrating or deconvolving the phase differential image in step S34 is also described. However, the phase differential image can also be processed as a two-dimensional phase image. In this case, instead of performing step S34, a kernel including the kernel used for the deconvolution in step S34 can be used in the deconvolution in the refractive index distribution calculation step S6, thereby obtaining the three-dimensional refractive index distribution of the observation object from the phase differential image (two-dimensional phase image) generated in step S33.

[0163] Next, the refractive index distribution measurement method B is described. FIG. B01 to FIG. B03 are diagrams showing the structures of observation devices 1D to 1F that can be used when measuring the refractive index distribution based on the refractive index distribution measurement method B. The observation device 1D shown in FIG. B01 has the same optical system from the light source 11 to the imaging unit 43 as the observation device 1A shown in FIG. A01, but is different in that an analysis unit 60 is provided instead of the analysis unit 50. The observation device 1E shown in FIG. B02 has the same optical system from the light source 11 to the imaging unit 43 as the observation device 1B shown in FIG. A02, but is different in that an analysis unit 60 is provided instead of the analysis unit 50. The observation device 1F shown in FIG. B03 has the same optical system from the light source 11 to the imaging unit 43 as the observation device 1C shown in FIG. A03, but is different in that an analysis unit 60 is provided instead of the analysis unit 50.

[0164] The analyzing unit 60 is electrically connected to the imaging unit 43, and inputs the interference intensity image captured by the imaging unit 43. The analyzing unit 60 calculates the three-dimensional refractive index distribution of the observation object S by processing the input interference intensity image. The analyzing unit 60 may also be a computer. The analyzing unit 60 includes: an interference intensity image acquisition unit 61, a first complex amplitude image generation unit 62, a second complex amplitude image generation unit 63, a phase conjugate operation unit 64, a two-dimensional phase image generation unit 65, a three-dimensional phase image generation unit 66, a refractive index distribution calculation unit 67, a display unit 68, and a storage unit 69.

[0165] The interference intensity image acquisition unit 61 irradiates the observation object S along each of a plurality of light irradiation directions by changing the orientation of the reflection surface of the reflector 22. In addition, the interference intensity image acquisition unit 61 acquires the interference intensity image of the reference position from the shooting unit 43 for each of the plurality of light irradiation directions. The interference intensity image acquisition unit 61 includes a CPU, and has an output port for outputting a control signal for changing the orientation of the reflection surface of the reflector 22, and also has an input port for inputting the interference intensity image from the shooting unit 43. There is no need to move the objective lens 25 along the optical axis. The reference position is an image plane position that is in a conjugate relationship with the shooting surface of the shooting unit 43.

[0166] The first complex amplitude image generating unit 62, the second complex amplitude image generating unit 63, the phase conjugate calculating unit 64, the two-dimensional phase image generating unit 65, the three-dimensional phase image generating unit 66 and the refractive index distribution calculating unit 67 perform processing based on the interference intensity image, and include a processing device such as a CPU, a GPU, a DSP or an FPGA. The display unit 68 displays the image to be processed, the image in the process of processing and the image after processing, and includes, for example, a liquid crystal display. The storage unit 69 stores data of various images, and includes a hard disk drive, a flash memory, a RAM and a ROM, etc. The first complex amplitude image generating unit 62, the second complex amplitude image generating unit 63, the phase conjugate calculating unit 64, the two-dimensional phase image generating unit 65, the three-dimensional phase image generating unit 66, the refractive index distribution calculating unit 67 and the storage unit 69 can also be configured by cloud computing.

[0167] The storage unit 69 also stores a program for causing the interference intensity image acquisition unit 61, the first complex amplitude image generation unit 62, the second complex amplitude image generation unit 63, the phase conjugate operation unit 64, the two-dimensional phase image generation unit 65, the three-dimensional phase image generation unit 66, and the refractive index distribution calculation unit 67 to perform each process. The program may be stored in the storage unit 69 when the observation apparatuses 1D to 1F are manufactured or shipped, or the program acquired via a communication line after shipping may be stored in the storage unit 69, or the program stored in a computer-readable storage medium 2 may be stored in the storage unit 69. The storage medium 2 is any medium such as a floppy disk, a CD-ROM, a DVD-ROM, a BD-ROM, or a USB memory.

[0168] Details of the processing performed by the interference intensity image acquisition unit 61, the first complex amplitude image generation unit 62, the second complex amplitude image generation unit 63, the phase conjugate operation unit 64, the two-dimensional phase image generation unit 65, the three-dimensional phase image generation unit 66 and the refractive index distribution calculation unit 67 will be described later.

[0169] FIG. B04 is a flow chart of a refractive index distribution measurement method B. The refractive index distribution measurement method B can be implemented when using any of the observation devices 1D to 1F. The refractive index distribution measurement method B comprises: an interference intensity image acquisition step S61, a first complex amplitude image generation step S62, a second complex amplitude image generation step S63, a phase conjugation calculation step S64, a two-dimensional phase image generation step S65, a three-dimensional phase image generation step S66, and a refractive index distribution calculation step S67.

[0170] The interference intensity image acquisition step S61 is processed by the interference intensity image acquisition unit 61. The first complex amplitude image generation step S62 is processed by the first complex amplitude image generation unit 62. The second complex amplitude image generation step S63 is processed by the second complex amplitude image generation unit 63. The phase conjugate operation step S64 is processed by the phase conjugate operation unit 64. The two-dimensional phase image generation step S65 is processed by the two-dimensional phase image generation unit 65. The three-dimensional phase image generation step S66 is processed by the three-dimensional phase image generation unit 66. The refractive index distribution calculation step S67 is processed by the refractive index distribution calculation unit 67.

[0171] In the interference intensity image acquisition step S61, the interference intensity image acquisition unit 61 irradiates the observation object S along each of the plurality of light irradiation directions by changing the orientation of the reflection surface of the reflector 22. Furthermore, the interference intensity image acquisition unit 61 acquires the interference intensity image of the reference position from the imaging unit 43 for each of the plurality of light irradiation directions.

[0172] In the first complex amplitude image generation step S62, the first complex amplitude image generation unit 62 generates a complex amplitude image of the reference position for each of the plurality of light irradiation directions based on the interference intensity image of the reference position acquired by the interference intensity image acquisition unit 61. In the case of the observation device 1D (FIG. B01) and the observation device 1F (FIG. B03), the first complex amplitude image generation unit 62 can generate a complex amplitude image based on one interference intensity image by the Fourier fringe analysis method. In the case of the observation device 1E (FIG. B02), the first complex amplitude image generation unit 62 can generate a complex amplitude image based on three or more interference intensity images having different optical path length differences (phase differences) between the object light and the reference light by the phase shift method.

[0173] In the second complex amplitude image generating step S63 , the second complex amplitude image generating unit 63 generates complex amplitude images at a plurality of positions in the z direction for each of the plurality of light irradiation directions based on the complex amplitude image at the reference position (z=0) generated by the first complex amplitude image generating unit 62 .

[0174] The interference intensity image acquisition step S61, the first complex amplitude image generation step S62 and the second complex amplitude image generation step S63 in the refractive index distribution measurement method B perform the same processing as the interference intensity image acquisition step S1, the first complex amplitude image generation step S2 and the second complex amplitude image generation step S3 in the refractive index distribution measurement method A, respectively.

[0175] The phase conjugation operation step S64 is performed after the processing of the second complex amplitude image generating step S63. The phase conjugation operation step S64 may also be performed before the processing of the second complex amplitude image generating step S63 (described later). In addition, in the case where the second complex amplitude image generating step S63 generates a complex amplitude image of a certain z position from a complex amplitude image of a reference position through multiple stages, the phase conjugation operation step S64 may also be performed between a certain stage in the multiple stages and the next stage (described later). In the phase conjugation operation step S64, the phase conjugation operation unit 64 performs phase conjugation operation on the complex amplitude images of each of the multiple irradiation directions, and generates the complex amplitude images of each of the multiple irradiation directions when the relationship between light irradiation and shooting relative to the observation object is reversed.

[0176] In addition, the phase conjugate operation is an operation on the complex amplitude image based on the phase conjugate method, which is to calculate the transfer matrix representing the relationship between the light irradiation and the light output of the object, and includes the calculation of its inverse matrix and coordinate transformation. The phase conjugate method is sometimes also called phase conjugation, time reversal method, time reversal, digital phase conjugation, digital phase conjugation method, etc. Details will be described later.

[0177] In the two-dimensional phase image generation step S65, the two-dimensional phase image generation unit 65 generates a two-dimensional phase image for each of the plurality of positions based on the complex amplitude images in the plurality of light irradiation directions generated by the second complex amplitude image generation unit 63 or the phase conjugate calculation unit 64. The two-dimensional phase image generated here corresponds to a phase image centered on the focused z-direction position.

[0178] In the two-dimensional phase image generating step S65, when the phase image generated based on the complex amplitude image before the processing of the phase conjugation operation step S64 is set as the first phase image and the phase image generated based on the complex amplitude image obtained by the processing of the phase conjugation operation step S64 is set as the second phase image, the two-dimensional phase image is mainly generated based on the first phase image for the position relatively close to the imaging part among the multiple positions, and the two-dimensional phase image is mainly generated based on the second phase image for the position relatively far from the imaging part.

[0179] In addition, after all the complex amplitude images of the respective positions in the plurality of light irradiation directions are generated in the second complex amplitude image generation step S63, the processing after the phase conjugation operation step S64 may be performed. In addition, in the second complex amplitude image generation step S63, a complex amplitude image of a certain position in the z direction may be generated for each of the plurality of light irradiation directions, and the processing of generating a two-dimensional phase image of the position in the two-dimensional phase image generation step S65 may be repeated while scanning the z direction position. In the latter case, it is good in that the capacity of the image data to be stored in the storage unit 69 can be reduced.

[0180] In the three-dimensional phase image generation step S66, the three-dimensional phase image generation unit 66 generates a three-dimensional phase image based on the two-dimensional phase images of each of the plurality of positions generated by the two-dimensional phase image generation unit 65. The three-dimensional phase image generated here is an image having positions x and y in the two-dimensional phase image and position z of the two-dimensional phase image as variables.

[0181] In the refractive index distribution calculation step S67 , the refractive index distribution calculation unit 67 obtains the three-dimensional refractive index distribution of the observation object by deconvolution based on the three-dimensional phase image generated by the three-dimensional phase image generation unit 66 .

[0182] The two-dimensional phase image generation step S65, the three-dimensional phase image generation step S66 and the refractive index distribution calculation step S67 in the refractive index distribution measurement method B perform the same processing as the two-dimensional phase image generation step S4, the three-dimensional phase image generation step S5 and the refractive index distribution calculation step S6 in the refractive index distribution measurement method A, respectively.

[0183] FIG. B05 is a diagram for explaining the order and images of each process of the second complex amplitude image generation step S63 and the two-dimensional phase image generation step S65. This figure shows a method in which the process of the phase conjugation operation step S64 is not performed. In this method, in the second complex amplitude image generation step S63, for each of the multiple light irradiation directions, the free propagation formula of the above-mentioned formula (1) and formula (2) is used to generate a complex amplitude image of each of the multiple z-direction positions (z=z1, z2, z3 in this figure) based on the complex amplitude image of the reference position (z=0) generated in the first complex amplitude image generation step S62. Then, in the two-dimensional phase image generation step S65, for each of the multiple positions, a complex differential interference image is generated based on the complex amplitude image of each of the multiple light irradiation directions generated in the second complex amplitude image generation step S63, and a phase differential image is further generated.

[0184] Figures B06 to B08 are diagrams for explaining the order and images of each process of the second complex amplitude image generation step S63, the phase conjugate operation step S64, and the two-dimensional phase image generation step S65. These diagrams show how the process of the phase conjugate operation step S64 is performed before, during, or after the process of the second complex amplitude image generation step S63.

[0185] The first method shown in FIG. B06 corresponds to the flowchart of FIG. B04. In the first method, the phase conjugation operation step S64 is performed after the processing of the second complex amplitude image generation step S63. In the second complex amplitude image generation step S63, for each of the multiple light irradiation directions, the free propagation formula of the above formula (1) and formula (2) is used to generate a complex amplitude image of each of the multiple z-direction positions (z=z1, z2, z3 in the figure) based on the complex amplitude image of the reference position (z=0) generated in the first complex amplitude image generation step S62.

[0186] In the first mode, next, in the phase conjugate calculation step S64, for each of the plurality of positions, the complex amplitude images for each of the plurality of irradiation directions are subjected to phase conjugate calculation, and the complex amplitude images for each of the plurality of irradiation directions are generated when the relationship between light irradiation and imaging with respect to the observation object is reversed. Then, in the two-dimensional phase image generation step S65, for each of the plurality of positions, a complex differential interference image is generated based on the complex amplitude images for each of the plurality of light irradiation directions generated in the phase conjugate calculation step S64, and a phase differential image is further generated.

[0187] In the second mode shown in FIG. B07, the phase conjugate operation step S64 is performed before the processing of the second complex amplitude image generating step S63. In the phase conjugate operation step S64, for each of the plurality of light irradiation directions, the complex amplitude image of the reference position (z=0) generated in the first complex amplitude image generating step S62 is subjected to phase conjugate operation, and a complex amplitude image of each of the plurality of irradiation directions is generated when the relationship between light irradiation and imaging with respect to the observation object is reversed.

[0188] In the second method, then, in the second complex amplitude image generating step S63, for each of the plurality of light irradiation directions, based on the complex amplitude image of the reference position (z=0) generated in the phase conjugation operation step S64, a complex amplitude image is generated for each of the plurality of z-direction positions (z=z1, z2, z3 in the figure). Furthermore, in the two-dimensional phase image generating step S65, for each of the plurality of positions, based on the complex amplitude images for each of the plurality of light irradiation directions generated in the second complex amplitude image generating step S63, a complex differential interference image is generated, and a phase differential image is further generated.

[0189] In the third method shown in FIG. B08 , when the second complex amplitude image generating step S63 generates complex amplitude images for respective positions based on the complex amplitude image of the reference position through two stages, a phase conjugation operation step S64 is performed between the first stage and the second stage of the two stages.

[0190] In the third aspect, in the first stage of the second complex amplitude image generating step S63, for each of the plurality of light irradiation directions, the free propagation formula of the above formula (1) and formula (2) is used to generate a complex amplitude image of each of the plurality of z-direction positions (z=z1, z3, z5 in the figure) based on the complex amplitude image of the reference position (z=0) generated in the first complex amplitude image generating step S62. Next, in the phase conjugate computing step S64, the phase conjugate computing is performed on the complex amplitude images of each of the plurality of irradiation directions, and the complex amplitude images of each of the plurality of irradiation directions are generated when the relationship between the light irradiation and the photographing with respect to the observation object is reversed.

[0191] In the third embodiment, in the second stage of the second complex amplitude image generating step S63, for each of the plurality of light irradiation directions, a complex amplitude image of each of the z-direction positions (z=z2, z4, z6) is generated based on the complex amplitude image of the z-direction position (z=z1, z3, z5) generated in the phase conjugation calculation step S64 by the free propagation formula of the above formula (1) and formula (2). Furthermore, in the two-dimensional phase image generating step S65, for each of the plurality of positions, a complex differential interference image is generated based on the complex amplitude image of each of the plurality of light irradiation directions generated in the second complex amplitude image generating step S63, and a phase differential image is further generated.

[0192] The first, second and third methods differ in the number of times of phase conjugation calculation on the complex amplitude image in the phase conjugation calculation step S64. The third method is shorter than the first method in terms of the overall processing time of the phase conjugation calculation step S64, and the second method is even shorter.

[0193] FIG. B09 is a diagram for explaining the order and images of each process of the three-dimensional phase image generation step S66 and the refractive index distribution calculation step S67. In the three-dimensional phase image generation step S66, a three-dimensional phase image is generated based on the two-dimensional phase images of each of the plurality of positions generated in the two-dimensional phase image generation step S65. At this time, for a position relatively close to the imaging unit, a two-dimensional phase image generated based on the complex amplitude image before the process of the phase conjugation operation step S64 (a two-dimensional phase image generated in the manner of FIG. B05) is mainly used. On the other hand, for a position relatively far from the imaging unit, a two-dimensional phase image generated based on the complex amplitude image after the process of the phase conjugation operation step S64 (a two-dimensional phase image generated in any manner of FIG. B06 to FIG. B08) is mainly used. Next, in the refractive index distribution calculation step S67, based on the three-dimensional phase image generated in the three-dimensional phase image generation step S66, the three-dimensional refractive index distribution of the observation object is obtained by deconvolution. In addition, each refractive index distribution data constituting the three-dimensional refractive index distribution (for example, in FIG. B09 , the two-dimensional refractive index distribution data constituting the three-dimensional refractive index distribution) can become refractive index tomography data.

[0194] There are three ways to generate a two-dimensional phase image at each position in the z direction. A phase image generated based on the complex amplitude image before the phase conjugation operation step S64 (a phase image generated in the manner of FIG. B05) is set as the first phase image φ1. A phase image generated based on the complex amplitude image after the phase conjugation operation step S64 (a phase image generated in any of the manners of FIG. B06 to FIG. B08) is set as the second phase image φ2. A weighting function α whose derivative with respect to a variable z representing the distance from the imaging unit along the light propagation path is less than 0 is used. The value of the weighting function is greater than or equal to 0 and less than or equal to 1.

[0195] In the first approach, the weight function α is the threshold z at z th The following range is a positive value (eg, 1), and the values ​​in other ranges are 0. That is, the two-dimensional phase image is represented by the following equation (8).

[0196]

[0197] In the second aspect, the value of the weight function α changes continuously in at least a part of the range in the z direction. That is, the two-dimensional phase image is expressed by the following equation (9).

[0198] φ(x,y,z)=α(z)·φ1(x,y,z)+[1-α(z)]φ2(x,y,z) (9)

[0199] In the third aspect, the weight function α has a value corresponding to the position (x, y) on the plane perpendicular to the optical axis (z direction). That is, the two-dimensional phase image is expressed by the following equation (10).

[0200] φ(x,y,z)=α(x,y,z)·φ1(x,y,z)+[1-α(x,y,z)]φ2(x,y,z) (1o)

[0201] Next, the content of the phase conjugate calculation in the phase conjugate calculation step S64 will be described using FIG. B10 and FIG. B11.

[0202] FIG. B10 is a diagram showing the input light U when the interference intensity image is captured by the imaging unit. in (k in ) and output light out (r out ). in (k in ) represents the wave number k of the light irradiating the object to be observed in The complex amplitude of u out (r out ) represents the position r of the light output from the observation object out The complex amplitude of U in (k in ) and u out (r out ) is represented by the following equation (11). in The nth element U in (k in n ) represents the wave number k in n The complex amplitude of the plane wave. The column vector u out The nth element u out (r out n ) means at position r out n The complex amplitude of the observed light. The matrix T(r out , k in ) represents U in (k in ) and u out (r out The linear relationship between the two is called the transfer matrix. Through such a transfer matrix, the scattering process of light in the observed object can be represented. The matrix T(r out , k in )'s element T at row n1 and column n2 n1，n2 Indicates that the wave number k in n2 When a plane wave with amplitude 1 is input at position rout n1 The complex amplitude of the observed light.

[0203]

[0204] FIG. B11 is a diagram showing the input light U when the relationship between light irradiation and imaging is reversed. out (k out ) and output light in (r in ). In this case, U out (k out ) represents the wave number k of the light irradiating the object to be observed out The complex amplitude of u in (r in ) represents the position r of the light output from the observation object in The complex amplitude of U out (k out ) and u in (r in ) is expressed by the following equation (12). out The nth element U out (k out n ) represents the wave number k out n The complex amplitude of the plane wave. The column vector u in The nth element u in (r in n ) means at position r in n The complex amplitude of the observed light. The matrix S(r in , k out ) indicates U out (k out ) and u in (r in ) is the transfer matrix when the relationship between light irradiation and shooting is reversed.

[0205]

[0206] U in (k in ) As shown in the following formula (13), u in (r in ) is represented by the Fourier transform. out (k out ) As shown in the following formula (14), u out (r out) is represented by Fourier transform. If equations (11) to (14) are used, the transfer matrix S(r in , k out ) using the matrix representing the inverse Fourier transform and the transfer matrix T(r out , k in ), expressed by the following formula (15).

[0207]

[0208] In the phase conjugate calculation step S64, first, based on the complex amplitude image, the transfer matrix T (r out , k in Then, based on the transmission matrix T(r out , k in ) and the above formula (15), the transfer matrix S (r in , k out Then, based on the transmission matrix S(r in , k out ) to obtain the complex amplitude image when the relationship between light irradiation and shooting is reversed.

[0209] For each of the plurality of light irradiation directions, the vector U of the input light in the n-th light irradiation direction when the interference intensity image is captured by the imaging unit is in n (k in ) is represented by the following formula (16), where only the value of the nth element is 1 and the values ​​of the other elements are 0. in n (k in ), output light u out n (r out ) is expressed by the following formula (17). This formula (17) corresponds to the complex amplitude obtained when the n-th light is irradiated in the direction.

[0210]

[0211] Based on this equation (16) and the above equation (11), the following equation (18) is obtained. Furthermore, if the same equation is obtained for each of the plurality of light irradiation directions, the following equation (19) is obtained. Thus, the transfer matrix T(r out , k in ). Furthermore, according to the formula (19) and the above formula (15), the transfer matrix S (r in , k out ).

[0212]

[0213] Input light U in the n-th light irradiation direction among a plurality of light irradiation directions when the relationship between light irradiation and imaging is reversed out n (k out ) is represented by the following formula (20), where only the value of the nth element is 1, and the values ​​of the other elements are 0. According to this formula, with respect to the input light U out n (k out ) output light u in n (r in ) is represented by the following formula (21). This formula (21) represents the complex amplitude when the relationship between light irradiation and imaging is reversed. Thus, the complex amplitude image when the relationship between light irradiation and imaging is reversed can be obtained.

[0214]

[0215]

[0216] The transfer matrix S(r in , k out ), as shown in the above formula (15), it is necessary to calculate the transmission matrix T(r out , k in ). Therefore, the transmission matrix T needs to be a square matrix in which the number of row elements and the number of column elements are equal to each other. That is, the dimension of the matrix of the wave number space on the light irradiation side of the observation object and the number of pixels of the complex amplitude image in the interference intensity image acquisition step S61 need to be equal to each other.

[0217] In order to make the two equal to each other, the dimension of the matrix of the wave number space on the light irradiation side of the observation object in the interference intensity image acquisition step S61 is made consistent with the number of pixels, or only a part of the image obtained by the imaging unit is used for subsequent processing. However, in general, since the number of pixels of the image obtained by the imaging unit is, for example, 1024×1024, it is difficult to make the dimension of the matrix of the wave number space on the light irradiation side of the observation object the same as the number of pixels. In addition, using only a part of the image obtained by the imaging unit for subsequent processing leads to a decrease in resolution.

[0218] Therefore, it is preferable that, as shown in FIG. B12, in the phase conjugate operation step S64, the complex amplitude image is divided into a plurality of partial images each having the same number of pixels as the dimension of the matrix of the wave number space on the light irradiation side relative to the observation object, a phase conjugate operation is performed on each of the plurality of partial images, and then the plurality of partial images are combined. In this case, any two or more of the plurality of partial images may have a common region.

[0219] Next, the refractive index distribution measurement method C is described. FIG. C01 to FIG. C03 are diagrams showing the structures of observation devices 1G to 1I that can be used when measuring the refractive index distribution based on the refractive index distribution measurement method C. The observation device 1G shown in FIG. C01 has the same optical system from the light source 11 to the imaging unit 43 as the observation device 1A shown in FIG. A01, but is different in that an analysis unit 70 is provided instead of the analysis unit 50. The observation device 1H shown in FIG. C02 has the same optical system from the light source 11 to the imaging unit 43 as the observation device 1B shown in FIG. A02, but is different in that an analysis unit 70 is provided instead of the analysis unit 50. The observation device 1I shown in FIG. C03 has the same optical system from the light source 11 to the imaging unit 43 as the observation device 1C shown in FIG. A03, but is different in that an analysis unit 70 is provided instead of the analysis unit 50.

[0220] The analyzing unit 70 is electrically connected to the imaging unit 43, and inputs the interference intensity image captured by the imaging unit 43. The analyzing unit 70 calculates the three-dimensional refractive index distribution of the observation object S by processing the input interference intensity image. The analyzing unit 70 may also be a computer. The analyzing unit 70 includes an interference intensity image acquiring unit 71, a first complex amplitude image generating unit 72, a second complex amplitude image generating unit 73, a two-dimensional phase image generating unit 74, a three-dimensional phase image generating unit 75, a refractive index distribution calculating unit 76, a third complex amplitude image generating unit 77, a display unit 78, and a storage unit 79.

[0221] The interference intensity image acquisition unit 71 irradiates the observation object S along each of a plurality of light irradiation directions by changing the orientation of the reflection surface of the reflector 22. In addition, the interference intensity image acquisition unit 71 acquires the interference intensity image of the reference position from the shooting unit 43 for each of the plurality of light irradiation directions. The interference intensity image acquisition unit 71 includes a CPU, and has an output port for outputting a control signal for changing the orientation of the reflection surface of the reflector 22, and also has an input port for inputting the interference intensity image from the shooting unit 43. There is no need to move the objective lens 25 along the optical axis. The reference position is an image plane position that is in a conjugate relationship with the shooting surface of the shooting unit 43.

[0222] The first complex amplitude image generating unit 72, the second complex amplitude image generating unit 73, the two-dimensional phase image generating unit 74, the three-dimensional phase image generating unit 75, the refractive index distribution calculating unit 76 and the third complex amplitude image generating unit 77 perform processing based on the interference intensity image, and include a processing device such as a CPU, a GPU, a DSP or an FPGA. The display unit 78 displays the image to be processed, the image in the process of processing and the image after processing, etc., and includes, for example, a liquid crystal display. The storage unit 79 stores data of various images, and includes a hard disk drive, a flash memory, a RAM and a ROM, etc. The first complex amplitude image generating unit 72, the second complex amplitude image generating unit 73, the two-dimensional phase image generating unit 74, the three-dimensional phase image generating unit 75, the refractive index distribution calculating unit 76, the third complex amplitude image generating unit 77 and the storage unit 79 can also be configured by cloud computing.

[0223] The storage unit 79 also stores a program for causing the interference intensity image acquisition unit 71, the first complex amplitude image generation unit 72, the second complex amplitude image generation unit 73, the two-dimensional phase image generation unit 74, the three-dimensional phase image generation unit 75, the refractive index distribution calculation unit 76, and the third complex amplitude image generation unit 77 to perform each process. The program may be stored in the storage unit 79 when the observation apparatuses 1G to 1I are manufactured or shipped, or the program acquired via a communication line after shipping may be stored in the storage unit 79, or the program stored in a computer-readable storage medium 2 may be stored in the storage unit 79. The storage medium 2 is any medium such as a floppy disk, a CD-ROM, a DVD-ROM, a BD-ROM, or a USB memory.

[0224] Details of the processing performed by the interference intensity image acquisition unit 71, the first complex amplitude image generation unit 72, the second complex amplitude image generation unit 73, the two-dimensional phase image generation unit 74, the three-dimensional phase image generation unit 75, the refractive index distribution calculation unit 76, and the third complex amplitude image generation unit 77 will be described later.

[0225] Figures C04 and C05 are flowcharts of a refractive index distribution measurement method C. Figure C05 shows a portion of the flowchart of Figure C04. The refractive index distribution measurement method C can be implemented when using any of the observation devices 1G to 1I. The refractive index distribution measurement method C includes: an interference intensity image acquisition step S71, a first complex amplitude image generation step S72, a second complex amplitude image generation step S73, a two-dimensional phase image generation step S74, a three-dimensional phase image generation step S75, a refractive index distribution calculation step S76, and a third complex amplitude image generation step S77.

[0226] The process of the interference intensity image acquisition step S71 is performed by the interference intensity image acquisition unit 71. The process of the first complex amplitude image generation step S72 is performed by the first complex amplitude image generation unit 72. The process of the second complex amplitude image generation step S73 is performed by the second complex amplitude image generation unit 73. The process of the two-dimensional phase image generation step S74 is performed by the two-dimensional phase image generation unit 74. The process of the three-dimensional phase image generation step S75 is performed by the three-dimensional phase image generation unit 75. The process of the refractive index distribution calculation step S76 is performed by the refractive index distribution calculation unit 76. The process of the third complex amplitude image generation step S77 is performed by the third complex amplitude image generation unit 77.

[0227] In the interference intensity image acquisition step S71, the interference intensity image acquisition unit 71 irradiates the observation object S along each of the plurality of light irradiation directions by changing the orientation of the reflection surface of the reflection mirror 22. Furthermore, the interference intensity image acquisition unit 71 acquires the interference intensity image of the reference position from the imaging unit 43 for each of the plurality of light irradiation directions.

[0228] In the first complex amplitude image generation step S72, the first complex amplitude image generation unit 72 generates a complex amplitude image for each of the plurality of light irradiation directions based on the interference intensity image acquired by the interference intensity image acquisition unit 71. In the case of the observation device 1G (FIG. C01) and the observation device 1I (FIG. C03), the first complex amplitude image generation unit 72 can generate a complex amplitude image based on one interference intensity image by Fourier fringe analysis. In the case of the observation device 1H (FIG. C02), the first complex amplitude image generation unit 72 can generate a complex amplitude image based on three or more interference intensity images having different optical path length differences (phase differences) between the object light and the reference light by a phase shift method. The complex amplitude image generated in the first complex amplitude image generation step S72 can also be a complex amplitude image of the same reference position as the interference intensity image, or a complex amplitude image of another position generated based on the complex amplitude image of the reference position.

[0229] In the second complex amplitude image generation step S73, the second complex amplitude image generation unit 73 generates, for each of the multiple light irradiation directions, a complex amplitude image of each of the multiple z-direction positions between the first position and the second position based on the complex amplitude image of the first position with respect to the distance from the imaging unit 43 along the light propagation path.

[0230] In the two-dimensional phase image generation step S74, the two-dimensional phase image generation unit 74 generates a two-dimensional phase image for each of the plurality of positions based on the complex amplitude images in the plurality of light irradiation directions generated by the second complex amplitude image generation unit 73. The two-dimensional phase image generated here corresponds to a phase image centered on the focused z-direction position.

[0231] In the three-dimensional phase image generation step S75, the three-dimensional phase image generation unit 75 generates a three-dimensional phase image from the first position to the second position based on the two-dimensional phase images of the plurality of positions generated by the two-dimensional phase image generation unit 74. The three-dimensional phase image generated here is an image having positions x and y in the two-dimensional phase image and position z of the two-dimensional phase image as variables.

[0232] In the refractive index distribution calculation step S76 , the refractive index distribution calculation unit 76 obtains the three-dimensional refractive index distribution of the observation object between the first position and the second position by deconvolution based on the three-dimensional phase image generated by the three-dimensional phase image generation unit 75 .

[0233] The interference intensity image acquisition step S71, the first complex amplitude image generation step S72, the second complex amplitude image generation step S73, the two-dimensional phase image generation step S74, the three-dimensional phase image generation step S75 and the refractive index distribution calculation step S76 in the refractive index distribution measurement method C respectively perform processing that is substantially the same as the interference intensity image acquisition step S1, the first complex amplitude image generation step S2, the second complex amplitude image generation step S3, the two-dimensional phase image generation step S4, the three-dimensional phase image generation step S5 and the refractive index distribution calculation step S6 in the refractive index distribution measurement method A.

[0234] In the third complex amplitude image generating step S77, the third complex amplitude image generating unit 77 generates a complex amplitude image at the second position for each of the plurality of light irradiation directions based on the complex amplitude image at the first position used in the second complex amplitude image generating step S73 and the three-dimensional refractive index distribution of the observation object between the first position and the second position calculated in the refractive index distribution calculating step S76.

[0235] In step S83 including the second complex amplitude image generation step S73, the two-dimensional phase image generation step S74, the three-dimensional phase image generation step S75, and the refractive index distribution calculation step S76, the three-dimensional refractive index distribution of the observation object between the first position and the second position is obtained based on the complex amplitude image of the first position with respect to the distance from the imaging unit 43 along the light propagation path. The processing of step S83 and the third complex amplitude image generation step S77 is repeated. This is explained using FIGS. C04 to C07.

[0236] Figure C06 is a diagram for explaining the relationship between the region containing the observation object and the 1st to Jth blocks. As shown in the figure, the region containing the observation object is divided into the 1st to Jth blocks based on the distance from the imaging unit along the light propagation path (z direction). In the figure, J=3. The jth block among the 1st to Jth blocks is the block from z=z j-1 to z = z jIn each j-th block, z=z j-1 The position (near end) is set as the first position, and the z farthest from the imaging unit is set as z j The position (distal end) is set as the second position.

[0237] FIG. C07 is a diagram for explaining the order of processing of the 1st to Jth blocks. As shown in the figure, for each jth block, in step S83, based on the complex amplitude image of the first position, complex amplitude images and two-dimensional phase images of multiple z-direction positions between the first position and the second position are generated, and a three-dimensional phase image between the first position and the second position is generated, and a three-dimensional refractive index distribution is further obtained. For each jth block, in the third complex amplitude image generation step S77, based on the complex amplitude image of the first position and the three-dimensional refractive index distribution calculated in step S83, a complex amplitude image of the second position is generated.

[0238] The complex amplitude image at the second position of the (j+1)th block generated in the third complex amplitude image generation step S77 is used as the complex amplitude image at the first position of the next jth block, and the jth block is subjected to each of the processes of step S83 and the third complex amplitude image generation step S77. If the three-dimensional refractive index distribution is obtained for each of the 1st to Jth blocks, the three-dimensional refractive index distribution of the entire observation object is obtained by combining them. In addition, the three-dimensional refractive index distribution of each of the 1st to Jth blocks (for example, the refractive index distribution of the 1st block, the refractive index distribution of the 2nd block, and the refractive index distribution of the 3rd block in Figure C07) can become refractive index tomographic data.

[0239] As shown in FIG. C04 and FIG. C05, in step S81 after the first complex amplitude image generation step S72, j=0 is set, and in the next step S82, the value of j is increased by 1 and set to j=1, and each process of step S83 and the third complex amplitude image generation step S77 is performed on the first block. That is, for the first block closest to the imaging unit, based on the complex amplitude image generated in the first complex amplitude image generation step S72, the position z=z0 closest to the imaging unit (near end) is set as the first position, and the position z=z1 farthest from the imaging unit (far end) is set as the second position, and the respective processes of step S83 (second complex amplitude image generation step S73, two-dimensional phase image generation step S74, three-dimensional phase image generation step S75, refractive index distribution calculation step S76) and the third complex amplitude image generation step S77 are performed in sequence. Thereafter, the process returns to step S82.

[0240] For the j-th block (here, j is greater than or equal to 2 and less than J), based on the complex amplitude image generated for the (j-1)-th block in the third complex amplitude image generation step S77, z=z closest to the imaging unit is converted to j-1 The position (near end) is set as the first position, and the z farthest from the imaging unit is set as z jThe position (far end) is set as the second position, and the processing of step S83 (second complex amplitude image generation step S73, two-dimensional phase image generation step S74, three-dimensional phase image generation step S75, refractive index distribution calculation step S76) and third complex amplitude image generation step S77 are performed in sequence. After that, the process returns to step S82.

[0241] For the last segment block farthest from the imaging unit, i.e., the Jth block, based on the complex amplitude image generated for the (J-1)th block in the third complex amplitude image generating step S77, z=z J-1 The position (near end) is set as the first position, and the z farthest from the imaging unit is set as z J The position (far end) is set as the second position, and the processing of step S83 (second complex amplitude image generating step S73, two-dimensional phase image generating step S74, three-dimensional phase image generating step S75, refractive index distribution calculating step S76) is performed.

[0242] For the Jth block, in step S84 after step S83, it is determined to be the final segment block, and the process ends without entering the third complex amplitude image generation step S77. In addition, for the Jth block, after the three-dimensional phase image generation step S75, it is determined to be the final segment block, and the process ends without entering the refractive index distribution calculation step S76. In this case, a three-dimensional phase image of the entire observation object is obtained.

[0243] Furthermore, the region including the observation object may be divided into two blocks in sequence based on the distance from the imaging unit along the light propagation path (z direction), in which case the above-mentioned processing for the first block and the processing for the Jth block in the final stage may be performed. In addition, the region including the observation object may not be divided into a plurality of blocks, in which case the processing of each of step S83 (second complex amplitude image generation step S73, two-dimensional phase image generation step S74, three-dimensional phase image generation step S75, refractive index distribution calculation step S76) and third complex amplitude image generation step S77 may be performed in sequence only once.

[0244] Next, the third complex amplitude image generation step S77 will be described in detail. When the observation object is irradiated with light and the interference intensity image is acquired, in each j-th block, the second position (z=z j ) propagates inside the jth block and reaches the first position (z = z j-1 ), and further propagates to the imaging unit. Therefore, in the third complex amplitude image generation step S77, by numerical calculation taking into account the refractive index distribution of the j-th block, the first position (z=z j-1 ) propagates backward inside the jth block, and the second position (z = z jThat is, in the third complex amplitude image generation step S77, for each of the plurality of light irradiation directions, based on the first position (z=z j-1 ) and the refractive index distribution of the jth block, and generate the second position (z = z j ). When performing this processing, a method of numerically calculating the propagation of the light wavefront in consideration of the refractive index distribution of the medium is used. As such numerical calculation methods for propagation in inhomogeneous media, BPM (Beam Propagation Method) and SSNP (Split-Step Non-Paraxial) are known. The following describes the processing using BPM in the third complex amplitude image generation step S77.

[0245] Figure C08 is a diagram for explaining the processing content of BPM. This figure shows an arbitrary j-th block. As shown in this figure, the j-th block is divided into M (7 in this figure) slices (1st to Mth slices) based on the distance from the imaging unit along the light propagation path (z direction). The thickness of each slice is about the wavelength.

[0246] The thickness of each slice may also be constant. Here, the thickness of each slice is set to a constant value Δz. The mth slice among the 1st to Mth slices of the jth block is from position (z j-1 +(m-1)Δz) to position (z j-1 +mΔz). From the first position of the jth block (z=z j-1 ) towards the second position (z=z j ) sequentially imparts phase changes corresponding to the refractive index distribution in the 1st to Mth slices, causing the light wavefront to propagate in the reverse direction by Δz.

[0247] Furthermore, the thickness Δz of each slice in the third complex amplitude image generating step S77 may be different from or identical to the position interval when generating complex amplitude images at a plurality of z-direction positions between the first position and the second position in the second complex amplitude image generating step S73 .

[0248] The phase change o(x, y, z) imparted to the light wavefront when the slice of thickness Δz at position z propagates in the reverse direction is expressed by the following equation (22). v is the wave number of light in a vacuum. δn(x, y, z) is the refractive index distribution n(x, y, z) of the object under observation at position z and the refractive index n of the background (medium) b The difference of is expressed by the following formula (23). In addition, cosθ is expressed by the following formula (24).

[0249]

[0250] δn(x,y,z)=n(x,y,z)-n b (twenty three)

[0251]

[0252] If the position of the mth slice (z = z j-1 +(m-1)Δz) is assumed to be u(x, y, z), then the complex amplitude of the light at the position (z+Δz) after the light propagates backward inside the mth slice is u(x, y, z+Δz) as expressed by the following equation (25). x , k y ; △z) is expressed by the following equation (26). Equation (25) expresses that the product of the complex amplitude u(x, y, z) of the light and the phase change o(x, y, z) is Fourier transformed, and the Fourier transform result and P(k x , k y ; △z) is inversely Fourier transformed to obtain the complex amplitude u(x, y, z+Δz) of the light at position (z+Δz) after propagation in the slice of thickness △z. △z It is a function that performs calculation on light propagation of △z.

[0253]

[0254] The propagation of the light wavefront in each slice of the j-th block is expressed by the following equations (27) to (29). That is, if the first position (z = z j-1 ) is assumed to have a complex amplitude u(x, y, z j-1 ), then the complex amplitude u(x, y, z) of the light after propagation in the first slice of the jth block is j-1 +Δz) is expressed by the following equation (27). If the complex amplitude of the light after propagating through the (m-1)th slice of the jth block is u(x, y, z j-1 +(m-1)Δz), then the complex amplitude u(x, y, z) of the light propagating in the mth slice of the jth block is j-1 +mΔz) is expressed by the following equation (28). If the complex amplitude of the light after propagating through the (M-1)th slice of the jth block is u(x, y, z j-1 +(M-1)Δz), then the second position (z=z j ) is the complex amplitude u(x, y, z j ) is expressed by the following formula (29).

[0255] u(x,y,z j-1 +Δz)=P Δz [o(x,y,z j-1)·u(x,y,z j-1 )] (27)

[0256] u(x,y,z j-1 +mΔz)=P Δz [o(x,y,z j-1 +(m-1)Δz)·u(x,y,z j-1 +(m-1)Δz)] (28)

[0257] u(x,y,z j )=P Δz [o(x,y,z j-1 +(M-1)Δz)·u(x,y,z j-1 +(M-1)Δz)] (29)

[0258] Thus, in the third complex amplitude image generation step S77, by numerical calculation taking into account the refractive index distribution of the j-th block, the first position (z=z j-1 ) propagates backwards for each slice in the jth block, and the second position (z = z j ) of the light wave surface.

[0259] FIG. C09 is a flowchart of the third complex amplitude image generation step S77. In step S41, the position z is initialized to the first position of the jth block (z=z j-1 ). In step S42, the interaction between the complex amplitude u(x, y, z) of the light at position z and the phase change o(x, y, z) is calculated. In step S43, the wavefront of the light after the interaction is propagated a distance Δz, and the complex amplitude u(x, y, z+Δz) of the light at position z+Δz is calculated. In step S44, z obtained by adding Δz is set as the new z. In step S44, if it is determined that position z has not yet reached the second position of the jth block (z=z j ), then return to step S42 and repeat steps S42 to S44. In step S44, if it is determined that position z has reached the second position of the jth block (z = z j ), the processing of the third complex amplitude image generation step S77 is terminated. The complex amplitude of the light acquired at the end becomes the second position (z=z j )'s complex amplitude.

[0260] Any of the above-described refractive index distribution measurement methods A to C can realize three-dimensional refractive index tomography with reduced influence of multiple scattered light even when the observation object is a multiple scatterer. Any of the above-described refractive index distribution measurement methods A to C is suitable for measuring the refractive index distribution of a three-dimensional culture body as the observation object.

[0261] In addition, it is also possible to use an observation device and a refractive index distribution measurement method using self-interference. For example, the observation device 1J shown in FIG. C10 includes: a light source 11, a lens 12, a lens 21, a reflector 22, a lens 23, a focusing lens 24, an objective lens 25, a reflector 44, a lens 42, a shooting unit 43, and an analysis unit 70. Compared with the structure of the observation device described so far, the difference is that in the observation device 1J, after the light output from the light source 11 is guided by the optical fiber 14, it is not branched into two parts and is emitted from the light output end 18. In addition, the observation device 1J is different in that a reflector 44 is provided instead of the beam splitter 41. The observation device 1J does not have an interference optical system. The shooting unit 43 can capture an interference intensity image of a reference position formed by self-interference of light irradiated on the observation object S along multiple light irradiation directions and passing through the observation object S. The analysis unit 70 can use such an interference intensity image formed by self-interference to perform the same image processing as described so far.

[0262] In addition, the three-dimensional refractive index distribution of the observation object S between the first position and the second position may not be the refractive index distribution based on the three-dimensional phase image, but may be obtained using a refractive index distribution acquisition device capable of acquiring the refractive index distribution. In this case, the observation device may include the following functional units: (1) an interference intensity image acquisition unit that acquires interference intensity images of reference positions of the plurality of light irradiation directions from a photographing unit that photographs interference intensity images of reference positions of light irradiated to the observation object and passing through the observation object in each of the plurality of light irradiation directions; (2) a first complex amplitude image generation unit that generates a complex amplitude image based on the interference intensity image for each of the plurality of light irradiation directions; (3) a refractive index distribution acquisition unit that acquires the three-dimensional refractive index distribution of the observation object between the first position and the second position with respect to the distance from the photographing unit along the light propagation path; (4) a second complex amplitude image generation unit (equivalent to the third complex amplitude image generation unit of the observation devices 1A to 1D) that generates a complex amplitude image of the second position for each of the plurality of light irradiation directions based on the complex amplitude image of the first position and the three-dimensional refractive index distribution.

[0263] In addition, in this case, the refractive index distribution measuring method only needs to include the following steps: (1) an interference intensity image acquiring step, acquiring interference intensity images of reference positions of each of the multiple light irradiation directions from a capturing unit that captures interference intensity images of reference positions of light irradiated on the observation object and passing through the observation object along each of the multiple light irradiation directions; (2) a first complex amplitude image generating step, generating a complex amplitude image based on the interference intensity image for each of the multiple light irradiation directions; (3) a refractive index distribution acquiring step, acquiring a three-dimensional refractive index distribution of the observation object between a first position and a second position with respect to the distance from the capturing unit along the light propagation path; and (4) a second complex amplitude image generating step, generating a complex amplitude image of the second position based on the complex amplitude image of the first position and the three-dimensional refractive index distribution for each of the multiple light irradiation directions.

[0264] The first aspect of the present disclosure is a method for determining a bile ductule region contained in an observation object using refractive index distribution data of an observation object containing hepatocytes. In one embodiment, the determination of the bile ductule region is performed based on the characteristic that a region in the observation object has bile ductules. That is, in one embodiment, a region having bile ductule characteristics as a region in the observation object is determined as the bile ductule region. Furthermore, in one embodiment, the determination of the bile ductule region can also be performed by using a machine learning model that utilizes a feature quantity corresponding to the characteristic that a region in the observation object has bile ductules.

[0265] The characteristics of the bile ductules of one embodiment include: one or more characteristics selected from the characteristics that the region is a roughly circular or roughly tubular region existing between hepatocytes, the characteristics that the refractive index of the region is lower than the refractive index of hepatocytes, and the characteristics that the refractive index of the peripheral portion of the region is higher than the refractive index of hepatocytes. The characteristics of the bile ductules of one embodiment include: the characteristics that the region is a roughly circular or roughly tubular region existing between hepatocytes, the characteristics that the refractive index of the region is lower than the refractive index of hepatocytes, and the characteristics that the refractive index of the peripheral portion of the region is higher than the refractive index of hepatocytes. The refractive index of a region refers to a statistical value such as the average value, central value, minimum value or maximum value of the refractive index of the region. In addition, the refractive index of hepatocytes refers to a statistical value such as the average value, central value, minimum value or maximum value of the refractive index of the whole or a part of the hepatocytes. The roughly circular region represents an area with a roundness above a threshold value. The above threshold value can be determined based on the type and state of the observation object and the type and content of hepatocytes contained in the observation object, or can be determined in a manner that an area having characteristics of other bile ducts in the refractive index distribution data of an observation object (reference) different from the observation object (sample) for determining the bile duct is determined as the bile duct area. For example, in the case where the sample is a hepatocyte mass, other hepatocyte masses can be used as a reference, or a two-dimensional liver culture can be used. The above threshold value can be, for example, 65%, 70%, 75%, 80%, or 85%. The roughly tubular area is a tubular area, and is an area in which the cross section perpendicular to the tubular flow axis of the area or the cross section observed on the tomographic plane in the refractive index tomography data is roughly circular. The tubular flow axis refers to an axis formed by the axis formed by the collection of cross-sectional centers (centers of gravity) of the tubes. The peripheral portion of a region refers to the concept of a portion adjacent to the region (adjacent portion) and a portion at a predetermined distance from the adjacent portion, which includes the portion of the observation object not included in the region. It is equivalent to the portion near the lumen of the portion occupied by hepatocytes that form the wall of the capillary bile duct by close adhesion.

[0266] In one embodiment, the characteristics of the bile ductules further include: having a region that is derived from microvilli and has a refractive index higher than the refractive index of the region as a whole. Generally speaking, the region derived from microvilli is roughly linear or curved (bent, spiral, etc.) when the refractometer distribution data is three-dimensional, and roughly linear, curved, or dotted when the refractometer distribution data is two-dimensional such as refractive index tomography data. When the region derived from microvilli is roughly linear or curved, its length is, for example, about 1 to 2 μm, and may be longer or shorter depending on the type and state of the observed object and the type and content of hepatocytes contained in the observed object.

[0267] The second aspect of the present disclosure is a method for evaluating a bile duct region based on parameters of bile ducts obtained from refractive index distribution data of an observation object containing hepatocytes. In one embodiment, the bile duct region is evaluated based on the parameters of the bile ducts. That is, in one embodiment, a bile duct region that is a bile duct region in the observation object and has a better value of the bile duct parameter is evaluated as a high-quality bile duct. In addition, in one embodiment, the bile duct region can also be evaluated by using a machine learning model that uses a feature value corresponding to the parameter of the bile duct to learn the bile duct region with a better value as a higher-quality bile duct.

[0268] In one embodiment, the parameters of the bile ductules include: one or more parameters selected from the area or perimeter of the cross section perpendicular to the axis of the tube flow or the cross section observed on the tomographic plane, the refractive index, the central value of the refractive index, the number of microvilli, and the average length of the microvilli. In one embodiment, the parameters of the bile ductules include: one or more parameters selected from the area of ​​the cross section perpendicular to the axis of the tube flow or the cross section observed on the tomographic plane, the refractive index, the number of microvilli, and the average length of the microvilli. In one embodiment, regarding the area of ​​the cross section, it is considered that the thicker the bile ductules are, the more mature the bile ductules are, so the bile ductules region with a large cross-sectional area can also be judged as a high-quality bile ductule. The refractive index is the refractive index of the bile ductule region. In one embodiment, it can be considered that the more space occupied by the microvilli and the amount of contents, the higher the refractive index, so the bile ductules region with a high refractive index can also be evaluated as a high-quality bile ductule. In one embodiment, regarding the number and length of microvilli, it is believed that the larger the surface area of ​​the microvilli, the greater the secretion of bile acid into the bile ductules and the excretion of liver metabolites. Therefore, a bile ductule region with a larger number of microvilli can be judged as a high-quality bile ductule, and a bile ductule region with a longer average length of microvilli can be judged as a high-quality bile ductule.

[0269] In one embodiment, the method for evaluating the bile duct region may also include, before the evaluation, the step of determining the bile duct region contained in the observation object using the refractive index distribution data of the observation object. That is, the method for evaluating the bile duct region of one embodiment includes: the step of determining the bile duct region contained in the observation object using the refractive index distribution data of the observation object; and the step of evaluating the bile duct region based on the parameters of the bile duct in the determined bile duct region. In one embodiment, the method for evaluating the bile duct region includes: the step of determining the bile duct region contained in the observation object using the refractive index distribution data of the observation object, wherein the determination of the bile duct region is based on the fact that the region in the observation object has the characteristics of the bile duct, and the characteristics of the bile duct include the characteristics of the roughly circular or roughly tubular region existing between the hepatocytes, the characteristics of the refractive index of the region being lower than the refractive index of the hepatocytes, and the characteristics of the refractive index of the peripheral portion of the region being higher than the refractive index of the hepatocytes; and the bile duct region is evaluated based on the parameters of the bile duct in the determined bile duct region.

[0270] The third aspect of the present disclosure is a method for evaluating an observation object containing hepatocytes based on the number of bile duct regions determined by the method of the first aspect of the present disclosure (a method for determining the bile duct region contained in the observation object using the refractive index distribution data of the observation object containing hepatocytes) and / or the evaluation of the bile duct region obtained by the method of the second aspect of the present disclosure (a method for evaluating the bile duct region based on the parameters of the bile duct obtained from the refractive index distribution data of the observation object containing hepatocytes). The quality of the observation object containing hepatocytes is usually evaluated using the number of bile ducts as an indicator, and it goes without saying that the higher the quality of the bile ducts contained in the observation object containing hepatocytes, the higher the quality of the bile ducts itself. Therefore, the observation object containing hepatocytes itself can be evaluated by the method for determining the bile duct region and the method for evaluating the bile duct region. For example, in Fig.35 In the example shown, the hepatocyte mass A having a large number of bile duct regions can be evaluated as a hepatocyte mass of high quality.

[0271] The evaluation method of an observation object including hepatocytes according to one embodiment can be used, for example, as follows.

[0272] For example, it is known that the formation of bile ducts can be used as one of the criteria for differentiation of an observation object containing hepatocytes, and therefore, by evaluating the number of bile duct regions contained in the observation object containing hepatocytes, the differentiation of the observation object containing hepatocytes can be evaluated. That is, another aspect is a method for evaluating the differentiation of the observation object containing hepatocytes using the refractive index distribution data of the observation object containing hepatocytes.

[0273] For example, the evaluation method of an observation object containing hepatocytes according to one embodiment can be used to evaluate the culture method of hepatocytes. As a culture method of hepatocytes, in addition to the usual planar culture, various culture methods such as sandwich culture method, three-dimensional spheroid culture method, hollow fiber bioreactor method, micro-pattern co-culture method, perfusion multi-well plate culture method, microfluidic liver biochip culture method and microfluidic multi-organ device culture method have been developed. By evaluating the number of bile duct capillary regions contained in the observation object containing hepatocytes cultured by these methods, the culture method of hepatocytes can be evaluated. That is, the fourth aspect of the present disclosure is a method for evaluating a method for culturing hepatocytes, comprising: a step of culturing hepatocytes; and a step of evaluating an observation object obtained by culturing using the method of the third aspect of the present disclosure (a method for evaluating an observation object containing hepatocytes based on the number of bile duct regions determined by the method of the first aspect of the present disclosure (a method for determining the bile duct region contained in the observation object using refractive index distribution data), and / or an evaluation of the bile duct region obtained by the method of the second aspect of the present disclosure (a method for evaluating the bile duct region by using parameters of the bile duct obtained from refractive index distribution data)). The evaluation of the method for culturing hepatocytes takes advantage of the ability to non-invasively determine and evaluate the bile duct region, for example Fig.36 As shown, it is also possible to perform the analysis based on the number of bile capillary regions corresponding to the number of days the hepatocyte aggregates are cultured.

[0274] For example, an evaluation method for an observation object containing hepatocytes in one embodiment can be used for drug screening. In addition to being the secretion destination of bile acids synthesized in hepatocytes, bile ductules are also the excretion pathway of liver metabolism as the main excretion method of drugs. In addition, drug-induced liver injury (DILI) is one of the main reasons for the removal of approved drugs and restrictions on use, as well as the inability to approve new drugs for safety. It is known that there are hepatocellular injury type, cholestatic type and mixed type, etc., especially in cholestatic DILI, morphological abnormalities of bile ductules are observed. Therefore, by screening using an evaluation method for an observation object containing hepatocytes in one embodiment, for example, drugs that cause an abnormal increase in the refractive index of the bile ductule region can be used as drugs that can cause abnormal secretion or abnormal excretion of bile acids or liver metabolites, and drugs that cause abnormal morphology of bile ductules can be excluded from development candidates as drugs that can cause cholestatic DILI. If these phenomena that have been clarified in the past at the clinical stage can be clarified in vitro, it is possible to prevent the failure of clinical trials due to the huge cost and time investment caused by the subsequent withdrawal of approved drugs from the market or use restrictions and the failure to obtain approval for new drugs due to safety. That is, the fifth aspect of the present disclosure is a method for screening drugs, which includes: the step of adding drugs to an observation object containing hepatocytes; and the step of evaluating the observation object obtained by culture using the method of the third aspect of the present disclosure (a method of evaluating the number of capillary bile duct areas determined by the method of the first aspect of the present disclosure (a method of determining the capillary bile duct area contained in the observation object using refractive index distribution data) and / or (an evaluation of the capillary bile duct area obtained by the method of the second aspect of the present disclosure (a method of evaluating the capillary bile duct area by parameters of the capillary bile duct obtained from refractive index distribution data)). Drug screening can be performed, for example, using morphological abnormalities of the capillary bile duct as an indicator. Fig.37 In the example shown, based on the cross-sectional area of ​​the bile canaliculi region, drug A that causes dilation of bile canaliculi in hepatocyte masses and drug B that causes contraction can be excluded from the development candidates as drugs that may cause cholestatic DILI. Alternatively, for example, morphological abnormalities of bile canaliculi focusing on microvilli can be used as an indicator. Fig.38 In the example shown, drug A that causes shortening of microvilli, drug B that causes reduction of microvilli, and drug C that causes shortening and reduction of microvilli can be excluded from the development candidates as drugs that may cause cholestatic DILI. These screening methods can be used to non-invasively identify and evaluate the capillary bile duct area and can be performed based on changes over time at multiple time points. Fig.39In the example shown, a drug that causes shortening of microvilli corresponding to the number of culture days after drug addition can be excluded from development candidates as a drug that may cause cholestatic DILI.

[0275] Furthermore, the method of specifying the bile duct region according to one embodiment can be used, for example, in the following manner.

[0276] For example, the method for determining the bile duct area of ​​one embodiment can be used to evaluate the formation of the bile duct network. It is known that the bile duct forms a three-dimensional network after formation and is responsible for the secretion or excretion of liver metabolites. By using the method for determining the bile duct area of ​​one embodiment to comprehensively determine the bile duct area contained in the observation object containing hepatocytes, the formation of the bile duct network can be evaluated. In addition, in one embodiment, as Fig.40 As shown in the example, the formation of the bile duct network induced by adding bile acids that promote network formation such as taurocholic acid can also be evaluated. In addition, in one embodiment, the formation of the bile duct network caused by adding bile acids that hinder network formation such as lithocholic acid can also be evaluated. These evaluations can be used to determine the advantages of the bile duct region non-invasively, and can also be carried out based on the time changes at multiple time points, and the volume of the bile duct region can also be used as an evaluation index.

[0277] For example, the method of determining the bile duct region of one embodiment can also be used as a guide for recovering contents from the bile duct. The bile duct is one of the main pathways of liver metabolism. If the bile duct region in an observation object containing hepatocytes can be determined non-invasively, for example, Fig.41 As in the example shown, a drain or catheter can be inserted into a specific bile capillary to recover the contents. The recovered contents can be analyzed by mass spectrometry, liquid chromatography, or gas chromatography, for example.

[0278] In addition, the method of evaluating the bile ductule area according to one embodiment can also be used as a method for diagnosing liver diseases in clinical specimens of the liver, a method for assisting in the diagnosis of liver diseases, or a method for collecting data for the diagnosis of liver diseases. It is obvious that abnormalities in the bile ductules may occur in a liver that has developed liver disease. For example, in a liver with Byler's disease, a lesion called "Byler's Bile" in which the microvilli in the bile ductules become extremely few has been confirmed. In addition, in cholestasis due to liver disease, enlargement of the bile ductules, reduction of microvilli, and formation of blebs have been confirmed. These insights are insights that have become clear through electron microscopy. According to one embodiment, a non-invasive method for evaluating the bile ductule area using refractive index distribution data can easily detect abnormalities in the bile ductules in clinical specimens of the liver without antagonism with other pathological stains such as HE staining. If abnormalities in the bile ductules can be detected based on refractive index distribution data, it is expected that more clinical specimens can be processed more quickly. For example, in Fig.42 In the example shown, comparison of the bile duct regions in clinical specimens A to C can assist in the diagnosis that clinical specimen B with short microvilli and clinical specimen C with short and few microvilli are clinical specimens with a high probability of having liver disease.

[0279] Hereinafter, the present disclosure will be described in detail by way of examples and the like, but the present disclosure is not limited thereto.

[0280] Example

[0281] [Example 1: Determination of bile duct region in two-dimensional liver culture using phase contrast and fluorescence microscopy]

[0282] Human hepatoma-derived cells (HepG2) were seeded in a glass-bottomed dish (MatTek), and cultured for 5 days in a DMEM medium containing 10% fetal bovine serum under the conditions of 5% CO2, 37 degrees, and 100% humidity to prepare a two-dimensional liver culture. After removing the medium, a Hanks' balanced salt solution (final concentration 2 μM) containing calcium chloride and magnesium chloride, which is a fluorescently labeled bile acid known to be secreted into the bile capillaries, CLF (cholyl-lysyl-fluorescein, Corning), was added, and cultured for 45 minutes under the conditions of 5% CO2, 37°C, and 100% humidity.

[0283] Phase contrast microscope images and fluorescence images were taken in the same field of view using an inverted microscope DM IL LED (manufactured by Leica). A 20x objective lens was used. Fluorescence images were obtained under the conditions of an excitation wavelength of 470 nm and a fluorescence wavelength of 500 to 550 nm. The results are shown in Fig.43 .

[0284] Phase contrast microscopy images confirmed the presence of luminal structures between cells (arrows 1 and 2 in the figure), and fluorescence images confirmed the accumulation of CLF in the luminal structures. Therefore, the areas indicated by arrows 1 and 2 in the figure were identified as bile duct capillaries.

[0285] [Example 2: Evaluation of bile duct region in two-dimensional liver culture using optical diffraction tomography (ODT)]

[0286] For the two-dimensional liver culture used in Example 1, refractive index distribution data was obtained using the observation apparatus 1A and the ODT described in the refractive index distribution measurement method A2. An objective lens with a magnification of 60 times was used. Representative refractive index tomographic data extracted from the obtained refractive index distribution data are shown in FIG. Fig.44 and Fig.45 .

[0287] exist Fig.44 In the region identified as the bile duct region in Example 1 (arrows 1 and 2 in the figure), the following characteristics were confirmed: a roughly circular region (circularity of 86%; the numerical values ​​in parentheses in the paragraphs below represent the values ​​of the bile duct 2), a refractive index (1.336) lower than that of the hepatocytes (1.344), and a refractive index (1.359) of the peripheral portion higher than that of the hepatocytes (1.344). In addition, Fig.45 In the region identified as the bile duct region in Example 1 (arrows 1 and 2 in the figure), the following characteristics were also confirmed, namely, a region derived from microvilli (MV), that is, a region with a refractive index higher than the refractive index (1.333) of the lumen of the bile duct (BC) (1.344). Fig.45 In the figure, G represents the centroid of the bile capillary region.

[0288] [Example 3: Evaluation of the bile duct region of hepatocyte clusters using optical diffraction tomography (ODT)]

[0289] Human hepatoma-derived cells (HepG2) were cultured at 1.00×10 5 The cells were seeded at a density in a cell culture container with a partition having low cell adhesion (AGC Technoglass, "EZSPHERE"), and cultured for 3 days in a DMEM medium containing 10% fetal bovine serum under the conditions of 5% CO2, 37°C, and 100% humidity. Dimethyl sulfoxide (Sigma Aldrich) was added to the medium to a final concentration of 0.05%, and the cells were cultured for another day to prepare hepatocyte aggregates.

[0290] For the obtained hepatocyte mass (hepatocyte mass 1), refractive index distribution data was obtained using the observation apparatus 1A and the ODT shown in the refractive index distribution measurement method C. An objective lens with a magnification of 60 times was used. Representative refractive index tomographic data extracted from the obtained refractive index distribution data are shown in FIG. Fig.46 and Fig.47 The refractive index distribution data obtained in Example 2 and the parameters obtained from the refractive index distribution data obtained for the hepatocyte mass 1 in Example 3 are shown in Tables 1 to 4. In addition, for another hepatocyte mass (hepatocyte mass 2) prepared in the same manner, representative refractive index tomographic data extracted from the refractive index distribution data obtained in the same manner are shown in Tables 1 to 4. Fig.48 .

[0291] [Table 1]

[0292]

[0293] [Table 2]

[0294]

[0295] [Table 3]

[0296]

[0297] [Table 4]

[0298]

[0299] exist Fig.46 In the figure, it is confirmed that structures (capillaries) having the same characteristics as those observed in the two-dimensional liver culture body also exist in the hepatocyte mass. It is shown that if an arbitrary cross section is observed, for example, 5 capillaries are confirmed at Z = 20.8 μm, 1 capillary is confirmed at Z = 28.8 μm, and 2 capillaries are confirmed at Z = 66.8 μm, so the capillary region can be determined in the hepatocyte mass using the refractive index distribution data.

[0300] exist Fig.47 In FIG. 1 , it is shown that, in addition to the existence of bile ducts, the size of the lumen, the number of microvilli, and the refractive index can be confirmed to be different in each bile duct, and therefore, the bile duct region can be evaluated using the refractive index distribution data in the hepatocyte mass. Furthermore, it is suggested that, for example, in the bile duct 7, the intraluminal structure X is confirmed inside the bile duct region, and therefore, there is the possibility that the existence of features and parameters useful for determining or evaluating the bile duct region, evaluating the hepatocyte mass, or drug screening can be discovered by using the refractive index distribution data in addition to the features and parameters used in the embodiment.

[0301] exist Fig.48In the figure, as an evaluation example of the bile duct area, for example, if the parameters of two bile duct areas in the refractive index tomography data are compared, the bile duct 1 in the figure has a larger cross-sectional area and a larger cross-sectional perimeter, a larger number of microvilli, and a larger refractive index and a larger central value of the refractive index than the bile duct 2. Therefore, it can be judged that the bile duct 1 is a high-quality bile duct.

[0302] Description of Reference Numerals

[0303] 1A to 1J…Observation device; 2…Storage medium; 11…Light source; 12…Lens; 13…Light incident end; 14…Optical fiber; 15…Fiber coupler; 16, 17…Optical fiber; 18, 19…Light exit end; 21…Lens; 22…Reflector; 23…Lens; 24…Converging lens; 25…Objective lens; 31…Lens; 32…Reflector; 33…Drive unit; 34…Lens; 41…Beam splitter; 42…Lens; 43…Photography unit; 44…Reflector; 50…Analysis unit; 51…Interference intensity image acquisition unit; 52…First complex amplitude image generation unit; 53…Second complex amplitude image generation unit; 54…Two-dimensional phase image generation unit; 55…Three-dimensional phase image generation unit; 56…Refractive index distribution meter calculation unit; 57…display unit; 58…storage unit; 60…analysis unit; 61…interference intensity image acquisition unit; 62…first complex amplitude image generation unit; 63…second complex amplitude image generation unit; 64…phase conjugation operation unit; 65…two-dimensional phase image generation unit; 66…three-dimensional phase image generation unit; 67…refractive index distribution calculation unit; 68…display unit; 69…storage unit; 70…analysis unit; 71…interference intensity image acquisition unit; 72…first complex amplitude image generation unit; 73…second complex amplitude image generation unit; 74…two-dimensional phase image generation unit; 75…three-dimensional phase image generation unit; 76…refractive index distribution calculation unit; 77…third complex amplitude image generation unit; 78…display unit; 79…storage unit.

Claims

1. A method, wherein: Using the refractive index distribution data of the observation object including hepatocytes, the bile capillary region included in the observation object is identified.

2. The method according to claim 1, wherein: The determination of the bile duct capillary region is based on the fact that the region in the observed object has characteristics of bile duct capillaries, and the characteristics of the bile duct capillaries include: characteristics of a roughly circular or roughly tubular region existing between hepatocytes; characteristics of a refractive index of the region being lower than the refractive index of hepatocytes; and characteristics of a refractive index of the peripheral portion of the region being higher than the refractive index of hepatocytes.

3. The method according to claim 2, wherein: The bile duct capillaries are further characterized by having a region derived from microvilli and having a refractive index higher than the refractive index of the entire region.

4. The method according to claim 3, wherein: The refractive index distribution data is refractive index slice data in a predetermined direction.

5. A method wherein The method is a method for evaluating the capillary bile duct area based on the parameters of the capillary bile duct obtained from the refractive index distribution data of the observation object containing hepatocytes, and the parameters of the capillary bile duct include: the area of ​​the cross section perpendicular to the tube flow axis or the cross section observed on the tomographic plane, the refractive index, the number of microvilli and at least one parameter of the average length of the microvilli.

6. The method according to any one of claims 1 to 5, wherein: The observation object containing hepatocytes is a hepatocyte mass.

7. The method according to claim 5, wherein: The method comprises the steps of identifying a bile duct region included in an observation object by the method according to any one of claims 1 to 4, and evaluating the bile duct region based on parameters of the bile duct in the identified bile duct region.

8. A method, wherein: The observation object containing hepatocytes is evaluated based on the number of bile duct areas determined by the method of any one of claims 1 to 4 and / or the evaluation of bile duct areas obtained by the method of claim 5.

9. A method for evaluating a method for culturing hepatocytes, wherein: Include: a step of culturing hepatocytes; and A step of evaluating the observation object obtained by culturing using the method according to claim 8.

10. A method for screening a drug, wherein: Include: a step of adding a drug to an observation object including hepatocytes; and A step of evaluating the observation object to which the drug is added using the method according to claim 8.

11. The method according to claim 8, wherein: The hepatocyte-containing observation object is a hepatocyte mass.

12. A device for determining a bile duct area, wherein: The method comprises: a data acquisition unit for acquiring refractive index distribution data of an observation object including hepatocytes; and a specifying unit for specifying a bile duct region included in the observation object by the method according to any one of claims 1 to 4.

13. A device for evaluating a bile duct region, wherein: The method comprises: a data acquisition unit for acquiring refractive index distribution data of an observation object containing hepatocytes; a determination unit for determining a bile duct capillary region contained in the observation object by the method according to any one of claims 1 to 4; and an evaluation unit for evaluating the bile duct capillary region by the method according to claim 5.

14. A program, wherein: The computer is caused to execute the following steps: a data acquisition step of acquiring refractive index distribution data of an observation object including hepatocytes; and a determination step of determining a bile duct region included in the observation object by the method according to any one of claims 1 to 4.

15. A program, wherein: The computer is caused to execute the following steps: a data acquisition step, which acquires the refractive index distribution data of the observation object containing hepatocytes; a determination step, which determines the capillary bile duct area contained in the observation object by the method described in any one of claims 1 to 4; and an evaluation step, which evaluates the capillary bile duct area by the method described in claim 5.

Images