Method for observing biological sample

Abstract

To provide a method for extracting an evaluation target region of a biological sample.SOLUTION: A method for three-dimensionally extracting an evaluation target region of a biological sample based on images in which an image of the biological sample embedded in a wax block captured by an X-ray microscope using X-rays having an energy of 4 to 12 keV and an image of a same measurement location of the same biological sample captured by an optical microscope after the image capturing by the X-ray microscope are matched, comprises the steps of: extracting a region of interest including the evaluation target region from the image from the X-ray microscope that has been matched with the image from the optical microscope; and binarizing the region of interest, and further extracting the evaluation target region from the region of interest by complementarily using image information from the optical microscope and the X-ray microscope.SELECTED DRAWING: Figure 23

Description

Technical Field

[0001] The present invention relates to a method for stereoscopically extracting an evaluation target region of a biological sample from images of the same measurement location of the same biological sample imaged by an X-ray microscope and an optical microscope and precisely matching them, and practically evaluating the region at the cellular level with spatial resolution.

Background Art

[0002] The pathological evaluation of biological tissues mainly focuses on the pathological evaluation in a plane (two-dimensional) cut out from a pathological specimen. However, pathological changes progress three-dimensionally within the tissue. In addition, when preparing a pathological specimen, a part of the specimen is discarded and evaluation becomes impossible. From such a situation, the development of a method for three-dimensionally evaluating a specimen and a method for evaluating all of the specimen has been an important problem to be solved. Conventionally, as methods for observing biological samples, a method of imaging a biological sample using a microscopy method (Patent Document 1), a method of imaging the kidney (renal tubule) of a rat (Patent Document 2), a method of infiltrating a contrast agent into a biological sample and solidifying it for imaging (Patent Document 3), etc. are known.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, at present, there is no practical observation method that can three-dimensionally evaluate the pathological changes of a specimen with spatial resolution at the cellular level. Although optical microscopes have relatively high two-dimensional resolution of about 0.2 μm and the advantage of accurate tissue identification by various staining methods, there are limitations in microscopic observation, such as specimen deformation (destructive deformation) caused by the application of physical force during specimen preparation and insufficient depth resolution caused by a specimen thickness of about 4 μm. On the other hand, although X-ray microscopes have the advantages of isotropic submicron three-dimensional resolution and non-destructiveness that allows specimen reuse, since the grayscale of X-ray microscope images is determined by reflecting the electron density of relatively light elements that make up biological specimens, they provide grayscale images with low contrast, which causes difficulties in tissue identification. Therefore, there has been a demand for the development of a practical method for three-dimensional observation of the same measurement location of the same biological specimen at the cellular level of spatial resolution by combining an X-ray microscope and an optical microscope to complement the above limitations and difficulties.

[0005] As a result of intensive studies to solve the above problems, the present inventor succeeded in three-dimensionally extracting the evaluation target region of the biological specimen from images of the same measurement location of the same biological specimen that were precisely matched and imaged by an X-ray microscope and an optical microscope by binarizing the X-ray microscope image and complementarily using the image information of the optical microscope and the X-ray microscope, and practically evaluating it with spatial resolution at the cellular level, thus completing the present invention.

Means for Solving the Problems

[0006] That is, the present invention is as follows. [1] A method for three-dimensionally extracting the evaluation target region of a biological specimen based on both images in which an image of a wax block-embedded biological specimen imaged by an X-ray microscope using X-rays with an energy of 4 to 12 keV is matched with an image of the same measurement location of the same biological specimen imaged by an optical microscope after the imaging, comprising: a step of extracting a region of interest including the evaluation target region from the X-ray microscope image matched with the optical microscope image; Binarizing the target region of interest, and further extracting an evaluation target region from within the target region of interest by complementarily using the image information of the optical microscope and the X-ray microscope; The method, including the above. [2] A method for evaluating a biological sample, including a step of evaluating the volume or shape of an evaluation target region extracted by the method according to [1]. [3] The method according to [2], wherein the volume evaluation is performed by multiplying the number of voxels in the image of the binarized evaluation target region by the voxel volume. [4] A method for displaying an evaluation target region of a biological sample, including a step of stereoscopically displaying the evaluation target region extracted or evaluated by the method according to any one of [1] to [3]. [5] A microscope image processing apparatus for stereoscopically extracting an evaluation target region of a sample based on both images in which an image of a wax block-embedded biological sample captured by an X-ray microscope using X-rays with an energy of 4 to 12 keV and an image of the same measurement location of the same biological sample captured by an optical microscope after the imaging are matched, means for extracting an observation target region (region of interest) including the evaluation target region from the images of the biological sample captured and matched by the X-ray microscope and the optical microscope, and means for binarizing the extracted region of interest and further extracting the evaluation target region from within the region of interest; The apparatus, comprising the above. [6] The apparatus according to [5], wherein the extraction of the region of interest and the extraction of the evaluation target region are performed by comparing the optical microscope image and the X-ray microscope image using image analysis software having a binarized image creation function and a shape or region selection function. [7] The apparatus according to [5], further including means for evaluating the volume or shape of the extracted evaluation target region. [8] The apparatus according to [7], wherein the volume evaluation is performed by multiplying the number of voxels in the image of the binarized evaluation target region by the voxel volume. [9] The apparatus according to any one of [5] to [8], further comprising means for three-dimensionally displaying the extracted or evaluated region of interest.

[10] A program for three-dimensionally extracting a region of interest of a sample based on two images in which an image of a wax-embedded biological sample imaged by an X-ray microscope using X-rays with an energy of 4 to 12 keV is matched with an image of the same measurement location of the same biological sample imaged by an optical microscope after the imaging, the program causing a computer to extract an observation region (region of interest) including the region of interest from the images of the biological sample imaged and matched by the X-ray microscope and the optical microscope, and binarize the extracted region of interest and further extract the region of interest from within the region of interest. A program for causing it to function as such.

Advantages of the Invention

[0007] According to the present invention, it has become possible to three-dimensionally extract a region of interest of a sample from images of the same measurement location of the same biological sample imaged and precisely matched by an X-ray microscope and an optical microscope, and practically evaluate it with spatial resolution at the cellular level. The present invention provides a method for practically three-dimensionally observing the same measurement location of the same biological sample with spatial resolution at the cellular level by complementary use of an X-ray microscope and an optical microscope.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12A

Figure 12B

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Modes for Carrying Out the Invention

[0009] The present invention relates to a method for three-dimensionally extracting an evaluation target region of a sample based on two images in which an image of a wax block-embedded biological sample imaged by an X-ray microscope using X-rays with an energy of 4 to 12 keV and an image of the same measurement location of the same biological sample imaged by an optical microscope after the imaging are matched. The method of the present invention includes the following steps. A step of extracting a region of interest including the evaluation target region from the X-ray microscope image matched with the optical microscope image; A step of binarizing the region of interest and further extracting the evaluation target region by complementarily using the image information of the optical microscope and the X-ray microscope from within the region of interest.

[0010] The following shows an aspect of extracting a region of interest including the evaluation target region from two images in which an image of a wax block-embedded biological sample imaged by an X-ray microscope using X-rays with an energy of 4 to 12 keV and an image of the same measurement location of the same biological sample imaged by an optical microscope after the imaging are matched, binarizing the region of interest, and further extracting the evaluation target region by complementarily using the image information of the optical microscope and the X-ray microscope from within the region of interest.

[0011] 1. Preparation of a block for X-ray imaging of a biological sample (1) Preparation of a biological sample In order to observe the three-dimensional structure of a biological sample with submicron resolution by an X-ray microscope, in the present invention, a wax contrast method described in WO2022 / 234844 is used with some modifications. The "wax contrast method" described in the above publication is a method of increasing the contrast against X-rays by utilizing the negative contrast effect obtained by infiltrating a wax such as paraffin into a biological sample and replacing the water in the sample with the wax, further embedding this wax-infiltrated biological sample in the same wax to prepare a block, and irradiating this block with X-rays to perform X-ray microscope imaging of the embedded biological sample.

[0012] Since unstained biological samples generally have low contrast against X-rays, it is difficult to observe them at the cellular level with spatial resolution using an X-ray microscope as they are. The reason is that in unstained biological samples, water, which is their main component, and other components have similar X-ray transmittance. In the wax contrast method, the water in the biological sample is replaced with wax having a higher X-ray transmittance to improve the contrast and enable X-ray microscopic observation at the cellular level. The wax contrast method is generally a type of method called negative contrast.

[0013] In the present invention, a plate-shaped wax block with a very large maximum optical path length of X-rays is used, and since the extra wax around the test biological sample is not removed, it is different from the wax contrast method described in WO2022 / 234844 in this regard.

[0014] That is, in the method described in WO2022 / 234844, in order to obtain better spatial resolution, it was necessary to remove as much wax as possible around the tissue to be observed from the X-ray microscopic observation sample and specify the maximum optical path length of X-rays to 2 mm or less. In contrast, in the present invention, the biological sample to be observed is embedded in a wax block, but the maximum optical path length of X-rays exceeds 2 mm. When the maximum optical path length exceeds 2 mm, the spatial resolution becomes slightly inferior, but by setting the minimum optical path length of X-rays to 2 mm or less, submicron spatial resolution can be maintained.

[0015] Therefore, by performing X-ray microscopic observation using the block of the present invention, while maintaining the spatial resolution necessary for complementary observation with an optical microscope, the compatibility with the conventional biological sample observation process of performing optical microscope observation can be significantly improved. Therefore, the wax block of the present invention has a biological sample embedded therein and has a minimum value of the optical path length of X-rays of 2 mm or less and a maximum value exceeding 2 mm. Although there is no theoretical limit to the maximum value of the maximum optical path length of X-rays of the block, for example, by setting it to 35 mm or less in consideration of the size of a general embedding dish, the compatibility with the biological sample observation process using an optical microscope can be further enhanced.

[0016] The preparation of the biological sample used in the present invention can follow the steps of preparing a sample for general optical microscopy observation. Figure 1 is a process diagram from cutting out a biological sample to immersing the biological sample in wax such as paraffin for wax penetration. In Figure 1, for a sample (biological sample) cut out and collected from a living body, chemical fixation is performed using a fixative, for example, formalin, glutaraldehyde, alcohol, Bouin's solution, etc. Subsequently, dehydration using alcohol, xylene, etc. and wax penetration are performed. Defatting and decalcification are optional steps and either one or both can be performed as necessary. Wax means a lipophilic compound that is solid at normal temperature (20°C to 30°C) and has a melting point of 40°C to 80°C. Examples of wax include paraffin, other petroleum waxes, synthetic waxes, etc., and paraffin is preferred.

[0017] (2) Embedding dish for wax contrast for embedding the test biological sample After performing wax penetration with wax such as paraffin, the test biological sample is embedded. A schematic diagram of an embedding dish for embedding the test biological sample is shown in Figure 2. Figure 2 shows an embedding dish with a rectangular mouth and bottom as an example. The dish has a length in the longitudinal direction (lengthwise direction) L1 of 3 to 32 mm and a transverse direction L2 of 3 to 28 mm on the bottom surface of the inner wall, and a depth d of 2 mm or less.

[0018] However, in the present invention, the shapes of the mouth (upper surface) and bottom of the embedding dish may be square, circular or elliptical in addition to rectangular, and are not limited. Also, the side wall of the embedding dish may have a shape extending in the normal direction from the bottom surface, that is, a rectangular parallelepiped shape where the area of the bottom inner wall is the same as the area of the upper inner wall, or a shape where the area of the upper inner wall is larger than the area of the bottom inner wall, that is, a shape where the inner wall bottom surface appears to shrink when viewed from above (a shape where the extension line of the side wall facing the bottom is tapered).

[0019] The depth d of the embedding dish is 2 mm or less, preferably 1.0 to 1.5 mm, for example, 1.2 mm. The depth of this embedding dish becomes the thickness of the block embedding the biological sample. The area of the bottom surface of the inner wall of the embedding dish, that is, the area of the surface of the block embedding the biological sample that is shaped by the bottom surface of the inner wall of the embedding dish (defined as the upper surface of the wax block) is 9 to 900 mm 2 However, these areas are not limited to the above, and those skilled in the art can appropriately set the size of the block for X-ray microscopic imaging. Examples of the material of the embedding dish include stainless steel or other metals, glass or ceramics, hard or soft resins, paper or wood, etc., but are not limited thereto, and other materials may also be used.

[0020] As shown in FIG. 3, while placing the biological sample using an embedding dish with a depth of 2 mm or less, and pouring wax into the embedding dish, a plate-shaped wax block with a thickness of 2 mm or less embedding the test biological sample is produced. For example, place the biological sample approximately in the center of the embedding dish (FIGS. 3a and b), pour wax, and solidify the wax. By taking out the solidified wax, a block embedding the biological sample (hereinafter simply referred to as "block") can be produced (FIG. 3c).

[0021] However, the order of placing the biological sample in the embedding dish and pouring the wax is not limited to the above and is arbitrary. That is, it is also possible to previously contain the biological sample in the wax and pour the biological sample together with the wax into the embedding dish. Alternatively, the embedding dish can be filled with wax in advance and the biological sample can be placed therein. The biological sample placed in the embedding dish may be appropriately finely adjusted so as to be located at the center of the block before the wax solidifies. In the present invention, the biological sample may be suspended in the wax, but it is preferably arranged so as to contact the bottom of the embedding dish.

[0022] Furthermore, before the wax solidifies, a cassette for X-ray imaging (referred to as an "embedding cassette") is placed on the embedding dish in which the biological sample is embedded, and the wax is solidified with the surface of the wax poured into the embedding dish in contact with the embedding cassette. Then, by removing the block from the embedding dish, a combined body of the embedding cassette and the embedding block can be produced (Figure 4).

[0023] 2. Embedding Cassette for Wax Contrast Imaging for Placing the Test Biological Sample In the present invention, the embedding cassette for placing the block in which the test biological sample is embedded has a strip-shaped or rectangular cut (space) for allowing X-rays to pass through when irradiated with X-rays, as shown in Figure 5. The block is placed in the cassette so that the region to be irradiated with X-rays, that is, the embedded biological sample, is located within the cut region. The size of the cut in the cassette is made larger than the imaging site of the test biological sample. In other words, the size of the imaging site of the biological sample is adjusted so as to fit within the area of the cut in the cassette. And the above cut is designed so that when performing X-ray imaging using the cassette, the loss rotation angle range caused by the cassette blocking the X-rays is less than 30°, that is, the imaging rotation angle range is 150° or more and the X-rays are not blocked. Details of the imaging rotation angle range will be described later. Although the biological sample is placed in the embedding block on the embedding dish so as to satisfy an imaging angle range of 150° or more of the X-ray microscope, factors that block X-rays may include not only the blocking of X-rays by the cassette but also other factors such as the blocking of X-rays by the thickness of the wax itself that embeds the biological sample.

[0024] Figure 6 is a diagram showing that the embedding block is placed so that the biological sample is located in the cut region of the cassette and is arranged in the X-ray irradiation direction in the X-ray microscope. As shown in Figure 6, X-rays are irradiated from the X-ray generating device of the X-ray microscope toward the block, and the X-rays pass through the cut of the cassette (the cut cut out in a strip shape in Figure 6) and irradiate the biological sample in the block placed in the cassette.

[0025] FIG. 7 is a diagram showing a cassette with a block installed being fixed to a stage. In FIG. 7, a block 702 embedding a biological sample 701 is provided in a cassette 703 and is positioned such that the biological sample 701 is within the range of a cut region 706 of the cassette. That is, the biological sample is placed within the cut so as to satisfy an imaging possible angle range of 150° or more, which will be described later. Then, the cassette is fixed on the stage such that the block is positioned on the detector side (the side opposite to the side where X-rays enter). In FIG. 7, the longitudinal direction of the biological sample faces the x direction (horizontal), but it can also be the z direction (vertical and horizontal). In that case, it may be arranged such that the cut of the cassette faces upward, and as long as the biological sample 701 fits within the horizontally long cut region, it is also possible to arrange the longitudinal direction of the biological sample in the vertical direction (z direction).

[0026] The cassette 703 is fixed on a stage 704 via a fixing member 705. As shown in FIG. 7, the fixing member 705 can be provided on the stage with a fitting jig having a recess, and the end of the cassette can be fitted into the fitting jig for fixing. In FIG. 7, both ends of the cassette 703 are in a state of being fitted to the fixing member 705, but the member for fixing the cassette to the stage is not particularly limited, and it may be fixed by pasting with double-sided tape or clay (not shown), or may be fixed by clipping or screwing. When using a clip, it is fixed by utilizing the elasticity of an object such as a leaf spring. When using screwing, it is fixed by utilizing an Allen screw or the like.

[0027] After fixing the cassette 703 on the stage 704, the position adjustment mechanism of the stage 704 (details will be described later) can be used to move it in the left - right or front - back direction (x or y direction), or up - down direction (z direction) so that the approximate center of the biological sample is located on the X - ray irradiation path (optical path) p1. Also, although the X - ray irradiation path p1 is the normal direction with respect to the block 702, if a rotation axis q1 is provided at the approximate center of the biological sample 701 in the normal direction with respect to the xy plane and rotated around the rotation axis q1, p1 will be in an oblique direction with respect to the block 702. That is, the optical path of the X - ray is in an oblique direction with respect to the block 702. The rotation angle θ at this time is within the X - ray imaging rotation angle range and can be rotated by 150° or more, preferably 180° or more.

[0028] The relationship between the biological sample 701 and the X - ray irradiation direction is shown in FIG. 8. In FIG. 8, the left panel shows the minimum and maximum optical path lengths of X - ray imaging in a conventional test example. Conventionally, since X - ray imaging was performed on a biological sample with the wax removed from the surface after wax contrast, the maximum optical path length and the minimum optical path length were approximately the same as the thickness or length of the biological sample.

[0029] In contrast, in the present invention, since X - ray irradiation is performed on a plate - shaped wax block embedding a wax - contrasted biological sample, the optical path length depends on the length or thickness of the block. Even if the thickness of the block is 2 mm or less, when the sample is rotated by a certain angle θ from the rotation axis of the sample, the optical path becomes as long as the length close to the longitudinal direction of the block (test example of the present invention; right panel of FIG. 8). In the present invention, the minimum value of the X - ray optical path length of the block is 2 mm or less, and the maximum value is greater than 2 mm. Here, there is no theoretical limit for the maximum value of the X - ray maximum optical path length of the block, but considering the size of a general embedding dish, it is preferably 35 mm or less. Note that for the rotation of the sample, the sample can be fixed and the X - ray source and camera can be rotated with respect to the sample, or the X - ray source and camera can be fixed and the sample can be rotated. Also, the rotation axis (direction) can be either the longitudinal direction or the short - hand direction of the sample.

[0030] FIG. 9 shows a position adjustment mechanism 90 for adjusting the xy direction (horizontal direction), z direction (height direction), and θ (rotation angle) of the stage 704.

[0031] The position adjustment mechanism 90 is a mechanism for aligning the observation target position for optical microscope observation, and includes a support base 901 on which the stage 704 is placed, and an adjustment unit 902 for adjusting the xy direction (horizontal direction), z direction (height direction), and the horizontal rotation angle of the support base 901. Protrusions (not shown) may be provided on the bottom surface of the stage 704, and fixing holes 903 and the like for fitting and fixing the protrusions may be provided on the support base 901. The adjustment unit 902 includes an adjustment knob 902a for adjusting the x direction of the stage 901, an adjustment knob 902b for adjusting the y direction, an adjustment knob 902c for adjusting the z direction, and a knob 902d for adjusting the horizontal rotation angle of the support base 901. Needless to say, when referring to the x direction, y direction, z direction, and rotation angle, the negative direction (for example, if the x direction is rightward, then leftward, and the same applies to the others) can also be adjusted. In addition, the position adjustment mechanism 90 can be an electric mechanism that can be automatically controlled by a computer program or the like in addition to being manual.

[0032] 3. X-ray imaging For wax imaging, X-rays with an energy of 4 to 12 keV are used (for example, the X-ray microscope nano3DX of Rigaku Corporation is used with X-rays of Cu wavelength), and it is necessary to adjust the sample size so that the X-rays can sufficiently penetrate the sample. Therefore, in the present invention, a plate-shaped wax block with a thickness of 2 mm or less embedding the subject sample is installed so that the block plate surface is aligned with the rotation axis of the X-ray microscope for X-ray imaging. As described above, the block is mounted on the cassette of the present invention, and the cassette is further fixed to the stage with a fixing member or a detachable adhesive material (such as double-sided tape or clay) for X-ray imaging. However, in order to improve the installation accuracy and reproducibility, a jig with an adjustment mechanism designed to attach the cassette to the observation stage with a constant sample height and tilt angle may be used.

[0033] In the present invention, X-ray microscopic observation may be performed in the state of a bare plate-shaped wax block that is not combined with an embedding cassette. In this case, the plate-shaped wax block after X-ray microscopic observation is waxed onto the upper surface of a wax block that does not contain a biological sample and is combined with a general embedding cassette without a notch (Fig. 16). A combination of both in which a wax block that does not contain a biological sample is installed in a cassette without a notch is referred to as a two-piece combination. In the two-piece combination, it is desirable that the wax constituting the wax block that does not contain the biological sample is the same as the wax constituting the plate-shaped wax block, but another similar wax may also be used. "Waxing" means that by using a heat-melted wax called waxing wax as an adhesive, the plate-shaped wax block and the wax block that does not contain the biological sample are adhered, and further left standing at room temperature to solidify the waxing wax, thereby bonding both blocks. By waxing the biological sample-embedded wax block after X-ray microscopic observation onto the upper surface of the wax block in the two-piece combination, a combination of the two-piece combination and the biological sample-embedded wax block is obtained. This combination is referred to as a three-piece combination. It is desirable that the wax of the waxing wax when producing the three-piece combination is the same as the wax constituting the plate-shaped wax block or the wax block that does not contain the biological sample, but another similar wax may also be used. The wax block after waxing can be handled in the same manner as a biological sample embedded in wax and combined with a general embedding cassette. The plate-shaped wax block contained in this wax block after waxing maintains a solid state throughout the entire process of waxing and maintains the corresponding relationship of orientation before and after waxing, so the position information of the plate-shaped wax block is almost preserved, and the deformation of the embedded biological sample is suppressed. Thereafter, optical microscopic observation is performed according to the steps described later.

[0034] 4. Acquisition of position information and observation with an optical microscope Fig. 10 is a diagram showing the process until optical microscopic imaging is performed after X-ray microscopic imaging is completed. After imaging with an X-ray microscope, position information to be evaluated in detail with an optical microscope is acquired from the three-dimensional image thus captured.

[0035] FIG. 11 is a diagram showing an overview for acquiring position information from a three-dimensional image. In FIG. 11, panel A shows a biological sample embedded in a wax block. The three-dimensional image after X-ray imaging is a CT slice which is a two-dimensional image of a cross-section obtained by cutting the wax block containing the biological sample in an arbitrary plane, and is displayed in a form in which the CT slices are continuously stacked in a direction perpendicular to the slice plane. Among these, an arbitrary position of the biological sample is used as a reference plane, for example, a plane in contact with the upper end of the biological sample can be used as the reference plane. In this case, for example, when viewed from the side of the upper surface of the block (the surface formed by a1-a2-a3-a4 in FIG. 11A), a plane U (a plane formed by b1-b2-b3-b4 which is parallel to the plane formed by a1-a2-a3-a4 and is separated by a distance p and is in contact with the upper end of the biological sample) parallel to the reference plane is used as the reference plane, and when continuously cutting with a plane parallel to the reference plane (in the xy direction in the figure), a series of CT slice images from the upper end to the lower end of the biological sample can be obtained (panel B). Since the thickness of one CT slice can be set as appropriate, the number of CT slice images obtained from a biological sample of a predetermined size is determined. Therefore, an area to be observed (observation target area) is searched for from among the series of CT slice images. For example, in panel B of FIG. 11, since the thickness of one slice can be arbitrarily set, the distance d n from the position (d0) of the reference plane U (reference image) of the biological sample to the nth image can be obtained. Therefore, at the time of optical microscope observation, the sample is cut with a microtome until the reference plane of the biological sample appears, and from there, d nSlice it thinly in micron thickness so that slices up to [a certain point] can be obtained. After that, stretching, dewaxing, and staining are performed according to the process shown in FIG. 10, and optical microscopy is carried out. In FIG. 11, the plane parallel to the plane including the upper end of the biological sample and the upper surface of the block as viewed from the upper surface side of the block is used as the reference plane, but it is not limited to this, and any plane including the plane including the lower end of the biological sample, the upper and lower surfaces of the block, etc. may be used as the reference plane. Also, the reference plane may be a plane parallel to the plane including the block surface or a non-parallel plane.

[0036] In the stretching process, the thinly sliced section is picked up with tweezers, floated in warm water and stretched, and then scooped up and adhered onto a slide glass. In the dewaxing process, the glass with the section attached is dried and dewaxed by immersing it in xylene or the like. In the staining process, staining is performed using methods such as HE (hematoxylin-eosin) staining that differentiates the cell nucleus from other tissues, or PAS reagent (periodic acid Schiff’s reagent) staining that clearly stains the basement membrane, according to the observation purpose. In the optical microscopy imaging process, variously stained biological samples are imaged with an optical microscope along the position information obtained by imaging with an X-ray microscope to acquire digital images.

[0037] For example, regarding the acquisition of position information, as shown in FIG. 18A, the X-ray microscope image is resliced in a direction perpendicular to the surface of the wax block (changing the slice direction of the CT image; for example, using the reslice tool of ImageJ). If the position of the upper end of the sample is taken as the reference (0.0 μm), the exact depth of any object in the sample can be specified. In the examples of FIGS. 18B and C, the renal corpuscle 1 is at a position 115.8 μm from the upper end of the sample. Regarding the observation with an optical microscope, the sample after X-ray imaging is directly sectioned with a microtome in a general manner without wax re-embedding for optical microscope observation, stained in a general manner, and observed using a general optical microscope.

[0038] As a result, X-ray microscope images and optical microscope images of the same measurement location of the same biological sample can be easily obtained in a form where the orientations are substantially the same. The orientation error between the two images in this state is, for example, within 10°, but is not limited thereto, and may be 10° or more depending on the required precision.

[0039] When slicing the sample with a microtome to a constant sample thickness, if the sections where the upper end of the sample appears are recorded, it is possible to calculate which section contains the object of interest. In the examples of FIGS. 18B and C, if the section thickness is 4 μm, the optical microscope image corresponding to the X-ray microscope image of the renal corpuscle 1 can be obtained from approximately the 30th section from the upper end of the sample.

[0040] 5. Matching of X-ray Microscope Image and Optical Microscope Image When X-ray microscope images and optical microscope images with substantially the same orientation are obtained as input images in the initial state, the two can be matched and precisely aligned by adjusting the orientation of the X-ray microscope image by the following method.

[0041] (1) Selection of an Arbitrary Observation Target Region as a Position Marker in a Biological Sample First, an arbitrary observation target region that is included in the biological sample and can be used as a position marker for orientation adjustment is selected from the images of the biological sample captured by the X-ray microscope and the optical microscope. The "image" means both the data of the image itself (data displayed in pixels, etc.) and numerical data such as coordinates. The form of the arbitrary observation target region is not limited, but for example, the size, structure, and type of cells or tissues, cell nuclei, and defective forms can be selected as position markers.

[0042] The defective form is any one of cancer, fibrosis, calcification, stones, and other deposits, or a combination thereof. For example, using the cell nucleus as a position marker, one reference region serving as a reference for orientation adjustment is selected, and about 10 index regions for evaluating the degree of coincidence are selected. The selection of the position marker may be performed manually or automated using AI or the like.

[0043] (2) Step of collating and precisely matching the X-ray microscope image and the optical microscope image According to the flowchart of azimuth adjustment (details will be described later), it is performed by rotating the image of the CT rotation angle and / or the tilt angle in the X-ray microscope (using the rotation tool of ImageJ, etc.).

[0044] (3) Flowchart of azimuth adjustment The step of collating and precisely matching the X-ray microscope image and the optical microscope image will be described in more detail. Consider the case of precisely matching the optical microscope (LM) image and the X-ray microscope (XRM) image. In Fig. 12A, (a) is a perspective schematic diagram of the LM image, and (b) is a perspective schematic diagram of the XRM image. The figure of the flowchart of azimuth adjustment is shown in Fig. 12B.

[0045] (3-1) First, find a region of interest containing position markers (such as cell nuclei) that commonly exist in the LM image and the XRM image, and use this region as the fiducial region (step S1). The slice where the fiducial region exists is called the "fiducial slice". Note that the fiducial region may contain multiple position markers.

[0046] (3-2) Next, select several regions of interest containing position markers (such as cell nuclei) that exist in the LM image but do not exist in the fiducial slice of the XRM image but exist within approximately 20 slices before and after (step S2). This region is called the "marker region", and the slice where the marker region exists is called the "marker slice". The number of selected marker regions is not limited, but 8 to 20 locations, preferably about 10 locations, are selected. Note that one marker region may contain multiple position markers.

[0047] In FIG. 12A, (b) shows a mode in which the XRM image is shifted by an angle of θ1 counterclockwise from the x - coordinate or y - coordinate of the reference slice plane around the reference region of the reference slice, and is shifted by an angle of θ2 counterclockwise around the y - axis passing through the center of the reference region. Here, θ1 is called the CT rotation angle and θ2 is called the tilt angle. Note that the CT rotation angle may be a clockwise rotation angle or a counterclockwise rotation angle with respect to the rotation axis. Also, the rotation axis of the tilt angle may be the x - axis or the y - axis. When the angles of θ1 and θ2 are shifted, the position markers in the regions other than the reference region of the LM image become invisible in the XRM image, but they will exist in some slices above and below the reference slice (for example, within 10 slices or 20 slices above and below).

[0048] FIG. 12A shows a mode in which three regions (M1, M2, and M3) existing in the LM image are selected as index regions including position markers (such as cell nuclei) in the XRM image. For the sake of simplicity of explanation, the number of index regions is set to three. In FIG. 12A(b), the slice where the index region M1 exists is defined as the index slice 1, and the slices where the index regions M2 and M3 exist are defined as the index slice 2.

[0049] Next, when these index regions are z - projected onto the reference slice, the index region M1 of the index slice 1 and the index regions M2 and M3 of the index slice 2 are projected onto the reference slice (FIG. 12A(c), step S3). When selecting the index regions, it is advisable that when the index regions are z - projected onto the reference slice, the projected index regions (projection index regions) be scattered as evenly as possible over the entire reference slice.

[0050] (3 - 3) The projection index area is a virtual existence. Since the actual XRM image is shifted by θ1 and θ2 degrees when viewed from the reference slice of the LM image, there are no position markers in the projection index area of the reference slice. Therefore, azimuth adjustment is performed using a rotation tool such as ImageJ (step S4) so that position markers appear in the projection index area of the reference slice. Step S4 includes steps S5 to S10. The azimuth adjustment is performed as follows.

[0051] First, calculate the rotation angle such that position markers appear in the projection index area of the reference slice. If an index area is added, calculate only for that added part. For a single index area, let the distances in the x and y directions (in pixel units) between the reference area and the projection index area be dx (= x_index - x_reference) and dy (= y_index - y_reference), and the distance between the reference slice and the index slice be dz (= z_index - z_reference). Then,

[0052] I) Select the axis that is closer to being parallel to the straight line connecting the projection index area and the reference area among the x or y axes (in the order of selection), and calculate the z-angle such that the axis and the straight line are parallel (step S5). z-angle = -tan -1 (dy / dx), or z-angle = tan -1 (dx / dy) For example, in Fig. 12A(c), since the axis closer to being parallel to the straight line connecting M1 and the reference area is the x-axis, the x-axis is selected, and thus the z-angle is calculated using the former of the above two formulas.

[0053] II) Next, calculate the y-angle or x-angle such that position markers appear in the projection index area of the reference slice (using the following formula) (step S6). For example, adjust the angle of θ2 (the pitching angle around the x-axis or y-axis) so that the cell nucleus etc. selected as the position marker included in the index area appears on the reference slice. That is, y-angle = {if dx > 0 tan -1 (dz / sqrt(dx 2+dy 2 ) else -tan -1 (dz / sqrt(dx 2 +dy 2 ))}, or x-angle = {if dy > 0 -tan -1 (dz / sqrt(dx 2 +dy 2 )) else tan -1 (dz / sqrt(dx 2 +dy 2 ))} For example, for M1 in Fig. 12A(c), since the x-axis is selected, the y-angle is calculated by the former of the above two equations. The sign is determined by the value of dx.

[0054] (3 - 4) Furthermore, among the combinations of the two projection index regions, select a pair in which the angle sandwiching the reference region is closer to a right angle (step S7). For example, in Fig. 12A(c), among the angles formed by (i) M1 - reference region - M2, (ii) M2 - reference region - M3, and (iii) M1 - reference region - M3, select the pair in which the angle is closer to a right angle. If the angle of (i) is 89°, the angle of (ii) is 80°, and the angle of (iii) is 180°, the pair in which the angle is closer to a right angle is the pair of M1 and M2 in (i).

[0055] As an example, in Fig. 19, in the pair of M4 and M9, since the angle formed by M4 - F - M9 is 87.1° and is close to a right angle, this pair is selected.

[0056] Calculate the average z-angle with the selected pair of projection index regions (the following equation) (step S8). That is, z-angle = (z-angle_1 + z-angle_2) / 2 As an example, in Fig. 19, since the z-angles of M4 and M9 are 40.6° and 37.6° respectively, the average z-angle is 39.1°. Conversely, by selecting pairs of target regions with close z-angles, pairs in a nearly orthogonal relationship can be found. When adding target regions, in order to improve the matching degree, pairs of target regions with a closer orthogonal relationship may be reselected based on the z-angle.

[0057] (3-5) For the pair of projection target regions selected in the above (3-4), input the values of x-angle, y-angle, and z-angle into the ImageJ TransformJ Rotate plugin to perform a rotation operation (the following command) (step S9). That is, run(“TransformJ Rotate”, “z-angle=z-angle y-angle=y-angle x-angle=x-angle interpolation=Linear background=0.0 adjust resample anti-alias”); For example, in Fig. 19, since the z-angle (average) is 39.1°, the y-angle (M4) is 2.58°, and the x-angle (M9) is 2.62°, input these values.

[0058] (3-6) If necessary, search for (x and y)-angles around the current orientation (step S10).

[0059] (3-7) If the matching number is equal to or greater than the evaluation criterion, the orientation adjustment ends. If the matching number is less than the evaluation criterion, return to the step (step S2) of the above (3-2) to add a target region.

[0060] (3-8) Evaluation of orientation adjustment based on the matching number of target regions As a result of performing orientation adjustment according to the above flowchart, for a certain index region, when a position marker appears in the projection index region of the reference slice, it is evaluated that the images of the X-ray microscope and the optical microscope match for that index region. The evaluation criterion for orientation adjustment is set to 6 to 8 matching index regions. That is, orientation adjustment is terminated when 6 or more, 7 or more, or 8 or more index regions match, and 8 or more matches are desirable. In addition, in 3 cases of successful orientation adjustment with 8 or more matching index regions, the average residual of the x-angle or y-angle of these matching index regions was 0.21° to 0.34°. Therefore, it can be considered that the precision matching accuracy obtained by this method is generally within 0.5°.

[0061] Note that the optical microscope image includes deformations caused by sample preparation, and local structural differences may occur between the X-ray microscope image and the optical microscope image. Therefore, the comparison of the X-ray microscope image and the optical microscope image may further include a correction process (such as Rigid Registration in Fiji) of the image data of the optical microscope. Software for automating the above flowchart may be designed.

[0062] 6. Stereoscopic evaluation of the matched image data In one aspect of the present invention, a method for stereoscopically extracting an evaluation target region of a biological sample based on both the matched X-ray microscope image and the optical microscope image includes the following steps. A step of extracting an observation target region (region of interest) including the evaluation target region from the X-ray microscope image matched with the image of the optical microscope, and A step of binarizing the region of interest and further extracting the evaluation target region from the region of interest.

[0063] (1) Extraction of the region of interest including the region to be evaluated From the X-ray microscope image, an extraction target area is extracted as an observation target area including the area to be evaluated using image analysis software having a shape or area selection function (such as the Polygon Selection tool in ImageJ). At this time, by referring to the optical microscope image in which the basement membrane or the like is clearly stained according to the method of the present invention shown in the examples, the extraction accuracy can be improved by confirming the extraction boundary. In the present invention, the shape of the extraction target area may be three-dimensional (3D) or two-dimensional (2D). In the present invention, both images are used complementarily to extract the extraction target area including the evaluation target area, but the present invention is not limited thereto, and the extraction target area including the evaluation target area may be extracted only from the X-ray microscope image.

[0064] (2) Extraction of the evaluation target area The evaluation target area is binarized and extracted from the X-ray microscope image using image analysis software having a binarized image creation function and a shape or area selection function (such as the Threshold tool and the Polygon Selection tool in ImageJ). At this time, according to the method of the present invention shown in the examples, a continuous shape in the depth direction is confirmed in the X-ray microscope image having an isotropic spatial resolution, and the number of cells included in the evaluation target area is confirmed by referring to the optical microscope image in which the cell nuclei are clearly stained. By using the optical microscope and the X-ray microscope complementarily in this way, the extraction accuracy can be improved.

[0065] (3) Evaluation of the evaluation target area The step of evaluating the volume or shape of the extracted evaluation target area is as follows. For the evaluation target region extracted from the X-ray microscope image, the volume or three-dimensional shape of the region is evaluated by the method of the present invention shown in the examples. The volume is calculated by multiplying the number of voxels in the region by the voxel volume, enabling quantitative evaluation of the volume, which is difficult in optical microscope observation. The three-dimensional shape becomes an evaluation index that cannot be obtained from an optical microscope image when the evaluation target region has a complex three-dimensional structure or when the three-dimensional arrangement between the evaluation target region and other tissues is desired. Further, when the evaluation target region is located between sections of an optical microscope sample, X-ray microscope images that are continuous in the depth direction are advantageous.

[0066] Software for automating the above three-dimensional evaluation using AI or the like may be designed. The software may have a black box configuration that performs the extraction of the evaluation target region and the three-dimensional evaluation as a series of steps, or may have an interactive configuration that first proposes a list of candidates for a large number of evaluation target regions extracted and then performs a three-dimensional evaluation on the candidates specified by the user.

[0067] 7. Microscope Image Processing Apparatus, Program, and Recording Medium The present invention further provides a microscope image processing apparatus and a computer program for microscope image processing.

[0068] The apparatus of the present invention is a microscope image processing apparatus that stereoscopically extracts an evaluation target region of a sample based on two images in which an image of a wax-embedded biological sample imaged by an X-ray microscope using X-rays with an energy of 4 to 12 keV and an image of the same measurement location of the same biological sample imaged by an optical microscope after the imaging are matched, and includes the following means. Means for extracting an observation target region (region of interest) including the evaluation target region from the images of the biological sample imaged and matched by the X-ray microscope and the optical microscope, and Means for binarizing the extracted region of interest and further extracting the evaluation target region from the region of interest. The apparatus of the present invention is used for the analysis of biological samples.

[0069] The program of the present invention is a program for three-dimensionally extracting an evaluation target region of a biological sample based on two images in which an image of a wax block-embedded biological sample captured by an X-ray microscope using X-rays with an energy of 4 to 12 keV is matched with an image of the same measurement location of the same biological sample captured by an optical microscope after the imaging. The program causes a computer to extract an observation target region (region of interest) including the evaluation target region from the images of the biological sample captured and matched by the X-ray microscope and the optical microscope, and binarize the extracted region of interest and further extract the evaluation target region from within the region of interest. This is a program for causing the computer to function as such. The program of the present invention is used for analyzing biological samples.

[0070] FIG. 13 shows an overview of a microscope image processing apparatus 100 provided in the present invention. In FIG. 13, the microscope image processing apparatus 100 includes an X-ray microscope imaging apparatus 10, an optical microscope imaging apparatus 30, and an information processing apparatus 40. These apparatuses may be connected via a communication network 20 as shown in FIG. 13. Further, the X-ray microscope imaging apparatus 10, the optical microscope imaging apparatus 30, and the information processing apparatus 40 can be installed independently without being connected to a network. In this case, only necessary information such as image data can be taken into an independent information processing apparatus for processing.

[0071] The X-ray microscope imaging apparatus 10 writes basic information such as patient information, examination information, and image ID from which the sample is derived into the header of an image file and transmits it to the information processing apparatus 40 in accordance with, for example, the DICOM (Digital Imaging and Communications in Medicine) standard. In the present invention, the X-ray microscope imaging apparatus 100 preferably uses nano3DX (Rigaku Corporation).

[0072] The optical microscope imaging device 30 is a commonly used microscope such as a stereomicroscope, an inverted microscope, or a metallurgical microscope, but it may also be a digital microscope equipped with a high-resolution digital camera instead of an eyepiece. By using a digital microscope, the captured microscope image can be saved as a digital image. In this case, the optical microscope imaging information is transmitted to the information processing device 40.

[0073] The information processing device 40 is a computer device that analyzes the images transmitted from the X-ray microscope imaging device 10 and the optical microscope imaging device 30 to assist in the inspection of biological samples and outputs this image information. When an X-ray microscope image is obtained by irradiating a biological sample with X-rays and taking a picture, the information processing device 40 acquires the position information of the biological sample for observation with an optical microscope based on an arbitrary position of the biological sample. Once the position information is acquired, the information processing device outputs the position information.

[0074] As shown in FIG. 14, the information processing device 40 includes a control unit 101, a communication unit 102, an operation unit 103, a display unit 104, a storage unit 110, etc. The control unit 101 is composed of a CPU, a ROM, a RAM, etc., and comprehensively controls the processing operations of each part of the information processing device 40. Specifically, the CPU reads out various processing programs stored in the ROM and expands them in the RAM, and performs various processes in cooperation with the programs.

[0075] Furthermore, the information processing device 40 includes a storage unit 110 provided in the ROM or the RAM. The storage unit 110 includes an X-ray microscope information database (DB) 111, an optical microscope information DB 112, and an attached information DB 113. The attached information DB stores the tissue names, IDs, sample collection dates, reference planes, slice thicknesses, reference regions, index regions, etc. of humans and animals from which the biological samples are derived.

[0076] The control unit 101 executes a process of acquiring the X-ray microscope imaging information and the optical microscope imaging information, and in cooperation with the display unit 104, exhibits an image display function on a monitor or the like (not shown). The communication unit 102 is composed of a network interface or the like, and performs data transmission and reception with external devices connected to an X-ray microscope imaging device, an optical microscope imaging device, and an information processing device via a communication network. The operation unit 103 includes a keyboard having various input keys and function keys, and a mouse or the like, and outputs an operation signal input by a key operation on the keyboard or a mouse operation to the control unit 101. Further, the operation unit 103 can also be composed of a touch panel, and in that case, an operation signal is output to the control unit 101 according to a touch operation by the user. The display unit 104 is configured to include a monitor such as an LCD (Liquid Crystal Display), and displays various screens according to an instruction of a signal input from the control unit 101.

[0077] FIG. 15 is a flowchart showing an azimuth adjustment process by the microscope image processing apparatus of the present invention. First, the control unit 101 of the information processing apparatus 40 receives an input of an X-ray microscope image and sets it as a target image 1. The control unit 101 specifies an observation target region that can be used as a position marker for azimuth adjustment and is included in the biological sample from the target image 1 (first specifying means). The control unit 101 executes image analysis processing based on this target image (step S11). The image analysis processing is executed in cooperation with a program stored in the control unit 101 and the storage unit 110.

[0078] Next, position information to be evaluated in detail with an optical microscope is extracted from the target image 1 (step S12). As shown in FIG. 11, since the thickness of one slice can be arbitrarily set, the control unit 101 calculates the distance d from the position (d0) of the reference plane U (reference image) of the biological sample to the nth image. n The slice thickness may be input by the user or may be preset.

[0079] Next, the control unit 101 receives an input of an optical microscope image and sets it as a target image 2 (step S13). The control unit 101 selects an observation target area that can be used as a position marker for orientation adjustment from among the target image 1 and the target image 2 (step S14). That is, the control unit 101 identifies an observation target area corresponding to the observation target area identified by the first identification means from the image obtained by imaging a part of the biological sample included in the image captured by the X-ray microscope with an optical microscope image (second identification means).

[0080] For the selection of the form of the observation target area, the form discriminator can be pre-trained with the form, and a form that matches or approximates the form can be automatically selected, or an input of the form selected by the user can be accepted. In step S14, for example, for each slice of the microscope image, image feature amounts such as density, shape, texture, and multi-resolution features are extracted on a pixel-by-pixel basis. One type of image feature amount may be extracted, or a plurality of types may be extracted.

[0081] The first and second identification means identify the observation target area based on an observation target area designation operation by the user, a predetermined priority for the observation target area, and / or a priority preset by the user. The "priority" is the order for identifying the observation target area and indicates in what order the observation target areas are identified. For example, when selecting an observation target area as a position marker for orientation adjustment of an X-ray microscope image of a kidney biopsy sample embedded in a wax block, the cell nucleus has the first priority, and a particulate defect form (such as fibrosis) has the second priority.

[0082] Also, the orientation error between the optical microscope image and the X-ray microscope image is set to within 10°, but is not limited thereto, and may be 10° or more depending on the required precision.

[0083] Next, the control unit 101 executes the orientation adjustment process shown in FIG. 12B in order to collate the information on the observation target area identified by the first identification means and the information on the observation target area identified by the second identification means (step S15). According to the orientation adjustment process shown in FIG. 12B, the control unit 101 selects one reference region and several index regions from the observation target region including the position marker. During the operation of step S15, when there is an input for changing the designation of the reference region and the index region from the user, or when an index region is added, the control unit 101 calculates the CT rotation angle θ1 and the tilt angle θ2 of the index region from the XRM images and displays them on the display unit 104 (step S16).

[0084] Subsequently, the control unit 101 calculates the projected index regions obtained by z-projecting the index regions onto the reference slice, and selects a specific region from among the plurality of projected index regions. For example, as shown in FIG. 12A(c), when three projected index regions (M1, M2, and M3) are selected, two straight lines formed by one projected index region - reference region - the other projected index region are extracted, and it is calculated at which combination of the projected index regions the angle formed by these straight lines is closest to a right angle, and the candidate projected index region pairs are displayed (step S17). Next, the control unit 101 displays the form of the projected index regions (step S18). Then, the control unit 101 calculates the rotation angles x-angle, y-angle, and z-angle for orientation adjustment (step S19).

[0085] The control unit 101 inputs the values of x-angle, y-angle, and z-angle for a pair of projected index regions into the storage unit 113 to perform a rotation operation, or displays the rotation angles for the user to perform a rotation operation on the display unit 104 (step S20). As a result of the orientation adjustment, if the position marker appears in the projected index region of the reference slice for a certain index region, it is evaluated that the images of the X-ray microscope and the optical microscope match for that index region. If necessary, a search for (x and y)-angle is performed around the current orientation (step S21). If the number of matching index regions reaches or exceeds the evaluation criterion, the orientation adjustment is terminated. If the number of matching regions is less than the evaluation criterion, an index region is added and the process returns to step S16. The evaluation criterion is set to the default value of 8 and can be appropriately changed within the range of 6 to 8.

[0086] Fig. 20 shows a flowchart of the steps up to the three-dimensional shape evaluation of the evaluation target area. In Fig. 20, an observation target area (referred to as the "target area") including the area to be evaluated is extracted from the image azimuth-adjusted as described above (step S101). The target area is extracted using, as indicators, the size, structure, and type of cells or tissues, cell nuclei, and defect morphology in the biological sample. Defect morphologies include cancer, fibrosis, calcification, stones, and deposits. For example, in Fig. 21, the renal corpuscle of the mouse kidney and the glomerulus inside it are extracted as the target area. Here, the accuracy of extraction can be improved by using an optical microscope and an X-ray microscope complementarily. "Using complementarily" means compensating for the features and advantages of one of the optical microscope and the X-ray microscope with respect to the other microscope that does not have such features or advantages or has a lower degree. For example, the continuous shape in the depth direction is confirmed in the X-ray microscope image with isotropic spatial resolution, while the extraction boundary is confirmed with reference to the optical microscope image in which the basement membrane and the like are clearly stained. The extraction of the target area may be performed using a detection support system by a computer equipped with an image processing tool or the like, or the user may specify it from the target image displayed on the display unit 104 using the operation unit 103. In step S101, for example, for each slice of the X-ray microscope image, image feature amounts such as density, shape, and texture are extracted in pixel units.

[0087] In step S101, the control unit 101 associates the information on the extracted areas in the optical microscope image and the X-ray microscope image with reference to the optical microscope image and displays it on the display unit 104. For example, as shown in Figs. 21 to 23, the optical microscope image and the X-ray microscope image are displayed side by side. Note that the optical microscope image and the X-ray microscope image may be displayed in parallel or superimposed.

[0088] Next, the control unit 101 performs feature amount analysis on the target region, binarizes and extracts the evaluation target region for each slice of the X-ray microscope image, and obtains the information thereof (step S102). When the evaluation target regions within the target region have the same pathological features, they are extracted as the same type of evaluation target regions. However, when the region having a plurality of pathological features is included in the target region, they can also be extracted as a plurality of different types of evaluation target regions. Here, by using the optical microscope and the X-ray microscope complementarily, the accuracy of extraction can be improved. For example, in the X-ray microscope image having isotropic spatial resolution, the continuous shape in the depth direction is confirmed, and on the other hand, referring to the optical microscope image in which the cell nuclei are clearly stained, the number of cells included in the evaluation target region is confirmed.

[0089] Next, the control unit 101 calculates the volume of the evaluation target region included in the target region based on the binarized information (step S103). The volume calculation is performed by multiplying the number of voxels of the extracted evaluation target region image by the voxel volume.

[0090] Next, the control unit 101 displays the three-dimensional shape of the evaluation target region portion on the display unit (step S104). The control unit 101 can also pre-learn the shape by the shape discriminator and display the shape that matches or approximates the shape, or can also accept the input of the shape by the user. In these images, for example, as shown in FIG. 23, a surrounding line (annotation) may be displayed on the evaluation target region (lesion portion).

[0091] The program of the present invention can be stored in a computer-readable recording medium. Therefore, a computer-readable recording medium storing the program of the present invention is also included in the present invention. Examples of the recording medium or the storage means include, but are not limited to, magnetic media (flexible disks, hard disks, etc.), optical media (CDs, DVDs, etc.), magneto-optical media, flash memories, and the like.

[0092] Example Hereinafter, the present invention will be described more specifically by way of examples. However, the scope of the present invention is not limited by these examples.

Example

[0093] Method An experiment was conducted using a wax (paraffin)-contrasted mouse renal biopsy simulation sample to determine whether it is possible to three-dimensionally evaluate the tissue in the observed region by observing the same measurement site of the same biological sample with an X-ray microscope and an optical microscope. Kidneys of a kidney disease model mouse created according to the partial nephrectomy method described in the literature (Kunishima et al. (2022) Scientific Reports 12, 9436) were used in the experiment. Next, a wax-contrasted sample (wax block) was created from the kidneys of the same disease model mouse according to the method described in the same literature.

[0094] Next, an X-ray microscope observation sample was created from the same wax block according to the direct mounting method described in the same literature. This sample was attached to an X-ray microscope nano3DX (Rigaku Corporation) (CCD detector), and X-ray projection image data was taken under the conditions of Cu-target (40 kV / 30 mA), L0270-bin1-XD2 (0.27 μm / voxel), Step scan (60 s exposure per sheet for 3400 sheets) (required time: 57.7 hours).

[0095] After processing this projection image data by ring artifact correction, drift correction, and phase retrieval (Paganin method; δ / β = 100), CT reconstruction was performed using software based on the general FBP algorithm. Further, a renal biopsy simulation sample after X-ray microscopic observation was re-embedded in a wax (paraffin) block, serially sectioned to a thickness of 4 μm with a microtome, stained with PAS reagent (periodic acid Schiff’s reagent) to prepare an optical microscopic observation sample, and observed using an optical microscope for serial section observation Nano Zoomer C9600-03 (Hamamatsu Photonics). Coarse alignment of the X-ray microscopic image and the optical microscopic image was manually performed using general display software (ImageJ and Drishti). Thereafter, alignment was performed according to the alignment process described in the specification.

[0096] Using the precisely aligned X-ray microscopic image and optical microscopic image obtained as described above, a three-dimensional evaluation of the glomerular lesion was attempted by the method of the present invention. First, as shown in FIG. 21, a region of interest was extracted from the X-ray microscopic image using the Polygon Selection tool of the program ImageJ. In this example, the glomerulus and the glomeruli inside thereof were extracted from a partial X-ray microscopic image composed of 400 CT slices including one mouse glomerulus. At this time, the extraction boundary was confirmed with reference to the optical microscopic image in which the basement membrane was clearly stained. Note that the optical microscopic image includes some deformation due to sample preparation, and when there are differences in local structures between the X-ray microscopic image and the optical microscopic image, the X-ray microscopic image is considered to be closer to the true structure.

[0097] Next, as shown in Fig. 22, a diseased area where mesangial cells aggregated was binarized and extracted as an evaluation target area from the X-ray microscope image using the Threshold tool and the Polygon Selection tool of the program ImageJ. At this time, in the X-ray microscope image with isotropic spatial resolution, a continuous shape in the depth direction was confirmed, and a mass of mesangial cells protruding and aggregating from normal mesangial cells connected in a chain was selected as the diseased area. Also, with reference to the optical microscope image in which the nuclei of mesangial cells were clearly stained, the number of cells contained in the diseased area was confirmed. By using the optical microscope and the X-ray microscope complementarily as described above, the accuracy in the extraction of glomeruli, renal corpuscles, and diseased areas can be improved. Finally, the volume of the diseased area was calculated by multiplying the number of voxels of the extracted diseased area image by the voxel volume. Also, the three-dimensional shape of the diseased area was evaluated by three-dimensional display using the program Drishti.

[0098] Results: Fig. 23 shows the three-dimensional display of the diseased area created using the program Drishti. The volume of the diseased area surrounded by the dotted line was calculated to be 825 μm 3 Also, the three-dimensional shape of a multi-protrusion fusion type diseased area in which a plurality of diseased protrusions generated from a single normal mesangial cell chain fused and grew large was visualized. As a result of the above experiments, it was shown that the method of the present invention that complementarily uses the image information of the optical microscope and the X-ray microscope enables three-dimensional and practical evaluation of cell-level lesions in biological tissues.

Explanation of Signs

[0099] 10: X-ray microscope imaging device, 20: Communication network, 30: Optical microscope imaging device, 40: Information processing device, 100: Microscope image processing device 101: Control unit, 102: Communication unit, 103: Operation unit, 104: Display unit, 110: Storage unit 701: Biological sample, 702: Block, 703: Cassette, 704: Stage, 705: Fixing member, 706: Cut area of cassette 90: Position adjustment mechanism, 901: Support base, 902: Adjustment unit

Claims

1. A method for three-dimensionally extracting an evaluation target region of a sample based on two images in which an image of a wax block-embedded biological sample captured by an X-ray microscope using X-rays with an energy of 4 to 12 keV is matched with an image of the same measurement location of the same biological sample captured by an optical microscope after the imaging, comprising: a step of extracting a region of interest including the evaluation target region from the X-ray microscope image matched with the optical microscope image; a step of binarizing the region of interest and further extracting the evaluation target region from within the region of interest by complementarily using the image information of the optical microscope and the X-ray microscope; The method as described above, including the above steps.

2. A method for evaluating a biological sample, including a step of evaluating the volume or shape of the evaluation target region extracted by the method according to Claim 1.

3. The method according to Claim 2, wherein the evaluation of the volume is performed by multiplying the number of voxels of the binarized image of the evaluation target region by the voxel volume.

4. A method for displaying an evaluation target region of a biological sample, including a step of three-dimensionally displaying the evaluation target region extracted or evaluated by the method according to any one of Claims 1 to 3.

5. A microscope image processing apparatus for three-dimensionally extracting an evaluation target region of a sample based on two images in which an image of a wax block-embedded biological sample captured by an X-ray microscope using X-rays with an energy of 4 to 12 keV is matched with an image of the same measurement location of the same biological sample captured by an optical microscope after the imaging, comprising: means for extracting an observation target region (region of interest) including the evaluation target region from the images of the biological sample captured and matched by the X-ray microscope and the optical microscope, and means for binarizing the extracted region of interest and further extracting the evaluation target region from within the region of interest; The apparatus as described above, provided with the above means.

6. The apparatus according to Claim 5, wherein the extraction of the region of interest and the extraction of the evaluation target region are performed by comparing the optical microscope image and the X-ray microscope image using image analysis software having a binarized image creation function and a shape or region selection function.

7. The apparatus according to Claim 5, further including means for evaluating the volume or shape of the extracted evaluation target region.

8. The apparatus according to Claim 7, wherein the evaluation of the volume is performed by multiplying the number of voxels of the binarized image of the evaluation target region by the voxel volume.

9. The apparatus according to any one of Claims 5 to 8, further comprising means for three-dimensionally displaying the extracted or evaluated evaluation target region.

10. A program for three-dimensionally extracting an evaluation target region of a sample based on two images in which an image of a wax block-embedded biological sample imaged by an X-ray microscope using X-rays with an energy of 4 to 12 keV and an image of the same measurement location of the same biological sample imaged by an optical microscope after the imaging are matched, causing a computer to, means for extracting an observation target region (region of interest) including the evaluation target region from the images of the biological sample imaged and matched by the X-ray microscope and the optical microscope, and means for binarizing the extracted region of interest and further extracting the evaluation target region from within the region of interest, a program for causing the computer to function as such.

Images