Method for observing biological sample

Abstract

A method for checking and matching images of the same measurement site of the same biological sample captured by an X-ray microscope and an optical microscope, the method comprising: a step for acquiring an image obtained by capturing a biological sample embedded in a wax block by an X-ray microscope using X-rays having an energy of 4-12 keV; a step for acquiring an image in which a part of the biological sample included in the image captured by the X-ray microscope is captured by an optical microscope; and a step for selecting, from the acquired X-ray microscope and optical microscope images, an arbitrary observation target region of the image in the biological sample as a position marker, and matching the X-ray microscope image and the optical microscope image by using the position marker.

Description

Technical Field

[0001] The present invention relates to a method for checking and precisely matching images of the same measurement site of the same biological specimen photographed by an X-ray microscope and an optical microscope. Background Art

[0002] The pathological evaluation of biological tissues mainly focuses on the pathological evaluation on a plane (two-dimensional) cut out from a pathological specimen. However, pathological changes develop three-dimensionally (stereoscopically) within the tissue. In addition, when creating a pathological specimen, a part of the specimen is discarded and cannot be evaluated. In view of such a situation, the development of a method for three-dimensionally evaluating a specimen and a method for evaluating the entire specimen has become an important problem to be solved.

[0003] Conventionally, as methods for observing biological specimens, a method of imaging a biological specimen using a microscopy method (Patent Document 1), a method of photographing the kidney (renal tubule) of a rat (Patent Document 2), a method of infiltrating a contrast agent into a biological specimen, solidifying it, and performing imaging (Patent Document 3), etc. are known.

[0004] Prior Art Documents

[0005] Patent Documents

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2020-528557

[0007] Patent Document 2: Japanese Patent Application Laid-Open No. 2014-211448

[0008] Patent Document 3: Pamphlet of International Publication No. 2022 / 234844 Summary of the Invention

[0009] Problems to be Solved by the Invention

[0010] However, currently there is no practical observation method capable of three-dimensionally evaluating the pathological changes of a specimen with spatial resolution at the cellular level. An optical microscope has advantages such as a relatively high two-dimensional resolution of about 0.2 μm and accurate tissue identification through various staining methods. On the other hand, there are limitations in microscopic observation, such as deformation (destructive deformation) of the specimen caused by the application of physical force during specimen preparation, or insufficient resolution in the depth direction due to a specimen thickness of about 4 μm. On the other hand, an X-ray microscope has advantages such as isotropic sub-micron three-dimensional resolution and non-destructiveness enabling specimen reuse. However, on the other hand, since the contrast of an X-ray microscope image is determined by reflecting the electron density of relatively light elements constituting the biological specimen, it provides a gray-scale image with low contrast, and due to this, difficulties arise in tissue identification. Therefore, there has been a need to develop a practical method for stereoscopically observing the same measurement site of the same biological specimen at the cellular level of spatial resolution by combining an X-ray microscope and an optical microscope to complement each other with respect to the above limitations or difficulties.

[0011] To solve the above problems, the inventors of the present invention conducted intensive research, and as a result, by using cell nuclei and the like contained in a biological specimen as position markers and performing orientation adjustment based on rotation operations of X-ray microscope images, they successfully matched and precisely aligned the images of the same measurement site of the same biological specimen captured by an X-ray microscope and an optical microscope, thereby completing the present invention.

[0012] Solutions for Solving the Problems

[0013] That is, the present invention is as follows.

[0014] [1] A method for matching and aligning images of the same measurement site of the same biological specimen captured by an X-ray microscope and an optical microscope, comprising:

[0015] a step of obtaining an image obtained by photographing a biological specimen embedded in a wax block with an X-ray microscope using X-rays having an energy of 4 to 12 keV;

[0016] a step of obtaining an image obtained by photographing a part of the biological specimen included in the image captured by the X-ray microscope with an optical microscope; and

[0017] a step of selecting an arbitrary observation target region of the images in the biological specimen as a position marker from the obtained X-ray microscope and optical microscope images, and using the position marker to match and align the X-ray microscope image and the optical microscope image.

[0018] [2]According to the method described in [1], wherein,

[0019] The image captured by the optical microscope is an image captured of the same measurement site of the same biological specimen with an azimuth error within 10° from the image captured by the X-ray microscope.

[0020] [3]According to the method described in [1], wherein,

[0021] The selection of the position marker is based on at least one of the size, structure, and type of cells or tissues in the biological specimen, the cell nucleus, and the defect morphology.

[0022] [4]According to the method described in [3], wherein,

[0023] The defect morphology is at least one of cancer, fibrosis, calcification, stone, and deposit.

[0024] [5]According to the method described in [1], wherein,

[0025] The verification of the X-ray microscope image and the optical microscope image is performed by comparing the LM reference section (referred to as the "XRM reference section" and the "LM reference section" respectively) containing the reference area with the index section (referred to as the "XRM index section") containing the index area, and performing azimuth adjustment through rotation operation based on the XRM reference section. Wherein, the reference area is the observation object area containing the position marker that coexists in the X-ray microscope image and the optical microscope image, and the index area is the observation object area containing the position marker that is included in the LM reference section but not in the XRM reference section, and is an area that appears within the upper and lower specified ranges of the XRM reference section.

[0026] [6]According to the method described in [5], wherein,

[0027] The rotation operation is performed by rotating the image of the CT rotation angle and / or tilt angle in the X-ray microscope.

[0028] [7]According to the method described in [1], wherein,

[0029] The verification of the X-ray microscope image and the optical microscope image further includes a correction process for the image of the optical microscope.

[0030] [8]A microscope image processing apparatus, comprising:

[0031] A first determination unit that determines an arbitrary observation object area included in the biological specimen from an X-ray microscope image of a paraffin-embedded biological specimen captured using X-rays with an energy of 4 to 12 keV;

[0032] A second determination unit that acquires an image obtained by photographing a part of the biological sample included in the image photographed by the X-ray microscope with an optical microscope, and determines a region corresponding to the observation target region determined by the first determination unit;

[0033] A unit that checks the information of the region determined by the first determination unit and the information of the region determined by the second determination unit; and

[0034] An output unit that outputs the check result.

[0035] [9] The apparatus according to [8], wherein

[0036] The first determination unit and the second determination unit determine the observation target region from the X-ray microscope image and the optical microscope image through a region extraction process associated with at least one of the size, structure, and type of cells or tissues, cell nuclei, and defect morphology.

[0037]

[10] The apparatus according to [8], wherein

[0038] A unit that makes the observation target region corresponding to the observation target region of the second determination unit by using the position information of the observation target region determined by the first determination unit.

[0039]

[11] The apparatus according to [8], wherein

[0040] The first determination unit and the second determination unit determine the observation target region based on an observation target region designation operation performed by a user, a priority preset for the observation target region, and / or a priority preset by the user.

[0041]

[12] The apparatus according to [8], wherein

[0042] It further has a display unit,

[0043] The output unit outputs information related to the observation target region extracted from the optical microscope image and the X-ray microscope image to the display unit,

[0044] The display unit arranges or overlays and displays the information related to the observation target region.

[0045]

[13] The apparatus according to [8], wherein

[0046] It is provided with a verification unit which, when the observation target area determined by the first determination unit and the second determination unit includes a position marker, verifies the optical microscope image and the X-ray microscope image by azimuth adjustment based on the rotation operation using the position marker.

[0047]

[14] A program product comprising a microscope image processing program for causing a computer to function as the following units:

[0048] A first determination unit which determines an arbitrary observation target area included in a paraffin-embedded biological specimen imaged by X-rays using an energy of 4 to 12 keV from an X-ray microscope image of the paraffin-embedded biological specimen;

[0049] A second determination unit which obtains an image obtained by photographing a part of the biological specimen included in the image photographed by the X-ray microscope with an optical microscope, and determines an area corresponding to the observation target area determined by the first determination unit;

[0050] A unit for verifying the information of the area determined by the first determination unit and the information of the area determined by the second determination unit; and

[0051] An output unit which outputs the verification result.

[0052]

[15] A computer-readable recording medium recording a microscope image processing program for causing a computer to function as the following units:

[0053] A first determination unit which determines an arbitrary observation target area included in a paraffin-embedded biological specimen imaged by X-rays using an energy of 4 to 12 keV from an X-ray microscope image of the paraffin-embedded biological specimen;

[0054] A second determination unit which obtains an image obtained by photographing a part of the biological specimen included in the image photographed by the X-ray microscope with an optical microscope, and determines an area corresponding to the observation target area determined by the first determination unit;

[0055] A unit for verifying the information of the area determined by the first determination unit and the information of the area determined by the second determination unit; and

[0056] An output unit which outputs the verification result.

[0057] Effects of the Invention

[0058] According to the present invention, it is possible to check and precisely match images of the same measurement site of the same biological specimen captured by an X-ray microscope and an optical microscope. The present invention provides images that are precisely aligned in orientation, which are required for a method of practically three-dimensionally observing the same measurement site of the same biological specimen with a spatial resolution at the cellular level by utilizing the complementarity of an X-ray microscope and an optical microscope. By making the orientations of the two images, namely the X-ray microscope image and the optical microscope image, more precisely match, it becomes possible to utilize the complementarity of the X-ray microscope and the optical microscope. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 FIG. is a diagram showing a general specimen preparation process for observing a biological specimen with an X-ray microscope and an optical microscope.

[0060] Figure 2 FIG. is a diagram showing an embedding dish for embedding a biological specimen in wax.

[0061] Figure 3 FIG. is a diagram showing a process of embedding a biological specimen in wax to produce an embedded block.

[0062] Figure 4 FIG. is a diagram showing an embedded block disposed in a cassette.

[0063] Figure 5 FIG. is a diagram showing the shape of a wax contrast box.

[0064] Figure 6 FIG. is a diagram showing a scheme of specimen setting during X-ray microscope measurement.

[0065] Figure 7 FIG. is a diagram showing fixing a cassette with a block to a stage.

[0066] Figure 8 FIG. is a diagram showing the relationship between a biological specimen and the irradiation direction of X-rays.

[0067] Figure 9 FIG. is a diagram showing a position adjustment mechanism for adjusting the position of a stage.

[0068] Figure 10 FIG. is a diagram showing a process of optical microscope observation performed after X-ray microscope observation.

[0069] Figure 11 FIG. is a diagram showing an outline of obtaining position information from a three-dimensional image.

[0070] Figure 12A FIG. is a diagram showing an outline of an orientation adjustment process in an X-ray microscope image and an optical microscope image.

[0071] Figure 12B It is a diagram showing the process of azimuth adjustment.

[0072] Figure 13 It is a diagram showing an overview of the microscope image processing apparatus.

[0073] Figure 14 It is a block diagram showing the operation of the information processing apparatus.

[0074] Figure 15 It is a diagram showing the process of azimuth adjustment of the microscope image processing apparatus.

[0075] Figure 16 It is a diagram showing the process of preparing a wax block for optical microscope observation from a plate-shaped wax block specimen after X-ray microscope observation in a state not combined with an embedding cassette.

[0076] Figure 17 It is a diagram showing a plan for incorporating an X-ray microscope observation process into an organism specimen observation process by an optical microscope.

[0077] Figure 18 It is a diagram showing an overview of obtaining position information from an actual X-ray microscope three-dimensional image using an organism specimen. A: A CT slice edited to have the same viewpoint as optical microscope observation. The position of the renal corpuscle is indicated by an arrow. B: Local magnification of the renal corpuscle. The internal structure required for confirming kidney lesions can be confirmed. C: A locally magnified CT slice of the renal corpuscle obtained by reducing the imaging data to 150°. Essentially the same result as in the case of all data is obtained.

[0078] Figure 19 It is a diagram showing an example of observing the same measurement site of the same organism specimen by an X-ray microscope (upper) and an optical microscope (lower).

[0079] Figure 20 It is a diagram showing the process of designating a characteristic cell nucleus common to an optical microscope (upper) and an X-ray microscope (lower) as a reference region. The position of the cell nucleus used as the reference region is indicated by an ○ mark.

[0080] Figure 21It is a diagram showing a process of designating about 10 index regions, which are within the observation target area containing characteristic cell nuclei in the image of an optical microscope (above), and although the region containing cell nuclei does not exist in the reference section (below) of an X-ray microscope, it exists within about 20 sections before and after. The positions of each region are indicated by ○ marks, the fiducial region is labeled as F, and the marker regions are labeled as M1 to M10. As an example, index section 1 with marker region 1 is given in the center. The positions of the projected marker regions after z-projection of each marker region are shown in the reference section.

[0081] Figure 22 It is a diagram showing the images after the orientation adjustment of an optical microscope (above) and an X-ray microscope (below). The positions of each region are indicated by ○ marks, the fiducial region is labeled as F, and the marker regions are labeled as M1 to M10. In the reference section after orientation adjustment, cell nuclei appear in all the projected marker regions.

[0082] Explanation of Reference Numerals

[0083] 10: X-ray microscope imaging device, 20: communication network, 30: optical microscope imaging device, 40: information processing device, 100: microscope image processing device

[0084] 101: control unit, 102: communication unit, 103: operation unit, 104: display unit, 110: storage unit

[0085] 701: biological specimen, 702: block, 703: cassette, 704: stage, 705: fixing member, 706: cutout region of the cassette

[0086] 90: position adjustment mechanism, 901: support table, 902: adjustment unit. Detailed implementation mode

[0087] The present invention relates to a method for matching and aligning images of the same measurement site of the same biological specimen photographed by an X-ray microscope and an optical microscope. The method of the present invention includes the following processes:

[0088] A process of obtaining an image obtained by photographing a biological specimen embedded in a wax block with an X-ray microscope using X-rays with an energy of 4 to 12 keV;

[0089] A process of obtaining an image obtained by photographing a part of the biological specimen included in the image photographed by the X-ray microscope with an optical microscope; and

[0090] A step of selecting an arbitrary observation target area of the images in the biological sample as a position marker from the obtained X-ray microscope and optical microscope images, and using the position marker to check and match the X-ray microscope image and the optical microscope image.

[0091] When observing the same measurement part of the same biological sample with an X-ray microscope and an optical microscope, the observation target areas in the X-ray microscope image and the optical microscope image do not necessarily match with a precision that allows for easy comparison.

[0092] Therefore, the present invention is characterized in that by checking the optical microscope image and the X-ray microscope image, azimuth adjustment based on the rotation operation of the X-ray microscope image is performed to precisely match the two microscope images.

[0093] The following shows a solution: obtaining an image obtained by photographing a biological sample embedded in a wax block with an X-ray microscope using X-rays with an energy of 4 to 12 keV, and an image obtained by photographing a part of the biological sample included in the image photographed by the X-ray microscope with an optical microscope, selecting an arbitrary observation target area of the two images in the biological sample as a position marker from the two images, and using the position marker to check and precisely match the two images.

[0094] 1. Fabrication of a block for X-ray imaging of a biological sample

[0095] (1) Preparation of a biological sample

[0096] In order to observe the three-dimensional structure of a biological sample with sub-micron resolution using an X-ray microscope, in the present invention, a part of the wax contrast method described in WO2022 / 234844 is modified and used. The "wax contrast method" described in the above publication refers to the following method: using the negative contrast effect generated by replacing water in the sample with wax by impregnating the biological sample with wax such as paraffin to improve the contrast with respect to X-rays, and then embedding the wax-impregnated biological sample in the wax to make a block, and performing X-ray microscope imaging on the embedded biological sample by irradiating the block with X-rays.

[0097] Since the contrast of an unstained biological specimen with respect to X-rays is generally low, it is difficult to directly observe it at the cellular level of spatial resolution by an X-ray microscope. The reason is that in an unstained biological specimen, water, which is its main constituent, and other constituents have the same degree of X-ray transmittance. In the wax-impregnation method, the contrast is improved by replacing the water in the biological specimen with wax having a higher X-ray transmittance, making it possible to observe the specimen by an X-ray microscope at the cellular level. The wax-impregnation method is a method generally called negative contrast.

[0098] In the present invention, a plate-shaped wax block having a very large maximum optical path length of X-rays is used. In addition, the excess wax located around the biological specimen to be examined is not removed. Thus, it is different from the wax-impregnation method described in WO2022 / 234844 in this respect.

[0099] That is, in the method described in WO2022 / 234844, in order to obtain better spatial resolution, it is necessary to remove as much wax as possible around the tissue to be observed from the specimen for X-ray microscope observation, and the maximum optical path length of X-rays is specified to be 2 mm or less. In contrast, in the present invention, although the biological specimen to be observed is embedded in a wax block, the maximum optical path length of X-rays exceeds 2 mm. When the maximum optical path length exceeds 2 mm, although the spatial resolution is slightly worse, by setting the minimum optical path length of X-rays to 2 mm or less, sub-micron spatial resolution can be ensured.

[0100] Therefore, by observing the specimen with an X-ray microscope using the block of the present invention, it is possible to maintain the spatial resolution required for complementary observation with an optical microscope and to significantly improve the adaptability to the conventional biological specimen observation process for optical microscope observation. "Complementary observation" means an observation in which a feature or advantage possessed by one of an optical microscope and an X-ray microscope is mutually compensated with the other microscope that does not have such a feature or advantage or has a lower degree thereof. Therefore, the wax block of the present invention has a biological specimen embedded therein, and the optical path length of X-rays has a minimum value of 2 mm or less and a maximum value exceeding 2 mm. In addition, the maximum value of the maximum optical path length of X-rays of the block is not theoretically limited, but for example, by setting it to 35 mm or less in consideration of the size of a general embedding dish, the adaptability to the biological specimen observation process by an optical microscope can be further improved.

[0101] The preparation of the biological specimen used in the present invention can follow the procedures for preparing a specimen for general optical microscope observation.

[0102] Figure 1 It is a process diagram from cutting out a biological specimen to impregnating the biological specimen into wax such as paraffin.

[0103] In Figure 1 , a specimen (biological specimen) collected by cutting out cells or tissues from a living organism is fixed, for example, chemically fixed using formalin, glutaraldehyde, alcohol, Bouin's solution, etc. Next, dehydration and wax infiltration using alcohol, xylene, etc. are performed. Defatting and decalcification are optional processes and either one or both can be performed as needed. Wax refers to a lipophilic compound that is solid at normal temperature (20°C to 30°C) and has a melting point of 40°C to 80°C. Examples of wax include paraffin wax, other petroleum waxes, synthetic waxes, etc., but paraffin wax is preferred.

[0104] (2) Wax embedding dish for embedding a test biological specimen for wax imaging

[0105] After wax infiltration with paraffin wax or the like, the test biological specimen is embedded.

[0106] In Figure 2 , a schematic diagram of an embedding dish for embedding a test biological specimen is shown.

[0107] Figure 2 As an example, an embedding dish with a rectangular mouth and bottom is shown. It is a vessel with a longitudinal (long side direction) L1 of the inner wall bottom surface of 3 to 32 mm, a transverse L2 of 3 to 28 mm, and a depth d of 2 mm or less.

[0108] However, in the present invention, the shapes of the mouth (upper surface) and bottom of the embedding dish are not limited to rectangles, but can also be squares, circles, or ellipses. In addition, the side wall of the embedding dish can also be a shape that extends in the normal direction from the bottom surface, that is, the shape of a rectangular parallelepiped with the same area of the inner wall bottom surface and the inner wall upper surface, or a shape with a larger area of the inner wall upper surface than the inner wall bottom surface, that is, a shape in which the inner wall bottom surface narrows when viewed from the upper surface (the extension line of the side wall going to the bottom surface is conical).

[0109] Although the depth d of the embedding dish is 2 mm or less, it is preferably 1.0 to 1.5 mm, for example, 1.2 mm. The depth of this embedding dish becomes the thickness of the block in which the biological specimen is embedded.

[0110] The area of the bottom surface of the inner wall of the embedding dish, that is, the area of the surface of the block in which the biological specimen is embedded that is profiled by the bottom surface of the inner wall of the embedding dish (defined as the upper surface of the wax block) is 9 to 900 mm 2 . However, these areas are not limited to the above. Those skilled in the art can appropriately set the size of the block for X-ray microscopy imaging.

[0111] Examples of the material of the embedding dish include stainless steel or other metals, glass or ceramics, hard or soft resin, paper or wood, etc. However, it is not limited to these materials and can also be other materials.

[0112] As Figure 3 shown, by using an embedding dish with a depth of 2 mm or less to place the biological specimen, and injecting wax into the embedding dish, a plate-shaped wax block with a thickness of 2 mm or less embedding the test biological specimen is made.

[0113] For example, place the biological specimen approximately in the center of the embedding dish ( Figure 3 a and b), inject wax, and solidify the wax. By taking out the solidified wax, a block embedding the biological specimen (hereinafter also simply referred to as "block") can be made ( Figure 3 c).

[0114] However, the order of placing the biological specimen in the embedding dish and injecting wax is not limited to the above and is arbitrary. That is, the biological specimen can also be contained in the wax in advance, and the biological specimen and the wax can be injected into the embedding dish together. Or, the embedding dish can be filled with wax in advance, and the biological specimen can be placed therein. The biological specimen placed in the embedding dish can be appropriately pre-adjusted in advance so as to be located in the center of the block before the wax solidifies. In the present invention, the biological specimen can also be suspended in the wax, but it is preferably pre-placed in contact with the bottom of the embedding dish.

[0115] Moreover, by placing an X-ray imaging cassette (referred to as "embedding cassette") on the embedding dish embedding the biological specimen before the wax solidifies, and solidifying the wax in a state where the surface of the wax injected into the embedding dish is in contact with the embedding cassette, a combination of the embedding cassette and the embedded block can be made later by taking out the block from the embedding dish ( Figure 4 ).

[0116] 2. Embedding cassette for wax imaging of the test biological specimen

[0117] In the present invention, the embedding cassette for placing the block embedding the test biological specimen is as Figure 5It has a strip-shaped or rectangular cutout (space) that allows X-rays to pass through when irradiated with X-rays. The block is arranged in the cassette in such a way that the region to be irradiated with X-rays, i.e., the embedded biological specimen, is located within the cutout region. The size of the cutout of the cassette is larger than the imaging part of the biological specimen to be examined. In other words, the imaging part of the biological specimen is sized to fit within the area of the cutout of the cassette. Also, the above-mentioned cutout is designed such that when performing X-ray imaging using the cassette, even if the loss of the rotation angle range due to the cassette blocking the X-rays is less than 30°, i.e., the imaging rotation angle range becomes 150° or more, the X-rays will not be blocked. The details of the imaging rotation angle range will be described later. In addition, although the biological specimen is arranged in the embedding block such that the X-ray imaging angle range of the X-ray microscope in the embedding dish satisfies 150° or more, the X-ray blocking factors can consider not only the blocking of X-rays by the cassette but also other factors such as the thickness of the wax itself that embeds the biological specimen blocking the X-rays.

[0118] Figure 6 It is a diagram showing that the embedding block is arranged such that the biological specimen is located within the cutout region of the cassette and is disposed in the irradiation direction of the X-rays of the X-ray microscope. As Figure 6 shown, the X-rays are irradiated from the X-ray generating device of the X-ray microscope toward the block, and the X-rays pass through the cutout of the cassette (the cutout formed by cutting through in a strip shape in Figure 6 ) and irradiate the biological specimen in the block in the cassette.

[0119] Figure 7 It is a diagram showing the cassette with the block fixed to the stage.

[0120] In Figure 7 , the block 702 embedding the biological specimen 701 is installed in the cassette 703, and its position is set such that the biological specimen 701 is within the range of the cutout region 706 of the cassette.

[0121] That is, the biological specimen is arranged within the cutout such that the imaging angle range described later satisfies 150° or more. Also, the cassette is fixed to the stage such that the block is on the detector side (the side opposite to the side where the X-rays enter).

[0122] In Figure 7 , the long side direction of the biological specimen is in the x direction (horizontal), but it can also be set in the z direction (vertical and horizontal). In this case, the cutout of the cassette can be arranged facing upward as long as the biological specimen 701 is within the horizontally long cutout region, and the long side direction of the biological specimen can also be arranged longitudinally (z direction).

[0123] The cassette 703 is fixed to the stage 704 via the fixing member 705. AsFigure 7 As shown, the fixing member 705 can be provided with an embedding jig having a recess on the stage, and the end portion of the cassette is embedded and fixed to the embedding jig. In Figure 7 , the two ends of the cassette 703 are fitted to the fixing member 705, but the member for fixing the cassette to the stage is not particularly limited, and it can also be attached and fixed with double-sided tape or clay (not shown), or fixed by clamping or screwing. In the case of using a clip, it is fixed by utilizing the elasticity of an object such as a leaf spring. In the case of using screwing, it is fixed by utilizing a set screw or the like.

[0124] After the cassette 703 is fixed to the stage 704, by the position adjusting mechanism of the stage 704 (details will be described later), it can be moved in the left-right or front-back direction (x or y direction) or the up-down direction (z direction) so that the approximate center portion of the biological sample is located on the irradiation path (optical path) p1 of the X-ray. In addition, the irradiation path p1 of the X-ray is in the normal direction with respect to the block 702, but when a rotation axis q1 is provided at the approximate center portion of the biological sample 701 in the normal direction with respect to the xy plane and rotated around the rotation axis q1, p1 becomes a direction inclined with respect to the block 702. That is, the optical path of the X-ray becomes a direction inclined with respect to the block 702. The rotation angle θ at this time is the X-ray imaging rotation angle range, and it can be rotated to 150° or more, preferably 180° or more.

[0125] The relationship between the biological sample 701 and the irradiation direction of the X-ray is shown in Figure 8 .

[0126] In Figure 8 , the left panel shows the minimum and maximum optical path lengths of X-ray imaging in a conventional test example. Conventionally, since X-ray imaging was performed on a biological sample from which the surface wax had been removed after wax imaging, the maximum optical path length and the minimum optical path length were substantially the same as the thickness or length of the biological sample.

[0127] In contrast, in the present invention, since X-rays are irradiated on a plate-shaped wax block in which a biological sample after wax imaging is embedded, the optical path length depends on the length or thickness of the block. Even when the thickness of the block is 2 mm or less, when rotated by a certain amount of the angle θ from the rotation axis of the sample, the optical path becomes as long as a length close to the long side direction of the block (test example of the present invention; Figure 8(right image). In the present invention, the minimum value of the X-ray optical path length of the block is 2 mm or less, and the maximum value is set to a value greater than 2 mm. Here, although there is no theoretical limit to the maximum value of the X-ray maximum optical path length of the block, considering the size of a general embedding dish, it can be 35 mm or less. In addition, for the rotation of the specimen, it is possible to fix the specimen and rotate the X-ray source and the camera relative to the specimen, or to fix the X-ray source and the camera and rotate the specimen. Further, the rotation axis (direction) can be the long side direction of the specimen or the short side direction of the specimen.

[0128] Figure 9 Fig. 90 shows a position adjustment mechanism for adjusting the xy direction (horizontal direction), z direction (height direction), and θ (rotation angle) of the stage 704.

[0129] The position adjustment mechanism 90 is a mechanism for aligning the position of an observation object for optical microscope observation, and includes a support table 901 on which the stage 704 is placed, and an adjustment unit 902 for adjusting the xy direction (horizontal direction), z direction (height direction), and the horizontal rotation angle of the support table 901. It is also possible to provide a protrusion (not shown) on the bottom surface of the stage 704, and a fixing hole 903 for embedding and fixing the protrusion on the support table 901, etc. The adjustment unit 902 includes an adjustment knob 902a for adjusting the x direction of the stage 901, an adjustment knob 902b for adjusting the y direction of the stage 901, an adjustment knob 902c for adjusting the z direction of the stage 901, and a knob 902d for adjusting the horizontal rotation angle of the support table 901. In addition, when referring to the x direction, y direction, z direction, and rotation angle, it is of course also possible to adjust the negative direction (for example, if the x direction is to the right, then to the left, and the same applies to others). Further, the position adjustment mechanism 90 can be set not only to be manual but also to be an electric mechanism that can be automatically controlled by a computer program or the like.

[0130] 3. X-ray imaging

[0131] In order to perform wax imaging, X-rays with an energy of 4 to 12 keV are required (for example, using the X-ray microscope nano3DX of Rigaku Corporation with X-rays of Cu wavelength), and the specimen size is adjusted so that the X-rays can fully penetrate the specimen. Therefore, in the present invention, a plate-shaped wax block with a thickness of 2 mm or less embedding the test biological specimen is set so that the block plate surface is aligned with the rotation axis of the X-ray microscope to perform X-ray imaging. As described above, the block is mounted on the cassette of the present invention, and then the cassette is fixed to the stage with a fixing member or a detachable adhesive material (such as double-sided tape or clay) to perform X-ray imaging. However, in order to improve the setting accuracy and reproducibility, it is also possible to use a jig with an adjustment mechanism designed to assemble the cassette to the observation stage so that the specimen height and tilt angle are constant.

[0132] In the present invention, X-ray microscopy can also be performed on a bare plate-shaped wax block not combined with an embedding cassette. In this case, the plate-shaped wax block after X-ray microscopy is brazed to the upper surface of a wax block that does not contain a biological sample and is combined with a general embedding cassette without a notch ( Figure 16 ). The combination of the two formed by setting a wax block that does not contain a biological sample in a cassette without a notch is called a two-piece combination. In the two-piece combination, the wax constituting the wax block that does not contain a biological sample is preferably the same as the wax constituting the plate-shaped wax block, but it can also be another similar wax. "Brazing" means that by using the wax after being heated and melted, which is called brazing wax, as an adhesive, the plate-shaped wax block is bonded to the wax block that does not contain a biological sample, and then the brazing wax is cured by standing at room temperature to combine the two blocks. By brazing the wax block of the biological sample embedding after X-ray microscopy on the upper surface of the wax block in the two-piece combination, a combination of the two-piece combination and the wax block of the biological sample embedding can be obtained. This combination is called a three-piece combination. The wax of the brazing wax when making the three-piece combination is preferably the same as the wax constituting the plate-shaped wax block or the wax block that does not contain the biological sample, but it can also be another similar wax. The wax block after brazing can be treated in the same way as the biological sample that is wax-embedded and combined with a general embedding cassette. The plate-shaped wax block contained in the wax block after brazing remains in a solid state throughout the brazing process and maintains the corresponding relationship of the orientation before and after brazing. Therefore, the position information of the plate-shaped wax block is roughly preserved, and in addition, the deformation of the embedded biological sample is suppressed. Thereafter, optical microscopy is performed according to the procedures described below.

[0133] 4. Acquisition of position information and observation by optical microscope

[0134] Figure 10 It is a diagram showing the process from the end of X-ray microscope photography to the optical microscope photography.

[0135] After the photography by the X-ray microscope, the position information that is desired to be evaluated in detail by the optical microscope is obtained from the photographed three-dimensional image.

[0136] Figure 11 It is a diagram showing an overview of obtaining position information from a three-dimensional image.

[0137] In Figure 11In this case, Picture A shows a biological specimen embedded in a wax block. The three-dimensional image after X-ray imaging is displayed in such a way that the two-dimensional images of the cross-sections obtained by cutting the wax block containing the biological specimen in an arbitrary plane are continuously stacked in a direction perpendicular to the cutting plane, i.e., CT slices. Here, an arbitrary position of the biological specimen is set as the reference plane. For example, the plane in contact with the upper end of the biological specimen can be set as the reference plane. In this case, for example, when viewed from the side of the upper surface of the block (the surface formed by a1 - a2 - a3 - a4 in Figure 11 ), if the plane U (in Figure 11 is a plane that contains the upper end of the biological specimen and is parallel to the plane containing the upper surface of the block, and is separated from the plane formed by a1 - a2 - a3 - a4 by a distance p and is in contact with the upper end of the biological specimen (the surface formed by b1 - b2 - b3 - b4)) is used as the reference plane, and cutting is continuously performed with planes parallel to this reference plane (in the xy direction in the figure), a series of CT slice images (Picture B) of the biological specimen from the upper end to the lower end can be obtained. Since the thickness of one CT slice can be appropriately set, the number of CT slice images that can be obtained from a biological specimen of a specified size is determined. Therefore, the region to be observed (observation target region) is searched for among this series of CT slice images. For example, in Figure 11 Picture B, since the thickness of one slice can be arbitrarily set, the distance d n from the position (reference image) d0 of the reference plane U of the biological specimen to the nth image can be obtained. Therefore, when performing optical microscope observation, the biological specimen is cut with a microtome until the reference plane of the biological specimen appears, and then thin sections are cut at a thickness in micrometers from this point until d n is reached. After that, stretching, dewaxing, and staining are performed according to the procedure shown in Figure 10 , and optical microscope imaging is performed. In addition, in Figure 11 , the plane that contains the upper end of the biological specimen and is parallel to the plane containing the upper surface of the block when viewed from the side of the upper surface of the block is used as the reference plane, but it is not limited to this. Any plane, such as the plane containing the lower end of the biological specimen or the plane containing the upper and lower surfaces of the block, can also be used as the reference plane. In addition, the reference plane can be a plane parallel to the plane containing the surface of the block, or a plane not parallel to the plane containing the surface of the block.

[0138] In the stretching process, the thinly cut slice is picked up with forceps, floated on warm water and stretched, and then fished out and adhered to a slide glass.

[0139] In the dewaxing process, the slide glass with the attached slice is dried and immersed in xylene or the like for dewaxing treatment.

[0140] In the staining process, methods such as HE (hematoxylin-eosin) staining that differentially stains cell nuclei and other tissues, and PAS reagent (periodic acid Schiff’s reagent) staining that clearly stains the basement membrane are used to perform staining corresponding to the observation purpose.

[0141] In the optical microscope photographing process, in accordance with the position information obtained by photographing various stained biological specimens with an X-ray microscope, various stained biological specimens are photographed with an optical microscope to obtain digital images.

[0142] For example, regarding the acquisition of position information, as Figure 18 shown in A, if the X-ray microscope image is resliced in a direction perpendicular to the surface of the wax block (changing the slice direction of the CT image; for example, using the reslice tool of ImageJ) and the position of the upper end of the specimen is set as the reference (0.0 μm), then the exact depth of any object in the specimen can be specified. In Figure 18 the example of B in Figure 18 and C in

[0143] the renal corpuscle 1 is located at a position 115.8 μm from the upper end of the specimen. Regarding the observation with an optical microscope, in order to perform an optical microscope observation on the specimen after X-ray imaging, the specimen is sectioned with a microtome by a general method without re-embedding the wax, stained by a general method, and observed with a general optical microscope.

[0144] As a result, X-ray microscope images and optical microscope images of the same measurement site of the same biological specimen can be easily obtained in a form with substantially the same orientation. The orientation error between the two images in this state is, for example, within 10°, but is not limited thereto, and may also be 10° or more depending on the required precision. Figure 18 In the example of B in Figure 18 and C in

[0145] 5. Verification of X-ray Microscope Images and Optical Microscope Images

[0146] In the case where X-ray microscope images and optical microscope images with roughly the same orientation are obtained as input images in the initial state, by adjusting the orientation of the X-ray microscope images using the following method, the two can be checked to fit precisely.

[0147] (1) Selection of position markers for any observation target area in the biological sample

[0148] First, select any observation target area from the images of the biological sample captured by the X-ray microscope and the optical microscope, which is included in the biological sample and can be used as a position marker for orientation adjustment. "Image" refers to both the data of the image itself (data displayed in pixels, etc.) and numerical data such as coordinates. The form of the arbitrary observation target area is not limited. For example, the size, structure, and type of cells or tissues, cell nuclei, and defect forms can be selected as position markers.

[0149] The defect form is any one or a combination of cancer, fibrosis, calcification, stones, and other deposits. In the embodiment, the cell nucleus is used as the position marker, one reference area serving as the reference for orientation adjustment is selected, and ten index areas for evaluating the degree of fit are selected. In the embodiment, the position marker is selected manually, but it can also be automated using artificial intelligence (AI), etc.

[0150] (2) Process of checking the X-ray microscope image and the optical microscope image to make them fit precisely

[0151] According to the flowchart of orientation adjustment (detailed content will be described later), it is carried out by rotating the image of the CT rotation angle and / or tilt angle in the X-ray microscope (using the rotation tool of ImageJ, etc.).

[0152] (3) Flowchart of orientation adjustment

[0153] The process of checking the X-ray microscope image and the optical microscope image to make them fit precisely will be further described in detail.

[0154] Consider the case of making the optical microscope (LM) image fit precisely with the X-ray microscope (XRM) image. In Figure 12A (a) is a three-dimensional schematic diagram of the LM image, and (b) is a three-dimensional schematic diagram of the XRM image. The diagram of the orientation adjustment process is shown in Figure 12B

[0155] (3-1)

[0156] First, an observation target region including a position marker (such as a cell nucleus) that exists in both the LM image and the XRM image is found, and this region is set as the fiducial region (step S1). The slice in which the fiducial region exists is set as the "fiducial slice". In addition, multiple position markers may be included in the fiducial region.

[0157] (3-2)

[0158] Next, several regions are selected from the observation target regions including the position markers (such as cell nuclei) that exist in the LM image, where these regions do not exist in the fiducial slice of the XRM image but exist within approximately 20 slices before and after (step S2). This region is called the "marker region", and the slice in which the marker region exists is called the "marker slice". In the present invention, the number of selected marker regions is not limited, but 8 to 20 regions are selected, preferably around 10 regions. In addition, multiple position markers may be included in one marker region.

[0159] In Figure 12A , (b) shows a scheme in which the XRM image is offset by an amount of an angle θ1 counterclockwise from the x coordinate or y coordinate of the fiducial slice plane centered on the fiducial region of the fiducial slice, and by an amount of an angle θ2 counterclockwise centered on the y axis passing through the center of the fiducial region. In the present invention, θ1 is called the CT rotation angle, and θ2 is called the tilt angle. In addition, the CT rotation angle can be a clockwise rotation angle or a counterclockwise rotation angle with respect to the rotation axis. Also, the rotation axis of the tilt angle can be the x axis or the y axis.

[0160] When offset by the angles θ1 and θ2, the position markers in the regions other than the fiducial region of the LM image become invisible in the XRM image, but exist in several slices above and below the fiducial slice (for example, within 10 or 20 slices above and below).

[0161] Figure 12A shows a scheme in which three regions (M1, M2, and M3) that exist in the LM image are selected as marker regions including position markers (such as cell nuclei) in the XRM image. In addition, for simplicity of explanation, the number of marker regions is set to three. In Figure 12A 's (b), the slice in which the marker region M1 exists is set as the marker slice 1, and the slices in which the marker regions M2 and M3 exist are set as the marker slice 2.

[0162] Next, when the index regions are z-projected onto the reference slice, the index region M1 of the index slice 1, and the index regions M2 and M3 of the index slice 2 are projected onto the reference slice ( Figure 12A of (c), step S3). In addition, when selecting the index regions, it is only necessary to ensure that when the index regions are z-projected onto the reference slice, the projected index regions (the projected index regions) are as evenly distributed as possible over the entire reference slice.

[0163] (3-3)

[0164] The projected index regions are virtual entities. Since the actual XRM image is offset by amounts θ1 and θ2 when viewed from the reference slice of the LM image, there are actually no position markers in the projected index regions of this reference slice. Therefore, azimuth adjustment is performed using a rotation tool such as ImageJ (step S4) in such a way that position markers appear in the projected index regions of the reference slice. Step S4 includes steps S5 to S10. The azimuth adjustment is performed using the following method.

[0165] First, calculate the rotation angle such that position markers appear in the projected index regions of the reference slice. When the index regions are added, additional calculations are only performed for this part. When the distances in the x and y directions (in pixel units) between the reference region and the projected index region for a certain one index region are set as dx (=x_index - x_reference) and dy (=y_index - y_reference), and the distance between the reference slice and the index slice is set as dz (=z_index - z_reference),

[0166] I)

[0167] Select the axis among the x or y axes that is closer to being parallel to the line connecting the projected index region and the reference region (the selection order is the same), and calculate the z-angle such that the axis and the line become parallel (step S5).

[0168] z-angle = -tan -1 (dy / dx), or

[0169] z-angle = tan -1 (dx / dy)

[0170] For example, in Figure 12A of (c), since the axis closer to being parallel to the line connecting M1 and the reference region is the x-axis, the x-axis is selected, and thus the z-angle is calculated using the first of the above two equations.

[0171] II)

[0172] Next, calculate the y-angle or x-angle (the following formula) such that a position marker appears in the projection index region of the reference slice (step S6). For example, the angle θ2 (the tilt angle around the x-axis or y-axis) is adjusted so that a cell nucleus or the like selected as the position marker included in the index region appears on the reference slice. That is,

[0173] y-angle = {if dx > 0 tan -1 (dz / sqrt(dx 2 + dy 2 ))

[0174] else -tan -1 (dz / sqrt(dx 2 + dy 2 ))}, or

[0175] x-angle = {if dy > 0 -tan -1 (dz / sqrt(dx 2 + dy 2 ))

[0176] else tan -1 (dz / sqrt(dx 2 + dy 2 ))}

[0177] For example, since the x-axis is selected for M1 in (c) of Figure 12A , the y-angle is calculated by the first of the above two formulas. The sign is determined by the value of dx.

[0178] (3 - 4)

[0179] Moreover, select a pair among the combinations of two projection index regions that are in a relationship where the angle sandwiching the reference region is closer to a right angle (step S7).

[0180] For example, in (c) of Figure 12A , select the pair that is in a relationship closer to a right angle among (i) the angle formed by M1 - reference region - M2, (ii) the angle formed by M2 - reference region - M3, and (iii) the angle formed by M1 - reference region - M3. If the angle of (i) is 89°, the angle of (ii) is 80°, and the angle of (iii) is 180°, then the pair in the relationship closer to a right angle is the pair of M1 and M2 in (i).

[0181] As an example, in Figure 21 , in the pair of M4 and M9, the angle formed by M4 - F - M9 is 87.1°, which is close to a right angle, so this pair is selected.

[0182] Calculate the average z-angle (in the following formula) in the selected pair of projection index regions (step S8). That is,

[0183] z-angle = (z-angle_1 + z-angle_2) / 2

[0184] As an example, in Figure 21 the z-angles of M4 and M9 are 40.6° and 37.6° respectively. Therefore, the average z-angle is 39.1°. Conversely, by selecting pairs of index regions with z-angles close to each other, pairs in a nearly orthogonal relationship can be found. When additional index regions are added, in order to improve the degree of coincidence, pairs of index regions with a more nearly orthogonal relationship can also be reselected based on the z-angle.

[0185] (3 - 5)

[0186] Input the values of x-angle, y-angle, and z-angle for the pair of projection index regions selected in the above (3 - 4) into the TransformJ Rotate plugin of ImageJ and perform a rotation operation (the following command) (step S9). That is,

[0187] run(“TransformJ Rotate”, “z-angle = z-angle y-angle = y-angle x-angle = x-angle interpolation = Linear background = 0.0 adjust resample anti-alias”);

[0188] For example, in Figure 21 the z-angle (average) is 39.1°, the y-angle (M4) is 2.58°, and the x-angle (M9) is 2.62°. Therefore, input these values.

[0189] (3 - 6)

[0190] If necessary, search for x-angle and y-angle around the current orientation (step S10).

[0191] (3 - 7)

[0192] If the number of coincidences is above the evaluation criterion, end the orientation adjustment. If the number of coincidences is less than the evaluation criterion, return to the process of (3 - 2) (step S2) and add index regions.

[0193] (3 - 8) Evaluation of orientation adjustment by the number of coincidences of index regions

[0194] When the result of the orientation adjustment according to the above flow chart is that a position marker appears in the projection index region of a certain index region on the reference slice, for this index region, it is evaluated that the images of the X-ray microscope and the optical microscope match. The evaluation criterion for the orientation adjustment is set to 6 to 8 for the number of matching index regions. That is, when 6 or more, 7 or more, or 8 or more index regions match, the orientation adjustment is ended, but preferably 8 or more match. In addition, among the 3 cases where 8 or more index regions match and the orientation adjustment is successful, the average residual of x-angle or y-angle of these matching index regions is 0.21° to 0.34°. Therefore, the precision of the precise matching obtained by the method of the present invention can be considered to be approximately within 0.5°.

[0195] In addition, the optical microscope image contains deformations caused by sample preparation, and sometimes local structural differences occur between the X-ray microscope image and the optical microscope image. Therefore, the verification of the X-ray microscope image and the optical microscope image can also include a correction process of the image data of the optical microscope (such as Rigid Registration using Fiji).

[0196] Software for automating the above flow chart can also be designed.

[0197] 6. Microscope Image Processing Apparatus, Program, and Recording Medium

[0198] The present invention also provides a microscope image processing apparatus and a computer program for microscope image processing. In addition, the present invention provides a program product including the program.

[0199] The apparatus of the present invention includes:

[0200] A first determination unit that determines an arbitrary observation target region included in a biological sample embedded in a wax block imaged by an X-ray using X-rays with an energy of 4 to 12 keV;

[0201] A second determination unit that obtains an image obtained by photographing a part of the biological sample included in the image photographed by the X-ray microscope with an optical microscope, and determines a region corresponding to the observation target region determined by the first determination unit;

[0202] A unit that verifies the information of the region determined by the first determination unit and the information of the region determined by the second determination unit; and

[0203] An output unit that outputs the verification result.

[0204] The device of the present invention is used for the analysis of biological specimens.

[0205] In addition, the program of the present invention is a microscope image processing program for causing a computer to function as the following units:

[0206] A first determination unit that determines any observation target region included in the biological specimen from an X-ray microscope image of a paraffin-embedded biological specimen taken using X-rays with an energy of 4 to 12 keV;

[0207] A second determination unit that obtains an image obtained by photographing a part of the biological specimen included in the image photographed by the X-ray microscope with an optical microscope, and determines a region corresponding to the observation target region determined by the first determination unit;

[0208] A unit that checks the information of the region determined by the first determination unit and the information of the region determined by the second determination unit; and

[0209] An output unit that outputs the check result.

[0210] In addition, the program and program product of the present invention are used for the analysis of biological specimens.

[0211] Figure 13 The outline of the microscope image processing device 100 provided in the present invention is shown. In Figure 13 it, the microscope image processing device 100 includes an X-ray microscope photographing device 10, an optical microscope photographing device 30, and an information processing device 40. As Figure 13 shown, these devices can be connected via a communication network 20. In addition, the X-ray microscope photographing device 10, the optical microscope photographing device 30, and the information processing device 40 can be independently provided without being connected to the network. In this case, an independent information processing device can take in only necessary information such as image data for processing.

[0212] The X-ray microscope photographing device 10 writes basic information such as patient information, examination information, and image ID from which the specimen is derived into the header of the image file in accordance with, for example, the DICOM (Digital Imaging and Communications in Medicine) standard, and sends it to the information processing device 40. In the present invention, the X-ray microscope photographing device 100 preferably uses nano3DX (Rigaku Corporation).

[0213] The optical microscope imaging device 30 is a microscope commonly used such as a stereomicroscope, an inverted microscope, a metallurgical microscope, etc., but it can also be a digital microscope equipped with a high-resolution digital camera to replace the eyepiece. By using the digital microscope, the captured microscope image can be saved as a digital image. In this case, the optical microscope image information is sent to the information processing device 40.

[0214] The information processing device 40 is a computer device that analyzes the images sent from the X-ray microscope imaging device 10 and the optical microscope imaging device 30, assists in the inspection of biological specimens, and outputs their image information. When an X-ray microscope image is obtained by irradiating a biological specimen with X-rays and taking a picture, the information processing device 40 obtains the position information of the biological specimen for observation with an optical microscope based on an arbitrary position of the biological specimen. When the position information is obtained, the information processing device outputs the position information.

[0215] As Figure 14 shown, the information processing device 40 includes a control unit 101, a communication unit 102, an operation unit 103, a display unit 104, a storage unit 110, etc.

[0216] The control unit 101 is composed of a CPU, a ROM, a RAM, etc., and comprehensively controls the processing operations of each part of the information processing device 40. Specifically, the CPU reads various processing programs stored in the ROM and expands them into the RAM, and performs various processes in cooperation with this program.

[0217] In addition, the information processing device 40 has a storage unit 110 provided in the ROM or RAM. The storage unit 110 includes an X-ray microscope information database (DB) 111, an optical microscope information DB 112, and an accessory information DB 113. The accessory information DB stores the tissue name, ID, specimen collection date, reference plane, slice thickness, reference area, index area, etc. of the person or animal from which the biological specimen is derived.

[0218] The control unit 101 performs the process of obtaining the X-ray microscope image information and the optical microscope image information, and functions as an image display to a monitor, etc. (not shown) in cooperation with the display unit 104.

[0219] The communication unit 102 is composed of a network interface, etc., and performs data transmission and reception between the X-ray microscope imaging device or the optical microscope imaging device connected via a communication network and external devices connected to the information processing device.

[0220] The operation unit 103 is a keyboard equipped with various input keys and function keys, as well as a mouse, etc., and outputs the operation signals input through key operations on the keyboard or mouse operations to the control unit 101. In addition, the operation unit 103 can also be constituted by a touch panel. In this case, operation signals are output to the control unit 101 according to the touch operations performed by the user.

[0221] The display unit 104 is configured to include a monitor such as an LCD (Liquid Crystal Display), and displays various screens according to the instructions of the signals input from the control unit 101.

[0222] Figure 15 It is a flowchart showing the orientation adjustment process performed by the microscope image processing apparatus of the present invention.

[0223] First, the control unit 101 of the information processing apparatus 40 receives the input of the X-ray microscope image and uses it as the object image 1. The control unit 101 determines the observation object region (first determination unit) that can be used as a position marker for orientation adjustment included in the biological specimen from the object image 1. The control unit 101 performs image analysis processing based on this object image (step S11). The image analysis processing is executed through the cooperation of the control unit 101 and the program stored in the storage unit 110.

[0224] Next, the position information to be evaluated in detail with an optical microscope is extracted from the object image 1 (step S12).

[0225] As Figure 11 shown, since the thickness of one slice can be arbitrarily set, the control unit 101 calculates the distance d from the position (d0) of the reference plane U (reference image) of the biological specimen to the nth image n . The thickness of the slice can be input by the user or preset.

[0226] Next, the control unit 101 receives the input of the optical microscope image and uses it as the object image 2 (step S13).

[0227] The control unit 101 selects the observation object region that can be used as a position marker for orientation adjustment from the object image 1 and the object image 2 (step S14). That is, the control unit 101 determines the observation object region corresponding to the observation object region determined by the first determination unit from the image obtained by photographing a part of the biological specimen included in the image photographed by the X-ray microscope with the optical microscope (second determination unit).

[0228] The selection of the morphology of the observation object area can also enable the morphology recognizer to learn the morphology in advance, automatically select a morphology that matches or is similar to the morphology, or accept the input of the morphology selected by the user. In step S14, for example, for each slice of the microscope image, image feature quantities such as concentration, shape, texture, and multi-resolution features are extracted in units of pixels. One or more image feature quantities can be extracted.

[0229] The first and second determination units determine the observation object area based on the observation object area designation operation performed by the user, the priority preset for the observation object area, and / or the priority preset by the user. "Priority" refers to the order for determining the observation object area, indicating from which observation object area to determine in sequence. For example, in the case of selecting the observation object area as a position marker for azimuth adjustment of an X-ray microscope image of a paraffin-embedded renal biopsy specimen, the cell nucleus becomes the first priority, and the granular defect morphology (such as fibrosis) becomes the second priority.

[0230] In addition, the azimuth error between the optical microscope image and the X-ray microscope image is set to within 10°, but it is not limited to this. Depending on the required precision, it can also be 10° or more.

[0231] Next, in order to check the information of the observation object area determined by the first determination unit and the information of the observation object area determined by the second determination unit, the control unit 101 executes Figure 12B the azimuth adjustment process (step S15) shown.

[0232] The control unit 101 selects a reference area and a plurality of index areas from the observation object area including the position marker according to Figure 12B the azimuth adjustment process shown.

[0233] During the operation of step S15, when there is an input from the user to change the designation of the reference area and the index areas, or when an index area is added, the control unit 101 calculates the CT rotation angle θ1 and tilt angle θ2 of the index area from the XRM image and displays them on the display unit 104 (step S16).

[0234] Next, the control unit 101 calculates the projected index area after z-projecting the index area onto the reference slice, and selects a specific area from the multiple projected index areas. For example, as Figure 12AAs shown in (c) of FIG. , when three projection index regions (M1, M2, and M3) are selected, two straight lines formed by one projection index region - reference region - another projection index region are extracted, and the angle formed by these straight lines is calculated to determine which combination of projection index regions is close to a right angle, and the candidate projection index region pairs are displayed (step S17).

[0235] Next, the control unit 101 displays the shape of the projection index region (step S18). Then, the control unit 101 calculates x-angle, y-angle, and z-angle as the rotation angles for azimuth adjustment (step S19).

[0236] The control unit 101 inputs the values of x-angle, y-angle, and z-angle for a pair of projection index regions to the storage unit 113 and performs a rotation operation, or displays on the display unit 104 the rotation angles for the user to perform a rotation operation (step S20). If, as a result of the azimuth adjustment, a position marker appears in the projection index region of the reference slice for a certain index region, for that index region, it is evaluated that the images of the X-ray microscope and the optical microscope match. As needed, a search for x-angle and y-angle is performed around the current azimuth (step S21). If the number of matching index regions is equal to or greater than the evaluation criterion, the azimuth adjustment ends. If the number of matches is less than the evaluation criterion, an additional index region is added and the process returns to step S16. The default value of the evaluation criterion is set to 8, and it can be appropriately changed within the range of 6 to 8.

[0237] The program of the present invention can be stored in a computer-readable recording medium. Therefore, a computer-readable recording medium recording the program of the present invention is also included in the present invention. Examples of the recording medium or storage unit include magnetic media (such as floppy disks, hard disks, etc.), optical media (such as CDs, DVDs, etc.), magneto-optical media, flash memories, etc., but are not limited to these.

[0238] Examples

[0239] Hereinafter, the present invention will be further specifically described by way of examples. However, the scope of the present invention is not limited to these examples.

[0240] [Example 1]

[0241] Method

[0242] An experiment was conducted on a mouse kidney biopsy simulation specimen after wax (paraffin) imaging to determine whether the same measurement site of the same biological specimen can be observed by an X-ray microscope and an optical microscope. The experiment used the kidneys of nephropathy model mice created by the partial kidney removal method described in the literature (Kunishima et al. (2022) Scientific Reports 12, 9436). Subsequently, a wax-imaging specimen (wax block) was created from the kidneys of the nephropathy model mice using the method described in this literature. Then, an X-ray microscope observation specimen was created from this wax block using the direct mounting method described in this literature. The specimen was assembled into an X-ray microscope nano3DX (Rigaku Corporation) (CCD detector), and X-ray projection image data was taken under the conditions of Cu-target (40 kV / 30 mA), L0270-bin1-XD2 (0.27 μm / voxel), step scan (one exposure for 60 s, a total of 3400 exposures) (the required time was 57.7 hours).

[0243] After processing the projection image data with ring artifact correction, drift correction, and phase recovery (Paganin method; δ / β = 100), CT reconstruction was performed using software based on the general FBP algorithm. Moreover, the mouse kidney biopsy simulation specimen after X-ray microscope observation was re-embedded in a wax (paraffin) block, and it was continuously sectioned into 4-μm-thick slices using a microtome. An optical microscope observation specimen was created by staining with PAS reagent (periodic acid Schiff’s reagent), and observation was performed using an optical microscope for continuous section observation Nano Zoomer C9600-03 (Hamamatsu Photonics). The rough alignment of the X-ray microscope image and the optical microscope image was manually performed using general display software (ImageJ and Drishti). Subsequently, alignment was performed according to the alignment process of the present invention.

[0244] The alignment of the present invention was performed using ImageJ in the order described in (3-1) to (3-8).

[0245] Results:

[0246] As Figure 19 、 Figure 20As shown, one position marker, i.e., a characteristic cell nucleus, that exists in both the LM (light microscope) image and the original XRM (X-ray microscope) image was found, and it was set as the reference region. The sections of the XRM image and the LM image containing the reference region were respectively set as the XRM reference section and the LM reference section. In this example, the cell nucleus (the 481st out of 1791) in the blood vessel within the renal corpuscle of the observation object was set as the reference region (arrow in the figure). Next, as Figure 21 shown, 10 locations were selected from among the characteristic multiple cell nuclei, which are the position markers in the LM reference section, where such cell nuclei do not exist in the XRM reference section but exist within approximately 20 sections before and after, and they were set as the index regions. When the region after z-projection of the index regions onto the XRM reference section was set as the projected index region, the projected index region was dispersed as evenly as possible throughout the XRM reference section. At this time point, no cell nuclei existed in the projected index region of the XRM reference section (the number of coincidences was 0).

[0247] Next, as Figure 22 shown, the orientation was adjusted using the rotation tool of ImageJ in such a way that cell nuclei, which are position markers, appeared in the projected index region of the XRM reference section. In this example, when rotated by 4.0° around the horizontal axis within the screen and by 0.6° around the vertical axis within the screen, cell nuclei appeared in all 10 projected index regions of the XRM reference section. That is, the number of coincidences in the index regions became 10, which is above the evaluation criterion, so the orientation adjustment was completed. The results of the above experiment show that the method of the present invention can check and precisely align the images obtained by photographing the same measurement site of the same biological sample with an X-ray microscope and a light microscope.

Claims

1. A method for comparing images of the same measurement site of the same biological sample taken by an X-ray microscope and an optical microscope to make them coincide with each other, characterized in that: Include: A step of acquiring an image obtained by photographing a biological sample embedded in a wax block using an X-ray microscope using X-rays with an energy of 4 to 12 keV; a step of acquiring an image obtained by photographing, with an optical microscope, a portion of the biological sample included in the image photographed with the X-ray microscope; and A process of selecting an arbitrary observation target area of ​​the image in the biological sample as a position marker from the acquired X-ray microscope and optical microscope images, and using the position marker to check and match the X-ray microscope image and the optical microscope image.

2. The method according to claim 1, wherein: The image captured by the optical microscope is an image of the same measurement site of the same biological sample captured with an azimuth error within 10° from the image captured by the X-ray microscope.

3. The method according to claim 1, wherein: The position marker is selected based on at least one of the size, structure, type, cell nucleus, and defect morphology of cells or tissues in the biological sample.

4. The method according to claim 3, wherein: The defect morphology is at least one of cancer, fibrosis, calcification, stone and deposit.

5. The method according to claim 1, wherein: The comparison of the X-ray microscope image and the optical microscope image is performed by comparing the LM reference slice among the reference slices called "XRM reference slice" and "LM reference slice" respectively containing the reference area with the index slice called "XRM index slice" containing the index area, through an azimuth adjustment based on a rotation operation of the XRM reference slice, wherein the reference area is an observation object area containing position markers that exist in both the X-ray microscope image and the optical microscope image, and the index area is an observation object area containing position markers that are contained in the LM reference slice but not in the XRM reference slice, and is an area that appears within the upper and lower specified ranges of the XRM reference slice.

6. The method according to claim 5, wherein: The rotation operation is performed by rotating the image of the CT rotation angle and / or tilt angle in the X-ray microscope.

7. The method according to claim 1, wherein: The comparison between the X-ray microscope image and the optical microscope image also includes a correction step of the optical microscope image.

8. A microscope image processing device, characterized in that: have: a first determination unit for determining an arbitrary observation target region included in the biological sample from an X-ray microscope image of the biological sample embedded in a wax block photographed using X-rays with an energy of 4 to 12 keV; a second specifying unit that acquires an image obtained by photographing a portion of the biological sample included in the image photographed by the X-ray microscope with an optical microscope, and specifies an area corresponding to the observation target area specified by the first specifying unit; a unit for checking the information of the area determined by the first determining unit and the information of the area determined by the second determining unit; as well as An output unit outputs the checking result.

9. The device according to claim 8, wherein: The first determination unit and the second determination unit determine the observation object area from the X-ray microscope image and the optical microscope image by performing area extraction processing associated with at least one of the size, structure and type of cells or tissues, cell nuclei, and defect forms.

10. The device according to claim 8, wherein: A means for associating the observation target region specified by the first specifying means with the observation target region of the second specifying means is provided.

11. The device according to claim 8, wherein: The first determination unit and the second determination unit determine the observation target area based on an observation target area designation operation performed by a user, a priority predetermined for the observation target area, and / or a priority preset by a user.

12. The device according to claim 8, wherein: It also has a display unit, the output unit outputs information related to the observation target area extracted from the optical microscope image and the X-ray microscope image to the display unit, The display unit displays information related to the observation target area in an arranged or overlapping manner.

13. The device according to claim 8, wherein: A collating unit is provided for collating the optical microscope image and the X-ray microscope image by adjusting the orientation based on a rotation operation using the position marker when the observation target area specified by the first specifying unit and the second specifying unit includes the position marker.

14. A program product, characterized in that Contains a microscope image processing program for causing a computer to function as the following units: a first determination unit for determining an arbitrary observation target region included in the biological sample from an X-ray microscope image of the biological sample embedded in a wax block photographed using X-rays with an energy of 4 to 12 keV; a second specifying unit that acquires an image obtained by photographing a portion of the biological sample included in the image photographed by the X-ray microscope with an optical microscope, and specifies an area corresponding to the observation target area specified by the first specifying unit; a unit for checking the information of the area determined by the first determining unit and the information of the area determined by the second determining unit; as well as An output unit outputs the checking result.

15. A computer-readable recording medium, characterized in that: A microscope image processing program for causing the computer to function as the following means is recorded: a first determination unit for determining an arbitrary observation target region included in the biological sample from an X-ray microscope image of the biological sample embedded in a wax block photographed using X-rays with an energy of 4 to 12 keV; a second specifying unit that acquires an image obtained by photographing a portion of the biological sample included in the image photographed by the X-ray microscope with an optical microscope, and specifies an area corresponding to the observation target area specified by the first specifying unit; a unit for checking the information of the area determined by the first determining unit and the information of the area determined by the second determining unit; as well as An output unit outputs the checking result.

Images