Specimen image registration method, specimen image analysis method, program, and recording medium

Abstract

To objectively evaluate success or failure for results of registration processing for registration between multiple sample images obtained through different kinds of dying.SOLUTION: A sample image registration method includes: a step of setting multiple processing target areas with different fields of view, for a first image obtained by imaging a tissue sample with first dying applied thereto, and setting multiple processing target areas corresponding to the fields of view of the processing target areas in the first sample image, for a second sample image obtained by imaging the tissue sample with second dying applied thereto; a step of executing registration processing to determine a required registration amount for aligning the first sample image with the second sample image, for corresponding fields of view; and a step of determining whether the registration processing is successful or not, on the basis of the registration amount determined for each of the processing target areas.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image analysis technique for evaluating stained tissue specimens, and in particular to a registration technique between multiple images taken of the same specimen. [Background technology]

[0002] In recent years, in the field of immunohistochemistry, so-called single-cell analysis has been attracting attention, in which a specimen is not only viewed as a group of cells, but also the expression levels and locations of biological substances such as proteins and related substances are measured on a cell-by-cell basis, focusing on the individual cells that compose the specimen. A representative analytical technique that has been widely used is flow cytometry, in which cells are dispersed in a fluid and quantitatively measured one by one.

[0003] On the other hand, with the advancement of image analysis technology, image cytometry, which images tissue specimens without separation and analyzes them by image processing, has also been put to practical use. In the analysis technology of multiple immunostaining method using image cytometry, biological specimens such as tissue slices taken from living organisms and artificially cultured cell tissues are repeatedly stained, photographed, and decolorized to obtain multiple specimen images corresponding to each of multiple stains.

[0004] In this technique, in order to enable cross-referencing between a plurality of specimen images stained differently, it is necessary to perform registration between the specimen images. For example, in the technique described in Patent Document 1, registration between two specimen images stained differently is performed as follows. That is, unreliable parts of the two images due to tissue deformation or the like are extracted and excluded. Then, corresponding parts are detected from the remaining reliable parts using image processing techniques, thereby identifying the positional relationship between the two images.

[0005] Image cytometry may be performed by processing multiple partial regions extracted from a specimen image. Specifically, multiple partial regions are set as processing target regions for each of multiple specimen images, and registration processing and image processing for image cytometry are performed between partial regions that correspond to the same field of view. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2019-032314 A Summary of the Invention [Problem to be solved by the invention]

[0007] In multiple specimen images acquired by the multiple immunostaining method, the appearance of the staining pattern differs for each image, so it may be difficult to find corresponding parts between the images. This may result in an inappropriate result of the registration process. However, in such cases where the image contents are significantly different and it is difficult to grasp the correspondence between the positions of the images, it is difficult to objectively indicate the degree of accuracy of the result of the registration process. Therefore, it is necessary for a user with specialized knowledge to check the images and judge the success or failure of the registration process, and adjust the parameters of the registration process based on the result.

[0008] In particular, when multiple processing target regions are set for each of multiple specimen images, the amount of work required for the user to check the success or failure of each registration process can be enormous. For this reason, it is desirable to automatically perform some kind of objective evaluation regarding the success or failure of the results of the registration processes.

[0009] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a technology for performing registration processing between multiple specimen images obtained by different staining, which is capable of objectively evaluating the success or failure of the results of the registration processing. [Means for solving the problem]

[0010] One aspect of the present invention is a method for registering a specimen image, which, in order to achieve the above-mentioned object, includes the steps of: setting a plurality of processing target areas having different field of view in a first specimen image obtained by applying a first stain to a tissue specimen and imaging it; and setting a plurality of processing target areas corresponding to each field of view of the processing target areas in the first specimen image in a second specimen image obtained by applying a second stain different from the first stain to the tissue specimen and imaging it; performing a registration process between corresponding fields of view to determine the registration amount required to align the processing target areas in the first specimen image with the processing target areas in the second specimen image; and performing a success / failure determination to determine whether the registration process was successful based on the registration amount determined for each of the plurality of processing target areas.

[0011] In the invention configured in this way, the success or failure of the registration process is determined comprehensively from the results of the registration process in each of a plurality of processing target regions that are extracted from each specimen image and correspond to the same field of view.

[0012] In this invention, a registration process is performed between one processing target area set in a first specimen image and one processing target area set in a second specimen image as corresponding to the same field of view as the first specimen image, and the registration process is performed for each of a plurality of combinations of such processing target areas.

[0013] If the positional deviation in the translation direction of the image is due to the variation in the positional relationship between the imaging device and the specimen during imaging, the amount of positional deviation will be about the same at any position in the image, so it is thought that the registration amount obtained as a result of the registration process performed on multiple processing target regions will also be about the same value. However, since the positional deviation in the rotation direction may also be included, and specimen images with different staining patterns are compared, the registration amounts actually obtained do not necessarily match.

[0014] In other words, if the registration amounts calculated for multiple processing target regions set at different positions are consistent with each other, the results of the registration processes can be considered to be highly accurate. On the other hand, if the registration amounts calculated for multiple processing target regions differ significantly, some of the results of the registration processes can be considered to be low in accuracy.

[0015] Therefore, by comparing the results of the registration processes executed for each of the multiple processing target regions, it is possible to determine whether the registration processes have been successful or not. Specifically, the success or failure of the registration processes can be determined from the registration amounts calculated by each registration process.

[0016] Another aspect of the present invention is a method for analyzing a specimen image, comprising the steps of: executing the above-mentioned specimen image registration method; and executing image cytometry based on the image content of the processing target area in which it has been determined that the registration process has been successful. Note that in this invention, "image cytometry" refers to image processing that measures at least one of quantitative information such as the shape and size of cells in the specimen and the expression level of a biological substance (e.g., a protein) in the cells on a cell-by-cell basis based on image data of the specimen image.

[0017] In the invention configured in this manner, image cytometry is performed only on processing target areas for which registration processing has been successful, making it possible to prevent errors in the image cytometry results due to inappropriate registration processing. Effect of the Invention

[0018] As described above, according to the present invention, by making a judgment based on the registration amount obtained by the registration processing for each of a plurality of processing target regions, it becomes possible to objectively evaluate the success or failure of the registration processing. [Brief description of the drawings]

[0019] [Figure 1] 1 is a flow chart showing the flow of image cytometry to which the present invention can be applied. [Diagram 2] FIG. 1 is a diagram showing an example of a method for imaging a specimen. [Diagram 3] FIG. 1 is a diagram showing the relationship between a specimen image and an analysis target region. [Figure 4] 13 is a flowchart showing a process of preliminary positional deviation correction. [Diagram 5] 1 is a flow chart showing a first embodiment of registration. [Figure 6] 13 is a flow chart showing a second embodiment of registration. [Figure 7] 13 is a flow chart showing a third embodiment of registration. [Figure 8] 13 is a flowchart showing a success / failure determination process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Fig. 1 is a flow chart showing the flow of an image cytometry technique to which the method of analyzing a specimen image according to the present invention can be applied. First, a tissue slice taken from a living body or an artificially cultured tissue is sliced ​​to an appropriate thickness and fixed on an appropriate carrier such as a slide glass to prepare a tissue specimen to be analyzed (step S101). This biological specimen is stained by an appropriate staining method (step S102), imaged (step S103), and decolorized (step S104) a predetermined number of times according to the number of types of staining (step S105). That is, the specimen is multiplexed immunostained, and imaged for each staining to obtain image data of a plurality of specimen images corresponding to each staining.

[0021] Next, the positional deviation between those specimen images is corrected (step S106). Due to the repetition of staining, imaging, and decolorization, there is a deviation between the actual imaging ranges of multiple specimen images, even if the same location of the same specimen is imaged. The amount of relative positional deviation between such images is corrected. The specific processing content will be described later, but the positional deviation correction here is a relatively rough one using low-magnification images.

[0022] From each specimen image for which positional deviation has been corrected in this manner, a plurality of analysis target regions (processing target regions) are extracted (step S107). The analysis target regions are partial regions of the specimen image that are the subject of analysis processing by image cytometry. As will be described later, a plurality of analysis target regions are extracted from one specimen image. Furthermore, among the plurality of specimen images, regions that correspond to the same visual field are set as analysis target regions.

[0023] Next, registration is performed for each of the extracted analysis target regions (step S108). More specifically, the amount of relative positional deviation is calculated and corrected between analysis target regions that correspond to the same field of view among the analysis target regions extracted from each of the multiple specimen images. The positional deviation correction in step S106 is relatively rough, and does not necessarily provide sufficient accuracy for performing image cytometry. Therefore, in order to achieve more accurate alignment, registration is performed again for each analysis target region. The details of the registration process will be described later.

[0024] Image cytometry is performed based on the image contents of the analysis target regions thus aligned. Specifically, cell regions occupied by individual cells are extracted from each analysis target region (step S109), and image cytometry is performed to quantitatively analyze each extracted cell (step S110). Various image processing techniques are known for the extraction of cell regions and image cytometry, and these can be appropriately selected and applied in this embodiment. Therefore, detailed explanations of these techniques will be omitted.

[0025] Fig. 2 is a diagram showing an example of a method for imaging a specimen. Fig. 2(a) shows an example of the main configuration of an imaging device 1 that realizes imaging. For example, a specimen S prepared on a slide glass is supported in a substantially horizontal position by an appropriate supporting means such as a contact glass 11. An illumination light source 12 is disposed above the specimen S, and causes substantially uniform illumination light L to be incident on the specimen S. If the illumination light L is visible light, bright field imaging is possible, and if it is excitation light for a fluorescent substance, fluorescent imaging is possible.

[0026] An image scanner 13 is disposed below the specimen S supported by the contact glass 11. The image scanner 13 has a one-dimensional (linear) image sensor (not shown) whose longitudinal direction is perpendicular to the paper surface. As indicated by the dotted arrow, the image scanner 13 moves in a direction perpendicular to the longitudinal direction, i.e., in the lateral direction of the paper surface, to scan the specimen S, thereby acquiring digital image data corresponding to a two-dimensional image of the specimen S. Imaging may be performed using a two-dimensional image sensor.

[0027] FIG. 2(b) shows an example of a specimen image. As shown in FIG. 2(b), the specimen image Is is captured so that the entire tissue slice T in the specimen fits within the field of view. Such an image is sometimes called a virtual slide. A large tissue slice T may be divided into multiple images and captured. It is desirable that the imaging resolution is sufficiently finer than the size of a cell. In this sense, an imaging device that uses a high-resolution linear image scanner to scan the specimen S to capture an image is effective. It is desirable that the acquired image can be presented to the user at various display magnifications. For the following explanation, the horizontal direction of the specimen image Is will be referred to as the X direction and the vertical direction as the Y direction, as shown in FIG. 2(b).

[0028] Figure 2(c) shows an example of a multiple immunostaining image. In a set of multiple specimen images Is1 to Is3 captured while repeatedly staining and destaining, the imaging field of view for the tissue section T may not be completely consistent between images due to variations in the setting position of the specimen S during imaging. In other words, the range of the specimen S that is actually included in the image may differ for each imaging.

[0029] In the example shown in Fig. 2(c), when one specimen image Is1 is used as a reference, the imaging range of another specimen image Is2 is shifted to the upper left. That is, images Is1 and Is2 are in a translational movement relationship. Also, specimen image Is3 is in a rotational movement relationship around an axis perpendicular to the XY coordinate plane with respect to specimen image Is1. In reality, it is considered that any two specimen images have a positional shift relationship that combines translational movement and rotational movement.

[0030] In other words, by using one specimen image as a reference and performing a translation operation that combines appropriate translation and rotation on another specimen image, it is possible to match corresponding points between the two images. For example, the misalignment between images can be corrected by an operation that includes a coordinate transformation process such as a rigid transformation or an affine transformation.

[0031] The misalignment correction and registration processing described below are based on this principle. That is, the "registration processing" is a computational process for specifying a transformation matrix that represents the coordinate transformation required to overlay two images, and the "misalignment correction" is an image process for relatively shifting one image so that it overlaps the other image according to the registration amount indicated by the elements of the specified transformation matrix.

[0032] In this specification, the term "registration processing" is used in the narrow sense described above, and the processing for aligning images, which is a broader concept including this as an element, is simply referred to as "registration." Furthermore, the "preliminary misalignment correction" in step S106 is based on the same principle as the "registration" in step S108. However, to distinguish it from the "registration" in step S108, which is the alignment in units of analysis target regions cut out from the specimen image, the processing in step S106, which is the preliminary alignment in units of specimen images, is referred to as "preliminary misalignment correction."

[0033] More strictly speaking, in the actual specimen image Is, nonlinear deviations may occur due to distortion caused by the optical characteristics of the imaging optical system, warping of the specimen S or floating of the specimen S relative to the contact glass 11, deformation of the tissue slice T itself, etc. However, these effects need not be considered here.

[0034] 3 is a diagram showing the relationship between specimen images and analysis target regions. In this embodiment, multiple analysis target regions are set in each specimen image, and image cytometry is performed by combining analysis target regions corresponding to the same field of view among multiple specimen images.

[0035] As shown in Figures 3(a) to 3(c), the staining patterns of the tissue specimen including the tissue slice T are different between specimen images Is1 to Is3 corresponding to different stains. Consider a case where three analysis target regions R11 to R13 are set for specimen image Is1 as shown in Figure 3(a). Note that the number, positions and sizes of the analysis target regions to be set are not limited to those exemplified here and can be arbitrary.

[0036] Here, the coordinate ranges of the regions R21 (R22, R23) and R31 (R32, R33) in the specimen images Is2 and Is3, which correspond to the same field of view as the region R11 (R12, R13) to be analyzed, are set to the same as the region R11 (R12, R13) in the image plane, as shown by the dotted lines in Figures 3(b) and 3(c). In this case, since the shift between the images shown in Figure 2(c) is not taken into account, it cannot be said that they correspond to the same field of view in the strict sense. As shown by the solid lines in Figures 3(b) and 3(c), by setting the regions while taking into account the positional shift between the images, it becomes possible to extract the regions to be analyzed that correspond to the same field of view from each specimen image.

[0037] It is possible to find corresponding points between the analysis target regions and reduce the effect of misalignment between the specimen images by performing registration between the extracted analysis target regions that correspond to each other. However, if the misalignment between the extracted analysis target regions becomes large, the overlapping area of ​​the field of view between them becomes small, and the success rate of registration decreases.

[0038] When the tissue specimen to be analyzed is a specimen that has been subjected to multiple immunohistochemical staining, the staining patterns vary greatly from image to image depending on the type of staining, and it may be difficult to find corresponding points between the images. To address this problem, in this embodiment, a preliminary positional deviation correction (step S106) is performed on the entire specimen image as a preprocessing step prior to cutting out the region to be analyzed.

[0039] 4 is a flowchart showing the process of the preliminary positional deviation correction in step S106. First, each of the specimen images Is1 to Is3 is reduced in magnification by resolution conversion (step S201). The captured specimen images Is1 to Is3 are large images that include the entire tissue slice T, and have the high resolution required for image cytometry. Therefore, the amount of image data is very large. If such images are directly applied to the registration process, the amount of calculations becomes enormous, requiring large memory resources and long processing time.

[0040] Since the purpose of the process here is to reduce to some extent the positional deviation when cutting out the analysis target region from each specimen image, the image is reduced in magnification to reduce the amount of data, and then the registration process is performed (step S202). For example, the number of pixels per side can be thinned to about 1 / 10. In this way, the amount of calculation and memory capacity can be significantly reduced. In this case, the registration amount can be calculated by referring to a relatively large structure in the specimen, such as the contour or cavity of the tissue slice T. As the coordinate transformation for aligning the specimen images, various known techniques, such as rigid transformation and affine transformation, can be applied.

[0041] Based on the registration amount thus obtained, a correction amount for properly aligning the multiple specimen images is determined (step S203), and the positional deviation between the images is corrected (step S204). The correction can be, for example, a combination of translation and rotation of one image relative to the other image. Note that the purpose of the correction here is to optimize the cut-out range so that corresponding fields of view are cut out from each image in the subsequent cut-out of the analysis target area, and it is not necessary to create a corrected image. Therefore, as a more practical method, the image correction (step S204) is not performed at this stage, and the cut-out area may be set by taking into account the above correction amount when the analysis target area is cut out in the subsequent step S107.

[0042] In particular, it is preferable to correct the positional deviation in the direction involving image rotation to some extent at this stage, as between specimen image Is1 and specimen image Is3 shown in Fig. 2(c). As is clear from Fig. 3(c), the influence of the positional deviation in the rotational direction between images on the setting of the analysis target area varies greatly depending on the position in the image from which the analysis target area is taken. For example, in step S106, only the rotational component of the positional deviation may be corrected.

[0043] In this way, by correcting the positional deviation between the specimen images, particularly the rotational component, to some extent before extracting the analysis target area from each specimen image, it is possible to reduce the positional deviation between the analysis target areas extracted from each specimen image as corresponding to the same field of view. Therefore, the registration process between the analysis target areas executed in step S108 only needs to accommodate a relatively small positional deviation.

[0044] In the registration process of the analysis target region, which is a partial region cut out from the specimen image, it may not be possible to refer to the macroscopic structure of the tissue section T. In particular, if the types of staining are different, there is little similarity in appearance in a microscopic comparison, and registration may fail. If a certain degree of positional deviation correction is performed in advance prior to cutting out the analysis target region, highly accurate registration based on a more microscopic structure is possible.

[0045] In this specification, the terms "success" and "failure" in registration are used in the following sense. A known registration processing algorithm is configured to output some result regardless of the relationship between two images to be processed. Specifically, a numerical value corresponding to the registration amount for aligning the images, for example, a numerical value corresponding to the amount of translation and rotation, is calculated. However, the numerical value does not necessarily guarantee an appropriate registration result.

[0046] One way of thinking about whether registration is successful or not is as follows. For example, when two images are moved relatively based on the calculated registration amount and superimposed, if the distance between any points corresponding to the same position on the specimen in the images is smaller than a predetermined value, it can be said that the two images have been aligned with high accuracy, and the registration can be considered to be "successful". On the other hand, if the distance is larger than the predetermined value, that is, if the corresponding points in the two images cannot be brought close enough to each other, or if the corresponding points cannot be found, the registration can be considered to be "failed". The "predetermined value" can be expressed, for example, by the number of pixels.

[0047] In this way, it is possible to qualitatively define whether registration is successful or unsuccessful, and it is also possible to provide an index for quantitatively and objectively determining whether or not a registration has been successful, particularly in cases where corresponding points between images can be easily detected. On the other hand, when there is a large difference in content between the images, as in this case, the indexes described above do not necessarily function well, and it is currently difficult to quantitatively express the accuracy of the calculated registration amount.

[0048] As will be described below, in this embodiment, a comprehensive judgment based on the registration amount obtained in the registration process in each of the multiple analysis target regions set for one specimen image is made, thereby making it possible to quantitatively and objectively judge whether the registration process has been successful or not.

[0049] Figures 5 to 7 are flowcharts showing the registration between analysis target regions in this embodiment. Each process shown in Figures 5 to 7 shows the registration process content including the success / failure judgment of the registration process according to this embodiment. Although the manner in which the result of the success / failure judgment is used differs between these processes, the basic concept is the same.

[0050] First, one aspect of registration according to this embodiment will be described with reference to Fig. 5. This process is executed after a plurality of analysis target regions are cut out from each of the specimen images (step S107 in Fig. 1). First, a registration process is executed (step S301) to calculate the amount of registration required to overlap between analysis target regions cut out from different specimen images and corresponding to the same field of view.

[0051] For example, one of a plurality of specimen images can be used as a reference image, and a registration process can be performed between the analysis target regions having corresponding fields of view between this reference image and another specimen image. For example, in the example shown in Fig. 3, when the specimen image Is1 is used as the reference image, a registration process is performed between one analysis target region R11 set in the specimen image Is1 and an analysis target region R21 set in another specimen image Is2 as having a common field of view.

[0052] Similarly, registration processing is performed between the analysis target region R12 in the specimen image Is1 and the analysis target region R22 in the specimen image Is2 having the corresponding field of view, and between the analysis target region R13 in the specimen image Is1 and the analysis target region R23 in the specimen image Is2 having the corresponding field of view. That is, registration processing is performed in each of the three analysis target regions between the specimen images Is1 and Is2, and the registration amount is calculated for each analysis target region.

[0053] When there are three or more specimen images, similar processing is performed between specimen image Is1, which is the reference image, and other specimen images Is3, etc. That is, registration processing is also performed between analysis target region R11 in specimen image Is1 and analysis target region R31 in specimen image Is3 having a corresponding field of view, between analysis target region R12 in specimen image Is1 and analysis target region R32 in specimen image Is3 having a corresponding field of view, and between analysis target region R13 in specimen image Is1 and analysis target region R33 in specimen image Is3 having a corresponding field of view, and the registration amount is calculated for each.

[0054] In this way, for one pair of two specimen images (e.g., specimen images Is1 and Is2), registration processing is performed for three analysis target regions, and the registration amounts are calculated for each pair. The registration amounts are statistically processed to determine whether each registration processing is successful or not (step S302).

[0055] The specific processing contents will be described later, but the basic idea of ​​the success / failure judgment processing is as follows. Under the premise that the positional deviation between specimen images can be corrected by rigid body alignment, it is considered that there is a certain correlation between the results of the registration processing for multiple analysis target areas cut out from one specimen image. For example, if the parallel movement required to superimpose the analysis target area R21 on the analysis target area R11 is a movement to the right, it is considered that the movement to superimpose the other analysis target area R22 on the analysis target area R12 and the movement to superimpose the analysis target area R23 on the analysis target area R13 will also be in roughly the same direction. For example, it is difficult to imagine that only the movement to superimpose the analysis target area R23 on the analysis target area R13 will be in the left direction, which is the opposite of the above.

[0056] From this, if such a result that is inconsistent with the results of other registration processes is included, it can be considered that one of the registration processes outputs an inappropriate result, that is, the registration process has failed. Such an event indicates that there is a high possibility that at least one registration process has failed, but does not directly indicate which registration process was successful and which registration process failed.

[0057] However, for example, if most of the registration processes performed on a large number of analysis target regions show a common trend and only some show different trends, it is possible to estimate that the results obtained in most of the registration processes are more likely. Based on this idea, it is possible to tentatively determine that a registration that outputs a result similar to the results of other processes is successful, and that a registration that does not output a result is unsuccessful.

[0058] Quantitatively, the success or failure of each registration process can be determined by statistical processing of the registration amount values ​​obtained as a processing result. That is, if the registration amount value is close to other results, it can be said that the registration process is highly likely to have been successful. On the other hand, if the registration amount value is significantly different from other results, it can be said that the registration process is highly likely to have failed. If the registration amount values ​​vary greatly, it is difficult to identify which one was successful. In this case, in order to avoid analysis errors caused by inappropriate registration processes, all registration processes may be considered to have failed.

[0059] In the processing mode shown in Fig. 5, the user is presented with a judgment result indicating whether the registration process for each analysis target region was successful or unsuccessful (step S303). This allows the user to understand the frequency of failures and their occurrence trends. For example, the user can redo the setting of the analysis target region and the registration as necessary. Then, the analysis target regions for which the registration process was judged to be successful are extracted (step S304), and are used for subsequent image cytometry (steps S109 and S110 in Fig. 1).

[0060] On the other hand, the processing mode shown in Fig. 6 is the same as the processing mode shown in Fig. 5 up to the registration processing for each analysis target region (step S301) and the success / failure determination thereof (step S302). However, this processing mode differs from the above processing mode in that an analysis target region for which the registration processing is determined to have failed is extracted (step S313), and the processing result is corrected based on the registration amount calculated in the successful registration processing (step S314).

[0061] As a correction method, for example, a method of replacing the registration amount in the failed registration process with a value calculated based on the registration amounts in the successful registration processes (for example, their average value or median value) can be considered. Also, a method of correcting the registration amount in the failed registration process with a value calculated based on the registration amount in the successful registration process may be used.

[0062] Specifically, by adding (or subtracting) a value calculated based on the registration amount in a successful registration process to (or from) the registration amount in an unsuccessful registration process, or approximately half of that value, the registration amount can be brought closer to the value when the registration process was successful.

[0063] 7, similarly to the above embodiment, registration processing (step S301) and success / failure determination (step S302) are performed for each analysis target region, and it is determined whether there are any failure cases based on the success / failure determination results (steps S323, S324). If there are no failure cases, that is, if the registration processing is determined to be successful for all analysis target regions (YES in step S324), they are subjected to image cytometry.

[0064] On the other hand, if there is a failure, at least a part of the processing parameters of the registration process is modified for the analysis target region, and the analysis target region is subjected to the registration process again (step S325). By changing the parameters and re-executing the registration process in this way, the success rate of the registration process may be improved. The parameters may be changed automatically, or the user may input changes. If the process still fails after re-execution, the process may be repeated until it is successful, or the process may be terminated and the analysis target region where the process has failed may be excluded from the target of image cytometry, as in the embodiment shown in FIG. 5.

[0065] 5 to 7 differ from each other in how the results of the success / failure determination are used, but they are common in that registration processing is performed between analysis target areas that have a common field of view, and the success / failure of each registration processing is determined comprehensively from the results of the registration processing in the multiple analysis target areas. Note that, with regard to the processing after the success / failure determination, the processing modes shown in Fig. 5 to 7 may be appropriately combined and executed.

[0066] In the explanation of the principle above, the success or failure of the registration process is considered to be "success if the calculated registration amount is close to the processing result in other analysis target regions, and failure if there is a large deviation."

[0067] However, especially when the positional deviation between the specimen images includes a rotational component (hereinafter referred to as the "rotational component"), as can be seen from FIG. 3(c), the X-direction and Y-direction components (hereinafter referred to as the "XY-direction components") of the registration amount calculated for each analysis target region differ depending on the position where the analysis target region is cut out. In other words, even if the registration process is successful for all analysis target regions, the calculated registration amount values ​​are not necessarily the same. In other words, even if there is variation in the calculated registration amount values, this does not immediately indicate the possibility of a failure in the registration process.

[0068] In this embodiment, since the positional deviation correction between the specimen images is performed prior to setting the analysis target area, it is expected that the large positional deviation between the extracted analysis target areas is eliminated. However, since the positional deviation correction process is based on low-magnification images, the effect is limited, and even in the registration process performed for each analysis target area based on finer images, it is unavoidable that some deviation in the rotational component is included.

[0069] Taking this into consideration, in the success / failure determination process of this embodiment, a process for avoiding erroneous determination that may occur mainly due to the rotational component of the position shift is added to the process based on the above principle. Specifically, the process is as follows.

[0070] 8 is a flow chart showing the success / failure determination process. First, a reference value is calculated for the rotational component of the registration amount calculated for each analysis target region (step S401). This reference value is a value obtained by statistically processing the registration amount calculated for each analysis target region, and for example, the average or median of the registration amount can be used. At this time, the reference value may be calculated by excluding large and small values ​​at a certain rate.

[0071] Next, the registration amount of the rotational component calculated for each analysis target region is individually evaluated by comparing it with a reference value (step S402). Specifically, the difference between the registration amount of the rotational component calculated for each analysis target region and the reference value is calculated, and the difference is compared with a predetermined threshold value Th1 (step S403).

[0072] If the difference between the registration amount of the rotational component and the reference value is not within threshold Th1 (NO in step S403), the registration process for the analysis target area is determined to have failed (step S410). As can be seen from Fig. 3(c), the positional deviation in the XY direction varies greatly depending on the setting position of the analysis target area, but the positional deviation amount of the rotational component should not change much. Therefore, a registration process in which the registration amount of the rotational component deviates significantly from the reference value is deemed to have failed.

[0073] If the difference between the registration amount of the rotation component and the reference value is within the threshold Th1 (YES in step S403), the registration amount of the XY direction components is evaluated. However, if the reference value obtained in step S401 is not within a predetermined threshold Th2 (NO in step S404), it is determined that the evaluation of the XY direction components is impossible (step S409).

[0074] When the positional deviation of the rotational component between the specimen images is large, for example exceeding the threshold value Th2, the variation in the registration amount in the XY direction components between the analysis target regions increases as described above. Therefore, even if the variation in the value of the registration amount in the XY direction calculated for each analysis target region is large, it is not possible to distinguish whether it is due to a registration failure or due to a large positional deviation of the rotational component. Therefore, when the reference value of the rotational component is large like this, a determination result is returned that the evaluation of the XY direction components is not possible.

[0075] If the reference value of the rotational component is sufficiently small (YES in step S404), the evaluation of the XY direction components is performed. Specifically, similarly to the case of the rotational component, the reference value of the XY direction components of the registration amount calculated for each analysis target region is calculated (step S405), and the value of the registration amount of the XY direction components calculated for each analysis target region is individually evaluated by comparing with the reference value (step S406).

[0076] If the difference between the registration amount in the XY direction and the reference value is within a predetermined threshold Th3 (YES in step S407), it can be said that the registration process for the analysis target area has obtained results similar to those of other processes in terms of both the XY direction components and the rotational components. Therefore, in this case, it can be determined that the registration process has been successful (step S408).

[0077] On the other hand, if there is a large discrepancy between the registration amount value and the reference value (NO in step S407), it can be determined that the registration process has failed (step S410).

[0078] The manner in which the result of the success / failure determination thus performed is used is, for example, as described above, but is not limited to this.

[0079] As described above, in this embodiment, a registration process is performed between a plurality of analysis target regions cut out from each of a plurality of specimen images, which correspond to the same field of view. Then, the success or failure of each registration process is determined comprehensively from the registration amount calculated for each of the plurality of analysis target regions. Specifically, the result of each registration process is compared with the result of other registration processes, and if the results are similar, it is determined to be successful, and if there is a large deviation, it is determined to be unsuccessful.

[0080] The success / failure determination process takes into account the difference in the nature of the misalignment in the XY direction and the misalignment in the rotation direction. That is, the misalignment in the XY direction between specimen images appears in the same manner in the analysis target area, which may be set at any position in the specimen image. In contrast, the misalignment in the rotation direction between specimen images appears in the same amount of misalignment in the rotation component regardless of the position of the analysis target area set in the specimen image, but the misalignment in the XY direction components varies greatly depending on the position where the analysis target area is cut out.

[0081] Therefore, in this embodiment, the rotational component of the registration processing result is evaluated first, and the XY direction components are evaluated only when it is confirmed that the amount of positional deviation in the rotational direction is sufficiently small, thereby reducing the probability of erroneous determination caused by the influence of the rotational positional deviation on the XY direction components of the registration amount.

[0082] Furthermore, prior to cutting out the analysis target area from the specimen image, a preliminary positional deviation correction is performed between the specimen images. When cutting out the analysis target area from two specimen images, the relative positional deviation between the specimen images is corrected and the cutting range is set, so that it is expected that the positional deviation between the analysis target areas of corresponding fields of view can be kept small. This can improve the probability of successful registration between the analysis target areas.

[0083] In this preliminary misalignment correction, it is sufficient to correct the misalignment between the specimen images to a certain extent, and high-precision correction is not necessarily required. If preliminary misalignment correction is performed using an image obtained by converting the resolution of the original specimen image and reducing its magnification, the amount of calculation and the amount of data generated during the process can be reduced, thereby shortening the processing time.

[0084] In this way, by making it possible to automatically determine whether the registration process has been successful, which was previously done by the user checking the image, it is possible to greatly reduce the workload on the user.

[0085] The imaging operation by the image scanner 13 in the above embodiment and the above-mentioned various image processes can be executed by a computer device having a general hardware configuration. In other words, by installing a computer program describing the above-mentioned operation procedures in a general-purpose computer device, the computer device can function as the executing entity of the present invention.

[0086] The present invention is not limited to the above-mentioned embodiment, and various modifications can be made without departing from the spirit of the present invention. For example, in the above-mentioned embodiment, three or more analysis target regions are set for each specimen image, and the result of the registration process for one analysis target region is evaluated based on the result of the registration process for the other analysis target regions. This method cannot be applied when there are two analysis target regions cut out from each specimen image. In this case, for example, the following can be done.

[0087] The basic concept is the same as in the above embodiment, and if the registration amounts calculated for the two analysis target regions are very similar, it is possible to consider that the registration processes were successful. On the other hand, if the registration amounts are significantly different, it is impossible to determine whether one or both of the registration processes have failed. In such a case, it is sufficient to return as a judgment result whether the registration processes for both analysis target regions were successful or at least one of them has failed.

[0088] Also, for example, in the above embodiment, rough alignment between specimen images is performed prior to cutting out the analysis target region. However, even if such rough alignment is not performed, the success / failure determination according to the present invention can be applied. For example, if a large positional deviation is suppressed due to the size of the specimen or the structure of the device, such rough alignment may be omitted.

[0089] As described above by way of example and description of specific embodiments, in the method for registering a specimen image according to the present invention, the registration amount may include at least the amount of translation and rotation between the processing target regions. Specifically, for example, the registration process may include coordinate transformation by rigid transformation or affine transformation.

[0090] For example, the specimen image registration method according to the present invention may be configured to set three or more processing target regions for each of the first and second specimen images, perform the registration processes, and determine the success or failure of each registration process based on the magnitude of variation in the registration amounts obtained. In this case, for example, a reference value can be set based on the registration amount value obtained in each registration process, and the success or failure of each registration process can be determined based on the magnitude of deviation of the registration amount from the reference value.

[0091] More specifically, a registration process in which the deviation between the registration amount and a reference value is smaller than a predetermined threshold value can be determined to be successful, while a registration process in which the deviation is larger than the threshold value can be determined to be unsuccessful.

[0092] Here, for a registration process determined to be unsuccessful, the registration amount may be corrected based on the registration amount in a registration process determined to be successful. Also, the registration process may be re-executed with changed parameters. In these cases, the reference value may be a value based on the average or median of the registration amounts obtained in each registration process.

[0093] Furthermore, for example, prior to setting the processing target area, a pre-processing for correcting the relative positional shift between the first and second specimen images may be performed, and corresponding processing target areas may be set between the first and second specimen images after the pre-processing. In this case, in the pre-processing, the amount of positional shift may be found based on images in which the resolution of the first and second specimen images is reduced.

[0094] The present invention can be implemented as a program for causing a computer device to execute each step of the above-mentioned sample image registration method, or as a computer-readable recording medium on which the program is non-temporarily recorded. Each of the above steps can be realized by a computer device having a general hardware configuration, and by implementing the program in an existing computer device, it is possible to impart new functions to the computer device. [Industrial Applicability]

[0095] This invention is applicable in fields such as medical and biological research, where the same tissue specimen is stained and imaged multiple times, and the images obtained are then integrated and analyzed. It is particularly suitable for analyzing multiply immunostained specimens on a single cell basis. [Explanation of symbols]

[0096] Is, Is1, Is2, Is3 Sample image R11~R13, R21~R23, R31~R33 Analysis area (processing area) S tissue specimen T tissue section

Claims

1. setting a plurality of processing target regions having different fields of view with respect to a first specimen image obtained by applying a first stain to a tissue specimen and imaging the same, and setting a plurality of processing target regions corresponding to the respective fields of view of the processing target regions in the first specimen image with respect to a second specimen image obtained by applying a second stain different from the first stain to the tissue specimen and imaging the same; performing a registration process between corresponding fields of view to obtain a registration amount required for aligning the processing target area in the first specimen image with the processing target area in the second specimen image; performing a success / failure determination step of determining whether the registration process has been successful or not based on the registration amounts calculated for each of the plurality of processing target regions; 1. A method for registering a specimen image, comprising:

2. The method for registering a specimen image according to claim 1 , wherein the registration amount includes at least an amount of translation and an amount of rotation between the processing target regions.

3. 3. The method for registering a specimen image according to claim 1, wherein the registration process includes coordinate transformation using a rigid transformation or an affine transformation.

4. 4. The method for registering specimen images according to claim 1, further comprising the steps of: setting three or more processing target areas for each of the first specimen image and the second specimen image, and executing the registration process for each of the processing target areas; and determining whether each of the registration processes is successful or not based on a magnitude of variation in the registration amount thus obtained.

5. 5. The method for registering a specimen image according to claim 4, further comprising the steps of: setting a reference value based on the value of the registration amount obtained in each of the registration processes; and determining whether each of the registration processes is successful or not based on a deviation of the registration amount from the reference value.

6. 6. The method for registering a specimen image according to claim 5, further comprising determining that the registration process is successful when the deviation between the registration amount and the reference value is smaller than a predetermined threshold, and determining that the registration process is unsuccessful when the deviation is larger than the threshold.

7. 7. The method for registering a specimen image according to claim 6, further comprising correcting an amount of registration for the registration process determined to be unsuccessful based on an amount of registration in the registration process determined to be successful.

8. The method for registering a specimen image according to claim 6 , further comprising the step of changing parameters and re-executing the registration process determined to have failed.

9. 7. The method for registering a specimen image according to claim 5, wherein the reference value is a value based on an average value or a median value of the registration amounts obtained in each of the registration processes.

10. performing a pre-processing step of correcting a relative positional deviation between the first specimen image and the second specimen image prior to setting the processing target area; 10. The method for registering specimen images according to claim 1, further comprising the step of: setting the processing target regions between the first specimen image and the second specimen image after the preprocessing, the processing target regions corresponding to each other.

11. The method for registering specimen images according to claim 10 , wherein in the pre-processing, the amount of positional deviation is calculated based on images obtained by reducing the resolution of the first specimen image and the second specimen image.

12. 12. A method for registering specimen images according to any one of claims 1 to 11, performing image cytometry based on the image content of the processing target area where the registration process has been determined to be successful; A method for analyzing a specimen image comprising:

13. A program for causing a computer device to execute each step of the specimen image registration method according to any one of claims 1 to 11.

14. A computer-readable recording medium having the program according to claim 13 non-transitoryly recorded thereon.

Images