Specimen image registration method, program, and recording medium

Abstract

To improve the success rate of registration even when a dyed state significantly differs among a plurality of sample images obtained by multiple kinds of dying.SOLUTION: A sample image registration method pertaining to the present invention comprises: a step (S205) in which at least one instance of a third image different from a first and a second image that are the objects of registration is selected from a plurality of sample images which multiple kinds of dying have been applied to; steps (S206, S207) in which a registration amount between the first and the third images and a registration amount between the second and the third images are obtained, respectively; and a step (S208) in which a registration amount between the first and the second images is obtained on the basis of the registration amount between the first and the third images and the registration amount between the second and the third images.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an image analysis technique for evaluating stained biological specimens, and in particular to a registration technique between multiple images taken of the same specimen. [Background technology]

[0002] In recent years, in the field of immunohistochemistry, so-called single-cell analysis has been attracting attention, in which a specimen is not only viewed as a group of cells, but also the expression levels and locations of biological substances such as proteins and related substances are measured on a cell-by-cell basis, focusing on the individual cells that compose the specimen. A representative analytical technique that has been widely used is flow cytometry, in which cells are dispersed in a fluid and quantitatively measured one by one.

[0003] On the other hand, with the advancement of image analysis technology, image cytometry, which images tissue specimens without separation and analyzes them by image processing, has also been put to practical use. In the analysis technology of multiple immunostaining method using image cytometry, biological specimens such as tissue slices taken from living organisms and artificially cultured cell tissues are repeatedly stained, photographed, and decolorized to obtain multiple specimen images corresponding to each of multiple stains.

[0004] In this technique, in order to enable cross-referencing between a plurality of specimen images stained differently, it is necessary to perform registration between the specimen images. For example, in the technique described in Patent Document 1, registration between two specimen images stained differently is performed as follows. That is, unreliable parts of the two images due to tissue deformation or the like are extracted and excluded. Then, corresponding parts are detected from the remaining reliable parts using image processing techniques, thereby identifying the positional relationship between the two images. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2019-032314 A Summary of the Invention [Problem to be solved by the invention]

[0006] In the case of multiple immunostaining, where multiple staining methods are applied to the same specimen, the staining conditions, such as the locations in the specimen where the staining occurs and the shading, color, etc., vary widely depending on the type of staining. For this reason, when automatically registering two randomly selected specimen images using image processing, for example, it may be impossible to detect corresponding locations due to the large difference in staining conditions between the selected specimen images, resulting in failed registration.

[0007] The registration failure here means that the positional relationship between two specimen images derived by image processing does not correctly represent the positional relationship in the actual specimen. The positional relationship between specimen images can be quantitatively expressed, for example, by the amount of registration, which indicates the amount of relative positional deviation between the two images, or the amount of movement of one image relative to the other image required to correct the positional deviation.

[0008] The present invention has been made in consideration of the above-mentioned problems, and aims to improve the success rate of registration between specimen images with significantly different staining conditions in a technique for performing registration between multiple specimen images obtained by multiple types of staining. [Means for solving the problem]

[0009] One aspect of the present invention is a method for registering specimen images, which performs registration between a first image and a second image that are different from each other among a plurality of specimen images obtained by staining a biological specimen with a plurality of types of stains and capturing an image for each stain. In order to achieve the above-mentioned object, the method for registering specimen images includes the steps of: searching the plurality of specimen images to select at least one third image that is different from the first image and the second image; calculating an amount of registration between the first image and the third image, and an amount of registration between the second image and the third image, respectively; and calculating an amount of registration between the first image and the second image based on the amount of registration between the first image and the third image and the amount of registration between the second image and the third image.

[0010] In the invention configured in this manner, even if no corresponding points can be found between the first and second images where the staining states differ significantly, if a third image that is different from these is mediated and registration is successful between the first and third images, and between the second and third images, it is possible to indirectly register the first and second images from the registration amounts found therein. Effect of the Invention

[0011] As described above, according to the present invention, even if no corresponding points can be found between the first and second images due to a large discrepancy in staining state, the success rate of registration can be improved by using a third image that is different from these images as an intermediary. [Brief description of the drawings]

[0012] [Figure 1] 1 is a flow chart showing the steps of an image cytometry technique to which the present invention can be applied. [Diagram 2] 1A to 1C are diagrams illustrating examples of specimen images handled in the present embodiment. [Diagram 3] FIG. 2 is a diagram illustrating the principle of registration processing. [Figure 4] 5A to 5C are diagrams illustrating a registration process in the present embodiment. [Diagram 5] 4 is a flowchart showing a registration process according to the present invention. [Figure 6] FIG. 2 is a diagram illustrating a relationship between a reference image and a target image in a feature space. [Figure 7] FIG. 11 is a diagram showing an example of an evaluation result of the present embodiment. [Figure 8] FIG. 13 is a diagram showing another example of a method for selecting an intermediary image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] FIG. 1 is a flow chart showing the flow of an image cytometry technique to which the present invention can be applied. First, a tissue slice taken from a living body or an artificially cultured tissue is sliced ​​to an appropriate thickness and fixed on an appropriate carrier such as a slide glass to prepare a biological specimen to be evaluated (step S101). This biological specimen is stained by an appropriate staining method (step S102), imaged (step S103), and decolorized (step S104) a predetermined number of times according to the number of types of staining (step S105). That is, the specimen is multiplexed immunostained, and multiple specimen images corresponding to each staining are obtained by imaging for each staining.

[0014] Next, registration is performed to align the specimen images (step S106). Due to the repetition of staining, imaging, and decolorization, there is a shift in the actual imaging range between multiple specimen images, even if the same location of the same specimen is imaged. Registration processing is a process for determining the amount of positional shift between such images, eliminating the effect of the positional shift, and enabling image superimposition. This processing can be suitably applied to one embodiment of the specimen image registration method according to the present invention, and the specific processing content will be described in detail later.

[0015] After identifying the cell region occupied by each cell from one or more specimen images, the region is applied to other specimen images that are mutually aligned, thereby identifying the cell region of all specimen images (step S107). Then, cytometry is performed to quantitatively analyze each cell (step S108). Various methods for identifying the cell region and cytometry are known, and these can be appropriately selected and applied in this embodiment. Therefore, detailed explanations of these methods are omitted.

[0016] Fig. 2 is a diagram showing an example of a specimen image handled by this embodiment. Fig. 2(a) is a diagram showing a schematic diagram of an example of a biological specimen, in which a tissue specimen S containing a plurality of cells C is considered. Each cell C is composed of a cell membrane M, a cytoplasm P, a cell nucleus N, etc. In multiple immunostaining (immunohistological staining), a plurality of staining methods for selectively staining specific biological substances constituting the cells C and the tissues containing them are sequentially applied to the specimen.

[0017] By capturing images of at least a portion of the specimen within the imaging field of view FV for each staining, multiple specimen images can be obtained. Each of these specimen images is stained with a specific biological substance, and the staining conditions, such as the stained areas and concentration, vary depending on the type of staining. Figure 2(b) shows an example in which the entire tissue specimen S is stained uniformly, while Figure 2(c) shows an example in which only the cell nuclei N are selectively stained. Figure 2(d) shows an example in which the cytoplasm P is selectively stained, and Figure 2(e) shows an example in which a specific biological substance contained only inside some cells is stained.

[0018] For example, to analyze the ratio of cells expressing a specific biological substance or the relationship between the expression patterns of multiple biological substances, it is necessary to cross-reference specimen images stained by these various staining methods, and therefore the positional relationship between each specimen image must be clear. For this purpose, a registration process between specimen images (step S106 in FIG. 1) is performed.

[0019] Figure 3 shows the principle of registration processing. Since multiple specimen images are images of the same area of ​​the same specimen, it is desirable that the imaging fields of view on the specimen match. However, due to the nature of the work of repeatedly staining, imaging, and destaining the specimen, it is inevitable that some degree of positional deviation and distortion will occur.

[0020] In principle, if the positions of the regions that correspond to each other between two images are specified in each image, the relative positional relationship between the images can be determined unambiguously. For example, the amount of positional shift between images in the direction along the image plane and in the direction of rotation around an axis perpendicular to the plane can be obtained if at least two points that correspond to each other in both images are specified. For example, as shown in FIG. 3(a), it is possible to specify the positional relationship between each specimen image from the positions in the images of two points Pa and Pb that correspond to each other on multiple specimen images Ia, Ib, Ic, Id, Ie, .... The purpose of registration is achieved by aligning the two images so that all of these two points overlap.

[0021] For example, as shown in FIG. 3(b), it is possible to align points Pa and Pb so that they overlap with each other by using one specimen image Ia as a reference and moving other specimen images by combining movement in the X direction parallel to the image plane and the Y direction perpendicular to the X direction, and rotation in the θ direction around an axis perpendicular to the XY plane.

[0022] However, the staining conditions in each specimen image are different, and there may be a situation where a biological substance that is clearly stained in one specimen image is barely visible in another specimen image. For example, there may be cases where there is no correlation at all between the results of a staining method that stains a certain protein and the results of a staining method that does not stain that protein. For this reason, it may be impossible to identify corresponding points or areas between the two specimen images, and registration may fail.

[0023] As a specific method of registration processing, that is, a technique for detecting corresponding areas between two images and specifying the positional relationship between the two images based on the result, various image processing techniques such as a known pattern matching technique can be applied. If registration is attempted between two images that are substantially uncorrelated using these techniques, some result will be returned. However, the result does not necessarily correctly represent the positional relationship on the specimen. In addition, depending on the staining state and the combination of images, it may not be possible to find any significant common points between the images. In this way, a state in which information representing the correct positional relationship cannot be obtained as a result of registration by image processing is collectively referred to as a "registration failure" here.

[0024] 4A and 4B are diagrams for explaining the registration process in this embodiment. As shown in FIG. 4A, consider a case in which two characteristic points P1 and P2 in a first specimen image I1 cannot be detected in a second specimen image I2, while two characteristic points P3 and P4 in the specimen image I2 cannot be detected in the specimen image I1. In such a case, it is not possible to detect corresponding regions, and there is a high probability that registration will fail.

[0025] In this embodiment, as shown in Fig. 4(b), in addition to the two specimen images I1 and I2, a third specimen image I3 having intermediate features between the two is selected from the other specimen images, and registration is attempted by placing this image I3 between the specimen images I1 and I2. Specifically, registration is performed based on the features common to the first specimen image I1 and the third specimen image I3, while registration is also performed between the second specimen image I2 and the third specimen image I3 based on the features common to them.

[0026] As shown by the dotted arrows in Fig. 4(b), if the positional relationship between the first specimen image I1 and the third specimen image I3 and the positional relationship between the second specimen image I2 and the third specimen image I3 can be specified, the results can be used to indirectly represent the positional relationship between the first specimen image I1 and the second specimen image I2, as shown by the dashed arrows in Fig. 4(b). The specific contents of the registration process of this embodiment based on this principle will be described below. Note that the third specimen image may be referred to as an "intermediate image" below.

[0027] Fig. 5 is a flow chart showing the registration process according to the present invention. This process corresponds to step S106 in Fig. 1, and can be realized, for example, by a computer device executing a program created in advance. As a computer device for executing this process, for example, a commercially available personal computer having a general hardware configuration can be applied. Specifically, a device equipped with a calculation processing device, a storage means such as a memory or storage, an input device, and a display device can be used.

[0028] First, one image that is the basis for registration is selected as a reference image from among multiple specimen images, and another image that is the target of registration processing is selected as a target image (step S201). Here, in relation to Fig. 4, a case is considered in which the first specimen image I1 is the reference image and the second specimen image I2 is the target image. Note that, in terms of determining the relative positional relationship between the two images, it is technically equivalent to use either image as the reference image.

[0029] The selection criteria for the reference image and the target image are arbitrary. For example, one specimen image related to a specific staining method may be previously determined as the reference image, and one standard image randomly selected from other specimen images may be set as the target image. Both the reference image and the target image may be randomly selected. However, it is necessary that a combination that allows the relative positional relationship to be calculated for all images to be analyzed is included in the end. For example, the reference image and the target image may be selected according to a predetermined order. For example, the embodiment may be such that an operator's operation input to select the reference image and the target image from among the specimen images displayed on the display device is accepted.

[0030] A registration amount that quantitatively represents the positional relationship between the reference image and the target image thus selected is obtained. For example, the amount of relative positional deviation between the two images can be the registration amount. As shown in FIG. 2(b) and FIG. 4(b), when aligning images by moving in the X direction, the Y direction, and the θ direction, the amount of relative movement in the X direction, the Y direction, and the θ direction for aligning the reference image and the target image can be the registration amount between the two images. A technique for obtaining the registration amount can be appropriately selected from known techniques and applied.

[0031] Next, it is determined whether the registration has been successful (step S203). If there are common or similar features between the reference image and the target image, these can be used to successfully perform registration to obtain the correct positional relationship between the two. On the other hand, as described above, if such features cannot be found appropriately, the registration may fail. The following operations differ depending on whether the registration has been successful.

[0032] Various criteria can be used to determine whether registration has been successful. For example, when registration is performed using a pattern matching technique, the success or failure of registration can be determined based on the magnitude relationship between the matching score and an appropriately set threshold. In addition, for example, the result of registration may be displayed on a display device so that an operator can determine the success or failure.

[0033] If the registration fails, steps S204 to S209 are executed. On the other hand, if the registration is successful (YES in step S203), these processing steps are skipped. The processing in steps S204 to S209 is a registration process using another image based on the above-mentioned principle. The specific contents of the process will be described below with reference to Figs. 5 and 6.

[0034] Fig. 6 is a diagram showing a schematic diagram of the relationship between a reference image and a target image in a feature space. Each of the sample images, including the reference image and the target image, can be associated with a point in the feature space by expressing the image contents using various features. For convenience of explanation, the figure shows a two-dimensional feature space using feature 1 and feature 2, but more generally, the feature space is defined as a multidimensional hyperspace using multiple types of features.

[0035] Note that the concept of feature space is introduced here to explain the concept of this embodiment, but it does not mean that features are used in actual processing. Therefore, the types of features are not specified. To avoid complexity, only images related to processing are plotted in the figure, but other sample images should also be plotted at appropriate positions in the feature space according to the contents of each image.

[0036] Two images that are relatively close to each other in the feature space have a high degree of similarity in their image content, and the registration success rate is high. On the other hand, two images that are far apart in the feature space have a large discrepancy in their image content, and the registration is more likely to fail.

[0037] The technical idea of ​​the registration process of this embodiment is to introduce a third image I3 as an intermediate image when the distance in the feature space is large and direct registration cannot be established between the reference image I1 and the target image I2, as shown in Fig. 6(a). If registration is established between the reference image I1 and the intermediate image I3, and between the intermediate image I3 and the target image I2, it means that indirect registration between the reference image I1 and the target image I2 has been successful.

[0038] For this reason, the intermediary image I3 is required to be an image located in the middle of the reference image I1 and the target image I2 in the feature space. In other words, in the feature space, the distance between the reference image I1 and the intermediary image I3 and the distance between the target image I2 and the intermediary image I3 are both required to be smaller than the distance between the reference image I1 and the target image I2.

[0039] Therefore, in this embodiment, other specimen images in the vicinity of the target image I2 in the feature space are searched for. It is considered that such images have an extremely high probability of successful registration with the target image I2. Specifically, as shown in FIG. 6(b), other specimen images with the smallest distance D from the target image I2 in the feature space are searched for, and the image is provisionally set as the intermediate image I3.

[0040] Strictly speaking, the distance in the feature space can be expressed by, for example, the Mahalanobis distance calculated based on the value of the feature amount, and a method of searching for an intermediary image from the distance thus calculated is also conceivable. Here, as one example, a specific method of this embodiment is described, which is a relatively simple method and yet can achieve successful registration with a sufficiently high probability.

[0041] In this embodiment, the difference between the target image I2 and all other specimen images except for the reference image I1 and the target image I2 is calculated (step S204). In order to prevent a positional shift between images from appearing as a large difference, it is preferable to perform image processing as pre-processing prior to the calculation of the difference. As the pre-processing, for example, image reduction, grayscale conversion, contrast enhancement, various filter processes, etc. can be applied. This makes it possible to prevent minor differences caused by specimen dirt, noise, etc. from affecting the processing.

[0042] As an index value quantitatively expressing the difference (hereinafter referred to as "difference value"), for example, the difference in brightness between two images can be calculated pixel by pixel, and the absolute value of the difference can be summed up for all pixels. Here, the difference in brightness is used as an index value of the similarity between two images, but "similar" here means that the shapes of the staining patterns (the arrangement of stained and unstained areas) are similar, not that the staining densities are the same. Therefore, in order to suppress the influence of differences in staining densities on the results, an appropriate evaluation value standardized in consideration of the overall density of the image and its variation, for example, may be introduced in place of the above-mentioned difference value.

[0043] It can be assumed that the smaller the difference value thus obtained is, the higher the similarity of the image to the target image I2 is. Therefore, the images are selected as the intermediate images I3 in ascending order of the difference value obtained (step S205). Then, the registration amounts are calculated between the reference image I1 and the intermediate image I3, and between the target image I2 and the intermediate image I3 (steps S206, S207).

[0044] From these results, the amount of registration between the reference image I1 and the target image I2 is calculated (step S208). For example, by adding together the amount of movement of the intermediary image I3 in the X direction, the Y direction, and the θ direction required to fix the reference image I1 and align the intermediary image I3 with it, and the amount of movement of the target image I2 in the X direction, the Y direction, and the θ direction required to fix the intermediary image I3 and align the target image I2 with it, respectively, it is possible to obtain the amount of movement of the intermediary image I3 in the X direction, the Y direction, and the θ direction required to fix the reference image I1 and align the target image I2 with it, respectively.

[0045] From the result thus obtained, it is determined whether the registration between the reference image I1 and the target image I2 has been successful (step S209). The criteria for determining success or failure can be the same as those in step S203. That is, for example, a method can be applied in which a threshold is set for a difference value between the reference image I1 and the target image I2 after alignment based on the calculated registration amount or a matching score in pattern matching, and success or failure is determined by comparing with the threshold. Also, for example, after alignment, the superimposed images may be displayed on a display device, and the operator may determine success or failure.

[0046] Since the intermediary image I3 was selected as an image with a relatively high similarity to the target image I2, it is expected that registration will be established with a high probability between the intermediary image I3 and the target image I2. On the other hand, this is not the case with the reference image I1. In other words, the distance in the feature space between the selected intermediary image I3 and the reference image I1 is not small enough, and in some cases it may be larger than the distance between the reference image I1 and the target image I2, which may result in registration failure.

[0047] If it is determined that the registration has failed (NO in step S209), the process returns to step S205, and the image with the next smallest difference value is newly selected as the intermediate image I3. Then, registration is attempted in the same manner as above.

[0048] As shown in Figure 6(c), when several images I3a, I3b, I3c, ... are extracted from the images located around the target image I2 in feature space in order of their distance from the target image I2, in a situation where registration is successful between multiple sets of images, some of them are expected to be located closer to the reference image I1 than the target image I2 and to be in a position where they can be successfully registered with the reference image I1.

[0049] Therefore, by performing registration several times using the intermediate image I3 selected in ascending order of difference value, the probability of successful registration between the reference image I1 and the target image I2 can be increased.

[0050] When it is determined in step S203 or step S209 that registration has been successful, it is determined whether registration has been completed for all specimen images (step S210). If there are images remaining for which registration has not been completed, new reference images and target images are selected (step S201), and the above process is repeated. In this case, it is desirable to select as the reference image a specimen image for which registration has been completed with other specimen images, so that a combination is obtained that allows calculation of the relative positional relationship for all images to be finally analyzed. In this way, registration can be completed for all specimen images.

[0051] Fig. 7 is a diagram showing an example of the evaluation result of this embodiment. The inventors of this application prepared a total of 16 image sets consisting of 6 or 12 specimen images, including sets of images for which direct registration could not be performed, and performed the registration process of this embodiment on these image sets, obtaining the following results. The success or failure of registration was judged by an expert by visually evaluating the images superimposed as a result of registration, and a case in which the amount of positional deviation between images was suppressed to approximately 3 pixels was deemed successful.

[0052] In other words, when the number of selected intermediary images was limited to one, the registration success rate was about 63%, and when it was increased to two, the success rate increased to about 81%. Furthermore, when the selection of up to three intermediary images was allowed, registration was successful with a probability of almost 100%.

[0053] On the other hand, as shown in the comparative example, when the intermediary images are randomly selected from images other than the reference image and the target image, the success rate increases by increasing the number of intermediary images, but this does not reach the method of this embodiment. This result shows that this embodiment has the effect of improving the success rate of registration.

[0054] As described above, in the registration process of this embodiment, by interposing the intermediate image I3 between the reference image I1 and the target image I2, which cannot be directly registered, it is possible to indirectly establish registration via the intermediate image I3. In this way, it is possible to improve the success rate of registration even between images that have low similarity and cannot be directly registered.

[0055] As described above, in the above embodiment, the tissue specimen S corresponds to the "biological specimen" of the present invention. In addition, in the above embodiment, the reference image I1 corresponds to the "first image" of the present invention, the target image I2 corresponds to the "second image" of the present invention, and the intermediate image I3 corresponds to the "third image" of the present invention. The "specimen image registration method" of the present invention is applied to the registration process shown in step S106 of FIG. 3 and FIG. 5.

[0056] The present invention is not limited to the above-mentioned embodiment, and various modifications can be made without departing from the spirit of the present invention. For example, in the above-mentioned embodiment, the intermediate image I3 is searched for based on the difference value from the target image I2. However, as described above, even if the relationship between the reference image and the target image is reversed, it is technically equivalent, and in that sense, the same results can be obtained even if the intermediate image is searched for near the reference image.

[0057] In addition, in the above embodiment, the reference image I1 is not particularly taken into consideration when selecting the intermediate image I3, but as described below, a selection method is also possible that takes into consideration the characteristics of not only the target image I2 but also the reference image I1.

[0058] Fig. 8 is a diagram showing another example of the selection method of the intermediary image. This modification can be realized by replacing step S205 of the registration processing steps shown in Fig. 5 with steps S205a and S205b shown in Fig. 8(a). In step S205a, a predetermined number of images are selected as candidate images for the intermediary image from among the sample images excluding the reference image I1 and the target image I2, in order of the smallest difference value from the target image I2. Fig. 8(b) shows an example in which three images I3a, I3b, and I3c are selected as candidate images.

[0059] Next, the difference between these candidate images and the reference image I1 is calculated, and the candidate image with the smallest difference value is selected as the intermediary image I3 (step S205b). Then, as shown in FIG. 8(b), among the candidate images I3a to I3c, the one that is closest to the reference image I1, that is, the one that is most similar to the reference image I1 (in this example, image I3c) is selected as the intermediary image I3. In this way, by selecting an image that is highly similar to both the reference image I1 and the target image I2 as the intermediary image I3, it is possible to further improve the success rate of registration. Even in this case, multiple third images I3 may be selected.

[0060] In the above embodiment, the case where the misalignment between images can be expressed by translation and rotation has been described. However, more complicated misalignment may occur, such as misalignment between images caused by distortion during imaging or by the specimen itself. More generally, the positional relationship between two images can be expressed using a transformation matrix even when such misalignment exists. In this case, the registration amount between the first image I1 and the second image I2 can be expressed by the product of the transformation matrix from the first image I1 to the third image I3 and the transformation matrix from the third image I3 to the second image I2.

[0061] As described above by way of example and with reference to specific embodiments, in the specimen image registration method according to the present invention, the third image may include, for example, an image having the highest similarity to the second image among a plurality of specimen images excluding the first and second images. In this way, registration can be successfully performed at least between the second and third images with a high probability. This avoids the selection of an image that is not similar to either the first or second image and would result in a failure in registration with both of them as the third image.

[0062] For example, the third image may include an image that has the smallest difference from the second image among a plurality of specimen images excluding the first and second images. Images with similar image content have a small difference between them. In other words, by using the difference between images as a measure of similarity, it becomes possible to search for similar images based on quantitative information.

[0063] In this case, for example, the difference between each of the multiple specimen images excluding the first and second images and the second image is calculated, and the third image can be selected based on the result. This makes it possible to uniquely select from the multiple specimen images the one with the smallest difference from the second image.

[0064] For example, the difference can be evaluated by an index value based on the sum of the absolute values ​​of the luminance differences between pixels at corresponding positions in two images for all pixels. By evaluating the difference over the entire image, it becomes possible to treat the pixel-by-pixel comparison as a more macroscopic comparison.

[0065] Also, for example, multiple third images may be selected in ascending order of difference from the specimen images. While the third image is selected to be close to the second image, the similarity with the first image is not taken into consideration. Therefore, the image with the smallest difference with the second image is not necessarily the most suitable third image. By selecting multiple third images, the possibility that one having similarity with the first image is included among them is increased, making it possible to improve the success rate of registration.

[0066] In this case, the registration amount between the first image and each of the third images and the registration amount between the second image and each of the third images may be calculated. If the image combinations for which the registration amounts have been calculated in this way include images for which registration between the first image and the third image and between the second image and the third image has been successful, the registration between the first image and the second image can be performed using the results.

[0067] Also, for example, a plurality of specimen images may be selected in ascending order of difference, and at least one of the images may be selected as the third image in descending order of difference from the first image. In this way, by selecting the third image in consideration of not only the second image but also the similarity with the first image, it is possible to further improve the success rate of registration between the first image and the second image through this.

[0068] In addition, as a method for calculating the registration amount between the first image and the second image, for example, there is a method in which the registration amount of the third image when the first image is used as a reference and the registration amount of the second image when the third image is used as a reference are combined to obtain the registration amount of the second image when the first image is used as a reference. This allows registration to be established between the first image and the second image. For example, when the registration amount is expressed using a transformation matrix, the registration amount after combination can be obtained by calculating the product of the transformation matrices.

[0069] The present invention can be implemented as a program for causing a computer device to execute each step of the above-mentioned sample image registration method, or as a computer-readable recording medium on which the program is non-temporarily recorded. Each of the above steps can be realized by a computer device having a general hardware configuration, and by implementing the program in an existing computer device, it is possible to impart new functions to the computer device. [Industrial Applicability]

[0070] This invention is applicable in fields such as medical and biological research, where the same tissue specimen is stained and imaged multiple times, and the images obtained are then integrated and analyzed. It is particularly suitable for analyzing multiply immunostained specimens on a single cell basis. [Explanation of symbols]

[0071] C cells I1 Reference image (first image) I2 Target image (second image) I3 Intermediate image (third image) S Tissue specimen (biological specimen)

Claims

1. A method for registering specimen images, comprising the steps of: staining a biological specimen with a plurality of types of stains and capturing the images for each stain; and performing registration between a first image and a second image, the first image and the second image being different from each other, the first image and the second image being different from each other; searching the plurality of specimen images to select at least one third image different from the first image and the second image; determining an amount of registration between the first image and the third image, and an amount of registration between the second image and the third image, respectively; determining an amount of registration between the first image and the second image based on an amount of registration between the first image and the third image and an amount of registration between the second image and the third image; 1. A method for registering a specimen image, comprising:

2. The method for registering specimen images according to claim 1 , wherein the third image includes an image having the highest similarity to the second image among the plurality of specimen images excluding the first image and the second image.

3. The method for registering specimen images according to claim 1 or 2, wherein the third image includes an image having the smallest difference from the second image among the plurality of specimen images excluding the first image and the second image.

4. The method for registering specimen images according to claim 3 , further comprising the steps of: calculating a difference between the second image and each of the plurality of specimen images excluding the first image and the second image; and selecting the third image based on the result of the calculation.

5. 5. The method for registering specimen images according to claim 3, wherein the difference is evaluated by an index value based on a total value of absolute values ​​of luminance differences between pixels at corresponding positions between the two images for all pixels.

6. 6. The method for registering a specimen image according to claim 3, further comprising the step of selecting a plurality of the third images from among the specimen images in ascending order of the difference.

7. The method for registering a specimen image according to claim 6 , further comprising the step of calculating an amount of registration between the third image and the first image and an amount of registration between the third image and the second image, respectively, for each of the third images.

8. 6. A method for registering a specimen image according to claim 3, further comprising the steps of: selecting a plurality of specimen images in order of decreasing difference from the specimen images; and selecting at least one of the images in order of decreasing difference from the first image as the third image.

9. 9. The method for registering a specimen image according to claim 1, further comprising the step of combining an amount of registration of the third image when the first image is used as a reference and an amount of registration of the second image when the third image is used as a reference, to determine an amount of registration of the second image when the first image is used as a reference.

10. A program for causing a computer device to execute each step of the specimen image registration method according to any one of claims 1 to 9.

11. A computer-readable recording medium having the program according to claim 10 non-transitoryly recorded thereon.

Images