Image diagnosis support device, image diagnosis support method, and program

Abstract

An image diagnosis support device capable of more accurately calculating diagnosis support information for supporting diagnosis is provided. [Solution] A region A estimation unit 203 estimates a region A in a second image 308 that corresponds to a region A stained with a first stain in the first image 307, based on a first image 307 that depicts tissue collected from a patient subject and stained with a first stain, and a second image 308 that depicts tissue collected from the patient and stained with a second stain. A region B estimation unit 204 estimates a region B in the second image 308 that is stained with a second stain, based on the second image 308. Diagnostic assistance information that assists in diagnosing the patient is calculated based on regions A and B in the second image 308.

Description

[Technical Field]

[0001] The present disclosure relates to an image diagnosis support device, an image diagnosis support method, and a program. [Background technology]

[0002] Immunohistochemistry, such as PD-L1 staining, which stains for a protein called PD-L1, is often used as a companion diagnostic to predict the efficacy of immune checkpoint inhibitors in pathological diagnosis in oncology treatment. Immunohistochemistry predicts efficacy based on the tumor proportion score (TPS), which is the ratio of positive tumor cells to total tumor cells in an immunostained image generated from patient tissue. However, accurate differentiation of tumor cells requires the use of additional stained images, such as hematoxylin-eosin (HE)-stained images, in addition to the immunostained images. Furthermore, the need to count tumor cells and positive tumor cells in the immunostained images places a significant burden on pathologists. In particular, the number of immune checkpoint inhibitors and immunostaining methods is expected to increase in the future, which is likely to increase the burden on pathologists.

[0003] In response to this, Patent Document 1 discloses a technique for detecting an area depicting a first biological object such as a lymphocyte and an area depicting a second biological object such as a tumor cell from a pathology image showing a tissue slice taken from a patient, aligning the areas, and calculating a spatial distribution measurement that characterizes the positional relationship between the areas. This technique predicts the impact of a predetermined treatment on the patient based on the spatial distribution measurement. Therefore, if a positive rate or the like is calculated as a spatial distribution measurement, it is possible to reduce the burden on pathologists. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent Specification 2023 / 0143860 Summary of the Invention [Problem to be solved by the invention]

[0005] However, since the immunostained images and other stained images used in the above-described companion diagnosis are created from different sections, they are not always aligned as in the technique described in Patent Document 1, which poses a problem that it may be difficult to accurately calculate an index for predicting medication efficacy, such as a positive rate. Note that this problem is not limited to companion diagnosis, but can arise in general image diagnosis in which diagnostic support information for supporting diagnosis is calculated from multiple stained images.

[0006] An object of the present disclosure is to provide an image diagnosis support device, an image diagnosis support method, and a program that are capable of more accurately calculating diagnostic support information for supporting diagnosis. [Means for solving the problem]

[0007] An image diagnosis support device according to one aspect of the present disclosure is an image diagnosis support device that supports a diagnosis for a subject, and includes: a first input unit that accepts a first image of tissue collected from the subject and stained with a first staining agent; a second input unit that accepts a second image of tissue collected from the subject and stained with a second staining agent; a first estimation unit that estimates a corresponding first region in the second image that corresponds to a first region in the first image stained with the first staining agent based on the first image and the second image; a second estimation unit that estimates a second region in the second image stained with the second staining agent based on the second image; a calculation unit that calculates diagnostic support information that supports a diagnosis for the subject based on the corresponding first region and the second region; and an output unit that outputs the diagnostic support information. [Effects of the Invention]

[0008] According to the present invention, it is possible to more accurately calculate diagnostic assistance information for assisting diagnosis. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an example of a hardware configuration of an image diagnosis support apparatus according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating an example of a functional configuration of an image diagnosis support apparatus according to an embodiment of the present disclosure. [Figure 3] 10A and 10B are diagrams for explaining an example of a procedure for creating a first image and a second image. [Figure 4] FIG. 1 is a diagram for explaining a problem with the conventional technology. [Figure 5] FIG. 1 is a diagram for explaining a problem with the conventional technology. [Figure 6] FIG. 2 is a diagram for explaining the processing performed by each functional unit of the image diagnosis support device. [Figure 7] FIG. 2 is a diagram illustrating an example of the configuration of a region A estimation unit. [Figure 8] FIG. 10 is a diagram illustrating another example of the configuration of the area A estimation unit. [Figure 9] FIG. 10 is a diagram illustrating an example of an output screen. [Figure 10] FIG. 10 is a diagram showing another example of an output screen. [Figure 11] 10 is a flowchart illustrating an operation of the image diagnosis support device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0011] 1 is a diagram illustrating an example of the hardware configuration of an image diagnosis support device according to an embodiment of the present disclosure. The image diagnosis support device 101 illustrated in FIG. 1 includes an interface 110, a memory 111, and a computing unit 112, which are connected to each other via a bus 113 so as to be able to communicate with each other.

[0012] The interface 110 is communicably connected to an external device and transmits and receives data to and from the external device. In this embodiment, an imaging device 120 and a display device 121 are shown as the external devices. However, the external devices are not limited to this example. The imaging device 120 acquires multiple stained images based on different sections of tissue collected from a patient, who is a subject, and transmits them to the image diagnosis support device 101. The display device 121 displays various information such as processing results of the image diagnosis support device 101.

[0013] The memory 111 stores a program (computer program) that defines the operation of the arithmetic unit 112 and various data generated or used by the program. The arithmetic unit 112 is, for example, a processor that reads the program stored in the memory 111 and executes the read program to realize various functional units. At least a part of the program stored in the memory 111 may be stored in a storage medium 131 that non-temporarily stores data.

[0014] Fig. 2 is a diagram showing an example of the functional configuration of the image diagnosis support device 101. As shown in Fig. 2, the image diagnosis support device 101 has, as functional units realized by the calculator 112 executing a program, a first image input unit 201, a second image input unit 202, an area A estimation unit 203, an area B estimation unit 204, a diagnosis support information calculation unit 205, and an output unit 206.

[0015] The first image input unit 201 is a first input unit that accepts a first image as a stained image. The second image input unit 202 is a second input unit that accepts a second image as a stained image.

[0016] The region A estimation unit 203 is a first estimation unit that estimates a corresponding first region, region A, based on the first image and the second image. The region B estimation unit 204 is a second estimation unit that estimates a second region, region B, based on the second image.

[0017] The diagnostic assistance information calculation unit 205 is a calculation unit that calculates diagnostic assistance information for assisting in diagnosis of a patient based on the region A and the region B. The output unit 206 outputs the diagnostic assistance information.

[0018] FIG. 3 is a diagram illustrating an example of a procedure for creating the first image and the second image.

[0019] First, tissue 302 is collected from a patient 301. The method for collecting the tissue is not particularly limited. Examples include surgically excising the tissue or extracting the tissue by needle biopsy.

[0020] The collected tissue 302 is then fixed in a fixative such as formalin and further embedded in paraffin to produce embedded tissue 303. The embedded tissue 303 is cut into a plurality of sections 304.

[0021] One of the sections 304 is mounted on a first image glass slide 305 and stained with a first staining agent. Another section 304 is mounted on a second image glass slide 306 and stained with a second staining agent different from the first staining agent. The section 304 for the first image and the section 304 for the second image are preferably adjacent sections.

[0022] The sections 304 mounted on the first image glass slide 305 and the second image glass slide 306 are imaged by the imaging device 120, and a first image 307 and a second image 308 are created, respectively.

[0023] In this embodiment, first image 307 is an HE-stained image stained with an HE staining agent, and second image 308 is an immunostained image stained with an immunostaining agent. In the example of FIG. 3 , the stained region in first image 307 includes tumor region 310, which depicts tumor cells. Furthermore, the stained region in second image 308 includes tumor region 311, which depicts tumor cells, and peripheral region 312, which depicts specific non-tumor cells such as lymphocytes and macrophages that are located around tumor region 311. Furthermore, tumor region 311 includes positive tumor region 311A, which depicts positive tumor cells, and negative tumor region 311B, which depicts negative tumor cells. Positive tumor region 311A ​​and peripheral region 312 show strong staining, while negative tumor region 311B shows weak staining. Note that positive tumor cells are tumors that express a protein to be stained by the immunostaining agent, and negative tumor cells are tumors that do not express the protein to be stained.

[0024] First image 307 and second image 308 are created from different sections 304, so tumor region 310 and tumor region 311 may differ in size. Furthermore, first image 307 and second image 308 are different stained images, so the appearance of the stained region, such as the color, is different, and the objects stained are also different.

[0025] The diagnostic assistance information in the above example is expressed by, for example, the following equation 1.

number

[0026] However, the diagnostic assistance information is not limited to the positive rate, and is calculated based on region A and region B, and is not particularly limited as long as it contributes to the diagnosis of the patient. For example, the diagnostic assistance information may be a value other than the positive rate, such as the number of cells in the positive tumor region 311A ​​described above, or a predetermined determination result determined from a value such as the positive rate. An example of this type of determination is a positive determination on a sample-by-sample basis. In this case, the determination result indicates negative (e.g., a positive rate of 1% or less), low expression positive (e.g., a positive rate of 1% to 49%), high expression positive (e.g., a positive rate of 50% or more), etc.

[0027] Furthermore, when the first image is an HE-stained image and the second image is a specially stained image, the diagnostic assistance information may be information relating to a specific property according to the type of special stain used for the specially stained image (for example, information indicating whether the image is elastic fiber, collagen fiber, or mucus). Furthermore, the position of tissue exhibiting a specific property may be displayed superimposed on the first image 307 or the second image 308.

[0028] 4 and 5 are diagrams for explaining a conventional image diagnosis support device according to the technology described in Patent Document 1. FIG.

[0029] 4 and 5, in the conventional device, a region A estimation unit 503 estimates a region A508 of a first image 307 using only the first image 307 received by a first image input unit 501, and a region B estimation unit 504 estimates a region B509 of a second image 308 using only the second image 308 received by the first image input unit 501. A registration unit 505 registers the first image 307 and the second image 308 to estimate a region A510 in the second image 308. Then, a diagnostic assistance information calculation unit 506 calculates diagnostic assistance information (in the example shown in the figure, a positive rate) based on the region B509 and the region A510 in the second image 308. Note that a region 511 represents region A510 ∩ region B509 in the second image 308.

[0030] In this case, if tumor region 521 in first image 307 and tumor region 522 in second image 308 are similar as shown in FIG. 4, no particular problem occurs with the diagnostic support information.

[0031] However, as shown in FIG. 5 , if tumor region 521 in first image 307 and tumor region 522 in second image 308 differ significantly, registration may fail to properly estimate region A510 in second image 308. That is, region A510 in second image 308 differs significantly from tumor region 522 in second image 308. As a result, the positive rate, which is diagnostic support information, may not be accurately estimated. For example, as shown in the example of FIG. 5 , if tumor region 521 in first image 307 is larger than tumor region 522 in second image 308, region A510 will be larger than tumor region 522, and non-tumorous regions will also be referenced for calculating the positive rate. Conversely, if tumor region 521 in first image 307 is smaller than tumor region 522 in second image 308, region A510 will be smaller, and region 511 will be limited to that smaller region, and only a portion of the tumor will be referenced for calculating the positive rate.

[0032] FIG. 6 is a diagram for explaining an example of a method for calculating diagnostic support information by the image diagnosis support device 101 of this embodiment.

[0033] It may be difficult to directly estimate region A using only second image 308. For example, even medical professionals and other experts may find it difficult to correctly estimate all tumor cells using only an immunostained image. In this case, referring to both first image 307 and second image 308 is effective for accurate diagnosis. However, conventional devices that estimate and register region A from first image 307 have issues such as those shown in the example of Figure 5.

[0034] In contrast to this, in this embodiment, the area A estimation unit 203 estimates area A in the second image 308 by using both the first image 307 and the second image 308. Specifically, the area A estimation unit 203 estimates area A in the second image 308 by collecting information necessary to identify whether or not area A in the first image 307 is in the second image 308, based on feature amounts that are local information in the second image 308.

[0035] FIG. 7 is a diagram showing an example of the configuration of the area A estimation unit 203. As shown in FIG.

[0036] First, a region of interest 702 is set in the second image 308. The region A estimation unit 203 sets a region of interest 701 in the first image 307 based on the set region of interest 702. The region of interest 702 may be set manually or automatically. The region of interest 701 is set so as to include the region corresponding to the region of interest 702 and its surrounding region. In other words, the region of interest 701 is a larger region than the region of interest 702.

[0037] The region A estimation unit 203 inputs the regions of interest 701 and 702 to a feature extractor 703 .

[0038] Feature extractor 703 calculates and outputs feature maps of the input regions of interest 701 and 702. In the example of FIG. 7 , feature extractor 703 is divided into feature extractors 701A and 702A for generating a region-of-interest selection signal (described later) and feature extractors 701B and 702B for identifying region A, respectively, for regions of interest 701 and 702. However, the configuration of feature extractor 703 is not limited to this example. For example, there may be only one feature extractor for region of interest 701. In this case, the output of that feature extractor is used for both generating a region-of-interest selection signal and identifying region A. Furthermore, the output of one feature extractor 703 corresponding to the same region of interest may be used as the input of the other feature extractor 703.

[0039] Based on the feature amount map of the region of interest 701 and the feature amount map of the region of interest 702, the region of interest selection signal generator 704 estimates a region of interest 707, which is a region in the first image 307 used to estimate a region A in the second image 308 in the region of interest 701, and outputs a region of interest selection signal indicating the estimation result. The region A in the second image 308 is a corresponding region A, which is a region in the second image 308 that corresponds to the region A in the first image 307. Note that there may be multiple regions of interest 707.

[0040] The attention region selection signal generator 704 uses, for example, attention in machine learning, to estimate whether each region corresponding to each cell (grid) of the feature map in the region of interest 701 is an attention region. The attention region selection signal is, for example, a signal that indicates "1" for each region in the region of interest 701 if it is an attention region, and indicates "0" if it is not an attention region.

[0041] Based on the region of interest selection signal, the feature selector 705 extracts estimation features, which are features used to estimate region A in the region of interest 701, from the feature map of the region of interest 701. For example, the feature selector 705 extracts features of cells corresponding to regions where the region of interest selection signal is "1."

[0042] The classifier 706 classifies whether the region of interest 702 is region A based on the estimation feature of the region of interest 701 and the feature map of the region of interest 702. Note that if it is possible to classify whether the region of interest 702 is region A or not based solely on the estimation feature output by the feature selector 705, the feature map extracted from the region of interest 702 does not need to be used.

[0043] The region A estimation unit 203 performs the above process while shifting the position of the region of interest 702 to estimate the region A over the entire second image 308 and outputs the estimation result 708.

[0044] The feature extractor 703, the region-of-interest selection signal generator 704, the feature selector 705, and the classifier 706 can be realized using a machine learning model such as a neural network model. For model training, for example, there is a method that uses, as a loss function, whether the estimation result 708 is estimated to be region A when the region of interest 702 is region A, or whether the estimation result 708 is estimated to be not region A when the region of interest 702 is not region A.

[0045] 8 is a diagram showing another example of the configuration of the area A estimation unit 203. The area A estimation unit 203 shown in FIG. 8 includes a first image area A estimator 710 in addition to the configuration shown in FIG.

[0046] The first image area A estimator 710 estimates the area A in the first image 307 and outputs the estimation result, the first image area A estimation result 711 .

[0047] The region of interest selection signal generator 704 estimates the region of interest based on the feature amount map of the region of interest 701 and the feature amount map of the region of interest 702, as well as the region A in first image estimation result 711. In this case, if the region of interest 702 is a region that should be estimated as region A, the region of interest is selected from region A in the first image 307, and if the region of interest 702 is a region that should not be estimated as region A, the region of interest is selected from a region that is not region A in the first image 307, thereby improving the accuracy of the region of interest selection. Furthermore, when machine learning is performed, a loss function that evaluates whether the first image 307 is selected from region A or non-region A indicated in the region A in first image estimation result 711 is added to the loss function described in FIG. 7.

[0048] Instead of providing the first image area A estimator 710, a manually created annotation may be used as the first image area A estimation result 711. In this case, the first image area A estimation result 709 is not generated during inference, so the configuration in Fig. 8 is used during learning, and the configuration in Fig. 7 is used during inference.

[0049] FIG. 9 is a diagram showing an example of an output screen output by the output unit 206. As shown in FIG.

[0050] A result display window 801, which is an output screen shown in FIG. 9, includes an estimated result image display section 802, a control panel section 803, and a diagnostic assistance information display section 804.

[0051] 9, the estimation result image includes a corresponding area A805 (area surrounded by a double line) which is the estimation result of area A in second image 308, and a common area 806 (area surrounded by a dashed line) which is the estimation result of the intersection of corresponding area A805 and area B in second image 308.

[0052] The control panel unit 803 is an operation unit for operating the estimation result image display unit 802, and accepts operations by the user to instruct enlargement, reduction, movement, etc. of the estimated image result. The output unit 206 enlarges, reduces, moves, etc. the estimated image result in response to the operations.

[0053] The diagnostic assistance information display section 804 displays the diagnostic assistance information. In the example of Fig. 9, the positive rate ((the number of cells in the common area 806) / (the number of cells in the corresponding area A805)) is displayed as the diagnostic assistance information.

[0054] It should be noted that the output screen shown in FIG. 9 is merely an example and is not limited to this example. For example, the position of the common area 806 may be displayed as diagnostic assistance information. In this case, the estimation result image display unit 802 also serves as the diagnostic assistance information display unit 804. Alternatively, any area based on the corresponding area A, area B, or the common part of areas A and B may be displayed. Alternatively, there may be multiple types of diagnostic assistance information, and diagnostic assistance information may be displayed according to a user selection.

[0055] 10 is a diagram showing another example of the output screen. In the example of FIG. 10, a result display window 801 includes an estimated result image display section 802 and a first image display section 807.

[0056] The estimation result image display section 802 further shows a region of interest 808 in the second image 308 in addition to the configuration of the example in FIG.

[0057] 10, the first image display unit 807 displays a region of interest 809 and a region of attention 810 in the first image 307. Specifically, when a region of interest 808 is set by the user in the estimation result image display unit 802, a region of interest 809 corresponding to the region of interest 808 and a region of attention 810 within the region of interest 809 are displayed.

[0058] The region of interest 808 may not be set manually by the user, but may be set automatically by the output unit 206. For example, the output unit 206 may display the sum or average of the regions of interest 810 calculated by scanning the region of interest 808. Furthermore, the first image display unit 807 may further display the first image region A estimation result 709, i.e., region A.

[0059] FIG. 11 is a flowchart for explaining the operation of the image diagnosis support device 101.

[0060] In the image diagnosis support device 101, first, the first image input unit 201 accepts a first image (step 1001), and the second image input unit 202 accepts a second image (step 1002).

[0061] The region A estimation unit 203 estimates a corresponding region A, which is a region in the second image that corresponds to the region A stained with the first staining agent in the first image, based on the first image and the second image (step 1003). The region B estimation unit 204 estimates a region B stained with the second staining agent in the second image, based on the second image (step 1004).

[0062] The diagnostic assistance information calculation unit 205 calculates diagnostic assistance information for assisting in the diagnosis of the patient based on the corresponding area A and area B (step 1005).

[0063] The output unit 206 outputs the diagnostic assistance information (step 1006).

[0064] As described above, according to this embodiment, the region A estimation unit 203 estimates a region A in the second image 308 that corresponds to the region A stained with the first stain in the first image 307, based on a first image 307 that depicts tissue collected from a patient subject and stained with a first stain, and a second image 308 that depicts tissue collected from the patient and stained with a second stain. The region B estimation unit 204 estimates a region B in the second image 308 that is stained with the second stain, based on the second image 308. Diagnostic assistance information for assisting in the diagnosis of the patient is calculated based on the regions A and B in the second image 308. Therefore, since it is possible to estimate the region A in the second image 308, it is possible to more accurately calculate the diagnostic assistance information for assisting in the diagnosis.

[0065] In this embodiment, the area A estimation unit 203 estimates the corresponding first area based on the feature amount maps of the first image and the second image, which makes it possible to estimate the corresponding first area more accurately.

[0066] Furthermore, in this embodiment, the region A estimation unit 203 estimates, for each of a plurality of selected regions obtained by dividing the second image, a region of interest in the first image that is used to estimate whether the selected region is a corresponding first region, based on the feature amount maps of the first image and the second image, and estimates whether the selected region is a corresponding first region based on the feature amount of the region of interest. In this case, it is possible to more accurately estimate the corresponding first region.

[0067] In this embodiment, the region A estimation unit 203 estimates the region A in the first image, and selects a region of interest for each selected region based on the estimation result. In this case, it is possible to improve the accuracy of selecting the region of interest.

[0068] In this embodiment, the output unit 206 further outputs information indicating the region of interest in the first image, allowing the user to understand which part of the first image the region A estimation unit 203 focused on when estimating the corresponding region.

[0069] The above-described embodiments of the present disclosure are merely illustrative examples of the present disclosure, and are not intended to limit the scope of the present disclosure to these embodiments alone. Those skilled in the art may implement the present disclosure in various other forms without departing from the scope of the present disclosure. [Explanation of symbols]

[0070] 101: Image diagnosis support device 110: Interface 111: Memory 112: Computing unit 113: Bus 120: Imaging device 121: Display device 131: Storage medium 201: First image input unit 202: Second image input unit 203: Region A estimation unit 204: Region B estimation unit 205: Diagnostic support information calculation unit 206: Output unit 701: Region of interest 701A: Feature extractor 701B: Feature extractor 702: Region of interest 703: Feature extractor 704: Region of interest selection signal generator 705: Feature selector 706: Classifier 707: Region of interest 708: Estimation result 710: First image region A estimator 711: First image region A estimation result

Claims

1. An image diagnosis support device for supporting a diagnosis of a subject, a first input unit that receives a first image of a tissue sample taken from the subject and stained with a first stain; a second input unit that receives a second image of tissue collected from the subject and stained with a second stain; a first estimation unit that estimates a corresponding first region, which is a region in the second image that corresponds to a first region stained with the first staining agent in the first image, based on the first image and the second image; a second estimation unit that estimates a second region in the second image that is stained with the second staining agent based on the second image; a calculation unit that calculates diagnostic assistance information that assists in a diagnosis of the subject based on the corresponding first region and the second region; an output unit that outputs the diagnostic assistance information.

2. The image diagnosis support device according to claim 1 , wherein the first estimating unit estimates the corresponding first region based on feature maps of the first image and the second image.

3. 3. The image diagnosis support device according to claim 2, wherein the first estimation unit estimates, for each of a plurality of selected regions obtained by dividing the second image, a region of interest in the first image to be used for estimating whether the selected region is the corresponding first region, based on feature amount maps of the first image and the second image, and estimates whether the selected region is the corresponding first region based on feature amounts of the region of interest.

4. The image diagnosis support device according to claim 3 , wherein the first estimation unit estimates the first region in the first image, and selects the region of interest for each selected region based on the estimation result.

5. The image diagnosis support device according to claim 3 , wherein the output unit further outputs information indicating the region of interest in the first image.

6. the first image is an HE-stained image stained with hematoxylin and eosin as the first staining agent; the second image is an immunostained image stained with an immunostaining agent as the second staining agent, The image diagnosis support device according to claim 1 , wherein the diagnostic support information is a positive rate of tumor cells.

7. the first image is an HE-stained image stained with hematoxylin and eosin as the first staining agent; the second image is a specially stained image stained with a special stain as the second staining agent, The image diagnosis support device according to claim 1 , wherein the diagnostic support information is information about properties of tumor cells according to the type of the special staining agent.

8. 1. An image diagnosis support method using an image diagnosis support device that supports a diagnosis for a subject, comprising: receiving a first image of tissue taken from the subject and stained with a first stain; receiving a second image of tissue taken from the subject and stained with a second stain; estimating a corresponding first region, which is a region in the second image that corresponds to a first region in the first image, based on the first image and the second image; estimating a second region within the second image based on the second image; calculating diagnostic assistance information for assisting a diagnosis for the subject based on the corresponding first region and the second region; and outputting the diagnostic support information.

9. In estimating the corresponding first region, for each of a plurality of selected regions obtained by dividing the second image, an attention region is estimated, which is an area in the first image used to estimate whether the selected region is the corresponding first region, and whether the selected region is the corresponding first region is estimated based on a feature amount of the attention region; The image diagnosis support method according to claim 8 , wherein the output of the diagnosis support information further includes outputting information indicating the region of interest in the first image.

10. the first image is an HE-stained image stained with hematoxylin and eosin as the first staining agent; the second image is an immunostained image stained with an immunostaining agent as the second staining agent, The image diagnosis support method according to claim 8 , wherein the diagnostic support information is a positive rate of tumor cells.

11. the first image is an HE-stained image stained with hematoxylin and eosin as the first staining agent; the second image is a specially stained image stained with a special stain as the second staining agent, The image diagnosis support method according to claim 8 , wherein the diagnostic support information is information about properties of tumor cells according to the type of the special staining agent.

12. On the computer, a first input unit that receives a first image of tissue collected from the subject and stained with a first stain; a second input unit that receives a second image of tissue collected from the subject and stained with a second stain; a first estimation unit that estimates a corresponding first region, which is a region in the second image that corresponds to a first region stained with the first staining agent in the first image, based on the first image and the second image; a second estimation unit that estimates a second region in the second image that is stained with the second staining agent based on the second image; a calculation unit that calculates diagnostic assistance information that assists in a diagnosis of the subject based on the corresponding first region and the second region; an output unit that outputs the diagnostic assistance information; and a program for realizing this output unit.

13. the first estimation unit estimates, for each of a plurality of selected regions obtained by dividing the second image, a region of interest that is a region in the first image used to estimate whether the selected region is the corresponding first region, and estimates whether the selected region is the corresponding first region based on a feature amount of the region of interest; The program according to claim 12 , wherein the output unit further outputs information indicating the region of interest within the first image.

14. the first image is an HE-stained image stained with hematoxylin and eosin as the first staining agent; the second image is an immunostained image stained with an immunostaining agent as the second staining agent, The program according to claim 13 , wherein the diagnostic assistance information is a positive rate of tumor cells.

15. the first image is an HE-stained image stained with hematoxylin and eosin as the first staining agent; the second image is a specially stained image stained with a special stain as the second staining agent, The program according to claim 13 , wherein the diagnostic assistance information is information about properties of tumor cells according to the type of the special staining agent.

Images