Information processing device, information processing method, and information processing system

Abstract

The present invention makes it possible to more effectively use estimation results. An information processing device, according to the present invention, comprises: a derivation unit (100) for deriving estimation results of a diagnosis for a second pathological image by using a trained model trained by using teacher data including a plurality of first pathological images; and an identification unit (100) for identifying, from among the plurality of first pathological images, a basis image serving as the basis for the derivation of the estimated results by the trained model.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing system. [Background technology]

[0002] In recent years, technologies have been developed to assist doctors and others in making diagnoses by outputting estimated diagnostic results based on a learning model from medical images such as pathological images. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-38467 A [Non-patent literature]

[0004] [Non-Patent Document 1] Yun Liu, Krishna Gadepalli, Mohammad Norouzi, George E. Dahl, Timo Kohlberger, Aleksey Boyko, Subhashini Venugopalan, Aleksei Timofeev, Philip Q. Nelson, Gregory S. Corrado, Jason D. Hipp, Lily Peng, Martin C. Stumpe: Detecting Cancer Metastases on Gigapixel Pathology Images. CoRR abs / 1703.02442 (2017) Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the above-mentioned conventional technology, only the estimated result of diagnosis derived using a learning model or the like is output, and the grounds for deriving the estimated result are not shown. Therefore, users such as doctors cannot judge whether the estimated result derived by the learning model is based on a correct estimation, and cannot fully utilize the estimated result.

[0006] Therefore, the present disclosure proposes an information processing device, an information processing method, and an information processing system that enable more effective use of the estimation result. [Means for solving the problem]

[0007] In order to solve the above problems, an information processing device of one embodiment according to the present disclosure includes a derivation unit that derives an estimated diagnostic result for a second pathological image using a trained model trained using teacher data including a plurality of first pathological images, and an identification unit that identifies, from the plurality of first pathological images, a basis image that was the basis for derivation of the estimated result by the trained model. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a diagnosis support system according to an embodiment. [Diagram 2] FIG. 4 is a diagram for explaining an imaging process according to an embodiment. [Diagram 3] FIG. 4 is a diagram for explaining an imaging process according to an embodiment. [Figure 4] FIG. 13 is a diagram for explaining a process for generating a partial image (tile image). [Diagram 5] FIG. 2 is a diagram for explaining a pathological image according to an embodiment. [Figure 6] FIG. 2 is a diagram for explaining a pathological image according to an embodiment. [Figure 7] FIG. 13 is a diagram showing an example of a viewing manner of a pathological image by a viewer. [Figure 8] FIG. 4 is a diagram illustrating an example of a browsing history storage unit included in the server. [Figure 9A]FIG. 2 is a diagram illustrating a diagnosis information storage unit included in the medical information system. [Figure 9B] FIG. 2 is a diagram illustrating a diagnosis information storage unit included in the medical information system. [Figure 9C] FIG. 2 is a diagram illustrating a diagnosis information storage unit included in the medical information system. [Figure 10] FIG. 2 is a diagram illustrating an example of a derivation device and a display control device according to an embodiment. [Figure 11] 1 is a schematic diagram showing a flow up to providing diagnostic support information by a diagnostic support system according to an embodiment; [Figure 12] FIG. 1 is a schematic diagram for explaining a method for specifying a base image according to an embodiment. [Figure 13] 11 is a schematic diagram for explaining another method for specifying a basis image according to an embodiment. FIG. [Figure 14] FIG. 11 is a diagram showing the relationship between the type of case and the purpose of diagnosis and the magnification of a pathological image suitable therefor. [Figure 15] 11 is a schematic diagram for explaining another method for specifying a basis image according to an embodiment. FIG. [Figure 16] 11 is a schematic diagram for explaining another method for specifying a basis image according to an embodiment. FIG. [Figure 17] FIG. 2 is a diagram showing an example of a diagnosis support UI screen (1) according to an embodiment. [Figure 18] FIG. 2 is a diagram showing an example of a diagnosis support UI screen (2) according to an embodiment. [Figure 19] FIG. 13 is a diagram showing an example of a diagnosis support UI screen (3) according to an embodiment. [Figure 20] FIG. 13 is a diagram showing a modified example of a diagnostic support UI screen (1) according to an embodiment. [Figure 21] 10 is a flowchart illustrating a learning process according to an embodiment. [Figure 22] 11 is a flowchart illustrating a derivation process according to an embodiment. [Figure 23] FIG. 2 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the derivation device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals, and duplicated descriptions will be omitted.

[0010] The present disclosure will be described in the following order. 1. One embodiment 1-1. System configuration 1-2. Various information 1-2-1.Pathological images 1-2-2. Browsing history information 1-2-3. Diagnostic information 1-3. Derivation device 1-4. Flow of generating diagnostic support information 1-5. How to identify the source image 1-5-1. First method of identification 1-5-2. Second identification method 1-5-3. The third method of identification 1-5-4. The fourth method of identification 1-5-5. Other identification methods 1-6.Display control device 1-6-1. Diagnostic support UI screen example 1-6-2. Modified examples of diagnostic support UI screen 1-7. Processing Procedure 1-7-1. Learning process procedure 1-7-2. Derivation process procedure 1-8. Actions and Effects 2. Other embodiments 2-1.Display device 2-2. Imaging device 2-3.Server 2-4.Pathological images 2-5.Hardware configuration

[0011] 1. ONE EMBODIMENT [1-1. System configuration] First, a diagnosis support system (information processing system, information processing device) according to one embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing the diagnosis support system according to the present embodiment. As shown in Fig. 1, the diagnosis support system 1 includes a pathology system 10, a pathology system 20, a medical information system 30, and a derivation device 100.

[0012] The pathology system 10 is a system used mainly by pathologists, and is applied to, for example, laboratories and hospitals. As shown in FIG. 1, the pathology system 10 includes a microscope 11, a server 12, a display control device 13, and a display device 14.

[0013] The microscope 11 is an imaging device that has the function of an optical microscope, captures an image of a specimen, which is an object to be observed, placed on a glass slide, and obtains a pathological image (an example of a medical image), which is a digital image.

[0014] Here, the specimen to be observed may be a specimen derived from a living body such as a specimen or tissue sample taken from a human body (hereinafter referred to as a biological sample) prepared for the purpose of pathological diagnosis. The specimen may be a tissue slice, a cell, or a microparticle, and the specimen is not particularly limited in terms of the type of tissue (e.g., organ, etc.) used, the type of disease to be treated, the attributes of the subject (e.g., age, sex, blood type, race, etc.), or the lifestyle of the subject (e.g., diet, exercise, smoking, etc.). The tissue slice may include, for example, a slice of a stained tissue slice (hereinafter simply referred to as a slice) before staining, a slice adjacent to the stained slice, a slice different from the stained slice in the same block (sampled from the same location as the stained slice), a slice in a different block (sampled from a different location from the stained slice) in the same tissue, a slice taken from a different patient, etc.

[0015] The server 12 is a device that stores and preserves in a storage unit (not shown) the pathological images captured by the microscope 11. When the server 12 receives a viewing request from the display control device 13, it searches for the pathological image from the storage unit (not shown) and sends the searched pathological image to the display control device 13.

[0016] The display control device 13 transmits a request for viewing a pathological image received from a user such as a doctor or a pathologist to the server 12. Then, the display control device 13 controls the display device 14 to display the pathological image received from the server 12.

[0017] The display device 14 has a screen using, for example, a liquid crystal, an EL (Electro-Luminescence), a CRT (Cathode Ray Tube), or the like. The display device 14 may be compatible with 4K or 8K, or may be formed by a plurality of display devices. The display device 14 displays a pathological image controlled to be displayed by the display control device 13. The server 12 also stores browsing history information regarding an area of ​​the pathological image observed by a pathologist via the display device 14. The browsing history information may be, for example, information regarding the browsing history of pathological images obtained from past cases by users such as doctors and pathologists.

[0018] The pathology system 20 is a system applied to a hospital different from that of the pathology system 10. The pathology system 20 includes a microscope 21, a server 22, a display control device 23, and a display device 24. Each unit included in the pathology system 20 is similar to that of the pathology system 10, and therefore a description thereof will be omitted.

[0019] The medical information system 30 is a system that stores information on the diagnosis of a patient. For example, when it is difficult to diagnose a disease condition from an image of an endoscopy or the like at a certain hospital, a biopsy may be performed to make a definitive diagnosis by a pathological diagnosis. A specimen made from tissue collected from a patient is imaged by a microscope 11 of the pathology system 10, and the pathological image obtained by imaging is stored in a server 12. The pathological image is displayed on a display device 14 by a display control device 13, and a pathologist using the pathology system 10 makes a pathological diagnosis. The doctor makes a definitive diagnosis based on the pathological diagnosis result, and the definitive diagnosis result is stored in the medical information system 30. The medical information system 30 stores information on diagnosis such as information for identifying a patient, disease information of the patient, test information and image information used for the diagnosis, diagnosis results, and prescription drugs. The medical information system 30 is called an electronic medical record system or the like.

[0020] However, the accuracy of pathological diagnosis varies depending on the pathologist. Specifically, the diagnosis results based on pathological images may differ depending on the pathologist's years of experience and specialty. For this reason, it is desirable to derive diagnostic support information, which is information that supports diagnosis, using machine learning in order to support pathological diagnosis.

[0021] One example of diagnostic support using machine learning is to train a learning model using pathological images obtained from past cases and diagnostic information issued by doctors or pathologists for those cases as teacher data, and then input a new pathological image to be diagnosed into the trained learning model (hereinafter referred to as the trained model) to derive an estimated diagnosis for the pathological image.

[0022] However, it is difficult for a user, such as a doctor or a pathologist, to determine whether or not an inference result output from a trained model is correct if the user is only presented with the inference result. In other words, unless the user knows what features in the input image are the basis for deriving the inference result, the user cannot determine the reliability of the inference result, and therefore it is difficult for the user to determine whether or not the inference result can be used as a reference for diagnosis.

[0023] Therefore, in this embodiment, in addition to the estimation result based on the trained model, a pathology image or a partial area thereof that was important in deriving the estimation result is presented to the user as a basis image, which allows the user to determine the reliability of the estimation result based on the basis image, thereby enabling a more accurate diagnosis of the case.

[0024] In this embodiment, information on a diagnosis made in the past on a case of the pathological image used as training data may be linked as diagnostic information to the pathological image used as training data. This diagnostic information may include information on the doctor who made the diagnosis, the diagnosis result given by the doctor, the results of various tests performed for the diagnosis, information on the patient of the case, and information on the annotation area added to the corresponding pathological image. In this embodiment, the diagnostic information linked to the pathological image picked up as the basis image is presented to the user together with the estimation result and the basis image. This allows the user to make a diagnosis by referring to past cases, making it possible to make a more correct diagnosis on the case.

[0025] In addition, in this embodiment, the pathological images used as training data are pathological images included in a pyramid-structured tile image group created for each case. Details of the pyramid-structured tile image group will be described later, but generally speaking, the lower the layer, the higher the magnification the higher the resolution of the image group. For the same case, the image group in each layer represents the same entire specimen. In this description, the entire image in the lowest layer, in other words, the image group with the highest magnification, is called a whole slide image.

[0026] In this embodiment, a whole slide image of the specimen may be obtained from a pathology image that displays a part of the specimen picked up as a basis image, and the obtained whole slide image may be presented to the user, thereby enabling the user to make a more accurate diagnosis of the case.

[0027] Furthermore, in this embodiment, information regarding the browsing history of the tile image group (hereinafter referred to as browsing history information) may be presented to the user. This browsing history information may be information indicating, for example, which doctor viewed which pathology image in a past case and to what extent (for example, the number of times or the time, etc.) the doctor viewed the pathology image. In addition, this browsing history information may be presented to the user in a form such as a heat map for whole slide images.

[0028] An example of processing by the derivation device 100 will be described using the example of Fig. 1. In the example of Fig. 1, information regarding diagnoses by pathologists is accumulated daily in the server 12 of the pathology system 10. That is, the server 12 stores a first pathology image, which is a pathology image corresponding to a first diseased tissue, and browsing history information regarding the browsing history of the pathologist for the first pathology image. In addition, in the example of Fig. 1, the derivation device 100 provides diagnostic support information to the pathology system 20.

[0029] First, the derivation device 100 acquires a first pathological image accumulated daily and browsing history information for the first pathological image from the server 12 of the pathology system 10. The derivation device 100 also acquires diagnostic information on a diagnostic result corresponding to the first pathological image from the medical information system 30. The derivation device 100 trains a learning model using the first pathological image and the corresponding diagnostic information as teacher data, thereby generating a trained model for estimating a diagnostic result from a second pathological image corresponding to a second diseased tissue different from the first diseased tissue. The teacher data may include browsing history information for the first pathological image.

[0030] It is also assumed that in the pathology system 20, a second pathological image corresponding to a second diseased tissue is generated by the microscope 21. At this time, when the display control device 23 receives a display request for the second pathological image from a user such as a doctor or a pathologist, the display control device 23 sends the second pathological image to the derivation device 100. The derivation device 100 derives an estimated result of a diagnosis for the case from the second pathological image using the trained model (derivation unit), and outputs the derived estimated result to the display control device 23 as a part of the diagnosis support information.

[0031] Moreover, the derivation device (identification unit) 100 identifies the first pathological image that was important when deriving the inference result as a basis image. Furthermore, the derivation device 100 identifies diagnostic information linked to the identified basis image (first pathological image). Then, the derivation device 100 outputs the identified basis image and diagnostic information to the display control device 23 as a part of diagnostic support information. That is, in this embodiment, the diagnostic support information may include the inference result output from the trained model, the basis image, and the diagnostic information.

[0032] As described above, the derivation device 100 may identify a whole slide image of a group of tile images including a basis image and browsing history information related to the group of tile images of the whole slide image, and output the identified whole slide image and browsing history information to the display control device 23 as part of the diagnostic support information.

[0033] In this way, the derivation device 100 specifies the first pathological image that was important when deriving the estimated result of diagnosis from the second pathological image as a basis image, and further specifies diagnostic information linked to this basis image. Then, the specified basis image and diagnostic information are presented to the user as diagnostic support information together with the derived estimated result. This allows the user to know what pathological image and diagnostic information the estimated result was derived on the basis of, and therefore makes it possible to judge the reliability of the estimated result. This allows the user to judge whether the estimated result is likely to be correct, and therefore makes it possible to make a diagnosis by effectively using the estimated result.

[0034] At this time, by providing the user with whole slide images of the tile images including the evidence image and browsing history information, it is possible to assist the user in making a more accurate diagnosis.

[0035] In the above, an example is shown in which the learning model is trained using the pathological image stored in the server 12 of the pathology system 10 as teacher data, but the derivation device 100 may train the learning model using the pathological image stored in the server 22 of the pathology system 20 as teacher data, or may train the learning model using both the pathological image stored in the server 12 and the pathological image stored in the server 22 as teacher data. That is, the derivation device 100 can use as teacher data any pathological image that has been viewed in the past. In the above, an example is shown in which the derivation device 100 provides diagnostic support information to the display control device 23, but the derivation device 100 may provide diagnostic support information to the display control device 13.

[0036] In the above example, the pathology system 10 and the pathology system 20 are described separately, but the pathology system 10 and the pathology system 20 may be the same system. More specifically, the diagnosis support system 1 may include only the pathology system 10. In this case, the derivation device 100 trains the learning model using the first pathology image stored in the server 12 as teacher data, and provides the diagnosis support information to the display control device 13 in response to a request from the display control device 13. In addition, the diagnosis support system 1 may include three or more pathology systems. In this case, the derivation device 100 may collect pathology images stored in each pathology system to generate teacher data, and train the learning model using this teacher data. In the above example, the medical information system 30 may be the same system as the pathology system 10 or 20. That is, the diagnostic information may be stored in the server 12 or 22.

[0037] The derivation device 100 according to the present embodiment may be realized by a server or a cloud server arranged on a network, or may be realized by a server 12 / 22 arranged in the pathology system 10 / 20. Alternatively, the derivation device 100 may be realized by distributing the components on a system constructed via a network, such as by a part of the derivation device 100 (e.g., the pathology image acquisition unit 131, the diagnosis information acquisition unit 132, and the learning unit 133 in the control unit 130, and the storage unit 120, etc.) arranged on a network, and the remaining part (e.g., the derivation unit 134, etc.) being realized by the server 12 / 22 of the pathology system 10 / 20.

[0038] So far, the diagnosis support system 1 has been briefly described. Below, the configuration and processing of each device will be described in detail, but various information (pathological image data structure, pathological image browsing history information, and diagnosis information) that is the premise of these descriptions will be described first. In the following, an example will be shown in which the derivation device 100 trains a learning model using teacher data accumulated in the pathological system 10 and provides the pathological system 20 with diagnosis support information.

[0039] [1-2. Various information] [1-2-1.Pathological images] As described above, a pathological image is generated by imaging a specimen using the microscope 11 or the microscope 21. First, the imaging process using the microscope 11 or the microscope 21 will be described with reference to Figs. 2 and 3. Figs. 2 and 3 are diagrams for explaining the imaging process according to this embodiment. Since the microscopes 11 and 21 perform similar imaging processes, the microscope 11 will be described here as an example. The microscope 11 described below has a low-resolution imaging section for imaging at low resolution and a high-resolution imaging section for imaging at high resolution.

[0040] In FIG. 2, a glass slide G10 containing a specimen A10 is included in an imaging region R10, which is an area that can be photographed by the microscope 11. The glass slide G10 is placed on, for example, a stage not shown. The microscope 11 generates a whole slide image, which is a pathological image in which the specimen A10 is entirely photographed, by imaging the imaging region R10 using a low-resolution imaging unit. The label information L10 shown in FIG. 2 describes identification information (for example, a character string or a QR code (registered trademark)) for identifying the specimen A10. By associating the identification information described in the label information L10 with a patient, it becomes possible to identify the patient corresponding to the whole slide image. In the example of FIG. 2, "#001" is described as the identification information. Note that, for example, a brief description of the specimen A10 may be described in the label information L10.

[0041] Next, after generating the whole slide image, the microscope 11 identifies an area in which the specimen A10 exists from the whole slide image, and sequentially images each divided area obtained by dividing the area in which the specimen A10 exists by a predetermined size using the high-resolution imaging unit. For example, as shown in FIG. 3, the microscope 11 first images the area R11 and generates a high-resolution image I11 that is an image showing a partial area of ​​the specimen A10. Next, the microscope 11 moves the stage to image the area R12 using the high-resolution imaging unit and generates a high-resolution image I12 corresponding to the area R12. In the same manner, the microscope 11 generates high-resolution images I13, I14, ... corresponding to the areas R13, R14, .... Although only up to the area R18 is illustrated in FIG. 3, the microscope 11 sequentially moves the stage to image all divided areas corresponding to the specimen A10 using the high-resolution imaging unit and generates high-resolution images corresponding to each divided area.

[0042] Incidentally, when the stage is moved, the glass slide G10 may move on the stage. When the glass slide G10 moves, there is a risk that an unphotographed area of ​​the specimen A10 may occur. As shown in FIG. 3, the microscope 11 captures images using a high-resolution imaging unit so that adjacent divided areas partially overlap, thereby preventing the occurrence of an unphotographed area even when the glass slide G10 moves.

[0043] The low-resolution imaging section and the high-resolution imaging section may be different optical systems or may be the same optical system. In the case where they are the same optical system, the microscope 11 changes the resolution according to the imaging target. In the above, an example in which the imaging area is changed by moving the stage has been shown, but the microscope 11 may change the imaging area by moving the optical system (such as the high-resolution imaging section). In addition, in FIG. 3, an example in which the microscope 11 images the specimen A10 from the center. However, the microscope 11 may image the specimen A10 in an order different from the imaging order shown in FIG. 3. For example, the microscope 11 may image the specimen A10 from the outer periphery. In the above, an example in which only the area where the specimen A10 exists is imaged by the high-resolution imaging section has been shown. However, since there are cases in which the area where the specimen A10 exists cannot be accurately detected, the microscope 11 may divide the imaging area R10 shown in FIG. 2 or the entire area of ​​the glass slide G10 and image it by the high-resolution imaging section.

[0044] Next, each high-resolution image generated by the microscope 11 is divided into a predetermined size. As a result, partial images (hereinafter, referred to as tile images) are generated from the high-resolution image. This will be described with reference to FIG. 4. FIG. 4 is a diagram for explaining the process of generating partial images (tile images). FIG. 4 shows a high-resolution image I11 corresponding to the area R11 shown in FIG. 3. Note that, in the following description, it is assumed that the partial images are generated from the high-resolution images by the server 12. However, the partial images may be generated by a device other than the server 12 (for example, an information processing device mounted inside the microscope 11).

[0045] 4, the server 12 generates 100 tile images T11, T12, ... by dividing one high-resolution image I11. For example, if the resolution of the high-resolution image I11 is 2560 x 2560 [pixels], the server 12 generates 100 tile images T11, T12, ... with a resolution of 256 x 256 [pixels] from the high-resolution image I11. In a similar manner, the server 12 generates tile images of the other high-resolution images by dividing them into similar sizes.

[0046] In the example of FIG. 4, the regions R111, R112, R113, and R114 are regions that overlap with other adjacent high-resolution images (not shown in FIG. 4). The server 12 performs stitching processing on the adjacent high-resolution images by aligning the overlapping regions using a technique such as template matching. In this case, the server 12 may generate tile images by dividing the high-resolution image after the stitching processing. Alternatively, the server 12 may generate tile images of regions other than the regions R111, R112, R113, and R114 before the stitching processing, and generate tile images of the regions R111, R112, R113, and R114 after the stitching processing.

[0047] In this manner, the server 12 generates tile images that are the smallest units of the captured image of the specimen A10. The server 12 then generates tile images of different hierarchical levels by sequentially combining the smallest unit tile images. Specifically, the server 12 generates one tile image by combining a predetermined number of adjacent tile images. This will be described with reference to Figs. 5 and 6. Figs. 5 and 6 are diagrams for explaining a pathology image according to this embodiment.

[0048] The upper part of FIG. 5 shows a group of minimum unit tile images generated by the server 12 from each high resolution image. In the example in the upper part of FIG. 5, the server 12 generates one tile image T110 by synthesizing four adjacent tile images T111, T112, T211, and T212 among the tile images. For example, if the resolution of the tile images T111, T112, T211, and T212 is 256×256, the server 12 generates a tile image T110 having a resolution of 256×256. In the same manner, the server 12 generates a tile image T120 by synthesizing four adjacent tile images T113, T114, T213, and T214. In this manner, the server 12 generates a tile image by synthesizing a predetermined number of minimum unit tile images.

[0049] Furthermore, the server 12 generates a tile image by further combining adjacent tile images from among the tile images obtained after combining the minimum unit tile images. In the example of Fig. 5, the server 12 generates one tile image T100 by combining four adjacent tile images T110, T120, T210, and T220. For example, if the resolution of the tile images T110, T120, T210, and T220 is 256 x 256, the server 12 generates a tile image T100 with a resolution of 256 x 256. For example, the server 12 generates a tile image with a resolution of 256 x 256 from an image with a resolution of 512 x 512 obtained by combining four adjacent tile images by performing a four-pixel average, a weighting filter (a process that reflects nearby pixels more strongly than distant pixels), a 1 / 2 thinning process, or the like.

[0050] By repeating this synthesis process, server 12 ultimately generates a single tile image having the same resolution as the resolution of the smallest tile image. For example, as in the above example, if the resolution of the smallest tile image is 256 x 256, server 12 repeatedly performs the above-mentioned synthesis process to ultimately generate a single tile image T1 with a resolution of 256 x 256.

[0051] Fig. 6 shows a schematic diagram of the tile images shown in Fig. 5. In the example shown in Fig. 6, the tile images in the lowest layer are the smallest unit tile images generated by server 12. The tile images in the second layer from the bottom are tile images obtained after the tile images in the lowest layer are combined. Tile image T1 in the top layer indicates one tile image that is finally generated. In this way, server 12 generates, as a pathology image, a tile image group having a hierarchy like a pyramid structure shown in Fig. 6.

[0052] Note that area D shown in FIG. 5 is an example of an area displayed on a display screen of the display device 14 or 24 or the like. For example, assume that the resolution at which the display device can display is three tile images vertically and four tile images horizontally. In this case, as in area D shown in FIG. 5, the level of detail of the specimen A10 displayed on the display device changes depending on the layer to which the tile image to be displayed belongs. For example, when the tile image of the lowest layer is used, a narrow area of ​​the specimen A10 is displayed in detail. Also, the more tile images of higher layers are used, the more coarsely a wide area of ​​the specimen A10 is displayed.

[0053] Server 12 stores tile images of each layer as shown in FIG. 6 in a storage unit (not shown). For example, server 12 stores each tile image together with tile identification information (an example of partial image information) that can uniquely identify each tile image. In this case, when server 12 receives a request to acquire a tile image including the tile identification information from another device (for example, display control device 13 or derivation device 100), server 12 transmits the tile image corresponding to the tile identification information to the other device. Also, for example, server 12 may store each tile image together with layer identification information that identifies each layer and tile identification information that can uniquely identify the tile image within the same layer. In this case, when server 12 receives a request to acquire a tile image including the layer identification information and the tile identification information from the other device, server 12 transmits the tile image that corresponds to the tile identification information to the other device.

[0054] Note that server 12 may store the tile images of each layer as shown in Fig. 6 in a storage device other than server 12. For example, server 12 may store the tile images of each layer in a cloud server or the like. Furthermore, the tile image generation process shown in Figs. 5 and 6 may be executed by a cloud server or the like.

[0055] Furthermore, server 12 does not have to store tile images of all layers. For example, server 12 may store only the tile image of the lowest layer, or may store only the tile image of the lowest layer and the tile image of the top layer, or may store only tile images of a predetermined layer (for example, odd-numbered layers, even-numbered layers, etc.). In this case, when server 12 receives a request from another device for a tile image of a layer that it does not store, server 12 dynamically combines the stored tile images to generate the tile image requested by the other device. In this way, server 12 can prevent storage capacity from becoming overloaded by thinning out the tile images to be saved.

[0056] Although the imaging conditions are not mentioned in the above example, the server 12 may store tile images of each layer as shown in FIG. 6 for each imaging condition. An example of the imaging condition is the focal length of the subject (such as the specimen A10). For example, the microscope 11 may image the same subject while changing the focal length. In this case, the server 12 may store tile images of each layer as shown in FIG. 6 for each focal length. The reason for changing the focal length is that some specimens A10 are semitransparent, and therefore there are focal lengths suitable for imaging the surface of the specimen A10 and focal lengths suitable for imaging the inside of the specimen A10. In other words, the microscope 11 can generate a pathological image of the surface of the specimen A10 and a pathological image of the inside of the specimen A10 by changing the focal length.

[0057] Another example of the imaging condition is a staining condition for the specimen A10. To be more specific, in a pathological diagnosis, a specific portion of the specimen A10 (e.g., a cell nucleus) may be stained with a luminescent material. A luminescent material is, for example, a substance that emits light when irradiated with light of a specific wavelength. Different luminescent materials may be stained for the same specimen A10. In this case, the server 12 may store tile images for each layer as shown in FIG. 6 for each dyed luminescent material.

[0058] Furthermore, the number and resolution of tile images described above are merely examples and can be changed as appropriate depending on the system. For example, the number of tile images that server 12 combines is not limited to four. For example, server 12 may repeat the process of combining 3×3=9 tile images. Furthermore, although the above example shows an example in which the resolution of the tile images is 256×256, the resolution of the tile images may be other than 256×256.

[0059] The display control device 13 uses software that employs a system compatible with the hierarchically structured tile image group described above, and extracts a desired tile image from the hierarchically structured tile image group in response to a user's input operation via the display control device 13, and outputs the extracted tile image to the display device 14. Specifically, the display device 14 displays an image of an arbitrary portion selected by the user among images of an arbitrary resolution selected by the user. This process allows the user to obtain the sensation of observing a specimen while changing the observation magnification. In other words, the display control device 13 functions as a virtual microscope. The virtual observation magnification here actually corresponds to the resolution.

[0060] [1-2-2. Browsing history information] Next, the viewing history information of the pathological image stored in the server 12 or 22 will be described with reference to FIG. 7. FIG. 7 is a diagram showing an example of a viewing manner by a viewer of the pathological image. In the example shown in FIG. 7, it is assumed that a viewer such as a pathologist views the pathological image I10 in the order of areas D1, D2, D3, . . . , and D7. In this case, the display control device 13 first acquires the pathological image corresponding to area D1 from the server 12 in accordance with the viewing operation by the viewer. In response to a request from the display control device 13, the server 12 acquires one or more tile images forming the pathological image corresponding to area D1 from the storage unit, and transmits the acquired one or more tile images to the display control device 13. Then, the display control device 13 displays the pathological image formed from the one or more tile images acquired from the server 12 on the display device 14. For example, when there are multiple tile images, the display control device 13 displays the multiple tile images side by side. Similarly, each time the viewer changes the display area, the display control device 13 obtains from the server 12 the pathological image corresponding to the area to be displayed (areas D2, D3, ..., D7, etc.) and displays it on the display device 14.

[0061] In the example of FIG. 7, the viewer first views a relatively large area D1, and since there is no area in area D1 that the viewer wants to observe carefully, the viewer moves the viewing area to area D2. Then, since there is an area in area D2 that the viewer wants to observe carefully, the viewer enlarges a part of area D2 to view area D3. Then, the viewer further moves to area D4, which is a part of area D2. Then, since there is an area in area D4 that the viewer wants to observe more carefully, the viewer enlarges a part of area D4 to view area D5. In this manner, the viewer also views areas D6 and D7. For example, the pathological images corresponding to areas D1, D2, and D7 are display images at a magnification of 1.25, the pathological images corresponding to areas D3 and D4 are display images at a magnification of 20, and the pathological images corresponding to areas D5 and D6 are display images at a magnification of 40. The display control device 13 acquires and displays the tile images of the hierarchical levels corresponding to each magnification from the hierarchically structured tile image group stored in the server 12. For example, the tile images corresponding to areas D1 and D2 are at a higher level than the tile image corresponding to area D3 (that is, a level closer to tile image T1 shown in FIG. 6).

[0062] While the pathological image is being viewed as described above, the display control device 13 acquires viewing information at a predetermined sampling period. Specifically, the display control device 13 acquires the center coordinates and the display magnification of the viewed pathological image at each predetermined timing, and stores the acquired viewing information in the storage unit of the server 12.

[0063] This point will be described with reference to FIG. 8. FIG. 8 is a diagram showing an example of the browsing history storage unit 12a of the server 12. As shown in FIG. 8, the browsing history storage unit 12a stores information such as "sampling," "center coordinates," "magnification," and "time." "Sampling" indicates the order of timing for storing the browsing information. "Center coordinates" indicates the position information of the viewed pathological image. In this example, the center coordinates are coordinates indicated by the center position of the viewed pathological image, and correspond to the coordinates of the coordinate system of the tile image group of the lowest layer. "Magnification" indicates the display magnification of the viewed pathological image. "Time" indicates the elapsed time from the start of browsing. In the example of FIG. 8, it is shown that the sampling period is 30 seconds. That is, the display control device 13 saves the browsing information in the browsing history storage unit 12a every 30 seconds. However, the sampling period is not limited to this example, and may be, for example, 0.1 to 10 seconds, or may be outside this range.

[0064] In the example of FIG. 8, sampling "1" indicates the browsing information of area D1 shown in FIG. 7, sampling "2" indicates the browsing information of area D2, sampling "3" and "4" indicate the browsing information of area D3, sampling "5" indicates the browsing information of area D4, and sampling "6", "7" and "8" indicate the browsing information of area D5. That is, the example of FIG. 8 indicates that area D1 was viewed for about 30 seconds, area D2 was viewed for about 30 seconds, area D3 was viewed for about 60 seconds, area D4 was viewed for about 30 seconds, and area D5 was viewed for about 90 seconds. In this way, the browsing time of each area can be extracted from the browsing history information.

[0065] Also, the number of times each region has been viewed can be extracted from the browsing history information. For example, each time a change operation of the display region (for example, a movement operation of the display region, a change operation of the display size) is performed, the number of times each pixel of the displayed pathological image is displayed increases by one. For example, in the example shown in FIG. 7, when the region D1 is displayed first, the number of times each pixel included in the region D1 is displayed is one. When the region D2 is displayed next, the number of times each pixel included in both the region D1 and the region D2 is displayed is two, and the number of times each pixel included in the region D2 but not in the region D1 is displayed is one. Since the display region can be specified by referring to the center coordinates and magnification of the browsing history storage unit 12a, the number of times each pixel (which can also be called each coordinate) of the pathological image is displayed can be extracted by analyzing the browsing history information stored in the browsing history storage unit 12a.

[0066] The display control device 13 may suspend the storage process of the viewing information if the viewer does not perform an operation to change the display position for a predetermined time (e.g., 5 minutes). In the above example, the pathological image viewed by the center coordinates and the magnification are stored as the viewing information, but the present invention is not limited to this example, and the viewing information may be any information that can identify the area of ​​the viewed pathological image. For example, the display control device 13 may store tile identification information that identifies a tile image corresponding to the viewed pathological image, or information indicating the position of the tile image corresponding to the viewed pathological image, as the viewing information of the pathological image. Although not shown in FIG. 8, the viewing history storage unit 12a stores information that identifies patients, medical records, etc. That is, the viewing history storage unit 12a shown in FIG. 8 stores the viewing information in a manner that allows it to be associated with patients, medical records, etc.

[0067] [1-2-3. Diagnostic information] Next, the diagnostic information stored in the medical information system 30 will be described with reference to Figs. 9A to 9C. Figs. 9A to 9C are diagrams showing a diagnostic information storage unit possessed by the medical information system 30. Figs. 9A to 9C show examples of storing diagnostic information in different tables for each organ to be examined. For example, Fig. 9A shows an example of a table that stores diagnostic information related to breast cancer examinations, Fig. 9B shows an example of a table that stores diagnostic information related to lung cancer examinations, and Fig. 9C shows an example of a table that stores diagnostic information related to colon examinations.

[0068] The diagnostic information storage unit 30A shown in FIG. 9A stores information such as "patient ID", "doctor ID", "pathological image", "diagnosis result", "grade", "tissue type", "genetic test", "ultrasound test", and "medication". "Patient ID" indicates identification information for identifying a patient. "Doctor ID" indicates identification information for identifying a doctor. This doctor ID may be associated with the doctor's years of experience, field of specialty, and evaluation as a doctor. "Pathological image" indicates a pathological image saved by a pathologist at the time of diagnosis. "Pathological image" may store position information (such as center coordinates and magnification) indicating the image area to be saved with respect to a whole slide image, rather than the image itself. "Diagnosis result" is a diagnosis result by a pathologist, and indicates, for example, the presence or absence of a lesion site and the type of the lesion site. "Grade" indicates the degree of progression of the disease site. "Tissue type" indicates the type of the disease site. "Genetic test" indicates the result of a genetic test. "Ultrasound test" indicates the result of an ultrasound test. Medication indicates information regarding medication to a patient.

[0069] The diagnostic information storage unit 30B shown in Fig. 9B stores information on a "CT examination" performed in a lung cancer examination, instead of the "ultrasound examination" stored in the diagnostic information storage unit 30A shown in Fig. 9A. The diagnostic information storage unit 30C shown in Fig. 9C stores information on an "endoscopic examination" performed in a large intestine examination, instead of the "ultrasound examination" stored in the diagnostic information storage unit 30A shown in Fig. 9A.

[0070] In the examples of Figures 9A to 9C, when "Diagnosis Result" stores "Normal," it indicates that the result of the pathological diagnosis was negative, and when information other than "Normal" is stored in "Diagnosis Result," it indicates that the result of the pathological diagnosis was positive.

[0071] [1-3. Derivation device] Next, the derivation device 100 according to the present embodiment will be described. Here, the derivation device 100 and the display control device 23 will also be described. FIG. 10 is a diagram showing an example of the derivation device and the display control device according to the present embodiment. As shown in FIG. 10, the derivation device 100 is a computer having a communication unit 110, a storage unit 120, and a control unit 130.

[0072] The communication unit 110 is realized by, for example, a network interface card (NIC) or the like. The communication unit 110 is connected to a network (not shown) by wire or wirelessly, and transmits and receives information to and from the pathology system 10, the pathology system 20, the medical information system 30, and the like via the network. The control unit 130, which will be described later, transmits and receives information to and from these devices via the communication unit 110.

[0073] The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 stores the trained model 121 generated by the control unit 130. The trained model 121 will be described later.

[0074] The control unit 130 is realized, for example, by a central processing unit (CPU) or a micro processing unit (MPU) executing a program (an example of a diagnostic assistance program) stored inside the derivation device 100 using a random access memory (RAM) or the like as a working area. The control unit 130 may also be executed by an integrated circuit, for example, an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0075] 10, control unit 130 has pathological image acquisition unit 131, diagnostic information acquisition unit 132, learning unit 133, and derivation unit 134, and realizes or executes the functions and actions of information processing described below. Note that the internal configuration of control unit 130 is not limited to the configuration shown in FIG. 10, and may be other configurations as long as they perform information processing described below.

[0076] The pathological image acquisition unit 131 acquires, via the communication unit 110, a pathological image that is one of the teacher data used for training the learning model performed by the learning unit 133. Specifically, the pathological image acquisition unit 131 acquires a first pathological image corresponding to a first diseased tissue stored in the server 12 of the pathology system 10. In addition, when the browsing history information is used as the teacher data, the pathological image acquisition unit 131 also acquires browsing history information related to browsing of the first pathological image from the server 12. In this case, the browsing history information acquired by the pathological image acquisition unit 131 includes information on an area of ​​the first pathological image that the pathologist viewed by enlarging it, information on an area of ​​the first pathological image that the pathologist viewed for a relatively long time, and information on the number of times the pathologist viewed each area of ​​the first pathological image, as in the example shown in FIG. 8.

[0077] The diagnostic information acquiring unit 132 acquires, via the communication unit 110, diagnostic information that is one of the teacher data used for training the learning model performed by the learning unit 133. Specifically, the diagnostic information acquiring unit 132 acquires, from the medical information system 30, diagnostic information for the first affected tissue corresponding to the first pathological image.

[0078] The learning unit 133 trains a learning model from the correspondence between the first pathological image (and the browsing history information) acquired by the pathological image acquisition unit 131 and the diagnostic information acquired by the diagnostic information acquisition unit 132. In this way, the learning unit 133 generates a trained model 121 for obtaining an estimated result of a diagnosis from the second pathological image. Then, the learning unit 133 stores the trained model 121 in the storage unit 120.

[0079] For example, weakly supervised learning can be applied to the training of the learning model by the learning unit 133, and the following method can also be used. “WILDCAT: Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation”, CVPR 2017 (http: / / webia.lip6.fr / ~durandt / pdfs / 2017_CVPR / Durand_WILDCAT_CVPR_2017.pdf) “Attention-based Deep Multiple Instance Learning”, 2018(https: / / arxiv.org / abs / 1802.04712)

[0080] However, the training method of the learning model by the learning unit 133 may be based on any algorithm. For example, the learning unit 133 can generate the learned model 121 using various learning algorithms such as deep learning, which is a machine learning method using a multi-layered deep neural network, a support vector machine, clustering, and reinforcement learning.

[0081] Also, the learning unit 133 may not train the learning model using all the first pathological images. For example, the learning unit 133 may train the learning model using only the first pathological images that have been particularly noted. For example, the learning unit 133 may train the learning model using only the first pathological images including the area that has been viewed for a predetermined time or more, or may train the learning model using only the first pathological images including the area that has been viewed at a predetermined magnification, or may train using only the first pathological images including the area that has been viewed a predetermined number of times or more, based on the viewing history information of each pathological image. Also, for example, the learning unit 133 may train the learning model using only the area that has been viewed for a predetermined time or more, or may train using only the area that has been viewed at a predetermined magnification, or may train the learning model using only the area that has been viewed a predetermined number of times or more. Also, for example, the learning unit 133 may train the learning model by cutting out only the central area of ​​the first pathological image, assuming that the central area of ​​the first pathological image is the area that has been noted.

[0082] The derivation unit 134 acquires a second pathological image from the display control device 23 and inputs it to the trained model 121, thereby causing the trained model 121 to derive an estimated result of a diagnosis based on the second pathological image, and outputs the derived estimated result to the display control unit 23b.

[0083] The derivation unit 134 also identifies one or more first pathological images that were important in the derivation of the estimation result by the trained model 121 from the pathological image database 211, and outputs the identified first pathological images as basis images to the display control unit 23b. Furthermore, the derivation unit 134 identifies diagnostic information linked to the first pathological image identified as the basis image from the diagnostic information database 212, and outputs the identified diagnostic information to the display control unit 23b.

[0084] In this case, as described above, the derivation unit 134 may acquire a whole slide image of a group of tile images including the first pathological image identified as the basis image, and viewing history information regarding each of the first pathological images that constitute the whole slide image, from the pathological image database 211 or the server 12, and output the acquired whole slide image and viewing history to the display control unit 23b.

[0085] [1-4. Flow of generating diagnostic support information] In this way, the diagnosis support system 1 according to the present embodiment provides users, such as pathologists and doctors who diagnose patients, with diagnostic support information for supporting their diagnosis. The diagnostic support information may include an inference result derived by a trained model based on the second pathological image, a basis image that was important in deriving the inference result, and diagnostic information linked to the basis image.

[0086] FIG. 11 is a schematic diagram showing a flow up to providing diagnostic support information by the diagnostic support system according to this embodiment. As shown in FIG. 11, in this embodiment, first, a user 300 such as a pathologist or a doctor operates the display control device 23 to input a second pathological image 201 of a diagnostic target acquired by the image acquisition unit 23a to the diagnostic support system 1. The second pathological image 201 input to the diagnostic support system 1 is input to the derivation unit 134 of the control unit 130. The derivation unit 134 reads out the trained model 121 from the storage unit 120 and inputs the second pathological image 201 to the trained model 121 to derive an estimated result of a diagnosis based on the second pathological image 201. The derived estimated result is input to the display control unit 23b of the display control device 23 and displayed on the display device 24 as one piece of diagnostic support information, thereby being presented to the user 300.

[0087] In addition, the derivation unit 134 specifies, from the pathology image database 211, a first pathology image that was important when the trained model 121 derives an estimation result based on the second pathology image 201. Then, the derivation unit 134 inputs the specified first pathology image as a basis image to the display control unit 23b of the display control device 23. As a result, the basis image is displayed on the display device 24 as one of the diagnostic support information and presented to the user 300. Note that the pathology image database 211 may be a storage device that stores the first pathology image in the server 12 of the pathology system 10, or may be another storage device that accumulates the first pathology image collected from this storage device. In the former case, the pathology image database 211 may include a storage device that stores the second pathology image in the server 22 of the pathology system 20. In the latter case, the pathology image database 211 may accumulate the second pathology image collected from the storage device of the server 22.

[0088] Furthermore, the derivation unit 134 identifies diagnostic information linked to the first pathological image identified as the basis image from the diagnostic information database 212. Then, the derivation unit 134 inputs the identified diagnostic information to the display control unit 23b of the display control device 23. As a result, the diagnostic image is displayed on the display device 24 as one piece of diagnostic support information and presented to the user 300. Note that the diagnostic information database 212 may be, for example, a storage device that stores diagnostic information in the medical information system 30.

[0089] As described above, a whole slide image and viewing history information may be presented to user 300. In this case, derivation unit 134 identifies a whole slide image of a tile image group including a ground image from pathology image database 211 and inputs it to display control unit 23b, and also acquires a viewing history for a pathology image related to the whole slide image from a viewing history information database (not shown) and inputs it to display control unit 23b.

[0090] [1-5. How to identify the source image] Next, a method for specifying a base image by the derivation unit 134 will be described with some examples.

[0091] [1-5-1. First Identification Method] 12 is a schematic diagram for explaining a method for identifying a basis image according to the present embodiment. As shown in FIG. 12, in the present embodiment, a second pathological image 201 of a diagnosis target is input to a trained model 121, and an estimated result 202 of a diagnosis is output from the trained model 121. Then, a first pathological image that was important in deriving the estimated result 202 is identified from a pathological image database 211, and the identified first pathological image is presented to a user together with the estimated result 202. In the example shown in FIG. 12, a first pathological image #2 is identified as a basis image and presented to a user together with the estimated result 202.

[0092] In addition, in this embodiment, as shown in FIG. 12, diagnostic information #2 linked to the first pathological image #2 identified as the basis image is identified from the diagnostic information database 212, and the identified diagnostic information #2 is presented to the user together with the estimation result 202 and the first pathological image #2 (basis image).

[0093] The pathology image identified as the basis image is at least one of the first pathology images used in training the learning model. Therefore, the basis image can be identified by analyzing how each of the first pathology images influenced the derivation of the estimation result by the trained model 121.

[0094] For example, the non-patent document “Understanding Black-box Predictions via Influence Functions” (Pang Wei Koh and Percy Liang, ICML2017) discloses a method for measuring the importance of a training image in prediction (estimation) by calculating how much the prediction changes when the training image is deleted from the training data.

[0095] Therefore, in this embodiment, the extent to which the estimation result changes when each of the first pathological images used as training data is deleted from the training data is calculated, and the importance of each of the first pathological images in the estimation is measured based on the amount of change. Then, a basis image is specified based on the measured importance. There may be multiple first pathological images specified as the basis image.

[0096] For example, the amount of change in the estimation result may be the amount of change in the score obtained by evaluating the estimation result. However, the present invention is not limited to this, and various values ​​may be used as long as they are the amount of change in the value obtained by quantitatively evaluating the estimation result.

[0097] In addition, in identifying the basis image, one or more first pathological images diagnosed by a veteran diagnostician (such as a doctor or a pathologist) may be preferentially selected. For example, information on the diagnostician who diagnosed each first pathological image may be identified based on the doctor ID in the diagnostic information linked to each first pathological image, and the priority of each diagnostician may be determined from the identified diagnostician information. Then, when there are multiple first pathological images with the same degree of importance measured, the priority of each diagnostician may be identified from the diagnostic information linked to the first pathological image, and one or more first pathological images to be used as the basis image may be selected from the first pathological images with the same degree of importance measured based on the priority.

[0098] Alternatively, instead of priorities, a bias value (coefficient, offset, etc.) may be set for each diagnostician, and one or more basis images may be selected after the importance measured for each first pathological image is increased or decreased by the bias value.

[0099] In this way, by taking into consideration the diagnostician's information when identifying the basis image, it is possible to further improve the reliability of the basis image and diagnostic information presented to the user.

[0100] [1-5-2. Second identification method] Moreover, FIG. 13 is a schematic diagram for explaining another method for identifying a grounds image according to this embodiment, and FIG. 14 is a diagram showing the relationship between the type of case and the purpose of diagnosis and the magnification of a pathological image suitable therefor.

[0101] 13, it is also possible to identify a basis image for each layer of the network of the trained model 121, and present the identified basis image for each layer and its diagnostic information to the user. In other words, when the trained model 121 is configured with multiple layers such as deep learning, it is also possible to identify important basis images for each layer, and present the identified basis image for each layer and its associated diagnostic information to the user.

[0102] Here, in deep learning, it is known that the features of interest change from layer to layer. For example, the first layer immediately after the input layer responds to concrete features such as edges and lines, and as the layers progress, the response changes to abstract features such as texture and color. This is thought to correlate with the magnification of the pathological image. In other words, it is thought that the upper layers (layers close to the input layer) of the trained model 121 respond to weakly magnified (low resolution) pathological images, and the lower layers (layers close to the output layer) respond to strongly magnified (high resolution) pathological images.

[0103] On the other hand, as shown in FIG. 14, in a diagnosis based on a pathological image, there is an appropriate magnification of the pathological image depending on the type of case, the purpose of diagnosis, etc. Therefore, it is desirable that the layer for identifying the basis image is changed depending on the type of case, the purpose of diagnosis, etc. For example, in the example shown in FIG. 14, when the case is "clear cell renal cell carcinoma" of the "kidney" and the purpose of diagnosis is "cancer detection", the basis image is identified in a layer (relatively lower layer) that responds to the feature amount of the pathological image with a magnification of about 5 times to 20 times, and when the case is "clear cell renal cell carcinoma" of the "kidney" and the purpose of diagnosis is "grade detection", the basis image is identified in a layer (relatively higher layer) that responds to the feature amount of the pathological image with a magnification of about 20 times to 40 times. In this way, the layer for identifying the basis image may be switched depending on the type of case, the purpose of diagnosis, etc.

[0104] [1-5-3. The third method of identification] 15 is a schematic diagram for explaining another method for specifying a basis image according to the present embodiment. As illustrated in FIG. 15, a different trained model may be created for each magnification of the pathological image, and the estimation result, the basis image, and the diagnostic information may be specified for each model.

[0105] In the example shown in Figure 15, a trained model 121-10 that derives an inference result 202-10 from a 10x pathological image, a trained model 121-20 that derives an inference result 202-20 from a 20x pathological image, and a trained model 121-40 that derives an inference result 202-40 from a 40x pathological image are prepared, and the basis images and diagnostic information associated with them that were important in deriving each of the inference results 202-10, 202-20, and 202-40 are identified from the pathological image database 211 and the diagnostic information database 212.

[0106] Specifically, 10 times the first pathological image #1-10 that was important when the trained model 121-10 derived the estimation result 202-10 is identified from the pathological image database 211, 20 times the first pathological image #2-20 that was important when the trained model 121-20 derived the estimation result 202-20 is identified from the pathological image database 211, and 40 times the first pathological image #2-40 that was important when the trained model 121-40 derived the estimation result 202-40 is identified from the pathological image database 211. Then, diagnostic information #1-10, #2-20, and #2-40 linked to each of the identified first pathological images #1-10, #2-20, and #2-40, respectively, are identified from the diagnostic information database 212.

[0107] With this configuration, the user can make a diagnosis based on the estimation results derived using the trained model 121 with an appropriate magnification depending on the type of case, the purpose of diagnosis, etc., as well as the basis image and diagnostic information.

[0108] [1-5-4. The fourth method of identification] In addition, for example, a method for visualizing the basis of estimation, such as Grad-CAM or Guided Grad-CAM, can be used to specify the basis image. Fig. 16 is a schematic diagram for explaining another method for specifying the basis image according to the present embodiment.

[0109] 16, when a method for visualizing the basis of estimation such as Grad-CAM or Guided Grad-CAM is used, it is possible to identify the area on the second pathological image 201 to which the neuron corresponds from the most fired neuron in each layer of the trained model 121. Therefore, in this embodiment, it is possible to identify the basis image that was important in deriving the estimation result for each layer of the trained model 121, and to identify which area in the second pathological image 201 contributed to deriving the estimation result for each layer of the trained model 121, and to present them to the user in association with each other.

[0110] In the example shown in FIG. 16, the trained model 121 is composed of three layers from the first layer to the third layer (excluding the input layer and the output layer), and in the first layer, the first pathological image #1 is specified as the basis image and the neuron corresponding to the region R1 in the second pathological image 201 fires the most, in the second layer, the first pathological image #2 is specified as the basis image and the neuron corresponding to the region R2 in the second pathological image 201 fires the most, and in the third layer, the first pathological image #N is specified as the basis image and the neuron corresponding to the region R3 in the second pathological image 201 fires the most. Therefore, the display device 24 that presents the diagnostic support information to the user displays the first pathological images #1, #2, #N that were important in each layer and the diagnostic information #1, E2, #N linked thereto, and an image 201A showing the positions on the second pathological image 201 of the regions R1, R2, R3 corresponding to the neurons that fired the most in each layer. The background of image 201A showing the positions of regions R1, R2, and R3 may be, for example, second pathological image 201.

[0111] [1-5-5. Other identification methods] The basis image that was important in the diagnosis can also be regarded as a similar image of the second pathological image (image to be estimated), which is an input image. That is, in this embodiment, instead of the first pathological image that was important in the machine learning, or together with the first pathological image that was important in the machine learning, it is also possible to use the first pathological image similar to the second pathological image as the basis image as a result of similar image search in the pathological image database 211. However, the method of using the first pathological image that was important in the machine learning as the basis image is considered to be more explanatory than the method of searching for a similar image according to the distance of the color or texture feature amount, and is therefore more useful for diagnosis.

[0112] Furthermore, when there is another heterostained image (for example, an image obtained by immunostaining) different from the first pathological image and this heterostained image is stored in the pathological image database 211 etc. in association with the first pathological image, the heterostained image associated with the basis image may be presented to the user together, thereby making it possible to provide the user with information more useful for diagnosis.

[0113] Furthermore, the above-mentioned methods for specifying a basis image may be combined as appropriate.

[0114] [1-6. Display control device] Next, the display control device 23 will be described. The display control device 23 is equipped with a display control program for presenting the diagnostic assistance information generated as described above to users such as doctors and pathologists. However, without being limited to this, the display control program may be downloaded from a server or installed in a general-purpose computer from a storage medium such as a DVD (Digital Versatile Disc), thereby realizing the operation of the display control device 23 described below. In addition, the operation of the display control device 23 described below may be realized by performing processing on two or more devices, such as performing some processing on a server and other processing on a client computer such as the display control device 23. Furthermore, the operation of the display control device 23 described below may be realized by running the display control program on a cloud.

[0115] [1-6-1. Diagnostic support UI screen example] As shown in Fig. 10, the display control device 23 has an image acquisition unit 23a and a display control unit 23b, and realizes or executes the functions and actions of information processing described below. The internal configuration of the display control device 23 is not limited to the configuration shown in Fig. 10, and may be other configurations as long as they perform the information processing described below. The image acquisition unit 23a and the display control unit 23b are realized by, for example, a CPU or MPU executing a display control program stored in the display control device 23 using a RAM or the like as a working area. The image acquisition unit 23a and the display control unit 23b may be executed by an integrated circuit such as an ASIC or an FPGA.

[0116] The image acquiring unit 23a acquires a second pathological image from the server 22, and transmits the acquired second pathological image to the derivation device 100. For example, the image acquiring unit 23a transmits the second pathological image that the pathologist is going to diagnose to the derivation device 100 in accordance with an operation by the pathologist. Note that the image acquiring unit 23a and the display control unit 23b may be the same unit, and the process of transmitting the second pathological image to the derivation device 100 may be performed by the display control unit 23b.

[0117] The display control unit 23b receives diagnostic assistance information output from the trained model 121 that receives the second pathological image as an input from the derivation device 100. Then, the display control unit 23b controls the display device 24 to display the received diagnostic assistance information to the user. Note that, hereinafter, the display control unit 23b controlling the display device 24 to display various information such as diagnostic assistance information on the display device 24 may be simply expressed as the display control unit 23b displaying various information.

[0118] FIG. 17 is a diagram showing an example of a diagnostic support UI screen (1) according to this embodiment. FIG. 18 is a diagram showing an example of a diagnostic support UI screen (2) according to this embodiment, and FIG. 19 is a diagram showing an example of a diagnostic support UI screen (3) according to this embodiment. The diagnostic support UI screen (2) is a display screen for a whole slide image of a tile image group including a grounds image, and the diagnostic support UI screen (3) is a display screen for a heat map of a browsing history (also called a browsing heat map). The diagnostic support UI screens (1) to (3) shown in FIG. 17 to FIG. 19 are generated by, for example, the display control device 23 and displayed on the display device 24.

[0119] 17 shows an example of a diagnosis support UI screen (1) in which an input image 201, an inference result 202, a grounds image 203, and diagnostic information 204 are presented to a user side by side. Note that the input image is the second pathological image 201 of the diagnosis target, and therefore the same reference numeral '201' is used here. Note that in this embodiment, as exemplified in FIG. 17, the inference result 202 is displayed as an image based on the second pathological image 201. A hatched region R202 in the image of the inference result 202 may be, for example, a region in the second pathological image 201 that is inferred as a cancer cell.

[0120] In addition, when the user selects, for example, the basis image 203 by clicking on it on the diagnostic support UI screen (1), as shown in FIG. 18, the display device 24 may display a diagnostic support UI screen (2) that presents the user with a whole slide image 205 of a group of tile images including the first pathological image selected as the basis image 203.

[0121] Furthermore, when the user selects, for example, by clicking on the “Viewing Heat Map” button 215 displayed on the diagnostic support UI screen (2), as shown in FIG. 19, the display device 24 may display a viewing heat map 206 generated based on the viewing history information of the image group for the first pathological image identified as the basis image 203.

[0122] [1-6-2. Modified examples of diagnostic support UI screen] As described above in "1-5-5. Other Identification Methods", when a first pathological image similar to a second pathological image, which is an input image 201, is identified from a pathological image database 211 by a similar image search, as shown in FIG. 20, a first pathological image 213 identified as a pathological image similar to the input image 201 as a result of the similar image search in the pathological image database 211 and diagnostic information 214 associated therewith may be displayed as a basis image and diagnostic information in the diagnosis support UI screen (1) together with a first pathological image (basis image 203) identified as important in machine learning and diagnostic information 204 associated therewith. Note that, as described above, instead of the first pathological image identified as important in machine learning and its diagnostic information, a first pathological image identified by a similar image search and its diagnostic information may be displayed as a basis image 203 and diagnostic information 204 in the diagnosis support UI screen (1).

[0123] Furthermore, the diagnostic support UI screen (1) may display an evaluation result (also called a score) evaluating the reliability (also called the accuracy rate) of the estimation result derived by the trained model 121.

[0124] [1-7. Processing Procedure] Next, a processing procedure according to this embodiment will be described in detail with reference to the drawings.

[0125] [1-7-1. Learning process procedure] Fig. 21 is a flowchart showing a learning process procedure according to this embodiment. As shown in Fig. 21, in this operation, the derivation device 100 first judges whether it is time to learn (step S101), and if it is not time to learn (step S101; NO), the derivation device 100 waits for this operation. For example, if the learning date and time for learning is determined in advance, the derivation device 100 judges whether the current date and time is the learning date and time.

[0126] If it is the learning timing (step S101; Yes), the derivation device 100 acquires the first pathological image and the diagnostic information as teacher data from the pathological image database 211 and the diagnostic information database 212 (step S102). At that time, the derivation device 100 may acquire the browsing history information as a part of the teacher data from the server 12.

[0127] Next, the derivation device 100 trains the learning model in the storage unit 120 by using the first pathological image and the diagnosis information as teacher data (step S103). Then, the derivation device 100 stores the trained model 121 generated by the training in the storage unit 120 (step S104), and ends this operation. As a result, the trained model 121 for deriving an estimated result of diagnosis from the pathological image is arranged in the storage unit 120.

[0128] 21 illustrates an example in which the trained model 121 is newly generated, but the derivation device 100 can also retrain the trained model 121. In this case, a new pathological image may be used for retraining the trained model 121.

[0129] [1-7-2. Derivation process procedure] Fig. 22 is a flowchart showing a derivation process procedure according to the present embodiment. As shown in Fig. 22, in this operation, the derivation device 100 first determines whether or not a second pathological image has been received from the display control device 23 (step S201).

[0130] Next, the derivation device 100 acquires the trained model 121 from the storage unit 120 (step S202). Next, the derivation device 100 inputs the second pathological image received in step S201 into the trained model 121, thereby deriving an estimated result of a diagnosis for the second pathological image (step S203).

[0131] Next, the derivation device 100 identifies the first pathological image that was important in deriving the estimation result by the trained model 121 as a basis image (step S204). The method described in "1-5. Method for identifying basis image" above may be used to identify this basis image.

[0132] Next, the derivation device 100 acquires, from the diagnostic information database 212, diagnostic information linked to the first pathological image identified as the basis image in step S204 (step S205).

[0133] In addition, the derivation device 100 acquires a whole slide image of a tile image group including the first pathological image identified as the basis image in step S204 from the pathological image database 211 (step S206), and acquires browsing history information regarding the tile image group of this whole slide image from the server 12 (step S207).

[0134] Next, the derivation device 100 inputs the estimation result, the basis image, the diagnostic information, the whole slide image, and the browsing history information acquired as described above to the display control unit 23b of the display control device 23 (step S208), and ends this operation. In response to this, the display control unit 23b generates diagnostic support UI screens (1) to (3) as exemplified in Figs. 17 to 20, and inputs them to the display device 24. As a result, the display device 24 displays the diagnostic support UI screens (1) to (3) as exemplified in Figs. 17 to 20.

[0135] [1-8. Actions and Effects] As described above, according to this embodiment, when presenting to a user an inferred result of a diagnosis derived by the trained model 121 trained using pathology images acquired from past cases, it is possible to present to the user the pathology image on which the inferred result was derived, so that the user can determine on what pathology image the inferred result was derived. This allows the user to determine the reliability of the inferred result based on the basis of the image, so that a more correct diagnosis can be made for the case.

[0136] In addition, according to this embodiment, in addition to the estimation result and the basis image, it is possible to present to the user diagnostic information linked to the basis image, thereby enabling the user to make a more accurate diagnosis by referring to past cases.

[0137] Furthermore, according to this embodiment, it is also possible to present to the user whole slide images of tile images including evidence images and the viewing history of pathology images, thereby providing the user with information to make a correct diagnosis.

[0138] (2. Other embodiments) The processing according to the above-described embodiment may be implemented in various different forms other than the above-described configuration.

[0139] [2-1.Display device] In the above embodiment, an example has been shown in which the diagnostic support UI screens (1) to (3) are displayed on a stationary display device 24. However, the diagnostic support information may be displayed on a wearing device (such as a head-mounted display) worn by a viewer viewing the pathological image displayed on the display device 24. In this case, the diagnostic support information may be superimposed on the pathological image displayed on the display device 24. In addition, the diagnostic support information may be displayed on a transparent display that is worn so as to cover the front surface of the display device 24. In this case, the diagnostic support information may be displayed on the transparent display so as to be superimposed on the pathological image displayed on the display device 24.

[0140] [2-2. Imaging device] In the above embodiment, a microscope has been described as an example of a device for capturing images of a specimen, but the present invention is not limited to this. For example, the device for capturing images of a specimen may be a medical image acquisition device such as an endoscope for capturing images of the inside of a patient's body, a computed tomography (CT), or a magnetic resonance image (MRI). In this case, the server 12 or the server 22 stores medical images such as two-dimensional still images or moving images generated by an endoscope and three-dimensional images generated by a CT or an MRI. The server 12 or the server 22 may also store information related to the images, such as imaging conditions and diagnosis results for the images, in association with these images.

[0141] [2-3. Server] Further, other pathological images captured by other medical image acquisition devices such as an endoscope, CT, or MRI may be stored in the server 12 or the server 22 in association with the pathological image generated by the microscope. In this case, the display control unit 23b may display, for reference, the other pathological images captured by the other imaging devices side by side in addition to the pathological image generated by the microscope.

[0142] [2-4. Pathological images] Among the pathological images stored in the server 12 and the server 22, some have low resolution. That is, the resolution of the pathological image used as the teacher data may not have a resolution sufficient to properly train the learning model. Here, when a glass slide containing a specimen is stored, the glass slide may be re-imaged with a high-resolution microscope to generate a new high-resolution pathological image. For this reason, when the resolution of the first pathological image used as the teacher data is not sufficient to properly train the learning model and there is a re-imaged pathological image, the derivation device 100 may use the re-imaged pathological image as the teacher data to train the learning model.

[0143] [2-5. Hardware configuration] Information devices such as the derivation devices 100 and 200 and the display control device 23 according to the above-described embodiments are realized by a computer 1000 having a configuration as shown in FIG. 23, for example. The following describes the derivation device 100 according to the above-described embodiment as an example. FIG. 23 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the derivation device 100. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. Each unit of the computer 1000 is connected by a bus 1050.

[0144] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processes corresponding to the various programs.

[0145] The ROM 1300 stores boot programs such as a basic input output system (BIOS) executed by the CPU 1100 when the computer 1000 is started up, and programs that depend on the hardware of the computer 1000 .

[0146] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records a response generation program according to the present disclosure, which is an example of program data 1450.

[0147] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0148] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading a program or the like recorded on a predetermined computer-readable recording medium. The medium may be, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0149] For example, when the computer 1000 functions as the derivation device 100 according to the above embodiment, the CPU 1100 of the computer 1000 executes the diagnostic assistance program loaded on the RAM 1200 to realize functions such as the pathological image acquisition unit 131, the diagnostic information acquisition unit 132, the learning unit 133, and the derivation unit 134. The HDD 1400 stores the diagnostic assistance program according to the present disclosure and data in the storage unit 120. For example, when the computer 1000 functions as the display control device 23 according to the above embodiment, the CPU 1100 of the computer 1000 executes the display control program loaded on the RAM 1200 to realize functions such as the image acquisition unit 23a and the display control unit 23b. The HDD 1400 stores the display control program according to the present disclosure. The CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, the CPU 1100 may acquire these diagnostic assistance programs and display control programs from other devices via the external network 1550.

[0150] [others] Among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

[0151] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0152] Furthermore, the above-described embodiments and modifications can be appropriately combined as long as the processing contents are not contradictory.

[0153] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0154] The present technology can also be configured as follows. (1) a derivation unit that derives an estimated result of a diagnosis for the second pathological image by using a trained model trained using teacher data including a plurality of first pathological images; An identification unit that identifies a basis image that is a basis for deriving the estimation result by the trained model from among the plurality of first pathological images; An information processing device comprising: (2) The identifying unit further identifies diagnostic information associated with the first pathological image identified as the evidence image. The information processing device according to (1). (3) The trained model is composed of a plurality of layers, The derivation unit identifies, for each of the plurality of layers, the basis image that is the basis for deriving the estimation result from among the plurality of first pathological images. The information processing device according to (1) or (2). (4) the plurality of first pathological images include a plurality of first pathological images obtained by imaging a same specimen prepared from a biological sample at different magnifications, The derivation unit derives the estimation result for each magnification by using each trained model prepared for each magnification of the first pathological image; The identification unit identifies, from among the plurality of first pathological images, the basis image that was the basis for deriving the estimation result in each of the trained models for each of the magnifications. The information processing device according to any one of (1) to (3) above. (5) The trained model is composed of a plurality of layers, The identification unit identifies an area on the second pathology image corresponding to the most fired neuron in each of the plurality of layers. The information processing device according to any one of (1) to (4). (6) the identifying unit identifies one or more first pathological images as the basis image from among the plurality of first pathological images, and identifies the diagnostic information of each of the one or more first pathological images; The information processing device according to (2). (7) The diagnostic information includes information regarding a diagnostician who diagnosed the first pathological image linked to the diagnostic information, The identification unit selects one or more first pathological images from the one or more first pathological images based on information about the diagnostician. The information processing device according to (6). (8) The first and second pathological images are image data obtained by imaging a specimen prepared from a biological sample. The information processing device according to any one of (1) to (7) above. (9) The plurality of first pathological images includes an image group consisting of a plurality of first pathological images acquired by imaging the same specimen at different magnifications. The information processing device according to (8). (10) the plurality of first pathology images include whole slide images including an entire image of the specimen; The identification unit acquires the whole slide image included in the same image group as the first pathology image identified as the basis image from the plurality of first pathology images. The information processing device according to (9). (11) Further comprising a storage unit that stores a past viewing history for each of the plurality of first pathological images, The identification unit acquires a viewing history of each of the first pathology images included in the image group including the acquired whole slide image. The information processing device according to (10). (12) The image processing device further includes a display control unit that causes a display device to display the estimation result derived by the derivation unit and the ground image identified by the identification unit. The information processing device according to any one of (1) to (11) above. (13) deriving an estimated result of a diagnosis for the second pathological image using a trained model trained using training data including a plurality of first pathological images; Identifying a basis image that is a basis for deriving the estimation result by the trained model from among the plurality of first pathological images. An information processing method comprising the steps of: (14) an information processing device that derives an estimated result of a diagnosis for a pathology image obtained by imaging a specimen prepared from a biological sample, from the pathology image; The information processing device includes: Deriving an estimated result of a diagnosis for the second pathological image using a trained model trained using training data including a plurality of first pathological images; Identifying a basis image that is a basis for deriving the estimation result by the trained model from among the plurality of first pathological images; A program for executing the An information processing system comprising: [Explanation of symbols]

[0155] 1. Diagnostic support system 10, 20 Pathological Systems 11, 21 Microscope 12, 22 Server 13, 23 Display control device 14, 24 Display device 23a Image acquisition section 23b Display control unit 30 Medical Information Systems 100 Derivation device 110 Communications Department 120 Storage section 121, 121-10, 121-20, 121-40 Trained models 130 Control section 131 Pathological Image Acquisition Unit 132 Diagnostic information acquisition unit 133 Learning Department 134 Derivation part 201 Second Pathological Image 201A Images 202, 202-10, 202-20, 202-40 Estimation results 203 Image evidence 204, 214 Diagnostic information 211 Pathology Image Database 212 Diagnostic Information Database 213 First Pathological Image

Claims

1. a derivation unit that derives an estimated result of a diagnosis for the second pathological image by using a trained model trained using teacher data including a plurality of first pathological images; An identification unit that identifies one or more first pathology images from among the plurality of first pathology images as a basis image that serves as a basis for derivation of the estimation result by the trained model; Equipped with The identification unit identifies diagnostic information of each of the one or more first pathological images linked to the one or more first pathological images identified as the basis image, The diagnostic information includes information regarding a diagnostician who diagnosed the first pathological image linked to the diagnostic information, The identification unit is an information processing device that selects one or a plurality of first pathological images from the one or more first pathological images based on information about the diagnostician.

2. The first and second pathological images are image data obtained by imaging a specimen prepared from a biological sample. The information processing device according to claim 1 .

3. The plurality of first pathological images includes an image group consisting of a plurality of first pathological images acquired by imaging the same specimen at different magnifications. The information processing device according to claim 2 .

4. the plurality of first pathology images include whole slide images including an entire image of the specimen; The identification unit acquires the whole slide image included in the same image group as the first pathology image identified as the basis image from the plurality of first pathology images. The information processing device according to claim 3 .

5. A storage unit that stores a past viewing history for each of the plurality of first pathological images, The identification unit acquires a viewing history of each of the first pathology images included in the image group including the acquired whole slide image. The information processing device according to claim 4.

6. The image processing device further includes a display control unit that causes a display device to display the estimation result derived by the derivation unit and the ground image identified by the identification unit.

6. The information processing device according to claim 1.

7. deriving an estimated result of a diagnosis for the second pathological image using a trained model trained using training data including a plurality of first pathological images; Identifying one or more first pathological images from among the plurality of first pathological images as a basis image on which the trained model derives the estimation result; Identifying diagnostic information of each of the one or more first pathological images linked to the one or more first pathological images identified as the basis image; Including, The diagnostic information includes information regarding a diagnostician who diagnosed the first pathological image linked to the diagnostic information, Selecting one or more first pathology images from the one or more first pathology images based on information about the diagnostician. The information processing method further comprises:

8. an information processing device that derives an estimated result of a diagnosis for a pathology image obtained by imaging a specimen prepared from a biological sample, from the pathology image; The information processing device includes: Deriving an estimated result of a diagnosis for the second pathological image using a trained model trained using training data including a plurality of first pathological images; Identifying one or more first pathology images from among the plurality of first pathology images as a basis image that is a basis for deriving the estimation result by the trained model; Identifying diagnostic information of each of the one or more first pathological images associated with the one or more first pathological images identified as the basis image; A program for executing The diagnostic information includes information regarding a diagnostician who diagnosed the first pathological image linked to the diagnostic information, selecting one or more first pathology images from the one or more first pathology images based on information about the diagnostician; and a program for further executing the An information processing system comprising:

Images