DIAGNOSIS SUPPORT PROGRAM, DIAGNOSIS SUPPORT SYSTEM, AND DIAGNOSIS SUPPORT METHOD

The diagnosis support system addresses the inefficiency of manual annotation in pathological image diagnosis by using viewing history and diagnostic information to automatically identify regions of interest, improving diagnostic accuracy and efficiency.

JP7589679B2Active Publication Date: 2024-11-26SONY GROUP CORP
2 Cites 0 Cited by

Patent Information

Application Number
JP2021501643
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-02-28
Filing Date
2019-12-25
Publication Date
2024-11-26
Estimated Expiration
2039-12-25

AI Technical Summary

Technical Problem

Conventional methods for improving diagnostic accuracy in pathological images require significant manual effort and time for annotating lesions, making it difficult to efficiently enhance diagnostic precision.

Method used

A diagnosis support system that utilizes a derivation device to derive diagnostic support information based on pathological image viewing history and diagnostic information, generating a learning model to estimate regions of interest without manual annotation, thereby improving diagnostic accuracy.

Benefits of technology

The system reduces the time and effort required for annotation while enhancing diagnostic precision by automatically identifying regions of interest in pathological images, allowing for more accurate and efficient pathological diagnosis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007589679000001
    Figure 0007589679000001
  • Figure 0007589679000002
    Figure 0007589679000002
  • Figure 0007589679000003
    Figure 0007589679000003
Patent Text Reader

Abstract

The diagnostic assistance program causes a computer to execute a derivation procedure to derive diagnostic assistance information that assists in the diagnosis of a second pathological image corresponding to a second diseased tissue to be diagnosed, based on a first pathological image corresponding to a first diseased tissue, history information regarding the viewing history for the first pathological image, and diagnostic information for the first diseased tissue corresponding to the first pathological image.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a diagnostic support program, a diagnostic support system, and a diagnostic support method. [Background technology]

[0002] In recent years, a technology has been developed that marks (hereinafter referred to as annotation) areas where a lesion may exist as an area requiring attention in medical images such as pathological images. For example, Non-Patent Document 1 below discloses a technology that displays an area of ​​interest by learning annotated HE (hematoxylin and eosin, general tissue staining) image data. It is believed that this technology can improve the accuracy of diagnosis. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] "Annotation of Whole Slide Images Using Touchscreen Technology"; Jessical L. Baumann, Kari Garsha, Mike S. Flores, Faith Ough, Ehab A. ElGabry; Roche Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the above-mentioned conventional technology, a huge amount of work is required to improve diagnostic accuracy. Specifically, the work of annotating pathological images is performed manually, for example, and therefore requires a lot of time and workers. In other words, in the above-mentioned conventional technology, it is difficult to easily improve diagnostic accuracy.

[0005] Therefore, the present disclosure has been made in consideration of the above, and proposes a diagnostic support program, a diagnostic support system, and a diagnostic support method that can improve the accuracy of diagnosis. [Means for solving the problem]

[0006] In order to solve the above problem, a diagnostic support program according to an embodiment of the present disclosure includes: a first pathological image corresponding to a first diseased tissue; Specific areas of for At least one of the following: viewing time, number of views, and viewing magnification Historical information about your browsing history A region of interest of the first pathological image based on for a first diseased tissue corresponding to the first pathological image, The information includes at least one of the following: the presence or absence of a lesion, the type of the lesion, and the degree of progression of the lesion. and based on the diagnostic information, a second pathological image corresponding to a second affected tissue of the diagnosis target, which is acquired by a system different from the system that acquired the first pathological image. Indicate an area of ​​interest The computer is caused to execute a derivation procedure for deriving diagnostic assistance information. [Brief description of the drawings]

[0007] [Figure 1] 1 is a diagram showing a diagnosis support system 1 according to a first embodiment. [Diagram 2] FIG. 4 is a diagram for explaining an imaging process according to the first embodiment. [Diagram 3] FIG. 4 is a diagram for explaining an imaging process according to the first embodiment. [Figure 4] FIG. 13 is a diagram for explaining a process for generating a partial image (tile image). [Diagram 5] FIG. 2 is a diagram for explaining a pathological image according to the first embodiment. [Figure 6] FIG. 2 is a diagram for explaining a pathological image according to the first embodiment. [Figure 7] FIG. 13 is a diagram showing an example of a viewing manner of a pathological image by a viewer. [Figure 8] FIG. 4 is a diagram illustrating an example of a browsing history storage unit included in the server. [Figure 9A] FIG. 2 is a diagram illustrating a diagnosis information storage unit included in the medical information system. [Figure 9B]FIG. 2 is a diagram illustrating a diagnosis information storage unit included in the medical information system. [Figure 9C] FIG. 2 is a diagram illustrating a diagnosis information storage unit included in the medical information system. [Figure 10] 1 is a diagram illustrating an example of a derivation device and a display control device according to a first embodiment. [Figure 11] 3A to 3C are diagrams illustrating an example of a pathological image displayed by the display control device according to the first embodiment. [Figure 12] 5 is a flowchart showing a learning process procedure according to the first embodiment. [Figure 13] 5 is a flowchart showing a derivation process procedure according to the first embodiment. [Figure 14] 13A and 13B are diagrams illustrating an example of display of a pathological image according to a modified example. [Figure 15] 13A and 13B are diagrams illustrating an example of display of a pathological image according to a modified example. [Figure 16] 13A and 13B are diagrams illustrating an example of display of a pathological image according to a modified example. [Figure 17] 13A and 13B are diagrams illustrating an example of display of a pathological image according to a modified example. [Figure 18] 13A and 13B are diagrams illustrating an example of display of a pathological image according to a modified example. [Figure 19] 13A and 13B are diagrams illustrating an example of display of a pathological image according to a modified example. [Figure 20] 13A and 13B are diagrams illustrating an example of display of a pathological image according to a modified example. [Figure 21] 13A and 13B are diagrams illustrating an example of display of a pathological image according to a modified example. [Figure 22] 13A and 13B are diagrams illustrating an example of display of a pathological image according to a modified example. [Diagram 23] 11 is a diagram illustrating an example of a derivation device and a display control device according to a second embodiment. FIG. [Figure 24] FIG. 2 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the derivation device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals, and duplicated descriptions will be omitted.

[0009] The present disclosure will be described in the following order. <First embodiment> 1. System Configuration According to the First Embodiment 2. Various Information 2-1.Pathological images 2-2. Browsing history information 2-3.Diagnostic Information 3.Device configuration 3-1. Lead-out device according to the first embodiment 3-2. Display control device according to the first embodiment 4. Processing Procedure 4-1. Learning process procedure according to the first embodiment 4-2. Derivation process procedure according to the first embodiment 5. Variation 1 5-1. Display example (1) ~ 5-10. Display example (10) 5-11. Pathological images taken under different imaging conditions 5-12. Diagnostic prediction results 6. Variation 2 6-1. Learning process (1) to 6-8. Learning process (8) <Second embodiment> <Other embodiments>

[0010] (First embodiment) [1. Configuration of the system according to the first embodiment] First, a diagnosis support system 1 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing the diagnosis support system 1 according to the first embodiment. As shown in Fig. 1, the diagnosis support system 1 includes a pathology system 10, a pathology system 20, a medical information system 30, and a derivation device 100.

[0011] The pathology system 10 is a system used mainly by pathologists, and is applied to, for example, laboratories and hospitals. As shown in FIG. 1, the pathology system 10 includes a microscope 11, a server 12, a display control device 13, and a display device 14.

[0012] The microscope 11 is an imaging device that has the function of an optical microscope, captures an image of an observation object placed on a glass slide, and obtains a pathological image (an example of a medical image) that is a digital image. The observation object is, for example, tissue or cells collected from a patient, a piece of flesh from an organ, saliva, blood, etc.

[0013] The server 12 is a device that stores and preserves in a storage unit (not shown) the pathological images captured by the microscope 11. When the server 12 receives a viewing request from the display control device 13, it searches for the pathological image from the storage unit (not shown) and sends the searched pathological image to the display control device 13.

[0014] The display control device 13 sends the request for viewing the pathological image received from the user to the server 12. Then, the display control device 13 controls the display device 14 to display the pathological image received from the server 12.

[0015] The display device 14 has a screen using, for example, a liquid crystal, an EL (Electro-Luminescence), a CRT (Cathode Ray Tube), or the like. The display device 14 may be compatible with 4K or 8K, or may be formed by a plurality of display devices. The display device 14 displays a pathological image controlled to be displayed by the display control device 13. Note that, as will be described in detail later, the server 12 stores viewing history information regarding the area of ​​the pathological image observed by the pathologist via the display device 14.

[0016] The pathology system 20 is a system applied to a hospital different from the pathology system 10. The pathology system 20 includes a microscope 21, a server 22, a display control device 23, and a display device 24. Each unit included in the pathology diagnosis system 20 is similar to that of the pathology system 10, and therefore a description thereof will be omitted.

[0017] The medical information system 30 is a system that stores information on the diagnosis of a patient. For example, when it is difficult to diagnose a disease condition from an image of an endoscopy or the like at a certain hospital, a biopsy may be performed to make a definitive diagnosis by a pathological diagnosis. A tissue specimen made from tissue collected from a patient is imaged by a microscope 11 of the pathology system 10, and the pathological image obtained by imaging is stored in a server 12. The pathological image is displayed on a display device 14 by a display control device 13, and a pathologist using the pathology system 10 makes a pathological diagnosis. The doctor makes a definitive diagnosis based on the pathological diagnosis result, and the definitive diagnosis result is stored in the medical information system 30. The medical information system 30 stores information on diagnosis such as information for identifying a patient, disease information of the patient, test information and image information used for the diagnosis, diagnosis results, and prescription drugs. The medical information system 30 is called an electronic medical record system or the like.

[0018] Incidentally, the accuracy of pathological diagnosis varies depending on the pathologist. Specifically, the diagnosis results based on pathological images may differ depending on the pathologist, depending on the years of experience and the specialty of the pathologist. For this reason, in recent years, a technology has been developed to derive diagnostic support information, which is information that supports diagnosis, using machine learning for the purpose of supporting pathological diagnosis. Specifically, a technology has been proposed in which a plurality of pathological images in which noteworthy areas are annotated are prepared, and the plurality of pathological images are analyzed to estimate noteworthy areas in a new pathological image. This makes it possible to provide noteworthy areas in the pathological images to the pathologist, thereby supporting the pathologist's diagnosis.

[0019] However, when a pathologist performs a pathological diagnosis, he or she only observes the pathological image and rarely annotates areas that affect the diagnosis, such as lesions. For this reason, in the technology for deriving diagnostic support information using the above-mentioned machine learning, a large amount of learning data is prepared by annotating the pathological image, but this annotation work requires a lot of time and workers. If a sufficient amount of learning data cannot be prepared, the accuracy of the machine learning will decrease, and it will be difficult to derive diagnostic support information (i.e., notable areas in the pathological image) with high accuracy. In addition, there is a framework called weakly supervised learning that does not require detailed annotation data, but there is a problem that the accuracy is inferior to machine learning that uses detailed annotation data.

[0020] Therefore, the derivation device 100 of the diagnosis support system 1 according to the first embodiment derives diagnostic support information using viewing history information on the area of ​​the pathology image viewed by the pathologist at the time of pathological diagnosis. Specifically, the derivation device 100 derives diagnostic support information on viewing to support the diagnosis of a second pathology image corresponding to a second diseased tissue different from the first diseased tissue, based on a first pathology image corresponding to a first diseased tissue, viewing history information on the viewing by the pathologist of the first pathology image, and diagnostic information on the diseased tissue. The details will be described later, but conceptually, the derivation device 100 derives an area that is focused on by the pathologist in the first pathology image with a positive diagnosis result as diagnostic support information.

[0021] An example of processing by the derivation device 100 will be described using the example of Fig. 1. In the example of Fig. 1, information regarding diagnoses by pathologists is accumulated daily in the server 12 of the pathology system 10. That is, the server 12 stores a first pathology image, which is a pathology image corresponding to a first diseased tissue, and browsing history information regarding the browsing history of the pathologist for the first pathology image. In addition, in the example of Fig. 1, the derivation device 100 provides diagnostic support information to the pathology system 20.

[0022] First, the derivation device 100 acquires the first pathological image accumulated daily and the browsing history information for the first pathological image from the server 12 of the pathological system 10. The derivation device 100 also acquires the diagnostic information on the diagnostic result corresponding to the first pathological image from the medical information system 30. The derivation device 100 generates a learning model for estimating an attention area from a second pathological image corresponding to a second diseased tissue different from the first diseased tissue by learning in association with the first pathological image, the browsing history information, and the diagnostic information. Although the details will be described later, for example, the derivation device 100 generates a learning model for estimating an attention area of ​​a second pathological image similar to an area enlarged and displayed by a pathologist, an area viewed by the pathologist for a long time, or an area repeatedly viewed by the pathologist in a first pathological image with a positive diagnostic result.

[0023] It is also assumed that in the pathology system 20, a second pathological image corresponding to the second diseased tissue is generated by the microscope 21. At this time, when the display control device 23 receives a display request for the second pathological image from a user such as a doctor or a pathologist, the display control device 23 sends the second pathological image to the derivation device 100. The derivation device 100 derives diagnostic support information indicating a region of interest of the second pathological image from the second pathological image using a learning model, and outputs the derived diagnostic support information to the display control device 23. The display control device 23 controls the display device 24 to display the region of interest based on the diagnostic support information together with the second pathological image.

[0024] In this way, the derivation device 100 derives information on the attention area as diagnostic assistance information based on the browsing history information on the browsing of the pathology image, which is an action that the pathologist always takes when making a pathological diagnosis. That is, the derivation device 100 uses the browsing history information of the pathologist that is accumulated daily, so that the attention area can be estimated without requiring a lot of time and workers. Then, the derivation device 100 can improve the accuracy of the diagnosis by the pathologist by displaying the attention area on the display device 24. For example, although there may be variation in diagnosis depending on the pathologist, by displaying the attention area on the pathology image, any pathologist can carefully observe and diagnose the attention area, so that the variation in diagnosis can be reduced.

[0025] In addition, since the derivation device 100 performs learning using the diagnostic information, it can derive the focused area as diagnostic assistance information when the diagnostic result is positive. Therefore, the derivation device 100 can provide the area that has an effect on the diagnosis, thereby improving the accuracy of the diagnosis.

[0026] In the above description, the task of annotating a pathological image requires a lot of time and effort. However, the task of annotating a pathological image may be performed by an operator with reference to the diagnostic assistance information derived by the derivation device 100. This allows annotated pathological images to be generated efficiently, making it possible to perform machine learning based on a large amount of learning data.

[0027] In the above, an example is shown in which a learning model is generated using a pathological image stored in the server 12 of the pathology system 10 as learning data, but the derivation device 100 may generate a learning model using a pathological image stored in the server 22 of the pathology system 20 as learning data, or may generate a learning model using both a pathological image stored in the server 12 and a pathological image stored in the server 22 as learning data. That is, the derivation device 100 can use as learning data any pathological image that has been viewed in the past. In addition, in the above, an example is shown in which the derivation device 100 provides diagnostic support information to the display control device 23, but the derivation device 100 may provide diagnostic support information to the display control device 13.

[0028] In the above example, the pathology system 10 and the pathology system 20 are described separately, but the pathology system 10 and the pathology system 20 may be the same system. More specifically, the diagnosis support system 1 may include only the pathology system 10. In this case, the derivation device 100 generates a learning model from the first pathology image stored in the server 12, and provides the diagnosis support information to the display control device 13 in response to a request from the display control device 13. In addition, the diagnosis support system 1 may include three or more pathology systems. In this case, the derivation device 100 may collect pathology images stored in each pathology system to generate a learning model. In the above example, the medical information system 30 may be the same system as the pathology system 10 or 20. That is, the diagnosis information may be stored in the server 12 or 22.

[0029] So far, the diagnosis support system 1 has been briefly described. Below, the configuration and processing of each device will be described in detail, but various information (pathological image data structure, pathological image browsing history information, and diagnostic information) that is the premise of these descriptions will be described first. In the following, an example will be shown in which the derivation device 100 learns using learning data accumulated in the pathological system 10 and provides diagnostic support information to the pathological system 20.

[0030] [2. Various information] [2-1.Pathological images] As described above, a pathological image is generated by capturing an image of an observation object using the microscope 11 or the microscope 21. First, the imaging process performed by the microscope 11 or the microscope 21 will be described with reference to Figs. 2 and 3. Figs. 2 and 3 are diagrams for explaining the imaging process according to the first embodiment. Since the microscopes 11 and 21 perform similar imaging processes, the microscope 11 will be used as an example for the description. The microscope 11 described below has a low-resolution imaging section for capturing images at low resolution and a high-resolution imaging section for capturing images at high resolution.

[0031] In FIG. 2, a glass slide G10 containing an observation object A10 is included in an imaging region R10, which is an area that can be photographed by the microscope 11. The glass slide G10 is placed on, for example, a stage not shown. The microscope 11 generates an overall image, which is a pathological image in which the observation object A10 is entirely photographed, by imaging the imaging region R10 using a low-resolution imaging unit. The label information L10 shown in FIG. 2 describes identification information (for example, a character string or a QR code (registered trademark)) for identifying the observation object A10. By associating the identification information described in the label information L10 with a patient, it is possible to identify the patient corresponding to the overall image. In the example of FIG. 2, "#001" is described as the identification information. Note that, for example, a brief description of the observation object A10 may be described in the label information L10.

[0032] Next, after generating the entire image, the microscope 11 identifies the region in which the observation object A10 exists from the entire image, and sequentially images each divided region obtained by dividing the region in which the observation object A10 exists by a predetermined size using the high-resolution imaging unit. For example, as shown in FIG. 3, the microscope 11 first images the region R11 and generates a high-resolution image I11 that is an image showing a partial region of the observation object A10. Next, the microscope 11 moves the stage to image the region R12 using the high-resolution imaging unit and generates a high-resolution image I12 corresponding to the region R12. In the same manner, the microscope 11 generates high-resolution images I13, I14, ... corresponding to the regions R13, R14, .... Although only up to the region R18 is illustrated in FIG. 3, the microscope 11 sequentially moves the stage to image all divided regions corresponding to the observation object A10 using the high-resolution imaging unit and generates high-resolution images corresponding to each divided region.

[0033] Incidentally, when the stage is moved, the glass slide G10 may move on the stage. When the glass slide G10 moves, there is a risk that an unphotographed area of ​​the observation object A10 may occur. As shown in FIG. 3, the microscope 11 captures images using a high-resolution imaging unit so that adjacent divided areas partially overlap, thereby preventing the occurrence of an unphotographed area even when the glass slide G10 moves.

[0034] The low-resolution imaging section and the high-resolution imaging section may be different optical systems or may be the same optical system. In the case where they are the same optical system, the microscope 11 changes the resolution according to the imaging target. In the above, an example in which the imaging area is changed by moving the stage has been shown, but the microscope 11 may change the imaging area by moving the optical system (such as the high-resolution imaging section). In addition, in FIG. 3, an example in which the microscope 11 images the observation object A10 from the center. However, the microscope 11 may image the observation object A10 in an order different from the imaging order shown in FIG. 3. For example, the microscope 11 may image the observation object A10 from the outer periphery. In addition, in the above, an example in which only the area where the observation object A10 exists is imaged by the high-resolution imaging section has been shown. However, since there are cases in which the area where the observation object A10 exists cannot be accurately detected, the microscope 11 may divide the imaging area R10 shown in FIG. 2 or the entire area of ​​the glass slide G10 and image it by the high-resolution imaging section.

[0035] Next, each high-resolution image generated by the microscope 11 is divided into a predetermined size. As a result, partial images (hereinafter, referred to as tile images) are generated from the high-resolution image. This will be described with reference to FIG. 4. FIG. 4 is a diagram for explaining the process of generating partial images (tile images). FIG. 4 shows a high-resolution image I11 corresponding to the area R11 shown in FIG. 3. Note that, in the following description, it is assumed that the partial images are generated from the high-resolution images by the server 12. However, the partial images may be generated by a device other than the server 12 (for example, an information processing device mounted inside the microscope 11).

[0036] 4, the server 12 generates 100 tile images T11, T12, ... by dividing one high-resolution image I11. For example, if the resolution of the high-resolution image I11 is 2560 x 2560 [pixels], the server 12 generates 100 tile images T11, T12, ... with a resolution of 256 x 256 [pixels] from the high-resolution image I11. In a similar manner, the server 12 generates tile images of the other high-resolution images by dividing them into similar sizes.

[0037] In the example of FIG. 4, the regions R111, R112, R113, and R114 are regions that overlap with other adjacent high-resolution images (not shown in FIG. 4). The server 12 performs stitching processing on the adjacent high-resolution images by aligning the overlapping regions using a technique such as template matching. In this case, the server 12 may generate tile images by dividing the high-resolution image after the stitching processing. Alternatively, the server 12 may generate tile images of regions other than the regions R111, R112, R113, and R114 before the stitching processing, and generate tile images of the regions R111, R112, R113, and R114 after the stitching processing.

[0038] In this manner, the server 12 generates tile images that are the smallest units of a captured image of the observation object A10. The server 12 then generates tile images of different hierarchical levels by sequentially combining the smallest unit tile images. Specifically, the server 12 generates one tile image by combining a predetermined number of adjacent tile images. This will be described with reference to Figs. 5 and 6. Figs. 5 and 6 are diagrams for explaining a pathological image according to the first embodiment.

[0039] The upper part of FIG. 5 shows a group of minimum unit tile images generated by the server 12 from each high resolution image. In the example in the upper part of FIG. 5, the server 12 generates one tile image T110 by synthesizing four adjacent tile images T111, T112, T211, and T212 among the tile images. For example, if the resolution of the tile images T111, T112, T211, and T212 is 256×256, the server 12 generates a tile image T110 having a resolution of 256×256. In the same manner, the server 12 generates a tile image T120 by synthesizing four adjacent tile images T113, T114, T213, and T214. In this manner, the server 12 generates a tile image by synthesizing a predetermined number of minimum unit tile images.

[0040] Furthermore, the server 12 generates a tile image by further combining adjacent tile images from among the tile images obtained after combining the minimum unit tile images. In the example of Fig. 5, the server 12 generates one tile image T100 by combining four adjacent tile images T110, T120, T210, and T220. For example, if the resolution of the tile images T110, T120, T210, and T220 is 256 x 256, the server 12 generates a tile image T100 with a resolution of 256 x 256. For example, the server 12 generates a tile image with a resolution of 256 x 256 from an image with a resolution of 512 x 512 obtained by combining four adjacent tile images by performing a four-pixel average, a weighting filter (a process that reflects nearby pixels more strongly than distant pixels), a 1 / 2 thinning process, or the like.

[0041] By repeating this synthesis process, server 12 ultimately generates a single tile image having the same resolution as the resolution of the smallest tile image. For example, as in the above example, if the resolution of the smallest tile image is 256 x 256, server 12 repeatedly performs the above-mentioned synthesis process to ultimately generate a single tile image T1 with a resolution of 256 x 256.

[0042] Fig. 6 shows a schematic diagram of the tile images shown in Fig. 5. In the example shown in Fig. 6, the tile images in the lowest layer are the smallest unit tile images generated by server 12. The tile images in the second layer from the bottom are tile images obtained after the tile images in the lowest layer are combined. Tile image T1 in the top layer indicates one tile image that is finally generated. In this way, server 12 generates, as a pathology image, a tile image group having a hierarchy like a pyramid structure shown in Fig. 6.

[0043] Note that area D shown in FIG. 5 is an example of an area displayed on a display screen of the display device 14 or 24 or the like. For example, assume that the resolution at which the display device can display is three tile images vertically and four tile images horizontally. In this case, as in area D shown in FIG. 5, the level of detail of the observed object A10 displayed on the display device changes depending on the layer to which the tile image to be displayed belongs. For example, when the tile image of the lowest layer is used, a narrow area of ​​the observed object A10 is displayed in detail. Also, the more tile images of higher layers are used, the more coarsely a wide area of ​​the observed object A10 is displayed.

[0044] Server 12 stores tile images of each layer as shown in FIG. 6 in a storage unit (not shown). For example, server 12 stores each tile image together with tile identification information (an example of partial image information) that can uniquely identify each tile image. In this case, when server 12 receives a request to acquire a tile image including the tile identification information from another device (for example, display control device 13 or derivation device 100), server 12 transmits the tile image corresponding to the tile identification information to the other device. Also, for example, server 12 may store each tile image together with layer identification information that identifies each layer and tile identification information that can uniquely identify the tile image within the same layer. In this case, when server 12 receives a request to acquire a tile image including the layer identification information and the tile identification information from the other device, server 12 transmits the tile image that corresponds to the tile identification information to the other device.

[0045] Note that server 12 may store the tile images of each layer as shown in Fig. 6 in a storage device other than server 12. For example, server 12 may store the tile images of each layer in a cloud server or the like. Furthermore, the tile image generation process shown in Figs. 5 and 6 may be executed by a cloud server or the like.

[0046] Furthermore, server 12 does not have to store tile images of all layers. For example, server 12 may store only the tile image of the lowest layer, or may store only the tile image of the lowest layer and the tile image of the top layer, or may store only tile images of a predetermined layer (for example, odd-numbered layers, even-numbered layers, etc.). In this case, when server 12 receives a request from another device for a tile image of a layer that it does not store, server 12 dynamically combines the stored tile images to generate the tile image requested by the other device. In this way, server 12 can prevent storage capacity from becoming overloaded by thinning out the tile images to be saved.

[0047] Although the imaging conditions are not mentioned in the above example, the server 12 may store tile images of each layer as shown in FIG. 6 for each imaging condition. An example of an imaging condition is the focal length of a subject (such as the observed object A10). For example, the microscope 11 may image the same subject while changing the focal length. In this case, the server 12 may store tile images of each layer as shown in FIG. 6 for each focal length. The reason for changing the focal length is that some observed objects A10 are semitransparent, and therefore there are focal lengths suitable for imaging the surface of the observed object A10 and focal lengths suitable for imaging the inside of the observed object A10. In other words, the microscope 11 can generate a pathological image obtained by imaging the surface of the observed object A10 and a pathological image obtained by imaging the inside of the observed object A10 by changing the focal length.

[0048] Another example of the imaging condition is a staining condition for the observation object A10. To be more specific, in a pathological diagnosis, a specific portion (e.g., a cell nucleus) of the observation object A10 may be stained with a luminescent material. A luminescent material is, for example, a substance that emits light when irradiated with light of a specific wavelength. Different luminescent materials may be stained for the same observation object A10. In this case, the server 12 may store tile images of each layer as shown in FIG. 6 for each dyed luminescent material.

[0049] Furthermore, the number and resolution of tile images described above are merely examples and can be changed as appropriate depending on the system. For example, the number of tile images that server 12 synthesizes is not limited to four. For example, server 12 may repeat the process of synthesizing 3×3=9 tile images. Furthermore, although the above example shows an example in which the resolution of the tile images is 256×256, the resolution of the tile images may be other than 256×256.

[0050] The display control device 13 uses software that employs a system compatible with the hierarchically structured tile image group described above, and extracts a desired tile image from the hierarchically structured tile image group in response to a user's input operation via the display control device 13, and outputs this to the display device 14. Specifically, the display device 14 displays an image of an arbitrary portion selected by the user out of an image of an arbitrary resolution selected by the user. This process allows the user to obtain the sensation of observing the object while changing the observation magnification. In other words, the display control device 13 functions as a virtual microscope. The virtual observation magnification here actually corresponds to the resolution.

[0051] [2-2. Browsing history information] Next, the viewing history information of the pathological image stored in the server 12 or 22 will be described with reference to FIG. 7. FIG. 7 is a diagram showing an example of a viewing manner by a viewer of the pathological image. In the example shown in FIG. 7, it is assumed that a viewer such as a pathologist views the pathological image I10 in the order of areas D1, D2, D3, . . . , and D7. In this case, the display control device 13 first acquires the pathological image corresponding to area D1 from the server 12 in accordance with the viewing operation by the viewer. In response to a request from the display control device 13, the server 12 acquires one or more tile images forming the pathological image corresponding to area D1 from the storage unit, and transmits the acquired one or more tile images to the display control device 13. Then, the display control device 13 displays the pathological image formed from the one or more tile images acquired from the server 12 on the display device 14. For example, when there are multiple tile images, the display control device 13 displays the multiple tile images side by side. Similarly, each time the viewer changes the display area, the display control device 13 obtains from the server 12 the pathological image corresponding to the area to be displayed (areas D2, D3, ..., D7, etc.) and displays it on the display device 14.

[0052] In the example of FIG. 7, the viewer first views a relatively large area D1, and since there is no area in area D1 that the viewer wants to observe carefully, the viewer moves the viewing area to area D2. Then, since there is an area in area D2 that the viewer wants to observe carefully, the viewer enlarges a part of area D2 to view area D3. Then, the viewer further moves to area D4, which is a part of area D2. Then, since there is an area in area D4 that the viewer wants to observe more carefully, the viewer enlarges a part of area D4 to view area D5. In this manner, the viewer also views areas D6 and D7. For example, the pathological images corresponding to areas D1, D2, and D7 are display images at a magnification of 1.25, the pathological images corresponding to areas D3 and D4 are display images at a magnification of 20, and the pathological images corresponding to areas D5 and D6 are display images at a magnification of 40. The display control device 13 acquires and displays the tile images of the hierarchical levels corresponding to each magnification from the hierarchically structured tile image group stored in the server 12. For example, the tile images corresponding to areas D1 and D2 are at a higher level than the tile image corresponding to area D3 (that is, a level closer to tile image T1 shown in FIG. 6).

[0053] While the pathological image is being viewed as described above, the display control device 13 acquires viewing information at a predetermined sampling period. Specifically, the display control device 13 acquires the center coordinates and the display magnification of the viewed pathological image at each predetermined timing, and stores the acquired viewing information in the storage unit of the server 12.

[0054] This point will be described with reference to FIG. 8. FIG. 8 is a diagram showing an example of the browsing history storage unit 12a of the server 12. As shown in FIG. 8, the browsing history storage unit 12a stores information such as "sampling," "center coordinates," "magnification," and "time." "Sampling" indicates the order of timing for storing the browsing information. "Center coordinates" indicates the position information of the viewed pathological image. In this example, the center coordinates are coordinates indicated by the center position of the viewed pathological image, and correspond to the coordinates of the coordinate system of the tile image group of the lowest layer. "Magnification" indicates the display magnification of the viewed pathological image. "Time" indicates the elapsed time from the start of browsing. In the example of FIG. 8, it is shown that the sampling period is 30 seconds. That is, the display control device 13 saves the browsing information in the browsing history storage unit 12a every 30 seconds. However, the sampling period is not limited to this example, and may be, for example, 0.1 to 10 seconds, or may be outside this range.

[0055] In the example of FIG. 8, sampling "1" indicates the browsing information of area D1 shown in FIG. 7, sampling "2" indicates the browsing information of area D2, sampling "3" and "4" indicate the browsing information of area D3, sampling "5" indicates the browsing information of area D4, and sampling "6", "7" and "8" indicate the browsing information of area D5. That is, the example of FIG. 8 indicates that area D1 was viewed for about 30 seconds, area D2 was viewed for about 30 seconds, area D3 was viewed for about 60 seconds, area D4 was viewed for about 30 seconds, and area D5 was viewed for about 90 seconds. In this way, the browsing time of each area can be extracted from the browsing history information.

[0056] Also, the number of times each region has been viewed can be extracted from the browsing history information. For example, each time a change operation of the display region (for example, a movement operation of the display region, a change operation of the display size) is performed, the number of times each pixel of the displayed pathological image is displayed increases by one. For example, in the example shown in FIG. 7, when the region D1 is displayed first, the number of times each pixel included in the region D1 is displayed is one. When the region D2 is displayed next, the number of times each pixel included in both the region D1 and the region D2 is displayed is two, and the number of times each pixel included in the region D2 but not in the region D1 is displayed is one. Since the display region can be specified by referring to the center coordinates and magnification of the browsing history storage unit 12a, the number of times each pixel (which can also be called each coordinate) of the pathological image is displayed can be extracted by analyzing the browsing history information stored in the browsing history storage unit 12a.

[0057] The display control device 13 may suspend the storage process of the viewing information if the viewer does not perform an operation to change the display position for a predetermined time (e.g., 5 minutes). In the above example, the pathological image viewed by the center coordinates and the magnification are stored as the viewing information, but the present invention is not limited to this example, and the viewing information may be any information that can identify the area of ​​the viewed pathological image. For example, the display control device 13 may store tile identification information that identifies a tile image corresponding to the viewed pathological image, or information indicating the position of the tile image corresponding to the viewed pathological image, as the viewing information of the pathological image. Although not shown in FIG. 8, the viewing history storage unit 12a stores information that identifies patients, medical records, etc. That is, the viewing history storage unit 12a shown in FIG. 8 stores the viewing information in a manner that allows it to be associated with patients, medical records, etc.

[0058] [2-3. Diagnostic Information] Next, the diagnostic information stored in the medical information system 30 will be described with reference to Figs. 9A to 9C. Figs. 9A to 9C are diagrams showing a diagnostic information storage unit possessed by the medical information system 30. Figs. 9A to 9C show examples of storing diagnostic information in different tables for each organ to be examined. For example, Fig. 9A shows an example of a table that stores diagnostic information related to breast cancer examinations, Fig. 9B shows an example of a table that stores diagnostic information related to lung cancer examinations, and Fig. 9C shows an example of a table that stores diagnostic information related to colon examinations.

[0059] The diagnostic information storage unit 30A shown in FIG. 9A stores information such as "patient ID", "pathological image", "diagnostic result", "grade", "tissue type", "genetic test", "ultrasound test", and "medication". "Patient ID" indicates identification information for identifying a patient. "Pathological image" indicates a pathological image saved by a pathologist at the time of diagnosis. "Pathological image" may store position information (such as center coordinates and magnification) indicating the image area to be saved relative to the entire image, rather than the image itself. "Diagnosis result" is the diagnosis result by a pathologist, and indicates, for example, the presence or absence of a lesion site and the type of the lesion site. "Grade" indicates the progression of the disease site. "Tissue type" indicates the type of the disease site. "Genetic test" indicates the result of a genetic test. "Ultrasound test" indicates the result of an ultrasound test. Medication indicates information regarding medication to a patient.

[0060] The diagnostic information storage unit 30B shown in Fig. 9B stores information on a "CT examination" performed in a lung cancer examination, instead of the "ultrasound examination" stored in the diagnostic information storage unit 30A shown in Fig. 9A. The diagnostic information storage unit 30C shown in Fig. 9C stores information on an "endoscopic examination" performed in a large intestine examination, instead of the "ultrasound examination" stored in the diagnostic information storage unit 30A shown in Fig. 9A.

[0061] In the examples of Figures 9A to 9C, when "Diagnosis Result" stores "Normal," it indicates that the result of the pathological diagnosis was negative, and when information other than "Normal" is stored in "Diagnosis Result," it indicates that the result of the pathological diagnosis was positive.

[0062] [3. Equipment configuration] [3-1. Derivation device according to the first embodiment] Next, a derivation device 100 according to a first embodiment will be described. Here, the derivation device 100 and the display control device 23 will also be described. FIG. 10 is a diagram showing an example of the derivation device 100 and the display control device 23 according to the first embodiment. As shown in FIG. 10, the derivation device 100 is a computer having a communication unit 110, a storage unit 120, and a control unit 130.

[0063] The communication unit 110 is realized by, for example, a network interface card (NIC) or the like. The communication unit 110 is connected to a network (not shown) by wire or wirelessly, and transmits and receives information to and from the pathology system 10, the pathology system 20, the medical information system 30, and the like via the network. The control unit 130, which will be described later, transmits and receives information to and from these devices via the communication unit 110.

[0064] The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 stores a learning model 121 generated by the control unit 130. The learning model 121 will be described later.

[0065] The control unit 130 is realized, for example, by a central processing unit (CPU) or a micro processing unit (MPU) executing a program (an example of a diagnostic assistance program) stored inside the derivation device 100 using a random access memory (RAM) or the like as a working area. The control unit 130 may also be executed by an integrated circuit, for example, an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0066] 10, control unit 130 has history information acquisition unit 131, diagnosis information acquisition unit 132, learning unit 133, and derivation unit 134, and realizes or executes the functions and actions of information processing described below. Note that the internal configuration of control unit 130 is not limited to the configuration shown in FIG. 10, and may be other configurations as long as they perform information processing described below.

[0067] The history information acquisition unit 131 acquires information used in the learning process performed by the learning unit 133 via the communication unit 110. Specifically, the history information acquisition unit 131 acquires a first pathological image corresponding to a first diseased tissue and viewing history information regarding viewing of the first pathological image, both stored in the server 12 of the pathology system 10. The viewing history information acquired by the history information acquisition unit 131 includes, as in the example shown in FIG. 8, information on an area of ​​the first pathological image that the pathologist viewed by enlarging it, information on an area of ​​the first pathological image that the pathologist viewed for a relatively long time, and information on the number of times the pathologist viewed each area of ​​the first pathological image.

[0068] The diagnostic information acquisition unit 132 acquires information used for the learning process performed by the learning unit 133 via the communication unit 110. Specifically, the diagnostic information acquisition unit 132 acquires, from the medical information system 30, diagnostic information for the first affected tissue corresponding to the first pathological image.

[0069] Learning unit 133 learns the correspondence between the first pathological image and viewing history information acquired by history information acquisition unit 131, and the diagnostic information acquired by diagnostic information acquisition unit 132. As a result, learning unit 133 generates a learning model for estimating an attention area in the second pathological image. Then, learning unit 133 stores learning model 121 in storage unit 120.

[0070] The learning process by the learning unit 133 will be described in more detail. First, the learning unit 133 generates learning data. Specifically, the learning unit 133 generates a heat map image that reflects information about the attention area of ​​the first pathological image based on the viewing history information acquired by the history information acquisition unit 131. For example, the learning unit 133 associates each pixel of the first pathological image with a large index value (hereinafter, referred to as attention index value) that is larger as the pixel is more noticed by the viewer.

[0071] As an example, the learning unit 133 associates a higher attention index value with a pixel having a higher display magnification. Using the example shown in FIG. 7, the pathological image corresponding to the region D1 is a display image with a magnification of 1.25, the pathological image corresponding to the region D3 is a display image with a magnification of 20, and the pathological image corresponding to the region D5 is a display image with a magnification of 40. In this case, the learning unit 133 makes the attention index value of each pixel included in the region D3 higher than the attention index value of each pixel included in the region D1. Furthermore, the learning unit 133 makes the attention index value of each pixel included in the region D5 higher than the attention index value of each pixel included in the region D3.

[0072] Furthermore, for example, the learning unit 133 associates a higher attention index value with a pixel that has been viewed for a longer period of time. The example in Fig. 8 indicates that region D1 was viewed for approximately 30 seconds, region D3 was viewed for approximately 60 seconds, and region D5 was viewed for approximately 90 seconds. In this case, the learning unit 133 sets the attention index value of each pixel included in region D3 to be higher than the attention index value of each pixel included in region D1. Furthermore, the learning unit 133 sets the attention index value of each pixel included in region D5 to be higher than the attention index value of each pixel included in region D3.

[0073] Also, for example, the learning unit 133 associates a higher attention index value with a pixel that has been viewed more frequently. Note that the learning unit 133 may associate the attention index value using only one piece of information among the magnification rate, viewing time, and viewing count, or may associate the attention index value using two or more pieces of information. In this way, the learning unit 133 generates a heat map image in which an attention index value is associated with each pixel of each of the first pathological images acquired by the history information acquisition unit 131. Note that the heat map image is, for example, a collection of data that is a combination of pixels of the first pathological image and attention index values ​​corresponding to the pixels, and can be said to be viewing history information indicated by attention index values.

[0074] Next, the learning unit 133 performs machine learning using a combination of the first pathological image, a heat map image corresponding to the first pathological image, and diagnostic information corresponding to the first pathological image as learning data. For example, the learning unit 133 learns learning data in which the diagnostic result is positive (for example, the diagnostic result in the diagnostic information storage units 30A to 30C is other than "normal") as a positive example, and learning data in which the diagnostic result is negative (for example, the diagnostic result in the diagnostic information storage units 30A to 30C is "normal") as a negative example. Although the learning process by the learning unit 133 is a black box and therefore will be described conceptually, the learning unit 133 generates a learning model 121 that outputs a higher score (hereinafter, referred to as attention score) for an area in each area of ​​a new pathological image (second pathological image) that has a similar image feature amount to an area in which the attention index value tends to be high in the first pathological image group in which the diagnostic result was positive.

[0075] The attention score for each region of the second pathological image output by the learning model 121 can be said to indicate the degree to which a similar region to the region was previously noted by a pathologist. Here, since the pathologist pays more attention to a region with a lesion, the region that is noted by the pathologist can be said to be a region where a lesion is likely to exist. In other words, the attention score output by the learning model 121 can be said to indicate the probability that a lesion exists in the corresponding region. For example, when the first pathological image for learning data is an image used in a cancer examination, the attention score for each region can be said to indicate the probability that cancer tissue exists in each region, that is, the region that is estimated to have an effect on diagnosis.

[0076] In the above example, the first pathological image with a positive diagnosis is treated as a positive example, but not all observation objects depicted in the first pathological image are positive even if the diagnosis is positive, and observation objects depicted in a partial area of ​​the first pathological image are often lesions. For this reason, the learning unit 133 may learn only areas of the first pathological image with a positive diagnosis, in which the attention index value is equal to or greater than a predetermined threshold TH1, as positive examples, and areas in which the attention index value is smaller than the predetermined threshold TH1 as negative examples. The learning unit 133 may also learn using only first pathological images with a positive diagnosis.

[0077] For example, weakly supervised learning can be applied to the learning by the learning unit 133, and the following method can also be used. “WILDCAT: Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation”, CVPR 2017 (http: / / webia.lip6.fr / ~durandt / pdfs / 2017_CVPR / Durand_WILDCAT_CVPR_2017.pdf) “Attention-based Deep Multiple Instance Learning”, 2018(https: / / arxiv.org / abs / 1802.04712)

[0078] However, the learning method by the learning unit 133 may be based on any algorithm. For example, the learning unit 133 can generate a learning model using various learning algorithms such as deep learning, which is a machine learning method using a multi-layered deep neural network, a support vector machine, clustering, reinforcement learning, and the like.

[0079] Also, the learning unit 133 may not perform learning using all the first pathological images. For example, the learning unit 133 may perform learning using only the first pathological images that have been particularly noted. For example, the learning unit 133 may perform learning using only the first pathological images including the area viewed for a predetermined time or more, may perform learning using only the first pathological images including the area viewed at a predetermined magnification, or may perform learning using only the first pathological images including the area viewed a predetermined number of times or more. Also, for example, the learning unit 133 may perform learning using only the area viewed for a predetermined time or more, may perform learning using only the area viewed at a predetermined magnification, or may perform learning using only the area viewed a predetermined number of times or more. Also, for example, the learning unit 133 may perform learning by cutting out only the central area of ​​the first pathological image, assuming that the central area of ​​the first pathological image is the area that has been noted.

[0080] In the above example, the learning unit 133 generates a heatmap image as learning data. However, the process of generating a heatmap image may be performed by the microscope 11, the server 12, or the like. The heatmap image may be stored in the server 12. In this case, the history information acquisition unit 131 acquires the heatmap image from the server 12 as browsing history information.

[0081] The derivation unit 134 acquires the second pathological image from the display control device 23 and applies the learning model to derive diagnostic assistance information, which is information regarding the viewing of the second pathological image, based on the attention area of ​​the second pathological image estimated from the viewing history information. Specifically, the derivation unit 134 derives the diagnostic assistance information by determining that the area of ​​interest with a higher display magnification of the first pathological image is the area of ​​interest with a higher degree of attention, or the area of ​​the first pathological image with a longer viewing time is the area of ​​interest with a higher degree of attention, or the area of ​​the first pathological image with a higher number of viewings is the area of ​​interest with a higher degree of attention. Then, the derivation unit 134 outputs the diagnostic assistance information, which is the derivation result, to the display control device 23.

[0082] An example will be described. The derivation unit 134 receives a request from the display control device 23 to estimate the attention area of ​​the second pathological image together with the second pathological image. The second pathological image is, for example, a pathological image on which a pathologist is making a diagnosis. The derivation unit 134 then estimates the attention area of ​​the second pathological image using the learning model 121 stored in the storage unit 120. As described above, the learning model 121 outputs an attention score that is higher for an area similar to an area that was noticed by the pathologist among the pathological images with a positive diagnosis result. Such an attention score can be said to indicate the importance in the pathological diagnosis, that is, the degree of influence on the diagnosis. The derivation unit 134 specifies, as the attention area, an area of ​​the second pathological image in which the attention score calculated using the learning model 121 is higher than a predetermined threshold value TH2. The derivation unit 134 then transmits, to the display control device 23, diagnosis support information indicating position information regarding the attention area of ​​the second pathological image. For example, the derivation unit 134 transmits, as the position information, information relating to coordinates or tile images (tile identification information, or a combination of layer identification information and tile identification information) to the display control device 23.

[0083] [3-2. Display control device according to the first embodiment] Next, the display control device 23 will be described. The display control device 23 is equipped with a pathology image display control program for displaying pathology images such as the above-mentioned tile images. However, the present invention is not limited to this, and the pathology image display control program may be downloaded from a server or installed in a general-purpose computer from a storage medium such as a DVD (Digital Versatile Disc), thereby realizing the processing by the display control device 23 described below. In addition, the processing by the display control device 23 described below may be realized by performing processing by two or more devices, such as performing some processing on a server and performing other processing on a client computer such as the display control device 23. In addition, the processing by the display control device 23 described below may be realized by running the pathology image display control program on a cloud.

[0084] As shown in Fig. 10, the display control device 23 has an image acquisition unit 23a and a display control unit 23b, and is a computer that realizes or executes the functions and actions of information processing described below. The internal configuration of the display control device 23 is not limited to the configuration shown in Fig. 10, and may be other configurations as long as they perform the information processing described below. The image acquisition unit 23a and the display control unit 23b are realized by, for example, a CPU or an MPU executing a display control program stored in the display control device 23 using a RAM or the like as a working area. The image acquisition unit 23a and the display control unit 23b may be executed by an integrated circuit such as an ASIC or an FPGA.

[0085] The image acquiring unit 23a acquires a second pathological image from the server 22, and transmits the acquired second pathological image to the derivation device 100. For example, the image acquiring unit 23a transmits the second pathological image that the pathologist is going to diagnose to the derivation device 100 in accordance with an operation by the pathologist. Note that the image acquiring unit 23a and the display control unit 23b may be the same unit, and the process of transmitting the second pathological image to the derivation device 100 may be performed by the display control unit 23b.

[0086] The display control unit 23b receives diagnostic assistance information corresponding to the second pathological image from the derivation device 100. Then, the display control unit 23b controls the display device 24 to display the second pathological image in a state in which a region of interest, which is a region that is presumed to have an effect on the diagnosis, is visible based on the diagnostic assistance information. Note that hereinafter, the display control unit 23b controlling the display device 24 to display various information such as a pathological image on the display device 24 may be simply expressed as the display control unit 23b displaying various information.

[0087] Fig. 11 is a diagram showing an example of display of a pathological image by the display control device 23 according to the first embodiment. In the example shown in Fig. 11, the display control unit 23b displays the entire observation object A11 in the whole display region H10, displays an enlarged partial region of the observation object A11 in the enlarged display region H11, and further displays a support information button B10. For example, the display control unit 23b displays, in the whole display region H10, a pathological image I1 formed from a tile image of an arbitrary layer among the tile image group of the hierarchical structure shown in Fig. 6. In addition, the display control unit 23b displays, in the enlarged display region H11, a pathological image I2 formed from a tile image of a layer lower than the tile image displayed in the whole display region H10.

[0088] 11, the display control unit 23b displays an enlarged area frame FR indicating the area displayed in the enlarged display area H11 in the entire display area H10. A viewer such as a pathologist can change the area of ​​the observed object A11 to be displayed in the enlarged display area H11 by moving the enlarged area frame FR (e.g., dragging the mouse). This allows the viewer to check a part of the observed object A11 in more detail by viewing the pathological image in the enlarged display area H11 while grasping the entire observed object A11 in the entire display area H10.

[0089] Here, when the support information button B10 is selected by the viewer, the display control unit 23b displays a notable area of ​​the pathological image I2 displayed in the enlarged display area H11 in a visible state based on the diagnostic support information received from the derivation device 100. For example, the display control unit 231 acquires diagnostic support information corresponding to the pathological image I2 by transmitting the pathological image I2 displayed in the enlarged display area H11 to the derivation device 100. Then, the display control unit 23b displays a frame image RI10 in which the outer frame of the notable area is surrounded by a dotted line based on the diagnostic support information acquired from the derivation device 100. This allows the viewer to determine that the area indicated by the frame image RI10 should be observed more carefully among the pathological images displayed in the enlarged display area H11.

[0090] When the support information button B10 is selected again while the frame image RI10 is being displayed, the display control unit 23b may hide the frame image RI10 that has already been displayed. This allows the viewer to observe a pathology image to which no unnecessary information has been added.

[0091] In the example of FIG. 11, a frame image RI10 is displayed in the enlarged display region H11. However, the display control unit 23b may display a frame image corresponding to the pathological image I1 in the entire display region H10 instead of the enlarged display region H11. In this case, the image acquisition unit 23a acquires diagnostic support information corresponding to the pathological image I1 by transmitting the pathological image I1 to the derivation device 100. This allows the viewer to display the region in the pathological image I1 in which the frame image is displayed in the enlarged display region H11, so that the viewer can carefully observe all of the regions of interest in the observation object A11. Moreover, the display control unit 23b may display frame images in both the entire display region H10 and the enlarged display region H11.

[0092] In addition, the display control device 23 may acquire diagnostic assistance information in real time from the derivation device 100 when the pathologist views the pathological image, or may acquire diagnostic assistance information in advance by periodically (e.g., once a day) transmitting the pathological images stored in the server 22 to the derivation device 100.

[0093] [4. Processing Procedure] [4-1. Learning Processing Procedure According to the First Embodiment] Next, a processing procedure according to the first embodiment will be described with reference to Fig. 12 and Fig. 13. Fig. 12 is a flowchart showing a learning processing procedure according to the first embodiment. As shown in Fig. 12, the derivation device 100 determines whether or not it is time to learn (step S101). For example, when a learning date and time for learning is determined in advance, the derivation device 100 determines whether or not the current date and time is the learning date and time.

[0094] When it is the learning timing (step S101; Yes), the derivation device 100 acquires the first pathological image and the viewing history information of the first pathological image from the server 12 (step S102). In addition, the derivation device 100 acquires the diagnostic information corresponding to the first pathological image acquired from the server 12 from the medical information system 30 (step S103).

[0095] Next, the derivation device 100 learns the relationship between the heat map image and the diagnosis result based on the browsing history information (browsing time, browsing count, magnification, etc.) (step S104). As a result, the derivation device 100 generates a learning model 121 (step S105). Then, the derivation device 100 stores the learned learning model 121 in the storage unit 120.

[0096] 12 shows an example in which the learning model 121 is newly generated, the derivation device 100 may re-learn the learned learning model 121. In this case, the derivation device 100 corrects the learning model 121 by using new learning data.

[0097] [4-2. Derivation process procedure according to the first embodiment] Fig. 13 is a flowchart showing a derivation process procedure according to the first embodiment. As shown in Fig. 13, the derivation device 100 determines whether or not a second pathological image has been received from the display control device 23 (step S201).

[0098] Then, when the derivation device 100 receives the second pathological image (step S201; Yes), it acquires the trained learning model 121 from the storage unit 120 (step S202). Next, the derivation device 100 uses the learning model 121 to derive diagnostic assistance information, which is information indicating a notable area in the second pathological image (step S203). Then, the derivation device 100 transmits the diagnostic assistance information to the display control device 23 (step S204). As a result, the display control device 23 controls the display device 24 to display the second pathological image together with the diagnostic assistance information, as in the example shown in FIG. 11.

[0099] [5. Variation 1] The following describes modified examples of the display process with reference to Fig. 14 to Fig. 22. Fig. 14 to Fig. 22 are diagrams showing examples of display of pathological images according to the modified examples.

[0100] [5-1. Display example (1)] FIG. 11 shows an example of displaying information (frame image RI10) indicating one attention region, but the display control unit 23b may display information indicating an attention region corresponding to an attention score equal to or higher than each of a plurality of attention score thresholds. In this case, the display control unit 23b may change the display mode of the information indicating the attention region for each attention score. FIG. 11 also shows an example of displaying a frame image RI10 surrounded by a dotted line as information indicating the attention region. However, the display control unit 23b may display a mask image that semi-transparently covers the attention region as information indicating the attention region. The display control unit 23b may also display a pathology image on which information indicating the attention region is not superimposed and a pathology image on which information indicating the attention region is superimposed side by side.

[0101] This point will be described with reference to FIG. 14. In the example shown in FIG. 14, the display control unit 23b displays the pathological image I21 without superimposing information indicating the attention area on the left side of the display screen, and displays the pathological image I21 with superimposed information indicating the attention area on the right side of the display screen. Here, the display control unit 23b displays the mask images RI11, RI12, and RI13 filled semi-transparently as information indicating the attention area. Note that the mask images RI11, RI12, and RI13 have higher attention scores in this order. For example, the mask image RI11 is superimposed on an area where the attention score is equal to or greater than a threshold value TH11, the mask image RI12 is superimposed on a continuous area where the attention score is equal to or greater than a threshold value TH12 (>TH11), and the mask image RI13 is superimposed on a continuous area where the attention score is equal to or greater than a threshold value TH13 (>TH12). The display control unit 23b may change the saturation, brightness, or transparency of the mask images RI11, RI12, and RI13 according to the attention score.

[0102] 14, the display control unit 23b displays a display switching button B11. When the display switching button B11 is selected by the viewer, the display control unit 23b may hide the pathological image I21 on which information indicating the region of interest is not superimposed, as shown in the lower part of FIG. 14. When the display switching button B11 is selected in a state in which the pathological image I21 on which information indicating the region of interest is not superimposed is not displayed, the display control unit 23b may display the pathological image I21 on which information indicating the region of interest is not superimposed again, as shown in the upper part of FIG. This example is not limited to this, and the display control unit 23b may switch between displaying the pathological image I21 on which information indicating the region of interest is not superimposed and displaying the pathological image I21 on which information indicating the region of interest is superimposed, every time the display switching button B11 is selected.

[0103] [5-2. Display example (2)] In the example of FIG. 14, a plurality of mask images are displayed. However, the display control unit 23b may display a plurality of frame images for each threshold of the attention score. This point will be described with reference to FIG. 15. In the example shown in FIG. 15, the display control unit 23b displays frame images RI21, RI22, and RI23, which are not filled in, superimposed on the pathological image I21 as information indicating the attention area. Here, the frame images RI21, RI22, and RI23 have the highest attention scores in this order. As in the example of FIG. 14, the display control unit 23b may change the saturation, brightness, or transparency of the frame images RI21, RI22, and RI23 according to the attention score.

[0104] [5-3. Display example (3)] Furthermore, the display control unit 23b may automatically display pathological images corresponding to attention regions having a predetermined attention score or more. For example, the display control unit 23b may automatically display each pathological image in descending order of attention score. Furthermore, for example, the display control unit 23b may switch between pathological images corresponding to each attention region in the same display area and display them on the display device 24 in accordance with the operation of the viewer.

[0105] This point will be described with reference to FIG. 16. In the example shown in the upper part of FIG. 16, the display control unit 23b displays the mask images RI11, RI12, and RI13 superimposed on the pathological image I21, as in the example shown in FIG. 14, and also displays the automatic detection button B12. Here, the display control unit 23b automatically initially displays the pathological image I21 corresponding to a partial area of ​​the entire image as an image including an area with the highest attention score in the entire image. Then, when the automatic detection button B12 is selected by the viewer, the display control unit 23b switches the display to the pathological image I22 including an area with the second highest attention score in the entire image, as shown in the lower part of FIG. 16. Every time the automatic detection button B12 is selected, the display control unit 23b switches the display to the pathological image with the second highest attention score.

[0106] [5-4. Display example (4)] Furthermore, when a plurality of frame images or mask images are displayed as in the examples of FIGS. 14 and 15, the display control unit 23b may change the number of frame images or mask images to be displayed in accordance with an operation by the viewer.

[0107] This point will be described with reference to FIG. 17. In the example shown in the upper part of FIG. 17, the display control unit 23b displays the mask images RI11, RI12, and RI13 superimposed on the pathological image I21, as in the example shown in FIG. 14, and also displays the display change button B13. Here, the display control unit 23b initially displays the three mask images RI11, RI12, and RI13. Then, when the display change button B13 is selected by the viewer, the display control unit 23b hides the mask image RI13 having the lowest attention score among the mask images RI11, RI12, and RI13, as shown in the middle part of FIG. 17. Then, when the display change button B13 is further selected, the display control unit 23b hides the mask image RI12 having the second lowest attention score, as shown in the lower part of FIG. 17, thereby displaying only the mask image RI11. Then, when the display change button B13 is further selected, the display control unit 23b displays all the mask images RI11, RI12, and RI13 again as shown in the upper part of Fig. 17. Note that, when the display change button B13 is selected in the state shown in the lower part of Fig. 17, the display control unit 23b may hide the mask image RI11, and hide all the mask images RI11, RI12, and RI13.

[0108] [5-5. Display example (5)] Furthermore, the display controller 23b may simultaneously display information indicating the region of interest in a plurality of different regions of the pathological image. This point will be described with reference to Fig. 18. In the example shown in Fig. 18, the display controller 23b displays a pathological image I1 in which the observation object A10 is depicted as a whole, and also displays frame images RI21-RI25 superimposed on the pathological image I1. Since the frame images RI21-RI25 indicate the region of interest in the pathological image I1, the viewer can observe the regions corresponding to the frame images RI21-RI25 in detail while successively enlarging them.

[0109] [5-6. Display example (6)] In addition, in the example of FIG. 18, the display control unit 23b may display a frame image or a mask image indicating a non-attention region. For example, the display control unit 23b displays a frame image or the like indicating a non-attention region in a region where the attention score output by the learning model 121 is lower than a predetermined threshold value TH3. This point will be described with reference to FIG. 19. In the example shown in FIG. 19, the display control unit 23b displays a pathological image I1 in which the observation target A10 is depicted as a whole, displays frame images RI21 to RI23 indicating the attention region, and further displays frame images RI31 and RI32 indicating the non-attention region. Since the frame images RI31 and RI32 indicate that the region is not one that has been viewed with attention in the past, the regions corresponding to the frame images RI31 and RI32 can be observed simply, and therefore, pathological diagnosis can be performed efficiently. Note that the display control device 23 can specify the non-attention region by acquiring the attention score corresponding to each region of the pathological image I1 from the derivation device 100 as diagnostic support information.

[0110] [5-7. Display example (7)] The display control unit 23b may display pathological images corresponding to a plurality of attention regions side by side. This point will be described with reference to FIG. 20. As shown in FIG. 20, the display control unit 23b displays a screen including an entire display region H10 and an enlarged display region H11. In the example of FIG. 20, the display control unit 23b displays the pathological image I1 and frame images RI21 to RI25 shown in FIG. 18 in the entire display region H10. Then, the display control unit 23b displays the pathological images corresponding to the frame images RI21 to RI25 side by side in the enlarged display region H11. In the example of FIG. 20, the display control unit 23b displays the pathological image I41 corresponding to the frame image RI21, the pathological image I42 corresponding to the frame image RI22, the pathological image I43 corresponding to the frame image RI23, the pathological image I44 corresponding to the frame image RI24, and the pathological image I45 corresponding to the frame image RI25. At this time, the display control unit 23b may display the pathological images I41 to I45 in descending order of attention score.

[0111] [5-8. Display example (8)] In addition, in the example of FIG. 20, the display control unit 23b may display a plurality of pathological images corresponding to the region of interest in a selectable manner. Then, the display control unit 23b may store the selected pathological image in the storage unit of the server 22 as diagnosis result information. This point will be described with reference to FIG. 21. As shown in FIG. 21, the display control unit 23b displays the pathological image I1 in the whole display area H10 by superimposing frame images RI21 to RI25 on it, as in the example of FIG. 20. Furthermore, the display control unit 23b displays the pathological images I41 to I45 in a selectable arrangement in the enlarged display area H11. Specifically, the display control unit 23b displays check boxes CH11 to CH15 for selecting each of the pathological images I41 to I45. Then, when a save button (not shown) is pressed with these check boxes selected, the display control unit 23b stores the selected pathological image in the storage unit of the server 22 or the medical information system 30.

[0112] [5-9. Display example (9)] The display control unit 23b may display the learning results when the learning parameters are changed for a developer who generates a learning model. The display control unit 23b may display the learning results corresponding to different learning parameters side by side so that they can be compared. Examples of the learning parameters include a threshold value TH4 that sets a predetermined attention index value or less in a heat map image to 0, a batch size or input size for learning, and a network configuration for deep learning.

[0113] This point will be described with reference to FIG. 22. In the example of FIG. 22, the display control unit 23b displays two learning results side by side. Specifically, the display control unit 23b displays the learning results when the learning parameter A is "XXX", the learning parameter B is "YYY", and the learning parameter C is "ZZZ" on the left side of the screen. In addition, the display control unit 23b displays the learning results when the learning parameters A, B, and C are "PPP", "QQQ", and "RRR" on the right side of the screen. The display control unit 23b displays, as the learning result, information indicating the attention area (a mask image in the example of FIG. 22) superimposed on the pathological image I21. In addition, the display control unit 23b displays graphs G11 and G12 showing the relationship between the number of learning times in machine learning and the output of the loss function, as shown in FIG. 22. The vertical axis of the graphs G11 and G12 shown in FIG. 22 indicates the output of the loss function, and the horizontal axis indicates the number of learning times. The number of learning times corresponds to, for example, the number of backpropagation times (i.e., the number of times the weights are updated) performed when training a neural network. Since learning is performed at any time, the display control unit 23b displays information on the latest learning results when the learning result button B31 or B32 is selected.

[0114] [5-10. Display example (10)] In the above, a frame image and a mask image are described as examples of the diagnostic support information to be superimposed on the pathological image. However, the display control unit 23b may superimpose an arrow image pointing to the region of interest on the pathological image, or may display text information explaining the region of interest, as examples of the diagnostic support information.

[0115] [5-11. Pathological images with different imaging conditions] Also, as described in [2-1. Pathological Image], the tile image, which is a pathological image, may be generated for each imaging condition such as focal length and staining condition. When the tile image is generated for each imaging condition, the display control unit 23b displays, for example, a pathological image corresponding to a specific imaging condition (hereinafter, referred to as a specific pathological image) in the example described with reference to FIG. 11 and FIG. 14 to FIG. 22. At this time, the display control unit 23b may display, together with the specific pathological image, other pathological images corresponding to imaging conditions different from the specific imaging condition and having the same region as the specific pathological image, side by side. In this case, the display control unit 23b displays, among the other pathological images, information indicating the region of interest (such as a mask image or a frame image) at a position corresponding to the region of interest of the specific pathological image. Note that the specific imaging condition may be specified by the viewer. Furthermore, when a plurality of imaging conditions are specified by the viewer, the display control unit 23b may display the pathological images of the same region corresponding to each imaging condition side by side.

[0116] [5-12. Diagnostic prediction results] In addition, in the examples shown in Figs. 11 and 14 to 22, the display control unit 23b may display information predicted as a diagnosis result together with information indicating the region of interest (such as a mask image or a frame image). As described above, the attention score derived using the learning model 121 may indicate the probability that a lesion exists. That is, the attention score corresponding to each region of the second pathological image may indicate the probability that a lesion exists in each region. The display control unit 23b may display information related to the probability of the presence of a lesion determined from such attention scores. For example, in the examples shown in Figs. 11 and 14 to 22, the display control unit 23b may superimpose and display information related to the probability of the presence of a lesion together with information indicating the region of interest (such as a mask image or a frame image).

[0117] Furthermore, the information displayed by the display control unit 23b is not limited to the information described above, and the display control unit 23b may display additional information as appropriate. For example, the display control unit 23b may display the resolution of the pathological image. Furthermore, the display control unit 23b may display the recommended size for displaying the observation target based on the browsing history information. Furthermore, the display control unit 23b may display past pathological images of the same patient side by side. Furthermore, the display control unit 23b may display pathological images of other patients that are similar to the pathological image of the display target.

[0118] [6. Variation 2] A modified example of the learning process will now be described.

[0119] [6-1. Learning process (1)] In the above embodiment, an example was shown in which the learning unit 133 learns using the browsing history information acquired from the server 12. However, the learning unit 133 may acquire pathological images stored in the medical information system 30 as the browsing history information. Specifically, as described with reference to Figs. 9A to 9C, the pathological images stored by the pathologist at the time of diagnosis are stored in the diagnostic information storage unit of the medical information system 30. The pathologist often enlarges and stores the area of ​​the pathological image where the lesion was present. That is, the pathological image stored in the diagnostic information storage unit is an image of a partial area of ​​the entire image, and often corresponds to the area where the lesion was present or the area where the lesion is likely to be present. For this reason, the learning unit 133 learns that the pathological image stored in the medical information system 30 is an area that should be noted. For example, when generating a heat map image from the first pathological image stored in the server 12, the learning unit 133 adds a predetermined value to the attention index value of the area of ​​the first pathological image that corresponds to the pathological image acquired from the medical information system 30. This allows the learning unit 133 to learn by taking into account not only the browsing operations by the pathologist but also the pathological images that the pathologist actually paid attention to and saved, thereby making it possible to generate a highly accurate learning model 121.

[0120] [6-2. Learning process (2)] In the above embodiment, the learning unit 133 may learn using only the entire image of the first pathological image. The learning unit 133 may use the viewing history information used in the learning process as position information of the area viewed with attention in the entire image. This allows the learning unit 133 to generate a learning model 121 capable of estimating an area that is particularly noteworthy in the entire image corresponding to the entire observation target.

[0121] [6-3. Learning process (3)] Furthermore, the learning unit 133 may perform learning by weighting based on information about the observation object (here, the first diseased tissue). For example, the learning unit 133 may use, as the information about the observation object, information about the patient from whom the first diseased tissue was collected (gender, age, medical history, etc.), site information about the first diseased tissue (large intestine, lungs, etc.), the reagent that stained the first diseased tissue, information about the institution that stained the first diseased tissue, the type and progression of the diseased site of the observation object, etc.

[0122] For example, the learning unit 133 may learn by weighting the first pathological image corresponding to the patient more heavily as the patient's age increases. Also, for example, the learning unit 133 may learn by weighting the first pathological image corresponding to the first diseased tissue stained by an institution that has stained the first diseased tissue more heavily as the institution that has stained the first diseased tissue has higher reliability.

[0123] The learning unit 133 may also generate a learning model for each piece of information about the first diseased tissue by learning separately for each piece of information about the first diseased tissue. For example, the learning unit 133 generates a learning model for men based on the first pathological images of only men, and generates a learning model for women based on the first pathological images of only women. In this case, the derivation unit 134 performs the derivation process using a learning model that matches the gender of the patient corresponding to the second pathological image to be predicted. Similarly for other examples, the learning unit 133 may generate a learning model for the large intestine and a learning model for the lungs, or may generate a learning model for the dyeing agency A and a learning model for the dyeing agency B. For example, when generating a learning model for the large intestine, the learning unit 133 may learn the first pathological image of a man with a high incidence of large intestine by weighting it more heavily than the first pathological image of a woman. For example, when generating a learning model for breast cancer, the learning unit 133 may learn the first pathological image of a woman with a high incidence of breast cancer by weighting it more heavily than the first pathological image of a man.

[0124] In addition, when the learning unit 133 generates a learning model for each piece of information on the first diseased tissue, the derivation unit 134 may derive the diagnostic support information using each learning model. For example, the derivation unit 134 may derive the diagnostic support information X1 using a learning model for the dyeing agency A, and may derive the diagnostic support information X2 using a learning model for the dyeing agency B. Then, the display control unit 23b may superimpose and display both information indicating the attention area based on the diagnostic support information X1 (mask image, etc.) and information indicating the attention area based on the diagnostic support information X2 on the same pathological image. At this time, the display control unit 23b may change the display mode (saturation, brightness, transparency, etc.) of the information indicating the attention area according to the reliability of the diagnostic support information. For example, when the reliability of the dyeing agency B is higher than that of the dyeing agency A, the display control unit 23b determines that the reliability of the diagnostic support information X2 is higher than that of the diagnostic support information X1. Furthermore, in the example of FIG. 16, the display control unit 23b may switch the display of the pathological image according to the reliability of the diagnostic support information. For example, the display control unit 23b displays pathological images corresponding to regions identified by the diagnostic support information in descending order of reliability of the diagnostic support information. Furthermore, in the example of Fig. 20, the display control unit 23b may display pathological images in order according to the reliability of the diagnostic support information. For example, the display control unit 23b displays pathological images corresponding to regions identified by the diagnostic support information in order of reliability of the diagnostic support information.

[0125] Also, the learning unit 133 generates a learning model for each type of lesion site of the observation object (tissue type shown in Figs. 9A to 9C). In this case, the derivation unit 134 applies the second pathological image to each learning model, thereby being able to derive an attention score corresponding to each region of the second pathological image for each type of lesion site, that is, the probability that a lesion site exists. The display control unit 23b may display the probability that a lesion site exists for each type of lesion site together with information indicating the attention region (mask image, frame image, etc.) based on such attention score. Similarly, when the learning unit 133 generates a learning model for each stage of progression of the observation object (grade shown in Figs. 9A to 9C), the display control unit 23b may display information regarding the grade of the lesion site.

[0126] [6-4. Learning process (4)] Furthermore, the learning unit 133 may perform learning by weighting based on information related to the imaging of the first pathological image. For example, the learning unit 133 may use information related to the device that captured the first pathological image, information related to the institution that captured the first pathological image, and the like, as the information related to the imaging of the first pathological image.

[0127] For example, the learning unit 133 may learn by weighting the first pathology image captured by the device with a higher accuracy as the device that captured the first pathology image has a higher accuracy. Also, for example, the learning unit 133 may learn by weighting the first pathology image captured by the institution with a higher reliability as the device that captured the first pathology image has a higher reliability. Also, the learning unit 133 may generate a learning model for each piece of information related to the imaging of the first pathology image by learning separately for each piece of information related to the imaging of the first pathology image. Also, the display control unit 23b may display a plurality of pieces of diagnostic support information derived using each learning model. Also, the display control unit 23b may change the display mode of the information indicating the attention area according to the reliability of the diagnostic support information determined by the reliability of the imaging device or the imaging institution.

[0128] [6-5. Learning process (5)] Furthermore, the learning unit 133 may perform learning by weighting based on information related to the display of the first pathological image. For example, the learning unit 133 may use information related to the device used to display the first pathological image, the program used to display the first pathological image, the viewing time of the area where the first pathological image is displayed, or the number of times the area where the first pathological image is displayed is displayed.

[0129] For example, the learning unit 133 may learn by weighting the first pathological image displayed and diagnosed by a display device with a higher accuracy as the display device used to display the first pathological image is more accurate. Also, for example, the learning unit 133 may learn by weighting the first pathological image displayed and diagnosed by a program with a higher accuracy as the program used to display the first pathological image is more accurate. As already explained about the viewing time and the number of times of display, the learning unit 133 performs learning based on a heat map image in which the attention index value is reflected as a weight according to the viewing time and the number of times of display. Also, the learning unit 133 may generate a learning model for each information about the display of the first pathological image by learning separately for each information about the display of the first pathological image. Also, the display control unit 23b may display a plurality of pieces of diagnostic support information derived using each learning model. Also, the display control unit 23b may change the display mode of the information indicating the attention area according to the reliability of the diagnostic support information determined by the reliability of the display device and the display program.

[0130] [6-6. Learning process (6)] Furthermore, the learning unit 133 may perform learning by weighting based on information about a user corresponding to the browsing history information (i.e., a viewer of the first pathological image). For example, the learning unit 133 may use information that identifies a diagnostician who diagnosed the first pathological image, or information that indicates the ability rank of the diagnostician, as the information about a user corresponding to the browsing history information.

[0131] For example, the learning unit 133 may learn by weighting the first pathological image viewed by a diagnosing person who diagnosed the first pathological image with a higher diagnostic level or ability rank. The diagnostic level and ability rank of the diagnosing person are determined based on the years of experience, the number of cases experienced, and the like. The learning unit 133 may generate a learning model for each piece of information about the user corresponding to the browsing history information by learning separately for each piece of information about the user corresponding to the browsing history information. The display control unit 23b may display a plurality of pieces of diagnostic support information derived using each learning model. The display control unit 23b may change the display mode of the information indicating the attention area according to the reliability of the diagnostic support information determined by the reliability of the diagnostic level and ability rank of the diagnosing person.

[0132] [6-7. Learning process (7)] In the above embodiment, an example of learning using a heat map image has been shown, but the learning unit 133 may also perform learning without using a heat map image. For example, the learning unit 133 may learn using the pathologist's viewing time, number of views, magnification rate, etc. for each region of the first pathological image as explanatory variables, and an attention score (e.g., 0 to 1) indicating the degree of attention of the pathologist for each region of the pathological image as a target variable. In this case, the learning unit 133 learns so that the longer the viewing time of a region, the higher the attention score, the more frequently viewed a region, the higher the attention score, the more enlarged and viewed a region, and the higher the magnification rate, the higher the attention score.

[0133] [6-8. Learning process (8)] The viewing history storage unit 12a shown in FIG. 8 may store gaze information of the viewer when viewing the first pathological image. For example, the viewing history storage unit 12a stores, as gaze information, position information of an area of ​​the first pathological image that the viewer viewed for a predetermined time or more, or an area of ​​the first pathological image that the viewer viewed a predetermined number of times or more. Such gaze information can be acquired by a technique such as eye tracking. The learning unit 133 may learn by taking into account the gaze information stored in the viewing history storage unit 12a. For example, the learning unit 133 learns by reflecting an attention index value based on the gaze information in the above-mentioned heat map image.

[0134] Second embodiment In the above first embodiment, an example in which diagnostic assistance information is derived using a learning model has been described. In the second embodiment, an example in which diagnostic assistance information is derived without using a learning model will be described.

[0135] 23 is a diagram illustrating an example of a derivation device 200 and a display control device 23 according to the second embodiment. As illustrated in FIG. 23, the derivation device 200 includes a communication unit 110, a storage unit 220, and a control unit 230.

[0136] The storage unit 220 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 220 includes a history information storage unit 221.

[0137] The control unit 230 is realized, for example, by a CPU or an MPU executing a diagnostic assistance program stored in the derivation device 200 using a RAM or the like as a working area. The control unit 230 may also be executed by an integrated circuit such as an ASIC or an FPGA. As shown in FIG. 23, the control unit 230 has a history information acquisition unit 231, a diagnosis information acquisition unit 232, and a derivation unit 234.

[0138] The history information acquisition unit 231 acquires the first pathological image and viewing history information regarding viewing of the first pathological image from the server 12 of the pathology system 10, and stores the acquired first pathological image and viewing history information in the history information storage unit 221.

[0139] The diagnostic information acquisition unit 232 acquires, from the medical information system 30, diagnostic information for the first diseased tissue corresponding to the first pathological image, and stores the acquired diagnostic information in the history information storage unit 221.

[0140] When the derivation unit 234 receives a second pathological image from the display control device 23, the derivation unit 234 derives diagnostic assistance information based on the browsing history information related to the first pathological image having a similarity to the second pathological image higher than a predetermined threshold value TH4 by referring to the history information storage unit 221. Specifically, the derivation unit 234 identifies a first pathological image having a feature similarity to the second pathological image higher than the threshold value TH4 and a positive diagnosis result among the first pathological images stored in the history information storage unit 221. Then, the derivation unit 234 derives, as diagnostic assistance information, position information of an area of ​​the second pathological image similar to the area of ​​interest of the first pathological image, etc., based on the browsing history information corresponding to the identified first pathological image.

[0141] The derivation unit 234 may derive the diagnostic assistance information based on the viewing history information regarding the first pathological image having a correlation degree with the second pathological image higher than a predetermined threshold value TH5. The correlation degree here corresponds to the identity of the patient attributes (gender, age, etc.) and the identity of the affected tissue.

[0142] In this way, the derivation device 200 according to the second embodiment can derive diagnostic assistance information without using a learning model. The display process by the display control device 23 described in the first embodiment can be applied to the second embodiment.

[0143] (Other embodiments) The processing according to each of the above-described embodiments may be implemented in various different forms other than the above-described embodiments.

[0144] [Display device] In the above embodiment, an example has been shown in which information indicating the region of interest, such as a mask image, is superimposed on the pathology image displayed on the stationary display device 24. However, the information indicating the region of interest may be displayed on a wearing device (such as a head-mounted display) worn by a viewer viewing the pathology image displayed on the display device 24. In this way, the information indicating the region of interest displayed on the wearing device may be superimposed on the pathology image displayed on the display device 24. In addition, the information indicating the region of interest may be displayed on a transparent display worn so as to cover the front surface of the display device 24. In this way, the information indicating the region of interest displayed on the transparent display may be superimposed on the pathology image displayed on the display device 24.

[0145] [Imaging device] In the above embodiment, a microscope has been described as an example of a device for capturing an image of an observation target, but the present invention is not limited thereto. For example, the device for capturing an image of an observation target may be a medical image acquisition device such as an endoscope for capturing an image of the inside of a patient's body, a computed tomography (CT), or a magnetic resonance image (MRI). In this case, the server 12 or the server 22 stores medical images such as two-dimensional still images or moving images generated by an endoscope and three-dimensional images generated by a CT or an MRI. The server 12 or the server 22 may also store information related to the images, such as imaging conditions and diagnosis results for the images, in association with these images.

[0146] [server] Further, other pathological images captured by other medical image acquisition devices such as an endoscope, CT, or MRI may be stored in the server 12 or the server 22 in association with the pathological image generated by the microscope. In this case, the display control unit 23b may display, for reference, the other pathological images captured by the other imaging devices side by side in addition to the pathological image generated by the microscope.

[0147] [Pathological images] Among the pathological images stored in the server 12 or the server 22, some have low resolution. That is, although there is browsing history information, the resolution of the pathological image corresponding to the browsing history information may not be sufficient for proper learning. Here, when a glass slide containing an observation object is stored, the glass slide may be re-imaged with a high-resolution microscope to generate a new high-resolution pathological image. For this reason, when the resolution of the first pathological image for learning is not sufficient for proper learning and there is a re-imaged pathological image, the derivation device 100 may perform the above-described learning by associating the browsing history information of the first pathological image with the re-imaged pathological image. At this time, the derivation device 100 aligns the first pathological image with the re-imaged pathological image, and performs processing so that the attention area indicated by the browsing history information of the first pathological image corresponds to the area of ​​the re-imaged pathological image.

[0148] [Hardware configuration] The information devices such as the derivation devices 100 and 200 and the display control device 23 according to the above-described embodiments are realized by a computer 1000 having a configuration as shown in FIG. 24, for example. Hereinafter, the derivation device 100 according to the first embodiment will be described as an example. FIG. 24 is a hardware configuration diagram showing an example of the computer 1000 that realizes the functions of the derivation device 100. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. Each part of the computer 1000 is connected by a bus 1050.

[0149] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processes corresponding to the various programs.

[0150] The ROM 1300 stores boot programs such as a basic input output system (BIOS) executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000.

[0151] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records a response generation program according to the present disclosure, which is an example of program data 1450.

[0152] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0153] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading a program or the like recorded on a predetermined computer-readable recording medium. The medium may be, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0154] For example, when the computer 1000 functions as the derivation device 100 according to the first embodiment, the CPU 1100 of the computer 1000 executes the diagnostic assistance program loaded on the RAM 1200 to realize functions such as the history information acquisition unit 131, the diagnostic information acquisition unit 132, the learning unit 133, and the derivation unit 134. The HDD 1400 stores the diagnostic assistance program according to the present disclosure and data in the storage unit 120. For example, when the computer 1000 functions as the display control device 23 according to the first embodiment, the CPU 1100 of the computer 1000 executes the display control program loaded on the RAM 1200 to realize functions such as the image acquisition unit 23a and the display control unit 23b. The HDD 1400 stores the display control program according to the present disclosure. The CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, the CPU 1100 may acquire the diagnostic assistance program and the display control program from another device via the external network 1550.

[0155] [others] Of the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

[0156] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0157] Furthermore, the above-described embodiments and modifications can be appropriately combined as long as the processing contents are not contradictory.

[0158] (effect) As described above, the derivation device 100 (or 200) causes a computer to execute a derivation procedure for deriving diagnostic support information for supporting the diagnosis of a second pathological image corresponding to a second diseased tissue of a diagnosis target based on a first pathological image corresponding to a first diseased tissue, history information (the above-mentioned browsing history information) related to a browsing history for the first pathological image, and diagnostic information for the first diseased tissue corresponding to the first pathological image. The diagnosis support system 1 also includes a microscope 11 (or 21) and a derivation device 100 (or 200) that derives diagnostic support information, which is information for supporting a diagnosis. The processing by the derivation device 100 may be realized by software used to process a pathological image corresponding to an object imaged by the microscope 11 (or 21). In the following description of the effects, the derivation device 100 can be read as the derivation device 200.

[0159] As a result, according to the above embodiment, it is possible to improve diagnostic accuracy based on the diagnostic assistance information.

[0160] In addition, the derivation device 100 derives information related to browsing as diagnostic assistance information.

[0161] As a result, according to the above embodiment, it is possible to improve the accuracy of diagnosis during browsing.

[0162] In addition, the derivation device 100 derives diagnostic assistance information by using browsing history information corresponding to diagnostic information with a positive diagnostic result.

[0163] As a result, according to the above embodiment, diagnostic assistance information that takes into account browsing history information at the time of positive results is derived, thereby improving the accuracy of positive diagnosis.

[0164] The browsing history information also includes information on an image of a partial area of ​​the first pathological image that was saved when the first pathological image was diagnosed. The derivation device 100 also acquires the image of the partial area of ​​the first pathological image from a medical information system in which information on a patient corresponding to the first pathological image is registered.

[0165] As a result, according to the above embodiment, diagnostic assistance information is derived taking into account pathological images that the pathologist actually focused on and saved, thereby making it possible to further improve diagnostic accuracy.

[0166] The first pathological image is an entire image of the first affected tissue, and the viewing history information is position information indicating the position where the first pathological image was viewed.

[0167] As a result, according to the above embodiment, diagnosis support information for the entire first affected tissue can be derived, which can support diagnosis of the entire first affected tissue without omission.

[0168] The viewing history information is location information indicating the location where the first pathological image was viewed, and information indicating the time or number of times the first pathological image was viewed.

[0169] As a result, according to the above embodiment, diagnostic assistance information is derived based on the area that the pathologist viewed with attention, thereby making it possible to further improve diagnostic accuracy.

[0170] The first pathological image is an image formed by partial images corresponding to each region of the first diseased tissue. The viewing history information is partial image information indicating a viewed partial image among the partial images corresponding to each region. The partial image information is information indicating the position of the partial image or identifying the partial image.

[0171] As a result, according to the above embodiment, even in a system in which a pathological image is formed from partial images, it is possible to improve diagnostic accuracy.

[0172] Moreover, the derivation device 100 derives diagnostic assistance information based on the region of interest of the second pathological image estimated from the viewing history information.

[0173] As a result, according to the above embodiment, it is possible to provide a notable area in the second pathological image of the diagnosis target, thereby facilitating the diagnostic work and reducing variability in diagnosis.

[0174] In addition, based on the viewing history information, the derivation device 100 derives diagnostic assistance information by determining that an area with a higher display magnification is an area of ​​interest that has a higher degree of attention, or an area with a longer viewing time is an area of ​​interest that has a higher degree of attention, or an area with a higher number of views is an area of ​​interest that has a higher degree of attention.

[0175] As a result, according to the above embodiment, the region to be noted can be specified with high accuracy, and therefore the accuracy of diagnosis can be further improved.

[0176] The diagnosis information is information on the presence or absence of a lesion, the type of the lesion, or the progression of the lesion, and is acquired from a medical information system in which information on a patient corresponding to the first pathological image is registered.

[0177] As a result, according to the above embodiment, it is possible to derive diagnostic assistance information according to the presence or absence of a lesion, the type of lesion, or the progression of the lesion.

[0178] The derivation device 100 also learns the correspondence between the first pathological image, the viewing history information, and the diagnostic information. The derivation device 100 also derives diagnostic assistance information using the learning result.

[0179] As a result, according to the above embodiment, since browsing history information is used, diagnostic support information can be derived with high accuracy within a framework of supervised learning that does not require detailed annotation data, such as weakly supervised learning.

[0180] The derivation device 100 performs learning by weighting based on the information on the first diseased tissue, or performs learning by dividing each piece of information on the first diseased tissue. The information on the first diseased tissue is information on the patient from whom the first diseased tissue was collected, information on the site of the first diseased tissue, information on the reagent that stained the first diseased tissue, or information on the institution that stained the first diseased tissue.

[0181] As a result, according to the above embodiment, by using information on the first diseased tissue, it is possible to derive diagnostic assistance information with higher accuracy.

[0182] The derivation device 100 performs learning by weighting based on the information related to the imaging of the first pathological image, or performs learning by dividing the information related to the imaging of the first pathological image. The information related to the imaging of the first pathological image is information related to an apparatus that captured the first pathological image, or an institution that captured the first pathological image.

[0183] As a result, according to the above embodiment, by using information relating to the imaging of the first pathological image, it is possible to derive diagnostic assistance information with higher accuracy.

[0184] The derivation device 100 performs learning by weighting based on the information on the display of the first pathological image, or performs learning separately for each piece of information on the display of the first pathological image. The information on the display of the first pathological image is information on a device used to display the first pathological image, a program used to display the first pathological image, a viewing time of the area where the first pathological image is displayed, or a number of times the area where the first pathological image is displayed is displayed.

[0185] As a result, according to the above embodiment, by using information relating to the display of the first pathological image, it is possible to derive diagnostic assistance information with higher accuracy.

[0186] In addition, the derivation device 100 performs learning by weighting based on information on the user corresponding to the browsing history information, or performs learning by dividing information on the user corresponding to the browsing history information. In addition, the information on the user corresponding to the browsing history information is information for identifying a diagnostician who diagnosed the first pathological image, or information indicating the ability rank of the diagnostician.

[0187] As a result, according to the above embodiment, by using information about the user corresponding to the browsing history information, it is possible to derive diagnostic assistance information with higher accuracy.

[0188] Moreover, the derivation device 100 learns diagnostic information indicating a positive diagnostic result in association with browsing history information corresponding to the diagnostic information.

[0189] As a result, according to the above embodiment, diagnostic assistance information that takes into account browsing history information at the time of positive results is derived, thereby improving the accuracy of positive diagnosis.

[0190] Furthermore, the derivation device 100 learns diagnostic information indicating a negative diagnostic result in association with browsing history information corresponding to the diagnostic information.

[0191] As a result, according to the above embodiment, diagnostic assistance information that takes into account browsing history information when a negative result is obtained is derived, thereby improving the accuracy of a negative diagnosis.

[0192] Moreover, the derivation device 100 learns the region other than the region of interest in the second pathological image as diagnostic information indicating a negative diagnosis result.

[0193] As a result, according to the above embodiment, it is possible to estimate the region to be noted with higher accuracy, thereby further improving the accuracy of diagnosis.

[0194] In addition, the derivation device 200 derives diagnostic assistance information based on browsing history information regarding a browsing history corresponding to a first pathological image whose similarity to a second pathological image is higher than a threshold value, or a first pathological image whose correlation with a second pathological image is higher than a threshold value.

[0195] As a result, according to the above embodiment, it is possible to improve diagnostic accuracy based on the diagnostic assistance information.

[0196] Furthermore, the display control device 23 controls to display a region of interest, which is a region that is assumed to affect the diagnosis, superimposed on the second pathological image in a visibly recognizable state based on the diagnostic assistance information. Furthermore, the display control device 23 controls to display a mask image, which is an image covering the region of interest, a frame image, which is an image showing the outer frame of the region of interest, an arrow image pointing to the region of interest, or text information explaining the region of interest, superimposed on the second pathological image.

[0197] As a result, according to the above embodiment, a region that should be focused on for diagnosis in the second pathological image is displayed, so that it is possible to improve diagnostic accuracy and efficiency.

[0198] The display control device 23 also changes the display mode of the attention area according to the degree of influence on the diagnosis or the reliability of the diagnostic support information based on the diagnostic support information. The display mode is saturation, luminance, or transparency.

[0199] As a result, according to the above embodiment, information reflecting the degree of influence and the degree of reliability is displayed, so that it is possible to improve the accuracy of diagnosis and the efficiency of diagnosis.

[0200] Moreover, the display control device 23 controls so as to display a partial image of the second diseased tissue corresponding to the region of interest.

[0201] According to the embodiment described above, the region to be noted is itself displayed, so that it is possible to improve the accuracy of diagnosis and the efficiency of diagnosis.

[0202] Furthermore, when there are multiple regions of interest, the display control device 23 controls to display a pathological image corresponding to each region of interest. Furthermore, the display control device 23 controls to display the pathological images corresponding to each region of interest side by side on the display screen.

[0203] As a result, according to the above embodiment, it is possible to assist in diagnosing the entire first affected tissue without omission.

[0204] Furthermore, the display control device 23 controls so that the pathological images corresponding to the respective regions of interest are displayed on the display screen in order of the degree of influence on the diagnosis or the reliability of the diagnostic support information.

[0205] As a result, according to the above embodiment, information reflecting the degree of influence and the degree of reliability is displayed, so that it is possible to improve the accuracy of diagnosis and the efficiency of diagnosis.

[0206] Furthermore, the display control device 23 controls so that the pathological images corresponding to the respective regions of interest are switched and displayed in the same display area.

[0207] As a result, according to the above embodiment, it is possible to assist in diagnosing the entire first affected tissue without omission.

[0208] Furthermore, the display control device 23 controls so that the pathological images corresponding to the regions of interest are switched and displayed in the same display area in descending order of the degree of influence on the diagnosis or the reliability of the diagnostic support information.

[0209] As a result, according to the above embodiment, information reflecting the degree of influence and the degree of reliability is displayed, so that it is possible to improve the accuracy of diagnosis and the efficiency of diagnosis.

[0210] The diagnostic assistance information is a diagnostic prediction result that is predicted as a diagnostic result for the first diseased tissue. The diagnostic prediction result is the presence or absence of a lesion, the type of the lesion, the progression of the lesion, or the probability of the lesion.

[0211] As a result, according to the above embodiment, by deriving additional information, that is, a diagnostic prediction result, it is possible to improve the diagnostic accuracy.

[0212] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0213] The present technology can also be configured as follows. (1) an acquisition step of acquiring a first pathological image corresponding to a first diseased tissue, history information related to a viewing history of the first pathological image, and diagnostic information for the first diseased tissue corresponding to the first pathological image; a derivation step of deriving diagnostic assistance information for assisting in diagnosis of a second pathological image corresponding to a second diseased tissue of a diagnosis target based on the first pathological image, the history information, and the diagnosis information; A diagnostic support program that causes an information processing device to execute the above. (2) The derivation procedure is as follows: Deriving information regarding browsing as the diagnostic assistance information; The diagnostic support program according to (1) above. (3) The derivation procedure is as follows: deriving the diagnostic assistance information using history information corresponding to the diagnostic information with a positive diagnostic result; The diagnostic support program according to (1) or (2). (4) The history information includes information regarding an image of a partial area of ​​the first pathological image that was saved at the time of diagnosis of the first pathological image. The diagnostic support program according to any one of (1) to (3) above. (5) The derivation procedure is as follows: An image of a partial area of ​​the first pathological image is obtained from a medical information system in which information on a patient corresponding to the first pathological image is registered. The diagnostic support program according to (4) above. (6) the first pathological image is an overall image of the first affected tissue; The history information is location information indicating a location where the first pathological image was viewed. The diagnostic support program according to any one of (1) to (5) above. (7) The history information is location information indicating a location where the first pathological image was viewed, and information indicating a time or number of times the first pathological image was viewed. The diagnostic support program according to (6) above. (8) the first pathological image is an image formed by partial images corresponding to each region of the first diseased tissue, the history information is partial image information indicating a partial image that has been viewed among the partial images corresponding to each of the areas; The diagnostic support program according to any one of (1) to (7) above. (9) The partial image information is information indicating a position of the partial image or identifying the partial image. The diagnostic support program according to (8) above. (10) The derivation procedure is as follows: deriving the diagnostic assistance information based on a region of interest of the second pathological image estimated from the history information; The diagnostic support program according to any one of (1) to (9) above. (11) The derivation procedure is as follows: deriving the diagnostic assistance information based on the history information, the region having a higher display magnification is regarded as a region of interest having a higher degree of attention, or the region having a longer viewing time is regarded as a region of interest having a higher degree of attention, or the region having a larger number of viewings is regarded as a region of interest having a higher degree of attention; The diagnostic support program according to (10) above. (12) The diagnostic information is information regarding the presence or absence of a lesion, the type of the lesion, or the degree of progression of the lesion. The diagnostic support program according to any one of (1) to (11) above. (13) The diagnostic information is obtained from a medical information system in which information on a patient corresponding to the first pathological image is registered. The diagnostic support program according to any one of (1) to (12) above. (14) further causing the computer to execute a learning procedure for learning correspondence between the first pathological image, the history information, and the diagnosis information; The derivation procedure is as follows: deriving the diagnostic assistance information using a learning result from the learning procedure; The diagnostic support program according to any one of (1) to (13) above. (15) The learning procedure includes: Learning by weighting based on the information about the first affected tissue, or learning separately for each piece of information about the first affected tissue. The diagnostic support program according to (14) above. (16) The information about the first diseased tissue is information about a patient from whom the first diseased tissue was collected, information about the site of the first diseased tissue, a reagent that stained the first diseased tissue, or information about an institution that stained the first diseased tissue. The diagnostic support program according to (15) above. (17) The learning procedure includes: Learning is performed by weighting based on information related to the imaging of the first pathological image, or learning is performed separately for each piece of information related to the imaging of the first pathological image. The diagnostic support program according to any one of (14) to (16) above. (18) The information regarding the imaging of the first pathology image is information regarding an apparatus that imaged the first pathology image or an institution that imaged the first pathology image. The diagnostic support program according to (17) above. (19) The learning procedure includes: learning by weighting based on information regarding the display of the first pathological image, or learning by dividing the information regarding the display of the first pathological image The diagnostic support program according to any one of (14) to (18) above. (20) The information regarding the display of the first pathological image is information regarding a device used to display the first pathological image, a program used to display the first pathological image, a viewing time of the area where the first pathological image is displayed, or a number of times the area where the first pathological image is displayed is displayed. The diagnostic support program according to (19) above. (twenty one) The learning procedure includes: Learning is performed by weighting based on information about the user corresponding to the history information, or learning is performed separately for each piece of information about the user corresponding to the history information. The diagnostic support program according to any one of (14) to (20) above. (twenty two) The information about the user corresponding to the history information is information for identifying a diagnostician who diagnosed the first pathological image, or information indicating a capability rank of the diagnostician. The diagnostic support program according to (21) above. (twenty three) The learning procedure includes: learning the diagnostic information indicating a positive diagnosis result in association with the history information corresponding to the diagnostic information; The diagnostic support program according to any one of (14) to (22) above. (twenty four) The learning procedure includes: learning the diagnostic information indicating a negative diagnostic result in association with the history information corresponding to the diagnostic information; The diagnostic support program according to any one of (14) to (23) above. (twenty five) The learning procedure includes: A region other than the region of interest in the second pathological image is learned as the diagnostic information indicating a negative diagnosis. The diagnostic support program according to (24) above. (26) The derivation procedure is as follows: Deriving the diagnostic support information based on history information on a browsing history corresponding to the first pathological image having a similarity to the second pathological image higher than a threshold value or the first pathological image having a correlation to the second pathological image higher than a threshold value. The diagnostic support program according to any one of (1) to (25) above. (27) A microscope and A derivation device that derives diagnostic assistance information that is information that assists diagnosis; With The delivery device is deriving diagnostic assistance information for assisting in the diagnosis of a second pathological image corresponding to a second diseased tissue of a diagnostic subject imaged by the microscope, based on a first pathological image corresponding to a first diseased tissue, history information related to a viewing history of the first pathological image, and diagnostic information for the first diseased tissue; Diagnostic support system. (28) A diagnostic support system including a microscope and software used to process a pathological image corresponding to an object imaged by the microscope, The software comprises: causing an information processing device to execute a process of deriving diagnostic assistance information for assisting in a diagnosis of a second pathological image corresponding to a second diseased tissue of a diagnostic subject imaged by the microscope, based on a first pathological image corresponding to a first diseased tissue, history information related to a viewing history of the first pathological image, and diagnostic information for the first diseased tissue; Diagnostic support system. (29) The diagnosis support system further includes a display control device, The display control device includes: and controlling, based on the diagnostic assistance information, to display a region of interest, which is a region that is suspected to have an effect on a diagnosis, in a visibly recognizable state so as to be superimposed on the second pathological image. The diagnosis support system according to (27) or (28) above. (30) The display control device includes: and controlling to superimpose and display, on the second pathological image, a mask image which is an image covering the region of interest, a frame image which is an image showing an outer frame of the region of interest, an arrow image which points to the region of interest, or text information which explains the region of interest. The diagnosis support system described in (29) above. (31) The display control device includes: changing a display mode of the attention area according to a degree of influence on the diagnosis or a reliability of the diagnostic assistance information based on the diagnostic assistance information; The diagnosis support system according to (29) or (30) above. (32) The display aspect is saturation, brightness, or transparency. The diagnosis support system according to (31) above. (33) The display control device includes: Controlling to display a partial image of the second diseased tissue corresponding to the region of interest; The diagnosis support system according to any one of (29) to (32) above. (34) The display control device includes: When there are a plurality of regions of interest, control is performed so as to display a pathology image corresponding to each region of interest. The diagnosis support system according to any one of (29) to (33) above. (35) The display control device includes: Controlling to display pathological images corresponding to the respective regions of interest side by side on a display screen; The diagnosis support system according to (34) above. (36) The display control device includes: and controlling the display of the pathological images corresponding to the respective regions of interest on a display screen in order of their degree of influence on the diagnosis or their degree of reliability of the diagnostic assistance information. The diagnosis support system according to (35) above. (37) The display control device includes: Controlling the display of pathological images corresponding to the respective regions of interest in a single display area; The diagnosis support system according to (34) above. (38) The display control device includes: and controlling the display of the pathological images corresponding to the respective regions of interest in the same display area in order of the degree of influence on the diagnosis or the degree of reliability of the diagnostic assistance information. The diagnosis support system according to (37) above. (39) The diagnostic assistance information is a diagnostic prediction result predicted as a diagnostic result for the first affected tissue. The diagnosis support system according to any one of (28) to (38) above. (40) The diagnostic prediction result is the presence or absence of a lesion, the type of the lesion, the progression of the lesion, or the probability of the lesion. The diagnosis support system described in (39) above. (41) The computer Derive diagnostic assistance information for assisting in the diagnosis of a second pathological image corresponding to a second diseased tissue of a diagnosis target, based on a first pathological image corresponding to a first diseased tissue, history information related to a viewing history of the first pathological image, and diagnostic information for the first diseased tissue corresponding to the first pathological image. Diagnostic aid methods. (42) A diagnostic support system comprising a medical image acquisition device and software used to process a medical image corresponding to an object imaged by the medical image acquisition device, The software comprises: causing an information processing device to execute a process of deriving diagnostic assistance information for assisting in the diagnosis of a second medical image corresponding to a second diseased tissue of a diagnostic target imaged by the medical image acquisition device, based on a first medical image corresponding to a first diseased tissue, history information related to a viewing history of the first medical image, and diagnostic information for the first diseased tissue; Diagnostic support system. [Explanation of symbols]

[0214] 1. Diagnostic support system 11, 21 Microscope 12, 22 Server 13, 23 Display control device 14, 24 Display device 23a Image acquisition section 23b Display control unit 100, 200 derivation device 131, 231 History information acquisition unit 132, 232 Diagnostic information acquisition unit 133 Learning Department 134, 234 Derivation part

Claims

1. a derivation step of deriving diagnostic assistance information indicating a region of interest of a second pathological image corresponding to a second diseased tissue of a diagnosis target acquired by a system different from the system that acquired the first pathological image, based on a first pathological image corresponding to a first diseased tissue, a region of interest of the first pathological image based on history information regarding at least one of a viewing time, a number of viewings, and a magnification at the time of viewing for a specific region of the first pathological image, and diagnostic information including at least one of the presence or absence of a lesion site in the first diseased tissue corresponding to the first pathological image, a type of the lesion site, and a degree of progression of the lesion site; A diagnostic support program that causes a computer to execute the above.

2. The derivation procedure is as follows: Deriving information regarding browsing as the diagnostic assistance information; The diagnostic support program according to claim 1 .

3. the first pathological image is an overall image obtained by capturing an image of the entire first affected tissue, the history information is location information indicating a location where the first pathological image was viewed; The diagnostic support program according to claim 1 .

4. The history information is location information indicating a location where the first pathological image was viewed, and information indicating a time or a number of times the first pathological image was viewed. The diagnostic support program according to claim 3 .

5. the first pathological image is an image formed by partial images corresponding to each region of the first diseased tissue, the history information is partial image information indicating a partial image that has been viewed among the partial images corresponding to each of the areas; The diagnostic support program according to claim 1 .

6. The derivation procedure is as follows: The attention area of ​​the first pathological image is determined to have a higher degree of attention as the magnification at the time of viewing is larger, or as the viewing time is longer, or as the number of times the area is viewed is larger, The diagnostic support program according to claim 1 .

7. further causing the computer to execute a learning procedure for learning correspondence between the first pathological image, the history information, and the diagnosis information; The derivation procedure is as follows: deriving the diagnostic assistance information using a learning result from the learning procedure; The diagnostic support program according to claim 1 .

8. The learning procedure includes: learning the diagnostic information indicating a positive diagnosis result in association with the history information corresponding to the diagnostic information; The diagnostic support program according to claim 7.

9. The learning procedure includes: learning the diagnostic information indicating a negative diagnostic result in association with the history information corresponding to the diagnostic information; The diagnostic support program according to claim 7.

10. The learning procedure includes: A region other than the region of interest in the second pathological image is learned as the diagnostic information indicating a negative diagnosis. The diagnostic support program according to claim 9.

11. The derivation procedure is as follows: Deriving the diagnostic assistance information based on history information on a browsing history corresponding to the first pathological image having a similarity to the second pathological image higher than a threshold value or the first pathological image having a correlation to the second pathological image higher than a threshold value. The diagnostic support program according to claim 1 .

12. A plurality of systems, each having a microscope; A derivation device that derives diagnostic assistance information that is information that assists diagnosis; With The delivery device is deriving diagnostic assistance information indicating a region of interest of a second pathological image corresponding to a second diseased tissue of a diagnosis target imaged by the microscope possessed by a second system different from the first system among the multiple systems, based on a first pathological image corresponding to a first diseased tissue acquired by a first system among the multiple systems, a region of interest of the first pathological image based on history information regarding a viewing history of at least one of a viewing time, a viewing count, and a viewing magnification for a specific region of the first pathological image, and diagnostic information including at least one of the presence or absence of a lesion site in the first diseased tissue corresponding to the first pathological image, a type of the lesion site, and a progression level of the lesion site; Diagnostic support system.

13. The diagnosis support system further includes a display control device, The display control device includes: and controlling the display of a region of interest, which is a region that is assumed to have an effect on a diagnosis, on the second pathological image in a visibly recognizable state based on the diagnostic assistance information. The diagnosis support system according to claim 12.

14. The display control device includes: changing a display mode of the attention area according to a degree of influence on the diagnosis or a reliability of the diagnostic assistance information based on the diagnostic assistance information; The diagnosis support system according to claim 13.

15. The display control device includes: Controlling to display a partial image of the second diseased tissue corresponding to the region of interest; The diagnosis support system according to claim 13.

16. The diagnostic assistance information is a diagnostic prediction result predicted as a diagnostic result for the first affected tissue. The diagnosis support system according to claim 12.

17. The diagnostic prediction result is the presence or absence of a lesion, the type of the lesion, the progression of the lesion, or the probability of the lesion. The diagnosis support system according to claim 16.

18. The computer Based on a first pathological image corresponding to a first diseased tissue, a region of interest in the first pathological image based on history information regarding at least one of a viewing time, a number of views, and a magnification at the time of viewing a specific region of the first pathological image, and diagnostic information including at least one of the presence or absence of a lesion site in the first diseased tissue corresponding to the first pathological image, a type of the lesion site, and a degree of progression of the lesion site, diagnostic assistance information indicating a region of interest in a second pathological image corresponding to a second diseased tissue of a diagnosis target acquired by a system different from the system that acquired the first pathological image is derived. Diagnostic aid methods.

19. A diagnostic support system including a plurality of systems each having a microscope, and software used to process a pathological image corresponding to an object imaged by the microscope, The software comprises: and executing a process of deriving diagnostic assistance information indicating a region of interest of a second pathological image corresponding to a second diseased tissue of a diagnosis target imaged by the microscope possessed by a second system different from the first system among the multiple systems, based on a first pathological image corresponding to a first diseased tissue acquired by a first system among the multiple systems, a region of interest of the first pathological image based on history information regarding a viewing history of at least one of a viewing time, a viewing count, and a viewing magnification for a specific region of the first pathological image, and diagnostic information including at least one of the presence or absence of a lesion site in the first diseased tissue corresponding to the first pathological image, a type of the lesion site, and a progression level of the lesion site. Diagnostic support system.

Citation Information

Patent Citations

  • Image processing device and method, and program

    JP2012155455A

  • Diagnosis support device, diagnosis support method, and diagnosis support program

    JP2015116319A