Pathological diagnosis support device, pathological diagnosis support method, and pathological diagnosis support program

Abstract

A pathological diagnosis support device comprises: an acquisition unit that acquires pathological images including cells as subjects; a determination unit that determines whether malignant cells are included in the pathological images by inputting the pathological images to a machine-trained first classification model; a classification unit that classifies the pathological images by class by inputting the pathological images to a second classification model machine-trained to classify the pathological image into any of a plurality of classes including one or more features suggesting the presence of malignant cells; and an output unit that outputs a classification result and supports a user's decision making.

Description

Pathological Diagnosis Support Device, Pathological Diagnosis Support Method, and Pathological Diagnosis Support Program

[0001] The present disclosure relates to a pathological diagnosis support device, a pathological diagnosis support method, and a pathological diagnosis support program.

[0002] There is a known device that supports a diagnosis for determining a patient's medical condition. For example, Patent Document 1 discloses a diagnostic support device that extracts a plurality of characteristic regions suspected of being a lesion in the inner wall portion inside a living body, and color-codes and displays each position on the inner wall portion in the image according to the degree of risk based on the scores assigned to each characteristic region.

[0003] Japanese Patent Application Laid-Open No. 2012-157384

[0004] In the diagnostic support device described in Patent Document 1, pathological diagnosis is not assumed. In pathological diagnosis, a pathological specimen is thinly sliced, the sliced pathological specimen is attached to a slide glass and stained, and it is observed whether malignant cells are present. However, when malignant cells are unevenly distributed in the pathological specimen, there is a problem that malignant cells may not be found in the sliced pathological specimen and it may be diagnosed as benign. To solve this problem, there is a method of observing a plurality of sliced pathological specimens, but there is a problem that the cost increases by then. Therefore, a technique for suitably supporting pathological diagnosis is required.

[0005] The present disclosure has been made in view of the above problems, and an exemplary object thereof is to provide a technique for suitably supporting pathological diagnosis.

[0006] A pathology diagnostic support device relating to an exemplary aspect of this disclosure includes: an acquisition means for acquiring a pathology image of an arbitrary slice containing cells as a subject; a determination means for determining whether or not malignant cells are included in the pathology image of an arbitrary slice by inputting the pathology image of the arbitrary slice into a first classification model that has been trained to classify whether or not the cells included as a subject in the input image are malignant; a classification means for classifying the pathology image into each of the classes by inputting the pathology image determined by the determination means to include one or more features in the input image that suggest the presence of malignant cells in a neighboring slice, if the determination means determines that malignant cells are not included in the pathology image of the arbitrary slice; and an output means for outputting the classification results from the classification means.

[0007] A pathology diagnostic support method relating to an exemplary aspect of the present disclosure includes: an acquisition process in which at least one processor acquires a pathology image of an arbitrary slice containing cells as a subject; a determination process in which the at least one processor inputs the pathology image of the arbitrary slice to a first classification model that has been trained to classify whether or not the cells included as subjects in the input image are malignant, thereby determining whether or not malignant cells are included in the pathology image of the arbitrary slice; if the determination process determines that malignant cells are not included in the pathology image of the arbitrary slice, a classification process in which the at least one processor inputs the pathology image that was determined not to contain malignant cells in the determination process to a second classification model that has been trained to classify the image into one or more classes of features included in the input image that suggest the presence of malignant cells in a neighboring slice, thereby classifying the pathology image according to the class; and an output process in which the at least one processor outputs the classification result obtained by the classification process.

[0008] An exemplary aspect of the present disclosure relates to a pathology diagnostic support program, which is a program that causes a computer to function as a pathology diagnostic support device, and the computer functions as: an acquisition means for acquiring a pathology image of an arbitrary slice containing cells as a subject; a determination means for determining whether or not malignant cells are included in the pathology image of an arbitrary slice by inputting the pathology image of the arbitrary slice into a first classification model that has been trained to classify whether or not the cells included as a subject in the input image are malignant; a classification means for classifying the pathology image into each of the classes by inputting the pathology image that has been determined by the determination means to not contain malignant cells, if the determination means has determined that malignant cells are not included in the pathology image of the arbitrary slice, into one or more classes that include one or more features in the input image that suggest the presence of malignant cells in a neighboring slice; and an output means for outputting the classification results from the classification means.

[0009] A pathology diagnostic support device relating to an illustrative aspect of this disclosure includes an acquisition means for acquiring a pathology image of an arbitrary slice containing cells as the subject, and a classification means for classifying the pathology image of the arbitrary slice into each of several classes by inputting the pathology image of the arbitrary slice into a classification model that has been trained to classify the image into one or more classes that include one or more features in the input image that suggest the presence of malignant cells in a neighboring slice.

[0010] A pathology diagnostic support method relating to an illustrative aspect of this disclosure includes: an acquisition process in which at least one processor acquires a pathology image of an arbitrary slice containing cells as the subject; and a classification process in which the at least one processor inputs the pathology image of the arbitrary slice into a classification model that has been trained to classify the image into one of a plurality of classes, each class containing one or more features in the input image that suggest the presence of malignant cells in a neighboring slice.

[0011] An exemplary aspect of the present disclosure relates to a pathology diagnostic support program, which is a program that causes a computer to function as a pathology diagnostic support device, wherein the computer functions as an acquisition means for acquiring pathology images of arbitrary slices containing cells as subjects, and a classification means for classifying the pathology images of the arbitrary slices by inputting the pathology images of the arbitrary slices into a classification model that has been trained to classify the images into one or more classes of features contained in the input images, each of which features suggest the presence of malignant cells in neighboring slices.

[0012] One exemplary aspect of this disclosure is that it can provide a technology that suitably supports pathological diagnosis.

[0013] This is a block diagram showing the configuration of the pathology diagnostic support device related to this disclosure. This is a flowchart showing the flow of the pathology diagnostic support method related to this disclosure. This is a block diagram showing the configuration of the pathology diagnostic support device related to this disclosure. This is a flowchart showing the flow of the pathology diagnostic support method related to this disclosure. This is a block diagram showing the configuration of the pathology diagnostic support device related to this disclosure. This is a diagram showing an example of an image output by the output unit related to this disclosure. This is a diagram showing another example of an image output by the output unit related to this disclosure. This is a block diagram showing the configuration of the pathology diagnostic support device related to this disclosure. This is a diagram showing an example of a process performed by the expert network related to this disclosure. This is a block diagram showing the configuration of a computer that functions as a pathology diagnostic support device related to this disclosure.

[0014] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining some or all of the technologies (things or methods) employed in each of the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technologies employed in each of the exemplary embodiments shown below may also be included in the scope of the present invention. In addition, the effects mentioned in each of the exemplary embodiments shown below are examples of effects that can be expected in that exemplary embodiment and do not define the scope of the present invention. That is, embodiments that do not produce the effects mentioned in each of the exemplary embodiments shown below may also be included in the scope of the present invention.

[0015] [First Exemplary Embodiment] A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is the basic form for each of the exemplary embodiments described later. The scope of application of each technology adopted in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology adopted in this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems occur. Furthermore, each technology shown in the drawings referenced to explain this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems occur.

[0016] (Configuration of the Pathology Diagnostic Support Device 1) The configuration of the pathology diagnostic support device 1 will be described with reference to Figure 1. Figure 1 is a block diagram showing the configuration of the pathology diagnostic support device 1. As shown in Figure 1, the pathology diagnostic support device 1 includes an acquisition unit 11, a determination unit 12, a classification unit 13, and an output unit 14. In this exemplary embodiment, the acquisition unit 11, determination unit 12, classification unit 13, and output unit 14 realize an acquisition means, a determination means, a classification means, and an output means, respectively.

[0017] (Acquisition unit 11) The acquisition unit 11 acquires pathological images of arbitrary slices that include cells as subjects. The acquisition unit 11 supplies the acquired pathological images to the determination unit 12 and the classification unit 13.

[0018] (Determination Unit 12) The determination unit 12 inputs the pathology image of an arbitrary slice acquired by the acquisition unit 11 into a first classification model that has been trained to classify whether or not the cells included as subjects in the input image are malignant, thereby determining whether or not malignant cells are included in the pathology image of an arbitrary slice. The determination unit 12 supplies the determination result to the classification unit 13.

[0019] (Classification Unit 13) When the determination unit 12 determines that no malignant cells are present in the pathology image of an arbitrary slice, the classification unit 13 classifies the pathology image into classes by inputting the pathology image determined by the determination unit 12 to be free of malignant cells into a second classification model that has been trained to classify the image into one of several classes that include one or more features in the input image that suggest the presence of malignant cells in a neighboring slice. The classification unit 13 supplies the classification results to the output unit 14.

[0020] A neighboring slice refers to a slice that falls within a specified range from any given slice. While the specified range is not limited, one example is a range of approximately 6-20 μm. Since pathological specimens are typically sliced ​​to a thickness of 3-4 μm, this corresponds to approximately 2-5 such slices.

[0021] (Output unit 14) The output unit 14 outputs the classification result from the classification unit 13.

[0022] (Effects of the pathology diagnostic support device 1) As described above, the pathology diagnostic support device 1 employs a configuration comprising: an acquisition unit 11 that acquires a pathology image of an arbitrary slice containing cells as the subject; a determination unit 12 that determines whether or not malignant cells are included in the pathology image of an arbitrary slice by inputting the pathology image of the arbitrary slice acquired by the acquisition unit 11 to a first classification model that has been trained to classify whether or not the cells included as the subject in the input image are malignant; a classification unit 13 that, if the determination unit 12 determines that malignant cells are not included in the pathology image of an arbitrary slice, classifies the pathology image by class by inputting the pathology image that the determination unit 12 determined does not contain malignant cells to a second classification model that has been trained to classify the image into one of several classes that include one or more features in the input image that suggest the presence of malignant cells in a neighboring slice; and an output unit 14 that outputs the classification results from the classification unit 13.

[0023] Therefore, according to the pathology diagnostic support device 1, even if cells included as subjects in the pathology image of an arbitrary slice are classified as not malignant, they are classified into one of several classes that include one or more features suggesting the presence of malignant cells. Consequently, the pathology diagnostic support device 1 can suitably support pathology diagnosis.

[0024] (Example of implementation by program) When the pathology diagnostic support device 1 is composed of a computer including at least one processor and memory, the following program is stored in the memory. The program is a program that causes the computer to function as the pathology diagnostic support device 1, and causes the computer to function as: an acquisition unit 11 that acquires a pathology image of an arbitrary slice containing cells as the subject; a determination unit 12 that determines whether or not malignant cells are included in the pathology image of an arbitrary slice by inputting the pathology image of the arbitrary slice acquired by the acquisition unit 11 into a first classification model that has been trained to classify whether or not the cells included as the subject in the input image are malignant; a classification unit 13 that, when the determination unit 12 determines that malignant cells are not included in the pathology image of an arbitrary slice, classifies the pathology image by class by inputting the pathology image that the determination unit 12 determined does not contain malignant cells into one or more classes that include one or more features in the input image that suggest the presence of malignant cells in a neighboring slice; and an output unit 14 that outputs the classification results from the classification unit 13.

[0025] (Flowchart of Pathology Diagnosis Support Method S1) The flowchart of pathology diagnosis support method S1 will be explained with reference to Figure 2. Figure 2 is a flowchart showing the flowchart of pathology diagnosis support method S1. As shown in Figure 2, pathology diagnosis support method S1 includes acquisition processing S11, judgment processing S12, classification processing S13, and output processing S14.

[0026] (Acquisition process S11) In acquisition process S11, the acquisition unit 11 acquires a pathological image of an arbitrary slice containing cells as the subject. The acquisition unit 11 supplies the acquired pathological image to the determination unit 12 and the classification unit 13.

[0027] (Determination process S12) In determination process S12, the determination unit 12 inputs the pathology image of an arbitrary slice acquired by the acquisition unit 11 to a first classification model that has been trained to classify whether or not the cells included as subjects in the input image are malignant, thereby determining whether or not malignant cells are included in the pathology image of an arbitrary slice. The determination unit 12 supplies the determination result to the classification unit 13.

[0028] (Classification Processing S13) In classification processing S13, if the determination unit 12 determines that the pathology image of an arbitrary slice does not contain malignant cells, the classification unit 13 classifies the pathology image into classes by inputting the pathology image determined by the determination unit 12 to contain one or more features in the input image that suggest the presence of malignant cells in a neighboring slice. The classification unit 13 supplies the classification results to the output unit 14.

[0029] (Output processing S14) In output processing S14, the output unit 14 outputs the classification result from the classification unit 13.

[0030] (Effects of the pathology diagnosis support method S1) As described above, the pathology diagnosis support method S1 employs a configuration that includes: an acquisition process S11 in which the acquisition unit 11 acquires a pathology image of an arbitrary slice containing cells as the subject; a determination process S12 in which the determination unit 12 inputs the pathology image of the arbitrary slice acquired by the acquisition unit 11 to a first classification model that has been trained to classify whether or not the cells included as the subject in the input image are malignant, thereby determining whether or not malignant cells are included in the pathology image of the arbitrary slice; a classification process S13 in which, if the determination unit 12 determines that malignant cells are not included in the pathology image of the arbitrary slice, inputs the pathology image determined by the determination unit 12 to a second classification model that has been trained to classify into one of a plurality of classes that include one or more features in the input image that suggest the presence of malignant cells in a neighboring slice, thereby classifying the pathology image by class; and an output process S14 in which the output unit 14 outputs the classification results from the classification unit 13. Therefore, the pathology diagnostic support method S1 produces the same effect as the pathology diagnostic support device 1 described above.

[0031] (Configuration of the pathology diagnostic support device 2) The configuration of the pathology diagnostic support device 2 will be described with reference to Figure 3. Figure 3 is a block diagram showing the configuration of the pathology diagnostic support device 2. As shown in Figure 3, the pathology diagnostic support device 2 includes an acquisition unit 11 and a classification unit 13. In this exemplary embodiment, the acquisition unit 11 and the classification unit 13 realize an acquisition means and a classification means, respectively.

[0032] (Acquisition unit 11) The acquisition unit 11 acquires pathological images of arbitrary slices that include cells as subjects. The acquisition unit 11 supplies the acquired pathological images to the classification unit 13.

[0033] (Classification Unit 13) The classification unit 13 classifies pathological images into classes by inputting pathological images of arbitrary slices acquired by the acquisition unit 11 into a classification model that has been trained to classify the input images into one or more classes that include one or more features in the input image that suggest the presence of malignant cells in a nearby slice.

[0034] (Effects of the pathology diagnostic support device 2) As described above, the pathology diagnostic support device 2 employs a configuration that includes an acquisition unit 11 that acquires pathology images of arbitrary slices containing cells as subjects, and a classification unit 13 that classifies pathology images into classes by inputting the pathology images of arbitrary slices acquired by the acquisition unit 11 into a classification model that has been trained to classify the input images into one or more classes that include one or more features in the input image that suggest the presence of malignant cells in neighboring slices.

[0035] Therefore, the pathology diagnostic support device 2 can determine whether the pathology image of any slice contains at least one or more features that suggest the presence of malignant cells. Consequently, the pathology diagnostic support device 2 can effectively support pathology diagnosis.

[0036] (Example of implementation by program) When the pathology diagnostic support device 2 is composed of a computer including at least one processor and memory, the following program is stored in the memory. The program is a program that causes the computer to function as the pathology diagnostic support device 2, and is a pathology diagnostic support program that causes the computer to function as an acquisition unit 11 that acquires pathology images of arbitrary slices including cells as subjects, and a classification unit 13 that classifies pathology images into classes by inputting the pathology images of arbitrary slices acquired by the acquisition unit 11 into a classification model that has been trained to classify the input images into one or more classes that include one or more features in the input images that suggest the presence of malignant cells in neighboring slices.

[0037] (Flowchart of Pathology Diagnosis Support Method S2) The flowchart of Pathology Diagnosis Support Method S2 will be explained with reference to Figure 4. Figure 4 is a flowchart showing the flowchart of Pathology Diagnosis Support Method S2. As shown in Figure 4, Pathology Diagnosis Support Method S2 includes acquisition processing S11 and classification processing S13.

[0038] (Acquisition process S11) In acquisition process S11, the acquisition unit 11 acquires pathological images of arbitrary slices that include cells as subjects. The acquisition unit 11 supplies the acquired pathological images to the classification unit 13.

[0039] (Classification Processing S13) In classification processing S13, the classification unit 13 classifies the pathological images into classes by inputting the pathological images of arbitrary slices acquired by the acquisition unit 11 into a classification model that has been trained to classify the input images into one or more classes that include one or more features in the input image that suggest the presence of malignant cells in a neighboring slice.

[0040] (Effects of pathology diagnostic support method S2) As described above, the pathology diagnostic support method S1 employs a configuration that includes an acquisition process S11 in which the acquisition unit 11 acquires a pathology image of an arbitrary slice containing cells as the subject, and a classification process S13 in which the classification unit 13 classifies the pathology image into classes by inputting the pathology image of the arbitrary slice acquired by the acquisition unit 11 into a classification model that has been trained to classify the input image into one or more classes that include one or more features contained in the input image that suggest the presence of malignant cells in a neighboring slice. Therefore, the pathology diagnostic support method S2 produces the same effects as the pathology diagnostic support device 2 described above.

[0041] [Second Exemplary Embodiment] A second exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same function as those described in the above-described exemplary embodiment are denoted by the same reference numerals, and their descriptions are omitted as appropriate. The scope of application of each technology adopted in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology adopted in this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems arise. Furthermore, each technology shown in the drawings referenced to describe this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems arise.

[0042] (Overview of the Pathology Diagnosis Support Device 1A) The pathology diagnosis support device 1A is a device that determines whether cells included as subjects in a pathological image of an arbitrary slice (hereinafter, also simply referred to as a "pathological image") are malignant. Further, when the pathology diagnosis support device 1A determines that a pathological image does not contain malignant cells, it determines whether an arbitrary slice of the pathological image includes one or more features suggesting the presence of malignant cells in a slice adjacent to the arbitrary slice (hereinafter, also simply referred to as an "adjacent slice"). Further, the pathology diagnosis support device 1A determines whether at least any one of one or more features suggesting the presence of malignant cells in the adjacent slice is included in the pathological image of the arbitrary slice, and when at least any one of one or more features suggesting the presence of malignant cells in the adjacent slice is included in the pathological image of the arbitrary slice, outputs a classification result indicating which feature is included.

[0043] Examples of one or more features suggesting the presence of malignant cells in the adjacent slice include the following. - Dysplasia of tissue structure - Irregular arrangement of cells - Many microvessels - Strong inflammation - Presence of Helicobacter pylori As an example of a pathological image, an image of a pathological specimen obtained by thinly slicing a block-shaped pathological specimen and photographing the specimen stained and attached to a slide glass can be given. The pathological image may be an image (WSI: Whole Slide Imaging) of the entire pathological specimen, or a patch obtained by cutting out a part of the WSI.

[0044] (Configuration of the Pathology Diagnosis Support Device 1A) The configuration of the pathology diagnosis support device 1A will be described with reference to FIG. 5. FIG. 5 is a block diagram showing the configuration of the pathology diagnosis support device 1A. As shown in FIG. 5, the pathology diagnosis support device 1A includes a control unit 10, a storage unit 20, an input / output unit 30, and a communication unit 40.

[0045] (Storage Unit 20) Data referred to by the control unit 10 is stored in the storage unit 20. Examples of the storage unit 20 include, but are not limited to, a flash memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a combination thereof.

[0046] Examples of data stored in the memory unit 20 include the first classification model CM1 and the second classification model CM2. Note that the storage of the first classification model CM1 and the second classification model CM2 in the memory unit 20 indicates that the parameters defining each of the first classification model CM1 and the second classification model CM2 are stored in the memory unit 20.

[0047] The first classification model CM1 is a machine learning model that classifies whether or not the cells included as subjects in the input image are malignant. For example, the first classification model CM1 is machine learning-based using training data that includes images that do not contain malignant cells and the label "0", and training data that includes images that contain malignant cells and the label "1". In this case, the first classification model CM1 outputs "0" if it is highly likely that the input image does not contain malignant cells, and outputs "1" if it is highly likely that the input image contains malignant cells.

[0048] The second classification model CM2 is a machine - learned model that classifies into any of a plurality of classes including one or more features included in an input image, where the one or more features suggest the presence of malignant cells in neighboring slices. For example, the second classification model CM2 is machine - learned by the following training data to classify an input image into each class of one or more features that suggest the presence of malignant cells. - Training data including an image containing the feature of "cellular atypia" and the label "1" - Training data including an image containing the feature of "irregular cell arrangement" and the label "2" - Training data including an image containing the feature of "many microvessels" and the label "3" - Training data including an image containing the feature of "strong inflammation" and the label "4" - Training data including an image containing the feature of "Helicobacter pylori present" and the label "5" Furthermore, the second classification model CM2 is machine - learned by the following training data. - Training data including an image that does not include any of the one or more features that suggest the presence of malignant cells and the label "0" In this case, the second classification model CM2 outputs the class with the highest probability that the feature included in the input image corresponds to each of the one or more features that suggest the presence of malignant cells. For example, when the input image is most likely to correspond to not including any of the one or more features that suggest the presence of malignant cells (hereinafter also referred to as "no findings"), the second classification model CM2 outputs "0". Also, when the input image is most likely to correspond to including the feature of "cellular atypia", the second classification model CM2 outputs "1".

[0049] Also, the second classification model CM2 may output the probability that the feature included in the input image corresponds to each of the one or more features that suggest the presence of malignant cells.

[0050] As another example of the data stored in the storage unit 20, there are a pathological image, a determination result indicating whether or not the pathological image contains malignant cells, and a classification result described later.

[0051] (Input / Output Unit 30) The input / output unit 30 is an interface to an input device that accepts data input and an output device that outputs data. Examples of input devices include, but are not limited to, a microphone, camera, eye-tracking device, keyboard, and touchpad. Examples of output devices include, but are not limited to, a speaker and liquid crystal display.

[0052] (Communication Unit 40) The communication unit 40 is an interface for sending and receiving data over a network. Examples of the communication unit 40 include, but are not limited to, communication chips in various communication standards such as Ethernet®, Wi-Fi®, and wireless communication standards for mobile data communication networks, and USB-compliant connectors.

[0053] Furthermore, while the specific network configuration is not particularly limited, examples include wireless LAN (Local Area Network), wired LAN, WAN (Wide Area Network), public telephone network, mobile data communication network, or a combination of these networks.

[0054] (Control Unit 10) The control unit 10 controls each component of the pathology diagnostic support device 1A. As shown in Figure 5, the control unit 10 also includes an acquisition unit 11, a determination unit 12, a classification unit 13, and an output unit 14. In this exemplary embodiment, the acquisition unit 11, determination unit 12, classification unit 13, and output unit 14 implement the acquisition means, determination means, classification means, and output means, respectively.

[0055] (Acquisition Unit 11) The acquisition unit 11 acquires data supplied from the input / output unit 30 or the communication unit 40. The acquisition unit 11 stores the acquired data in the storage unit 20. As an example, the acquisition unit 11 acquires a pathological image of an arbitrary slice containing cells as the subject.

[0056] Furthermore, if the acquisition unit 11 acquires a WSI image of the entire pathological specimen, it may divide it into patches of a predetermined size. For example, if the acquired WSI has 120,000 x 90,000 pixels, the acquisition unit 11 divides it into 1024 x 1024 pixel patches and stores them in the storage unit 20.

[0057] (Determination Unit 12) The determination unit 12 determines whether or not malignant cells are present in the pathology image. For example, the determination unit 12 determines whether or not malignant cells are present in the pathology image by inputting the pathology image stored in the memory unit 20 into the first classification model CM1. The determination unit 12 stores the determination result in the memory unit 20.

[0058] (Classification Unit 13) The classification unit 13 classifies the pathological image into one of several classes that include one or more features in the pathological image that suggest the presence of malignant cells in a nearby slice. For example, the classification unit 13 classifies the pathological images into classes by inputting a pathological image that the determination unit 12 has determined does not contain malignant cells into the second classification model CM2.

[0059] As an example, as described above, the classification unit 13 classifies the pathological image into one of the following classes: • Class "No findings" • Class "Tissue structure dysplasia" • Class "Irregular arrangement of cells" • Class "Many microvessels" • Class "Severe inflammation" • Class "Presence of Helicobacter pylori" The classification unit 13 stores the classification result in the storage unit 20.

[0060] (Output Unit 14) The output unit 14 outputs data via the input / output unit 30 or the communication unit 40. For example, if the determination unit 12 determines that malignant cells are present in the pathology image, the output unit 14 outputs information indicating that the cells are malignant. For example, the output unit 14 outputs an image containing a message indicating that malignant cells are present in the pathology image.

[0061] Furthermore, the output unit 14 outputs the classification results stored in the memory unit 20. For example, the output unit 14 displays an image showing the classification results on a display device via the input / output unit 30.

[0062] For example, if the classification unit 13 classifies the pathological image of an arbitrary slice into at least one class of features that suggest the presence of malignant cells, the output unit 14 outputs a classification result that includes a message indicating that malignant cells may be present around the cells included as subjects in the pathological image of the arbitrary slice.

[0063] As another example, if the classification unit 13 classifies the pathological image of an arbitrary slice into at least one class of one or more features that suggest the presence of malignant cells, the output unit 14 outputs a classification result that includes a message indicating which of the one or more features that suggest the presence of malignant cells the pathological image of the arbitrary slice falls under.

[0064] As another example, if the classification unit 13 classifies the pathological image of an arbitrary slice into at least one class of features that suggest the presence of malignant cells, the output unit 14 indicates the areas corresponding to at least one of the features that suggest the presence of malignant cells on the pathological image of the arbitrary slice, and then outputs the classification result including the pathological image of the arbitrary slice.

[0065] As another example, if the second classification model CM2 outputs the probability that each of the features contained in the input image corresponds to one or more features that suggest the presence of malignant cells, the output unit 14 outputs a classification result that includes a message indicating that the higher the probability, the more likely it is that malignant cells are present near the cells contained in the pathological image of the arbitrary slice.

[0066] An example of an image output by the output unit 14 will be described later.

[0067] Furthermore, the output unit 14 may refer to other diagnostic results associated with the pathology image of an arbitrary slice. For example, if the pathology image is that of patient A, the output unit 14 will refer to other diagnostic results for patient A. As an example, the output unit 14 may refer to the diagnostic results of the clinical image of patient A.

[0068] In this case, if the classification unit 13 classifies the pathological image of the arbitrary slice into a class that does not fall under any of the one or more characteristics that suggest the presence of malignant cells, and other diagnostic results indicate that the pathological image of the arbitrary slice does not contain malignant cells, the output unit 14 outputs a classification result indicating that the cells contained in the pathological image of the arbitrary slice are benign.

[0069] On the other hand, even if the classification unit 13 classifies the pathological image into a class that does not fall under any of the one or more characteristics that suggest the presence of malignant cells, if other diagnostic results indicate that the pathological image contains malignant cells (for example, if cancer is strongly suspected in the CT scan), the output unit 14 outputs a classification result that includes a message indicating that malignant cells may be present in the pathological image.

[0070] With this configuration, even if no features suggesting the presence of malignant cells are found in the pathology images, the pathology diagnostic support device 1A can notify the user that malignant cells may be present by referring to other diagnostic results.

[0071] (Process flow performed by pathology diagnostic support device 1A) The process flow (pathology diagnostic support method S1A) performed by pathology diagnostic support device 1A will be explained with reference to Figure 6. Figure 6 is a flowchart showing the flow of pathology diagnostic support method S1A.

[0072] (Step S11) In step S11, the acquisition unit 11 acquires a pathological image that includes cells as the subject. The acquisition unit 11 stores the acquired pathological image in the storage unit 20.

[0073] (Step S12) In step S12, the determination unit 12 inputs the pathological image stored in the memory unit 20 to the first classification model CM1 to determine whether or not malignant cells are included in the pathological image. The determination unit 12 stores the determination result in the memory unit 20.

[0074] (Step S13) If, in step S12, it is determined that the pathological image does not contain malignant cells (Step S12: NO), then in step S13, the classification unit 13 inputs the pathological image determined by the determination unit 12 to not contain malignant cells into the second classification model CM2, thereby classifying the pathological image into classes. The classification unit 13 stores the classification results in the storage unit 20.

[0075] (Step S14) In step S14, the output unit 14 outputs the classification result stored in the memory unit 20. Also in step S14, as described above, the output unit 14 outputs a classification result including a message, or outputs a classification result including a pathology image.

[0076] (Step S15) If, in step S12, it is determined that the pathological image contains malignant cells (Step S12: YES), in step S15, the output unit 14 outputs information indicating that it is malignant.

[0077] (Example 1 of an image output by the output unit 14) An example of an image output by the output unit 14 in step S15 will be explained with reference to Figure 7. Figure 7 is a diagram showing an example of an image output by the output unit 14.

[0078] If the classification unit 13 classifies the pathological image into a class that includes at least one of one or more features, the output unit 14 may output a classification result that includes a message msg2 indicating that malignant cells may be present around the cells included as subjects in the pathological image, as shown in Figure 7.

[0079] With this configuration, the pathology diagnostic support device 1A can notify the user (physician, etc.) that the pathology specimen may be malignant, even if no malignant cells are present in the pathology image.

[0080] Furthermore, the output unit 14 may output a classification result that includes a message containing the relevant feature (tissue structural atypia) from among one or more features that suggest the presence of malignant cells in the pathological image, as shown in message msg1 and message msg2 in Figure 7.

[0081] With this configuration, the pathology diagnostic support device 1A can notify the user of the characteristics that led it to determine that the pathology specimen may be malignant.

[0082] Furthermore, the output unit 14 may indicate areas corresponding to at least one of the one or more features suggesting the presence of malignant cells with rectangles on the pathological image (Patch), as shown in Figure 7, and output the classification result including the pathological image. In addition, the output unit 14 may include a message msg1 in the classification result indicating that areas corresponding to at least one of the one or more features suggesting the presence of malignant cells have been indicated with rectangles on the pathological image, as shown in Figure 7.

[0083] With this configuration, the pathology diagnostic support device 1A can explicitly notify the user of which part of the pathology image contains at least one of one or more features that suggest the presence of malignant cells.

[0084] (Example 2 of an image output by the output unit 14) Another example of an image output by the output unit 14 in step S15 will be explained with reference to Figure 8. Figure 8 is a diagram showing another example of an image output by the output unit 14.

[0085] As described above, the output unit 14 may output a classification result that includes a message indicating that the higher the probability that each of the features in the input image corresponds to one or more features that suggest the presence of malignant cells, the higher the probability that malignant cells are present near the cells in the pathological image.

[0086] As an example, the output unit 14 will output a classification result that includes a message indicating that there is a high probability that malignant cells are present near the cells included in the pathological image, if the probability is 0.8 or higher.

[0087] In this configuration, we assume that the second classification model CM2 outputs the following probabilities for each class: - No findings: 0.29 - Tissue structure dysplasia: 0.40 - Irregular arrangement of cells: 0.84 - Abundant microvessels: 0.42 - Severe inflammation: 0.22 - Presence of Helicobacter pylori: 0.08 In this case, since the probability of "irregular arrangement of cells" is 0.8 or higher, the output unit 14 outputs a classification result that includes a message msg3 indicating that there is a high possibility that malignant cells are present near the cells included in the pathological image, as shown in Figure 8.

[0088] Furthermore, in this configuration, the output unit 14 may output classification results that include progressively different messages depending on the probability. For example, if the probability is 0.8 or higher, the output unit 14 outputs a classification result that includes the message, "There is a very high probability that malignant cells are nearby." If the probability is less than 0.8 and 0.6 or higher, the output unit 14 outputs a classification result that includes the message, "There is a high probability that malignant cells are nearby." If the probability is less than 0.6 and 0.4 or higher, the output unit 14 outputs a classification result that includes the message, "There may be malignant cells nearby."

[0089] Another example of a message indicating a high probability of malignant cells being present near cells in a pathology image is a message indicating a location where malignant cells are likely to be present. For example, if the probability is 0.8 or higher, the output unit 14 outputs a classification result that includes the message, "There is a possibility that malignant cells are present 2 mm below the slide being diagnosed." If the probability is less than 0.8 and 0.6 or higher, the output unit 14 outputs a classification result that includes the message, "There is a possibility that malignant cells are present 4 mm below the slide being diagnosed." If the probability is less than 0.6 and 0.4 or higher, the output unit 14 outputs a classification result that includes the message, "There is a possibility that malignant cells are present 6 mm below the slide being diagnosed."

[0090] With this configuration, the pathology diagnostic support device 1A can present the user with a message that more clearly indicates the likelihood of malignant cells being present, based on the probability output by the second classification model CM2.

[0091] (Effects of the pathology diagnostic support device 1A) As described above, in the pathology diagnostic support device 1A, even if the first classification model CM1 determines that no malignant cells are present in the pathology image, the device determines whether or not the pathology image contains features that suggest the presence of malignant cells. Furthermore, in the pathology diagnostic support device 1A, if the pathology image contains features that suggest the presence of malignant cells, the device classifies the pathology image according to the features.

[0092] Therefore, the pathology diagnostic support device 1A can notify the user that there is a possibility that malignant cells are present in the pathology specimen, even if the sliced ​​pathology specimen contains malignant cells, but the sliced ​​area does not contain malignant cells, and the pathology image of the sliced ​​pathology specimen does not contain malignant cells. Accordingly, the pathology diagnostic support device 1A can suitably support pathology diagnosis.

[0093] (Modification 1) The pathology diagnostic support device 1A may be configured to classify pathology images into classes by inputting the pathology images acquired by the acquisition unit 11 into a second classification model CM2 via the classification unit 13. In other words, the pathology diagnostic support device 1A may be configured to input pathology images into the second classification model CM2 without classifying whether the cells included as subjects in the pathology images are malignant or not.

[0094] Even in this configuration, the pathology diagnostic support device 1A can notify the user that the pathology specimen may contain malignant cells. Therefore, the pathology diagnostic support device 1A can suitably support pathology diagnosis.

[0095] [Third Exemplary Embodiment] A third exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same function as those described in the above-described exemplary embodiment are denoted by the same reference numerals, and their descriptions are omitted as appropriate. The scope of application of each technology adopted in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology adopted in this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems arise. Furthermore, each technology shown in the drawings referenced to describe this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems arise.

[0096] (Overview of Pathology Diagnostic Support Device 1B) Similar to the Pathology Diagnostic Support Device 1A described above, the Pathology Diagnostic Support Device 1B is a device that determines whether or not cells included as subjects in the pathology image of an arbitrary slice are malignant. Furthermore, if the Pathology Diagnostic Support Device 1B determines that malignant cells are not included in the pathology image of an arbitrary slice, it determines whether or not the pathology image of the arbitrary slice contains one or more features that suggest the presence of malignant cells in neighboring slices. The Pathology Diagnostic Support Device 1B also outputs a classification result indicating whether or not the pathology image of the arbitrary slice contains at least one of the one or more features that suggest the presence of malignant cells in neighboring slices, and, if the pathology image of the arbitrary slice contains at least one of the one or more features that suggest the presence of malignant cells in neighboring slices, which feature it contains.

[0097] (Configuration of Pathology Diagnostic Support Device 1B) The configuration of the pathology diagnostic support device 1B will be explained with reference to Figure 9. Figure 9 is a block diagram showing the configuration of the pathology diagnostic support device 1B. As shown in Figure 9, the pathology diagnostic support device 1B includes a control unit 10B, a storage unit 20B, an input / output unit 30, and a communication unit 40. The input / output unit 30 and the communication unit 40 are as described above, so their explanation will be omitted.

[0098] (Storage Unit 20B) The storage unit 20B stores the data that the control unit 10B references. Examples of the storage unit 20B include, but are not limited to, flash memory, HDD, SSD, or a combination thereof.

[0099] An example of data stored in the memory unit 20B is the first classification model CM1 described above.

[0100] Another example of data stored in the memory unit 20B is the second classification model CM2B. Similar to the second classification model CM2 described above, the second classification model CM2B is a machine learning model that classifies an input image of an arbitrary slice into one of several classes that contain one or more features suggesting the presence of malignant cells.

[0101] Furthermore, the second classification model CM2B, as shown in Figure 9, comprises at least one of the following: a gate network GN that has been trained to output the probability of belonging to each of several classes containing one or more features, and one or more expert networks EN that have been trained to output the probability of belonging to each of the one or more features.

[0102] For example, the gate network GN is machine-trained to output the probability of belonging to each of several classes containing one or more features, based on the following training data: • Training data containing images with "no findings" and label "0" • Training data containing images with the feature "abnormal tissue structure" and label "1" • Training data containing images with the feature "irregular arrangement of cells" and label "2" • Training data containing images with the feature "many microvessels" and label "3" • Training data containing images with the feature "severe inflammation" and label "4" • Training data containing images with the feature "presence of Helicobacter pylori" and label "5" The expert network EN includes the first expert network EN1 to the sixth expert network EN6, as shown in Figure 9. Each of the first expert network EN1 to the sixth expert network EN6 outputs the agreement rate between the features contained in the input image and a predetermined feature (in other words, the probability that the input image contains a predetermined feature). As an example, the first expert network EN1 to the sixth expert network EN6 each output the probability that the input image contains the following features: • First expert network EN1: Probability of containing "no findings" (probability of not containing any features) • Second expert network EN2: Probability of containing "abnormal tissue structure" • Third expert network EN3: Probability of containing "irregular arrangement of cells" • Fourth expert network EN4: Probability of containing "many microvessels" • Fifth expert network EN5: Probability of containing "severe inflammation" • Sixth expert network EN6: Probability of containing "presence of Helicobacter pylori" Details of expert network EN will be explained with reference to Figure 10. Figure 10 shows an example of the processing performed by expert network EN.

[0103] As shown in Figure 10, the expert network EN is configured to include a feature analyzer (Encoder) that takes an image as input and outputs a feature vector representing the features contained in the image.

[0104] During machine learning of the expert network EN, as shown in the upper part of Figure 10, a set of images containing predetermined features is input to the feature analyzer, and the expert network EN is trained to learn. For example, in the case of the first expert network EN1, a set of images corresponding to "no findings" is input to the feature analyzer, and the feature analyzer outputs a feature vector. In the case of the second expert network EN2, a set of images containing "tissue structure heteromorphism" is input to the feature analyzer, and the feature analyzer outputs a feature vector.

[0105] Furthermore, during inference by the expert network EN, pathological images are input to the feature analyzer, as shown in the lower part of Figure 10. The expert network EN then outputs the probability that the pathological image contains a predetermined feature, based on the feature vector output from the feature analyzer.

[0106] (Control Unit 10B) The control unit 10B controls each component of the pathology diagnostic support device 1B, similar to the control unit 10 described above. The control unit 10B also includes an acquisition unit 11, a determination unit 12, a classification unit 13, and an output unit 14, as shown in Figure 9. In this exemplary embodiment, the acquisition unit 11, determination unit 12, classification unit 13, and output unit 14 implement the acquisition means, determination means, classification means, and output means, respectively. The acquisition unit 11, determination unit 12, and output unit 14 are as described above.

[0107] (Classification Unit 13) The classification unit 13 classifies pathological images into classes by inputting pathological images that have been determined by the determination unit 12 to not contain malignant cells into the second classification model CM2B.

[0108] For example, the processing of the classification unit 13 when the second classification model CM2B outputs the following will be described.

[0109] Output of the gate network GN: Probability of class "0" (no findings) 0.14 Probability of class "1" (tissue structural atypia) 0.50 Probability of class "2" (irregular arrangement of cells) 0.10 Probability of class "3" (many microvessels) 0.20 Probability of class "4" (severe inflammation) 0.05 Probability of class "5" (presence of Helicobacter pylori) 0.01 Output of the expert network EN: Probability of 1st expert network EN1: "no findings" 0.29 Probability of 2nd expert network EN2: "tissue structural atypia" included 0.84 Probability of 3rd expert network EN3: "irregular arrangement of cells" included 0.40 Probability of 4th expert network EN4: "many microvessels" included 0.42 Probability of 5th expert network EN5: "severe inflammation" included 0.22 Probability of 6th expert network EN6: "presence of Helicobacter pylori" included 0.08 As an example, the classification unit 13 calculates the product of the output of the gate network GN and the output of the expert network EN. That is, the classification unit 13 calculates the following values: - Probability of "no findings": 0.14 × 0.29 = 0.04 - Probability of including "tissue structural atypia": 0.50 × 0.84 = 0.42 - Probability of including "irregular arrangement of cells": 0.10 × 0.40 = 0.04 - Probability of including "many microvessels": 0.20 × 0.42 = 0.08 - Probability of including "severe inflammation": 0.05 × 0.22 = 0.01 - Probability of including "presence of Helicobacter pylori": 0.01 × 0.08 = 0.00 Then, the classification unit 13 classifies the pathological image into "tissue structural atypia," which has the highest probability. The classification unit 13 may also classify it into a class with a probability higher than a predetermined value. In this case, if there are multiple classes with a probability higher than a predetermined value, the classification unit 13 classifies the pathological images into multiple classes.

[0110] As another example, the classification unit 13 may calculate the sum of the output of the gate network GN and the output of the expert network EN, and classify the result into the class with the highest probability. In this configuration as well, the classification unit 13 may classify the result into a class with a probability greater than a predetermined value.

[0111] As yet another example, the classification unit 13 may set weight coefficients for each output of the expert network EN, calculate a weighted sum of the outputs of the gate network GN and the expert network EN, and classify the data into the class with the largest probability of the calculated weighted sum. In this configuration as well, the classification unit 13 may classify the data into a class with a probability greater than a predetermined value.

[0112] As mentioned above, the second classification model CM2B only needs to include at least one of a gate network GN and one or more expert networks EN. For example, if the second classification model CM2B includes only a gate network GN, the classification unit 13 refers to the probabilities output from the gate network GN and classifies the pathological images into classes in the same manner as described above. On the other hand, if the second classification model CM2B includes only multiple expert networks EN, the classification unit 13 refers to the probabilities output from each of the multiple expert networks EN and classifies the pathological images into classes in the same manner as described above.

[0113] (Effects of the pathology diagnostic support device 1B) As described above, the pathology diagnostic support device 1B classifies pathological images by feature based on the probabilities output from the second classification model CM2B. When the second classification model CM2B includes the expert network EN, the accuracy of the output of the second classification model CM2B is increased. Therefore, the pathology diagnostic support device 1B can improve the accuracy of determining whether or not malignant cells are present in a pathological specimen.

[0114] (Modification 2) The pathology diagnostic support device 1B may also be configured to classify pathology images into classes by inputting the pathology images acquired by the acquisition unit 11 into the second classification model CM2B via the classification unit 13. In other words, the pathology diagnostic support device 1B may also be configured to input pathology images into the second classification model CM2B without classifying whether the cells included as subjects in the pathology images are malignant or not.

[0115] Even in this configuration, the pathology diagnostic support device 1B can improve the accuracy of determining whether or not malignant cells are present in the pathology specimen.

[0116] [Example of implementation by software] Some or all of the functions of pathology diagnostic support devices 1, 1A, 1B, and 2 (hereinafter also referred to as "each of the above devices") may be implemented by hardware such as integrated circuits (IC chips) or by software.

[0117] In the latter case, each of the above devices is implemented, for example, by a computer that executes instructions for a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 11. Figure 11 is a block diagram showing the hardware configuration of computer C, which functions as each of the above devices.

[0118] Computer C comprises at least one processor C1 and at least one memory C2. Memory C2 stores a program P for operating Computer C as each of the above-mentioned devices. In Computer C, the processor C1 reads and executes the program P from memory C2, thereby realizing each of the above-mentioned devices.

[0119] For processor C1, for example, a CPU (Central Processing Unit), GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Point Number Processing Unit), PPU (Physics Processing Unit), TPU (Tensor Processing Unit), quantum processor, microcontroller, or a combination thereof can be used. For memory C2, for example, flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), or a combination thereof can be used.

[0120] Furthermore, computer C may also be equipped with RAM (Random Access Memory) for loading program P at runtime and for temporarily storing various data. Computer C may also be equipped with a communication interface for sending and receiving data with other devices. Furthermore, computer C may also be equipped with an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0121] Furthermore, program P can be recorded on a non-temporary, tangible recording medium M that is readable by computer C. Such recording medium M can include, for example, tape, disk, card, semiconductor memory, or programmable logic circuitry. Computer C can acquire program P via such recording medium M. Program P can also be transmitted via a transmission medium. Such transmission mediums can include, for example, a communication network or broadcast waves. Computer C can also acquire program P via such transmission medium.

[0122] Furthermore, each of the above functions of each of the above devices may be implemented by a single processor in a single computer, by multiple processors in a single computer working together, or by multiple processors in each of multiple computers working together. In addition, the programs for implementing each of the above functions in each of the above devices may be stored in a single memory in a single computer, distributed and stored in multiple memories in a single computer, or distributed and stored in multiple memories in each of multiple computers.

[0123] [Addendum A] This disclosure includes the technologies described in the following addendums. However, the present invention is not limited to the technologies described in the following addendums, and various modifications are possible within the scope of the claims.

[0124] (Note A1) A pathology diagnostic support device comprising: an acquisition means for acquiring a pathology image of an arbitrary slice containing cells as a subject; a determination means for determining whether or not malignant cells are included in the pathology image of an arbitrary slice by inputting the pathology image of the arbitrary slice into a first classification model that has been trained to classify whether or not the cells included as a subject in the input image are malignant; a classification means for classifying the pathology image into each of the classes, if the determination means determines that malignant cells are not included in the pathology image of the arbitrary slice, by inputting the pathology image determined by the determination means to one or more features included in the input image that suggest the presence of malignant cells in a neighboring slice; and an output means for outputting the classification results from the classification means.

[0125] (Appendix A2) The pathology diagnostic support device according to Appendix A1, wherein the second classification model comprises at least one of: a gate network trained to output the probability of belonging to each of a plurality of classes including the one or more of the features; and one or more expert networks trained to output the probability of belonging to each of the one or more of the features.

[0126] (Appendix A3) The pathology diagnostic support device according to Appendix A1 or A2, wherein, if the pathology image of the arbitrary slice is classified by the classification means into at least one of the one or more characteristics, the output means outputs a classification result including a message indicating that malignant cells may be present around the cells included as subjects in the pathology image of the arbitrary slice.

[0127] (Appendix A4) If the pathological image of the arbitrary slice is classified by the classification means into at least one of the one or more features, the output means outputs a classification result including a message indicating which of the one or more features the pathological image of the arbitrary slice corresponds to, as described in any one of Appendix A1 to A3.

[0128] (Note A5) If the pathological image of the arbitrary slice is classified into at least one of the one or more features by the classification means, the output means indicates the location corresponding to at least one of the one or more features on the pathological image of the arbitrary slice, and then outputs the classification result including the pathological image of the arbitrary slice, as described in any one of Notes A1 to A4.

[0129] (Note A6) The pathology diagnostic support device according to any one of Notes A1 to A5, wherein the second classification model outputs the probability of belonging to each of the plurality of classes, and the output means outputs a classification result that includes a message indicating that the higher the probability, the higher the likelihood that malignant cells are present near the cells included in the pathology image of the arbitrary slice.

[0130] (Note A7) The pathology diagnostic support device according to any one of Notes A1 to A6, wherein the output means refers to other diagnostic results associated with the pathology image of the arbitrary slice, and when the classification means classifies the pathology image of the arbitrary slice into a class that does not fall under any of the one or more of the above characteristics, and the other diagnostic results indicate that the pathology image of the arbitrary slice does not contain malignant cells, it outputs a classification result indicating that the cells contained in the pathology image of the arbitrary slice are benign.

[0131] (Appendix A8) A pathology diagnostic support device comprising: an acquisition means for acquiring a pathology image of an arbitrary slice containing cells as the subject; and a classification means for classifying the pathology image of the arbitrary slice by inputting the pathology image of the arbitrary slice into a classification model that has been trained to classify the image into one or more classes that include one or more features in the input image that suggest the presence of malignant cells in a neighboring slice.

[0132] [Addendum B] This disclosure includes the technologies described in the following addendums. However, the present invention is not limited to the technologies described in the following addendums, and various modifications are possible within the scope of the claims.

[0133] (Note B1) A pathology diagnostic support method comprising: an acquisition process in which at least one processor acquires a pathology image of an arbitrary slice containing cells as a subject; a determination process in which the at least one processor inputs the pathology image of the arbitrary slice to a first classification model that has been trained to classify whether or not the cells included as subjects in the input image are malignant, thereby determining whether or not malignant cells are included in the pathology image of the arbitrary slice; if the determination process determines that malignant cells are not included in the pathology image of the arbitrary slice, a classification process in which, if the determination process determines that malignant cells are not included, the at least one processor inputs the pathology image that was determined not to contain malignant cells in the determination process to a second classification model that has been trained to classify the input image into one or more classes that include one or more features in the input image that suggest the presence of malignant cells in neighboring slices, thereby classifying the pathology image according to the class; and an output process in which the at least one processor outputs the classification result obtained by the classification process.

[0134] (Appendix B2) The pathology diagnostic support method according to Appendix B1, wherein the second classification model includes at least one of the following: a gate network trained to output the probability of belonging to each of a plurality of classes including the one or more of the features; and one or more expert networks trained to output the probability of belonging to each of the one or more of the features.

[0135] (Appendix B3) The pathology diagnostic support method according to Appendix B1 or B2, wherein, in the classification process, if the pathology image of the arbitrary slice is classified into at least one of the one or more features, at least one processor outputs a classification result in the output process that includes a message indicating that malignant cells may be present around the cells included as subjects in the pathology image of the arbitrary slice.

[0136] (Appendix B4) The pathology diagnostic support method according to any one of Appendix B1 to B3, wherein in the classification process, if the pathology image of the arbitrary slice is classified into at least one of the one or more features, at least one processor outputs a classification result in the output process that includes a message indicating which of the one or more features the pathology image of the arbitrary slice corresponds to.

[0137] (Appendix B5) In the classification process, if the pathological image of the arbitrary slice is classified into at least one of the one or more features, the pathological diagnostic support method according to any one of Appendices B1 to B4, wherein the at least one processor, in the output process, indicates the location corresponding to at least one of the one or more features on the pathological image of the arbitrary slice, and then outputs the classification result including the pathological image of the arbitrary slice.

[0138] (Appendix B6) The pathology diagnostic support method according to any one of Appendix B1 to B5, wherein the second classification model outputs the probability of belonging to each of the plurality of classes, and in the output processing, at least one processor outputs a classification result that includes a message indicating that the higher the probability, the more likely it is that malignant cells are present near the cells included in the pathology image of the arbitrary slice.

[0139] (Note B7) In the output processing, the at least one processor refers to other diagnostic results associated with the pathological image of the arbitrary slice, and if the classification processing classifies the pathological image of the arbitrary slice into a class that does not fall under any of the one or more of the above characteristics, and the other diagnostic results indicate that the pathological image of the arbitrary slice does not contain malignant cells, then outputs a classification result indicating that the cells contained in the pathological image of the arbitrary slice are benign, as described in any one of Notes B1 to B6.

[0140] (Appendix B8) A pathology diagnostic support method comprising: an acquisition process in which at least one processor acquires a pathology image of an arbitrary slice containing cells as the subject; and a classification process in which the at least one processor inputs the pathology image of the arbitrary slice into a classification model that has been trained to classify the image into one of a plurality of classes, each class containing one or more features in the input image that suggest the presence of malignant cells in a neighboring slice.

[0141] [Addendum C] This disclosure includes the technologies described in the following addendums. However, the present invention is not limited to the technologies described in the following addendums, and various modifications are possible within the scope of the claims.

[0142] (Note C1) A program for causing a computer to function as a pathology diagnostic support device, wherein the computer is configured to function as: an acquisition means for acquiring a pathology image of an arbitrary slice containing cells as a subject; a determination means for determining whether or not malignant cells are included in the pathology image of an arbitrary slice by inputting the pathology image of the arbitrary slice into a first classification model that has been trained to classify whether or not the cells included as a subject in the input image are malignant; a classification means for classifying the pathology image into each of the classes, by inputting the pathology image determined by the determination means to be free of malignant cells, if the determination means determines that malignant cells are not included in the pathology image of the arbitrary slice; and an output means for outputting the classification results from the classification means.

[0143] (Appendix C2) The pathology diagnostic support program according to Appendix C1, wherein the second classification model comprises at least one of: a gate network trained to output the probability of belonging to each of a plurality of classes including the one or more of the features; and one or more expert networks trained to output the probability of belonging to each of the one or more of the features.

[0144] (Appendix C3) The pathology diagnostic support program according to Appendix C1 or C2, wherein, if the pathology image of the arbitrary slice is classified by the classification means into at least one of the one or more characteristics, the output means outputs a classification result including a message indicating that malignant cells may be present around the cells included as subjects in the pathology image of the arbitrary slice.

[0145] (Note C4) The pathology diagnostic support program according to any one of Notes C1 to C3, wherein, if the pathology image of the arbitrary slice is classified by the classification means into at least one of the one or more features, the output means outputs a classification result including a message indicating which of the one or more features the pathology image of the arbitrary slice corresponds to.

[0146] (Note C5) If the pathological image of the arbitrary slice is classified into at least one of the one or more features by the classification means, the output means indicates the location corresponding to at least one of the one or more features on the pathological image of the arbitrary slice, and then outputs the classification result including the pathological image of the arbitrary slice, as described in any one of Notes C1 to C4.

[0147] (Note C6) The pathology diagnostic support program according to any one of Notes C1 to C5, wherein the second classification model outputs the probability of belonging to each of the plurality of classes, and the output means outputs a classification result that includes a message indicating that the higher the probability, the higher the likelihood that malignant cells are present near the cells included in the pathology image of the arbitrary slice.

[0148] (Note C7) The pathology diagnostic support program according to any one of Notes C1 to C6, wherein the output means refers to other diagnostic results associated with the pathology image of the arbitrary slice, and if the classification means classifies the pathology image of the arbitrary slice into a class that does not fall under any of the one or more of the above characteristics, and the other diagnostic results indicate that the pathology image of the arbitrary slice does not contain malignant cells, then outputs a classification result indicating that the cells contained in the pathology image of the arbitrary slice are benign.

[0149] (Appendix C8) A program that causes a computer to function as a pathology diagnostic support device, wherein the computer is configured to function as: an acquisition means for acquiring pathology images of arbitrary slices containing cells as subjects; and a classification means for classifying pathology images according to the class by inputting the pathology images of the arbitrary slices into a classification model that has been trained to classify the images into one or more classes of features contained in the input images, which include one or more features that suggest the presence of malignant cells in neighboring slices.

[0150] [Addendum D] This disclosure includes the technologies described in the following addendums. However, the present invention is not limited to the technologies described in the following addendums, and various modifications are possible within the scope of the claims.

[0151] (Note D1) A pathology diagnostic support device comprising at least one processor, the at least one processor performing: an acquisition process to acquire a pathology image of an arbitrary slice containing cells as a subject; a determination process to determine whether or not malignant cells are included in the pathology image of an arbitrary slice by inputting the pathology image of the arbitrary slice to a first classification model that has been trained to classify whether or not the cells included as a subject in the input image are malignant; a classification process to classify the pathology image determined by the determination process to not contain malignant cells, if the determination process determines that malignant cells are not included in the pathology image of the arbitrary slice, by inputting the pathology image determined by the determination process to one or more features included in the input image that suggest the presence of malignant cells in a neighboring slice; and an output process to output the classification results obtained by the classification process.

[0152] Furthermore, the pathology diagnostic support device may also include memory. The memory may also store a program for causing at least one processor to perform each of the aforementioned processes.

[0153] (Appendix D2) The pathology diagnostic support device according to Appendix D1, wherein the second classification model comprises at least one of: a gate network trained to output the probability of belonging to each of a plurality of classes including the one or more of the features; and one or more expert networks trained to output the probability of belonging to each of the one or more of the features.

[0154] (Note D3) In the classification process, if the pathological image of the arbitrary slice is classified into at least one of the one or more features, in the output process, at least one processor outputs a classification result that includes a message indicating that malignant cells may be present around the cells included as subjects in the pathological image of the arbitrary slice, according to Note D1 or D2.

[0155] (Note D4) In the classification process, if the pathological image of the arbitrary slice is classified into at least one of the one or more features, in the output process, at least one processor outputs a classification result including a message indicating which of the one or more features the pathological image of the arbitrary slice corresponds to, according to any one of Notes D1 to D3.

[0156] (Note D5) In the classification process, if the pathological image of the arbitrary slice is classified into at least one of the one or more features, the output process is as follows: the at least one processor indicates the location corresponding to at least one of the one or more features on the pathological image of the arbitrary slice, and then outputs the classification result including the pathological image of the arbitrary slice. The pathological diagnostic support device according to any one of Notes D1 to D4.

[0157] (Note D6) The pathology diagnostic support device according to any one of Notes D1 to D5, wherein the second classification model outputs the probability of belonging to each of the plurality of classes, and in the output processing, at least one processor outputs a classification result that includes a message indicating that the higher the probability, the more likely it is that malignant cells are present near the cells included in the pathology image of the arbitrary slice.

[0158] (Note D7) In the output processing, the at least one processor refers to other diagnostic results associated with the pathological image of the arbitrary slice, and if the classification processing classifies the pathological image of the arbitrary slice into a class that does not fall under any of the one or more of the above characteristics, and the other diagnostic results indicate that the pathological image of the arbitrary slice does not contain malignant cells, the pathological diagnostic support device according to any one of Notes D1 to D6 outputs a classification result indicating that the cells contained in the pathological image of the arbitrary slice are benign.

[0159] (Note D8) A pathology diagnostic support device comprising at least one processor, the at least one processor performing: an acquisition process to acquire a pathology image of an arbitrary slice containing cells as the subject; and a classification process to classify the pathology image of the arbitrary slice into each of several classes by inputting the pathology image of the arbitrary slice into a classification model that has been trained to classify the image into any of several classes that include one or more features contained in the input image that suggest the presence of malignant cells in a neighboring slice.

[0160] [Addendum E] This disclosure includes the technologies described in the following addendums. However, the present invention is not limited to the technologies described in the following addendums, and various modifications are possible within the scope of the claims.

[0161] (Appendix E1) A non-temporary recording medium that records a program for causing a computer to function as a pathology diagnostic support device, the program for causing the computer to perform: an acquisition process for acquiring a pathology image of an arbitrary slice containing cells as subjects; a determination process for determining whether or not malignant cells are included in the pathology image of an arbitrary slice by inputting the pathology image of the arbitrary slice into a first classification model that has been trained to classify whether or not the cells included as subjects in the input image are malignant; a classification process for classifying the pathology image into each of several classes, if the determination process determines that malignant cells are not included in the pathology image of the arbitrary slice, by inputting the pathology image determined by the determination process to be not included in the class by inputting the pathology image which has been determined by the determination process to be not included in the class by inputting one or more features included in the input image which are one or more features that suggest the presence of malignant cells in a neighboring slice; and an output process for outputting the classification results obtained by the classification process.

[0162] (Appendix E2) A non-temporary recording medium that records a program for causing a computer to function as a pathology diagnostic support device, the program which causes the computer to perform an acquisition process to acquire a pathology image of an arbitrary slice containing cells as the subject, and a classification process which classifies the pathology image of the arbitrary slice into one of several classes by inputting the pathology image of the arbitrary slice into a classification model that has been trained to classify the image into one or more classes that include one or more features in the input image that suggest the presence of malignant cells in a neighboring slice.

[0163] 1, 1A, 1B, 2 Pathology diagnostic support device 11 Acquisition unit 12 Judgment unit 13 Classification unit 14 Output unit CM1 First classification model CM2, CM2B Second classification model GN Gate network EN Expert network msg Message

Claims

1. A pathology diagnostic support device comprising: an acquisition means for acquiring a pathology image of an arbitrary slice containing cells as a subject; a determination means for determining whether or not malignant cells are included in the pathology image of an arbitrary slice by inputting the pathology image of the arbitrary slice into a first classification model that has been trained to classify whether or not the cells included as a subject in the input image are malignant; a classification means for classifying the pathology image into each of the classes, by inputting the pathology image that has been determined by the determination means to not contain malignant cells, if the determination means determines that malignant cells are not included in the pathology image of the arbitrary slice; and an output means for outputting the classification results from the classification means.

2. The pathology diagnostic support device according to claim 1, wherein the second classification model comprises at least one of: a gate network trained to output the probability of belonging to each of a plurality of classes including the one or more features; and one or more expert networks trained to output the probability of belonging to each of the one or more features.

3. The pathology diagnostic support device according to claim 1 or 2, wherein, if the classification means classifies the pathology image of the arbitrary slice into at least one of the one or more features, the output means outputs a classification result including a message indicating that malignant cells may be present around the cells included as subjects in the pathology image of the arbitrary slice.

4. If the pathological image of the arbitrary slice is classified by the classification means into at least one of the one or more features, the output means outputs a classification result including a message indicating which of the one or more features the pathological image of the arbitrary slice belongs to, according to any one of claims 1 to 3.

5. When the pathological image of the arbitrary slice is classified by the classification means into at least one of the one or more features, the output means indicates the location corresponding to at least one of the one or more features on the pathological image of the arbitrary slice, and then outputs the classification result including the pathological image of the arbitrary slice, according to any one of claims 1 to 4.

6. The pathology diagnostic support device according to any one of claims 1 to 5, wherein the second classification model outputs the probability of belonging to each of the plurality of classes, and the output means outputs a classification result that includes a message indicating that the higher the probability, the more likely it is that malignant cells are present near the cells included in the pathology image of the arbitrary slice.

7. The pathology diagnostic support device according to any one of claims 1 to 6, wherein the output means refers to other diagnostic results associated with the pathology image of the arbitrary slice, and if the classification means classifies the pathology image of the arbitrary slice into a class that does not fall under any of the one or more of the above characteristics, and the other diagnostic results indicate that the pathology image of the arbitrary slice does not contain malignant cells, the output means outputs a classification result indicating that the cells contained in the pathology image of the arbitrary slice are benign.

8. A pathology diagnostic support method comprising: an acquisition process in which at least one processor acquires a pathology image of an arbitrary slice containing cells as a subject; a determination process in which the at least one processor inputs the pathology image of the arbitrary slice to a first classification model that has been trained to classify whether or not the cells included as subjects in the input image are malignant, thereby determining whether or not malignant cells are included in the pathology image of the arbitrary slice; if the determination process determines that malignant cells are not included in the pathology image of the arbitrary slice, a classification process in which, if the determination process determines that malignant cells are not included, the at least one processor inputs the pathology image that was determined not to contain malignant cells in the determination process to a second classification model that has been trained to classify the input image into one or more classes that include one or more features in the input image that suggest the presence of malignant cells in neighboring slices, thereby classifying the pathology image according to the class; and an output process in which the at least one processor outputs the classification results obtained by the classification process.

9. A program for causing a computer to function as a pathology diagnostic support device, wherein the computer is configured to function as: an acquisition means for acquiring a pathology image of an arbitrary slice containing cells as a subject; a determination means for determining whether or not malignant cells are included in the pathology image of an arbitrary slice by inputting the pathology image of the arbitrary slice into a first classification model that has been trained to classify whether or not the cells included as a subject in the input image are malignant; a classification means for classifying the pathology image into each of the classes, if the determination means determines that malignant cells are not included in the pathology image of the arbitrary slice, by inputting the pathology image determined by the determination means to one or more features included in the input image that suggest the presence of malignant cells in a neighboring slice; and an output means for outputting the classification results from the classification means.

10. A pathology diagnostic support device comprising: an acquisition means for acquiring a pathology image of an arbitrary slice containing cells as the subject; and a classification means for classifying the pathology image of the arbitrary slice by inputting the pathology image of the arbitrary slice into a classification model that has been trained to classify the image into one or more classes, each class containing one or more features in the input image that suggest the presence of malignant cells in a neighboring slice.

11. A pathology diagnostic support method comprising: an acquisition process in which at least one processor acquires a pathology image of an arbitrary slice containing cells as the subject; and a classification process in which the at least one processor inputs the pathology image of the arbitrary slice into a classification model that has been trained to classify the image into one of a plurality of classes, each class containing one or more features in the input image that suggest the presence of malignant cells in a neighboring slice.

12. A program that causes a computer to function as a pathology diagnostic support device, wherein the computer is configured to function as: an acquisition means for acquiring pathology images of arbitrary slices containing cells as subjects; and a classification means for classifying the pathology images of the arbitrary slices by inputting the pathology images of the arbitrary slices into a classification model that has been trained to classify the images into one or more classes of features contained in the input images, each class containing one or more features that suggest the presence of malignant cells in neighboring slices.

Images