Information Processing Apparatus, Information Display Apparatus, Information Processing Method, Information Processing System, and Program

Abstract

To present information for enabling a user to determine validity about a method for treating a disease.SOLUTION: An information processing device disclosed in this specification includes estimation means for estimating a diagnostic result to a medical image obtained by photographing a subject by using a learning model obtained by learning with the medical image and the diagnostic result in the medical image as a pair, and output means for outputting evaluation values to candidates for a treatment method performed to the estimated diagnostic result and candidates for the treatment method.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The disclosure of this specification relates to an information processing apparatus, an information display apparatus, an information processing method, an information processing system, and a program.

Background Art

[0002] In recent years, due to the improvement in performance of imaging devices and the increase in the number of times of imaging, while the number of medical image data to be read has increased, the number of doctors is insufficient.

[0003] As a system for solving the above problems, a computer-aided diagnosis (CAD) system that analyzes medical images by a computer and presents information to assist doctors in reading is known.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, doctors' resources are still in short supply, and the diagnosis time for each subject is limited. Furthermore, there has been no means to solve the subject's desire to judge the validity of the treatment method presented by a doctor in view of other information.

[0006] Therefore, for a subject, it may be difficult to determine whether the treatment method of a disease presented by a doctor as a result of image diagnosis is an option suitable for the subject among the available treatment methods.

[0007] In view of the above problems, one of the purposes of the disclosure of this specification is to present information that enables a user to judge the validity of a treatment method for a disease.

[0008] Furthermore, not limited to the aforementioned objectives, the effects and benefits derived from each configuration shown in the embodiments for carrying out the invention described later, which cannot be obtained by conventional art, can also be considered as another objective of the disclosure in this specification. [Means for solving the problem]

[0009] The information processing device disclosed herein is characterized by comprising: estimation means for estimating a diagnostic result for a medical image of a subject using a learning model that has learned a pair of medical images and diagnostic results in the medical images; and output means for outputting candidate treatment methods to be applied to the estimated diagnostic result and evaluation values ​​for the candidate treatment methods. [Effects of the Invention]

[0010] According to the disclosures in this specification, information can be presented that allows users to judge the appropriateness of treatment methods for diseases. [Brief explanation of the drawing]

[0011] [Figure 1] A diagram showing an example of an information processing system according to the first embodiment. [Figure 2] This figure shows an example of the configuration of the information processing device according to the first embodiment. [Figure 3] A diagram showing an example of a flowchart of the information processing system according to the first embodiment. [Figure 4] This figure shows an example of the process for generating a learning model according to the first embodiment. [Figure 5] A diagram showing an example of the UI of an information display device according to the first embodiment. [Figure 6] A diagram showing an example of an information processing system related to Modification Example 1. [Modes for carrying out the invention]

[0012] Preferred embodiments of the information processing apparatus disclosed herein will be described in detail below with reference to the accompanying drawings. However, the components described in these embodiments are merely illustrative, and the technical scope of the information processing apparatus disclosed herein is determined by the claims and is not limited by the individual embodiments described below. Furthermore, the disclosure herein is not limited to the embodiments described below, and various modifications (including organic combinations of each embodiment) are possible based on the spirit of the disclosure herein, and these are not excluded from the scope of the disclosure herein. That is, configurations combining each of the embodiments described later and their modified examples are all included in the embodiments disclosed herein.

[0013] <First Embodiment> The information processing device according to this embodiment is characterized by presenting information that allows the user to judge the appropriateness of treatment methods derived from the diagnostic results of medical images acquired by various medical imaging devices (modalities) such as computed tomography scanners (hereinafter referred to as CT scanners).

[0014] Specifically, first, for example, a CT scanner is used to image the patient's chest, including the lesion. The CT images obtained from this imaging are then used as input data to estimate the diagnosis using a learning model. From the estimated diagnosis, similar diagnoses from the past are extracted, and the treatment methods used in the extracted diagnoses are identified. Finally, information about the identified treatment methods is presented to the user.

[0015] The medical imaging device is not limited to those listed above and may include MRI machines, 3D ultrasound machines, photoacoustic tomography machines, PET / SPECT machines, OCT machines, digital radiography machines, etc. Furthermore, the area to be imaged is not limited to those listed above and may include the brain, heart, lungs, liver, stomach, large intestine, etc.

[0016] The following explanation provides an example of using CT images of the chest as medical images for diagnosis.

[0017] FIG. 1 is a diagram showing the overall configuration of an information processing system including an information processing apparatus according to the present embodiment.

[0018] The information processing system includes a medical image capturing apparatus 101, a data server 102, an information processing apparatus 103, and an information display apparatus 104.

[0019] The medical image capturing apparatus 101 is installed in a medical institution such as a hospital, for example, and captures a subject to generate a medical image. Note that the image in the present embodiment includes not only the state displayed on the display unit but also the state stored in a database or a storage unit as image data.

[0020] The data server 102 holds and manages medical images of a subject captured by the medical image capturing apparatus 101 via a network and information associated with the medical images. For example, the medical image and the information associated with the medical image can be stored in a format compliant with the DICOM (Digital Imaging and Communication in Medicine) standard, which is an international standard that defines the format of medical images and communication procedures. However, it does not have to comply with the DICOM standard as long as the medical image and the information related to the medical image can be associated and stored. Also, the related information may be stored in a form separate from the medical image in a file or a database. In this case, the information processing apparatus 103 can access the file or the database as needed to refer to the related information. Further, the data server 102 may be a system within a hospital or a system outside the hospital.

[0021] As shown in FIG. 2, the information processing apparatus 103 can acquire medical images held by the data server 102 via a network. The information processing apparatus 103 includes a communication IF (Interface) 111, a ROM (Read Only Memory) 112, a RAM (Random Access Memory) 113, a storage unit 114, an operation unit 115, a display unit 116, and a control unit 117.

[0022] The communication interface 111 is implemented by a LAN card or the like and manages communication between an external device (e.g., a data server 102) and the information processing device 104. The ROM 112 is implemented by a non-volatile memory or the like and stores various programs. The RAM 113 is implemented by a volatile memory or the like and temporarily stores various information. The storage unit 114 is an example of a computer read storage medium and is implemented by a large-capacity information storage device such as a hard disk drive (HDD) or solid-state drive (SSD), and stores various information. The operation unit 115 is implemented by a keyboard or mouse or the like and inputs user instructions to the device. The display unit 116 is a device that displays various information generated by the control unit 117, and typically a liquid crystal display is used, but other types of displays such as plasma displays, organic EL displays, FEDs, etc. may also be used. In other words, among the functional configurations of the control unit 117, the display control means 121 causes the display unit 116 to display various information. The control unit 117 is implemented using a CPU (Central Processing Unit) or GPU (Graphical Processing Unit), and it provides overall control over each process in the information processing device 104.

[0023] The control unit 117 includes, as part of its functional configuration, an acquisition means 118, an estimation means 119, an identification means 120, a display control means 121, and a transmission means 122.

[0024] The acquisition means 118 reads and acquires medical images of the subject taken by the medical image acquisition device 101 and information associated with the medical images from the data server 102. The information associated with the medical images may include, for example, subject information such as subject ID, height, weight, age, sex, body fat, blood pressure, pregnancy status, heart rate, or body temperature, or examination information such as shooting conditions, imaging area, shooting date and time, or shooting location. The acquisition means 118 may acquire all the information associated with the medical images stored in the data server 102, or it may acquire only some of the items. Furthermore, when acquiring some items, the acquisition means 118 may automatically acquire predetermined information, or it may acquire information on items selected by the user via the operation unit 115. In addition, the data does not necessarily have to be acquired from the data server 102; for example, data transmitted directly from the medical image acquisition device 101 may be acquired. Also, the data server from which the information is acquired may differ depending on the information to be acquired. For example, the medical image and the information associated with the medical image may be acquired from different data servers.

[0025] The estimation means 119 estimates the diagnostic result from the medical image of the subject acquired by the acquisition means 118. In this embodiment, the diagnostic result is estimated from the medical image of the subject using a learning model that has undergone deep learning in advance. The learning model is constructed, for example, by performing supervised learning using a neural network with pairs of input data and labels as training data, as will be described in detail later. In this embodiment, the diagnostic result shows the identification result for items such as the presence or absence of disease, the severity (stage) of the disease, the type of disease, the presence or absence of metastasis, the location of metastasis, the location of the tumor, the size of the tumor, or the number of tumors. In this embodiment, a configuration using a learning model that estimates the severity (stage) of the disease is described as an example, but a configuration that estimates all information or a configuration that estimates any of the information is also possible. Furthermore, the learning model can iteratively perform learning based on training data including input data and labels. In addition, the learning model may be used to train other models through transfer learning or fine tuning, or further learning processing (additional learning) may be performed on the learning model. In this embodiment, the learning model used to estimate the diagnostic result may be generated by a learning means (not shown) provided by the information processing device 103, or it may be a model generated by an information processing device different from the information processing device 103. Furthermore, the specific algorithm used to generate the learning model is not limited to the above, and in addition to deep learning using a neural network, for example, a support vector machine, a Bayesian network, or a random forest may be used.

[0026] The identification means 120 identifies a treatment method for the disease based on the diagnostic result estimated by the estimation means 119.

[0027] The transmission means 121 transmits the treatment method identified by the identification means 120 to the information display device 104.

[0028] The information display device 104 is a device that displays various information transmitted from the information processing device 103, and typically uses a device such as a smartphone or tablet equipped with a liquid crystal display. However, other types of displays such as plasma displays, organic EL displays, and FEDs may also be used. Furthermore, it does not necessarily have to have a display as long as it is capable of displaying information; for example, it may be a device that uses technologies such as AR (Argumented Reality) to project information into space.

[0029] Next, the processing procedure of the information processing system 100 according to this embodiment will be explained using the flowchart in Figure 3.

[0030] (S301: Medical image capture and storage) In S301, the medical imaging device 101 captures a medical image of the subject and stores it in the data server 102 via the network along with the subject information or examination information.

[0031] (S302: Acquisition of medical images) In S302, the acquisition means 118 of the information processing device 103 acquires the medical image captured in S301 and the information associated with the medical image from the data server 102.

[0032] (S303: Estimation of diagnostic results) In S303, the estimation means 119 of the information processing device 103 estimates the diagnostic result by inputting the medical image of the subject acquired in S302 into a learning model.

[0033] Here, we will explain the method for generating the learning model used to estimate the diagnostic results using Figures 4A and 4B.

[0034] In this embodiment, the learning model is constructed by performing supervised learning using a neural network, with training data consisting of pairs of input data (medical images) and output results (diagnostic labels). The following example shows how the learning model is generated by the learning means (not shown) provided in the control unit 117, but it may also be generated by an information processing device other than the information processing device 103.

[0035] In S401, the learning means provided by the control unit 117 acquires medical images and labels to be used as training data from the data server 102. Note that the medical images and labels do not necessarily have to be acquired from the data server 102, and may be acquired from other data servers. Here, the labels in this embodiment are, for example, identification information such as the presence or absence of disease, the severity (stage) of the disease, the type of disease, the presence or absence of metastasis, the location of metastasis, the location of the tumor, the size of the tumor, or the number of tumors.

[0036] In S402, the learning method accepts the medical image and label pair acquired in S401 as training data 401.

[0037] In S403, the learning method generates a learning model by performing supervised learning using medical image and label pairs as training data 401.

[0038] The learning method involves providing pairs of input data and labels from the training data to a neural network 402 constructed by combining perceptrons, and performing forward propagation by changing the weights of each perceptron in the neural network so that the output of the neural network 402 is the same as the label. For example, in this embodiment, forward propagation is performed so that the identification information output by the neural network is the same as the identification information of the label.

[0039] Then, after performing forward propagation in this manner, the learning method adjusts the weight values ​​using a technique called backpropagation to reduce the error in the output of each perceptron. More specifically, the learning method calculates the error between the output of neural network 402 and the label, and modifies the weight values ​​to reduce the calculated error.

[0040] Here, the neural network 402 takes on a structure in which a large number of processing units 403 are arbitrarily connected. Examples of processing units 403 include convolution operations, normalization processes such as BatchNormalization, and processes using activation functions such as ReLU, Sigmoid, and Softmax, each having a set of parameters to describe the content of the process. These can be connected in layers of 3 to several hundred, for example, in sets of convolutional layers, pooling layers, fully connected layers, and output layers, to take on a structure called a convolutional neural network.

[0041] For example, in a convolutional layer, a filter with predetermined parameters is applied to the input image data in order to perform feature extraction such as edge detection. These predetermined parameters in the filter correspond to the weights of the neural network, and are learned by repeatedly performing the forward propagation and backpropagation described above.

[0042] In the pooling layer, the image output from the convolutional layer is blurred to allow for positional shifts of objects. This allows the image to be considered the same object even if its position changes. By combining these convolutional and pooling layers, features can be extracted from the image.

[0043] In a fully connected layer, image data from which feature portions have been extracted through convolutional and pooling layers is combined into a single node, and a value transformed by an activation function is output. Here, the activation function is a function that sets all output values ​​less than 0 to 0, and is used to send only the portion above a certain threshold as meaningful information to the output layer.

[0044] In the output layer, the output from the fully connected layer is converted into a probability using, for example, the softmax function, which is used for multi-class classification, and the classification information is output based on this probability. It should be noted that, similar to convolutional neural networks, forward propagation and backpropagation are repeated to minimize the error between the output and the label.

[0045] The learning method may use all the medical images acquired in S401 for training, or it may use only some of the acquired medical images for training. Furthermore, the learning method may use segmented images obtained by dividing the acquired medical images into multiple regions as input data, or it may use only some of the regions of interest extracted from the acquired medical images as input data.

[0046] Furthermore, the learning method may construct training data by associating multiple labels with a single input data. For example, training data can be constructed by associating the type of disease and the presence or absence of metastasis as labels with a single medical image, and a learning model can be generated that outputs the type of disease and the presence or absence of metastasis.

[0047] Alternatively, different learning models may be generated for each label associated with the input data. For example, a first learning model may be generated from training data in which the type of disease is used as a label for medical images, and a second learning model may be generated from training data in which the presence or absence of metastasis is used as a label for medical images.

[0048] Alternatively, multiple learning models may be generated by associating the same labels with different input data. For example, in S401, CT images and MRI images are acquired. Then, a first learning model may be generated from training data in which the severity (stage) of the disease is used as a label for the acquired CT images, and a second learning model may be generated from training data in which the severity (stage) of the disease is used as a label for the MRI images.

[0049] (S304: Identification of treatment method) In S304, the identification means 120 provided in the information processing device 103 identifies a treatment method based on the diagnostic result estimated in S303.

[0050] First, the identification means 120 extracts past diagnostic results that are highly similar to the estimation results described above. In addition to the estimation results, the similarity to at least one of the subject information and imaging information may also be calculated and diagnostic results extracted. For example, information on gender and pregnancy status may be added to the subject information to be used for similarity calculation.

[0051] Here, the similarity between the estimation result and the past diagnosis result is calculated such that the value increases as the degree of similarity between the content of the estimation result and the past diagnosis result increases. For example, the identification means 120 calculates the similarity based on the number of words that appear in common between the estimation result and the past diagnosis result. Alternatively, the identification means 120 obtains the feature vectors of the estimation result and the past diagnosis result from the words that appear in the estimation result and the past diagnosis result, and calculates the similarity as the distance between the feature vectors. Note that the method by which the identification means 120 calculates the similarity between the estimation result and the past diagnosis result may be any method. For example, in S303, the similarity between the medical image of the subject being estimated and the medical image previously taken and stored in the data server 102 may be calculated. Alternatively, for example, past diagnosis results may be classified into multiple classes, and in S303, it may be estimated which of the pre-classified classes the subject's medical image belongs to, and past diagnosis results within the estimated class may be extracted.

[0052] Furthermore, each item used to calculate similarity may be weighted. For example, if the test subject wants to know information about past patients who were the same sex (female) and pregnant at the time of the test, the weight of that item may be increased.

[0053] Then, the treatment methods applied in the extracted past diagnostic results are identified.

[0054] Here, the identification means 120 is desirable to identify multiple treatment methods, for example, because even if the diagnosis is the same (breast cancer), the treatment method to be selected will differ depending on the individual's health condition and the stage of disease progression.

[0055] More specifically, in this embodiment, the treatment methods for breast cancer are specified as total mastectomy, breast-conserving surgery, or neoadjuvant drug therapy. Note that the above treatment methods are examples only and are not limited thereto; various treatment methods may be specified depending on the suspected disease.

[0056] Furthermore, it is not always necessary to identify multiple treatment methods. For example, if the most probable treatment method is uniquely determined by the severity of the disease, then identifying only one treatment method is sufficient. In this case, it should be indicated that there are no comparable treatment methods.

[0057] (S305: Sending instructions for treatment) In S305, the transmission means 122 of the information processing device 103 transmits the treatment method identified in S304 and information regarding the treatment method via the network in response to a request from the information display device 104. The information regarding the treatment method will be described in more detail later, but for example, it may include evaluation values ​​for multiple indicators regarding the treatment method.

[0058] The transmission means 122 may initially transmit only the items that are candidates for treatment methods, and then transmit only the information of the treatment method selected by the user from among the candidate treatment methods displayed on the information display device 104. Alternatively, the transmission means 122 may be configured to transmit all the information regarding the treatment method specified in response to a request from the user.

[0059] (S306: Display of treatment method) In S306, the information display device 104 displays information regarding the treatment method transmitted from the information processing device 103.

[0060] Specifically, as shown in Figure 2, the information display device 104 includes, for example, a display unit 140 such as a liquid crystal display that displays a user interface for receiving instructions from the user, and a display control means 143 that controls the display of information on the display unit 140. The display control means 143 displays the information received by the receiving means 142 on the display unit 140, so that the user can confirm the information regarding the treatment method displayed on the display unit 140.

[0061] Figure 5 illustrates an example of a specific method for displaying information regarding treatment methods.

[0062] Display screen 501 shows a list of information regarding the treatment methods performed in past cases identified from the diagnosis estimated from the patient's medical images. For example, display screen 501 assumes that the patient's diagnosis of disease severity is "early stage," and that information has been transmitted regarding cases that were similarly diagnosed as "early stage" in the past, including cases that underwent mastectomy (95 cases), cases that underwent breast-conserving surgery (15 cases), and cases that underwent preoperative drug therapy (10 cases).

[0063] The display method shown on screen 501 is just an example and is not limited thereto. The display control means 143 may, for example, display candidate treatment methods that are likely to be chosen by the user in order, rather than displaying them in a list. The display control means 143 may also display candidate treatment methods in descending order of the number of cases in which they have been adopted, as shown on screen 501, or in ascending order. Alternatively, the display control means 143 may display candidate treatment methods in order from standard treatment. Furthermore, the display control means 143 may highlight treatment methods that have been adopted frequently. The control unit 141 may also be equipped with a search function that screens extracted past cases based on subject information, etc.

[0064] The reception means 144 of the information display device 104 accepts any selection from the user regarding the treatment method displayed on the display screen 501, and transitions the display screen according to the selection.

[0065] Display screen 502 shows an example of the display when breast-conserving surgery is selected on display screen 501. Specifically, display screen 502 displays evaluation values ​​for indicators such as high survival rate, cost, impact on fertility, impact on appearance, and low recurrence rate when breast-conserving surgery is selected, using a radar chart. Similar radar charts are displayed when other treatment methods shown on display screen 501 are selected, allowing the user to compare the evaluation values ​​shown in these radar charts for each treatment method. Note that the display format of the evaluation values ​​does not have to be a radar chart; they may be represented by various graphs or tables.

[0066] The evaluation values ​​for the indicators can be various statistical values ​​(mean, median, maximum, etc.) based on the results of evaluations in past cases. Note that different statistical values ​​may be used for each indicator, or the same statistical value may be used for all indicators.

[0067] Furthermore, in past cases, information on all indicators may not always be associated and stored. Therefore, for example, the number of cases used to calculate the evaluation value may be displayed alongside the evaluation value. Alternatively, if the sample size is small when calculating the evaluation value, it may be possible to highlight that the reliability is relatively low.

[0068] Then, when one of the displayed items is selected, supplementary information about the selected item is displayed, as shown in display screen 503. Display screen 503 shows an example where, when "Appearance" is selected from the displayed items, supplementary information comparing satisfaction with postoperative appearance to other treatment methods is displayed. Note that the supplementary information does not have to be information comparing each indicator to other treatment methods; for example, it may display testimonials from people who have previously chosen the displayed treatment method, or the results of surveys regarding satisfaction levels.

[0069] In this way, by comparing the evaluation values ​​and supplementary information of each indicator that makes up the chart for each treatment method, users can consider the treatment method that suits them best based on objective information.

[0070] Furthermore, the indicators displayed on display screen 502 do not necessarily have to be the five listed above; for example, they may include other indicators such as the magnitude of the risk of side effects and complications, or there may be fewer than five indicators. In addition, the system may accept the user's selection of indicators and display evaluation values ​​for the selected indicators, or it may display evaluation values ​​for predetermined indicators regardless of the user's selection.

[0071] The processing of the information processing system 100 is then carried out.

[0072] As described above, users can compare and examine multiple treatment methods and their evaluation values, or, even if there is only one treatment method, they can consider it based on information from past cases. This allows them to judge the appropriateness of the treatment method presented during a diagnosis with a doctor and to understand the treatment method that is right for them.

[0073] (Variation 1) In this embodiment, in S302, the acquisition means 118 of the information processing device 103 acquired the medical image of the subject and information associated with the medical image from the data server 102.

[0074] However, as shown in Figure 6, if the information display device 104 can access the data server 102 where the medical images or patient information of the subject are stored and managed, the information processing device 103 may be configured such that the acquisition means 118 acquires each piece of information transmitted from the information display device 104.

[0075] This allows users of the information display device 104 to obtain information about treatment methods without having to go to a medical institution, thus reducing constraints on location and time.

[0076] (Modification 2) In this embodiment, in S303, the estimation means 119 of the information processing device 103 is shown to estimate a diagnosis result (specifically, the severity of the disease) from a CT image of the subject's chest.

[0077] However, in S303, the estimation means 119 may correct the estimated diagnosis of the subject using the subject's subject information or the results of a detailed examination using a microscope. For example, by examining the subject's cells using a microscope, detailed information such as tumor size, presence or absence of lymph node metastasis, presence or absence of lymphatic vessel invasion, presence or absence of venous invasion, tumor type, cell proliferation capacity, malignancy, presence or absence of hormone receptors, or degree of HER2 protein expression can be obtained. Therefore, the estimation means 119 may correct the results estimated from the subject's medical images based on the results of the detailed examination. This makes it possible to estimate the subject's diagnosis more accurately.

[0078] In addition, for detailed examinations, the diagnostic result is estimated by inputting images of the subject taken with a microscope into the learning model, similar to the method described in S303.

[0079] In other words, the diagnostic result estimated by inputting a medical image captured by a first image capture device into a first learning model may be corrected using the output result obtained by inputting a medical image captured by a second image capture device into a second learning model.

[0080] (Variation 3) In this embodiment, in S306, the display control means 143 of the information display device 104 displays the evaluation values ​​of the indicators for each treatment method in a radar chart, and the user was able to compare these evaluation values ​​according to the treatment method selected.

[0081] However, the display control means 143 may display the evaluation values ​​of the indicators for each treatment method in a superimposed or parallel manner, rather than switching between them. For example, on the display screen 501, the reception means 144 receives selections of multiple treatment methods from the user, and the display control means 143 superimposes the evaluation values ​​of the indicators for the multiple treatment methods that have been accepted as a laser chart.

[0082] This allows users to compare different treatment methods without having to switch screens, thus improving visibility.

[0083] <Other Embodiments> The disclosures herein can also be realized by supplying a program that implements one or more of the functions of the embodiments described above to a system or device via a network or storage medium, and by a process in which one or more processors in the computer of that system or device read and execute the program. They can also be realized by a circuit that implements one or more of the functions.

[0084] A processor or circuit may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), or a field-programmable gateway (FPGA). It may also include a digital signal processor (DSP), a dataflow processor (DFP), or a neural processing unit (NPU).

[0085] The information processing device in each of the embodiments described above may be implemented as a single device, or as a combination of multiple devices that can communicate with each other to perform the above processing; both are included in the embodiments of the present invention. The above processing may also be performed using a common server device or group of servers. The information processing device and the multiple devices constituting the information processing system only need to be able to communicate at a predetermined communication rate, and do not need to be located in the same facility or in the same country.

[0086] Embodiments disclosed herein include a configuration in which a software program that implements the functions of the embodiments described above is supplied to a system or device, and the computer of the system or device reads and executes the code of the supplied program.

[0087] Therefore, the program code installed on the computer to implement the processing according to the embodiment is itself one of the embodiments of the present invention. Furthermore, based on the instructions contained in the program read by the computer, the operating system running on the computer may perform part or all of the actual processing, and the functions of the above-described embodiment can also be realized through that processing.

Claims

1. an estimation means for estimating a diagnosis result for a medical image of a subject using a learning model that has been trained by combining a medical image and a diagnosis result for the medical image; an output means for outputting candidate treatment methods to be applied to the estimated diagnosis results and evaluation values ​​for the candidate treatment methods; An information processing device comprising:

2. and further comprising a specifying means for specifying a treatment method performed in a past case extracted based on the similarity to the estimated diagnosis result, 2. The information processing apparatus according to claim 1, wherein the output means outputs the treatment method identified by the identification means as a candidate treatment method to be applied to the diagnosis result.

3. 3. The information processing device according to claim 1, wherein the output means outputs at least two candidate treatment methods when there are two or more candidate treatment methods, and when there is only one candidate treatment method, outputs the one candidate treatment method along with information indicating that there are no other candidate treatment methods.

4. The information processing device according to any one of claims 1 to 3, characterized in that the diagnostic result is an identification result for at least one of the following items: presence or absence of disease, severity of disease, type of disease, presence or absence of metastasis, location of metastasis, location of tumor, size of tumor, and number of tumors.

5. The information processing device according to any one of claims 1 to 4, characterized in that the learning model is constructed to include a neural network that has undergone deep learning using training data that pairs the medical image with a diagnostic result for the medical image.

6. An information processing device as described in any one of claims 1 to 5, characterized in that it further comprises a correction means for correcting the diagnostic result estimated by the estimation means by inputting a medical image of the subject taken by a first imaging device into a first learning model, using an output result obtained by inputting a medical image taken by a second imaging device into a second learning model.

7. An information processing device according to any one of claims 1 to 5, further comprising a correction means for correcting the diagnostic result estimated by the estimation means based on at least one of the diagnostic result of a medical image taken by an imaging device other than the imaging device used to take the medical image, subject information about the subject, and imaging conditions used to photograph the subject.

8. an acquisition means for acquiring candidate treatment methods to be applied to a diagnosis result estimated from a medical image of a subject and an evaluation value for the candidate treatment methods; a display control means for displaying the treatment method candidates acquired by the acquisition means and the evaluation values ​​for the treatment method candidates on a display unit; An information display device comprising:

9. the acquiring means acquires a first treatment method and an evaluation value for the first treatment method and a second treatment method and an evaluation value for the second treatment method as candidates for treatment methods to be applied to the diagnosis result; 9. The information display device according to claim 8, wherein the display control means displays the evaluation value for the first treatment method and the evaluation value for the second treatment method in parallel, superimposed, or switched displays on the display unit.

10. the acquiring means acquires a first treatment method, an evaluation value for the first treatment method, and information indicating that there are no other treatment method candidates; 9. The information display device according to claim 8, wherein the display control means displays, on a display unit, the evaluation value for the first treatment method and information indicating that there are no other treatment method candidates.

11. 11. The information display device according to claim 8, wherein the display control means further displays supplemental information related to the evaluation value.

12. an estimation means for estimating a diagnosis result for a medical image of a subject using a learning model that has been trained by combining a medical image and a diagnosis result for the medical image; an output means for outputting candidate treatment methods to be applied to the estimated diagnosis results and evaluation values ​​for the candidate treatment methods; an information processing device comprising: an acquisition means for acquiring candidate treatment methods to be applied to a diagnosis result estimated from a medical image of a subject, which are output from the information processing device, and an evaluation value for the candidate treatment methods; a display control means for displaying the treatment method candidates acquired by the acquisition means and the evaluation values ​​for the treatment method candidates on a display unit; an information display device comprising: An information processing system including:

13. The information display device further includes a transmission means for transmitting a medical image of the subject to the information processing device, 13. The information processing system according to claim 12, wherein the estimation means included in the information processing device estimates a diagnosis result for a medical image of the subject transmitted from the information display device.

14. an estimation means for estimating a diagnosis result of a breast disease from a medical image of a subject's chest using a learning model that has been trained by pairing a medical image and a diagnosis result from the medical image; an identification means for identifying a treatment method for a breast disease that was performed in a past case extracted based on the similarity with the estimated diagnosis result; a display control means for displaying the treatment method candidates identified by the identification means and the evaluation values ​​for the treatment method candidates on a display unit; An information processing system comprising:

15. The information processing system according to claim 14, characterized in that the display control means displays an evaluation value for at least one indicator of survival rate, cost, side effects, impact on appearance, impact on fertility, and low recurrence rate on the display unit.

16. an estimation step of estimating a diagnosis result from a medical image of a subject using a learning model that has been trained by pairing a medical image and a diagnosis result from the medical image; an identifying step of identifying a treatment method performed in a past case extracted based on the similarity to the estimated diagnosis result; a display control step of displaying the treatment method candidates identified by the identification means and the evaluation values ​​for the treatment method candidates on a display unit; An information processing method comprising:

17. 8. A program causing a computer to execute each of the means of the information processing apparatus according to claim 1.

Images