Computer program, information processing method, and information processing apparatus

Abstract

To provide a computer program and the like that can estimate a plurality of diseases according to symptoms and can present the basis of the estimation.SOLUTION: A computer program causes a computer to execute processing of: acquiring a plurality of symptoms; using a first learning model and a second learning model that are trained on the basis of training data to, when the plurality of symptoms are input thereto, output a plurality of diseases and the order of priority of the diseases, to acquire first output data including the plurality of diseases and the order of priority of the diseases that correspond to the acquired plurality of symptoms and are output from the first learning model, and second output data including the plurality of diseases and the order of priority of the diseases that are output from the second learning model; acquiring, from the training data, information on training data similar to the acquired plurality of symptoms on the basis of the acquired second output data; and associating and outputting the acquired first output data and the information on the training data similar to the plurality of symptoms.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to a computer program, an information processing method, and an information processing device. [Background technology]

[0002] Conventionally, in order to support doctors' diagnoses, diagnosis support systems have been proposed that perform analysis based on a patient's clinical information and present possible diseases. Doctors are expected to be able to make more accurate diagnoses by referring to the results output by the diagnosis support system. For example, the applicant has proposed a technology that can present multiple possible diseases for a patient's symptoms in candidate order (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2023-25506 A Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 makes it possible to estimate a number of diseases using a learning model. However, in a diagnosis support system using such a learning model, the processing by the diagnosis support system is a black box, so there is a drawback in that it is difficult to grasp the basis for the estimation, and improvement is desired.

[0005] An object of the present disclosure is to provide a computer program etc. that can infer multiple diseases based on symptoms and present the basis for the inference. [Means for solving the problem]

[0006] A computer program according to one embodiment of the present disclosure causes a computer to execute a process of acquiring a plurality of symptoms, and using a first learning model and a second learning model trained to output a plurality of diseases and a priority order of each disease when a plurality of symptoms are input based on training data, acquiring first output data including a plurality of diseases and a priority order of each disease output from the first learning model for the acquired plurality of symptoms, and second output data including a plurality of diseases and a priority order of each disease output from the second learning model, acquiring information regarding training data from the training data that is similar to the acquired plurality of symptoms based on the acquired second output data, and outputting the acquired first output data in association with information regarding the training data that is similar to the acquired plurality of symptoms.

[0007] An information processing method according to one embodiment of the present disclosure includes a process in which a computer acquires a plurality of symptoms, and uses a first learning model and a second learning model trained to output a plurality of diseases and a priority order of each disease when a plurality of symptoms are input based on training data, to acquire first output data including a plurality of diseases and a priority order of each disease output from the first learning model for the acquired plurality of symptoms, and second output data including a plurality of diseases and a priority order of each disease output from the second learning model, and acquires information regarding training data from the training data that is similar to the acquired plurality of symptoms based on the acquired second output data, and outputs the acquired first output data in association with information regarding the training data that is similar to the acquired plurality of symptoms.

[0008] An information processing device according to one embodiment of the present disclosure includes a control unit that executes a process of acquiring a plurality of symptoms, and using a first learning model and a second learning model that have been trained to output a plurality of diseases and a priority order of each disease when a plurality of symptoms are input based on training data, acquiring first output data including a plurality of diseases and a priority order of each disease output from the first learning model for the acquired plurality of symptoms, and second output data including a plurality of diseases and a priority order of each disease output from the second learning model, acquiring information regarding training data from the training data that is similar to the acquired plurality of symptoms based on the acquired second output data, and outputting the acquired first output data in association with information regarding the training data that is similar to the acquired plurality of symptoms. Effect of the Invention

[0009] According to the present disclosure, it is possible to deduce multiple diseases based on symptoms and provide reasons for the deduction. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram of a diagnosis support system. [Diagram 2] 1 is a block diagram showing an example of the configuration of an information processing device; [Diagram 3] FIG. 2 is a block diagram showing a configuration example of a terminal device. [Figure 4] FIG. 13 is a diagram showing an example of information stored in a training data DB. [Diagram 5] FIG. 1 is a diagram showing an example of contents of bibliographic information. [Figure 6] 1 is an explanatory diagram illustrating a method of estimation processing using a first learning model and a second learning model. FIG. [Figure 7] 1 is a flowchart illustrating an example of a processing procedure executed by the diagnosis support system. [Figure 8] 13 is a schematic diagram showing an example of a result screen displayed on a display unit of an information terminal device. FIG. [Figure 9] 10 is a flowchart illustrating an example of a processing procedure executed by an information processing apparatus according to a second embodiment. [Figure 10] FIG. 13 is a schematic diagram showing an example of a screen showing recommended information. [Figure 11] FIG. 13 is a block diagram showing an example of the configuration of an information processing device 1 according to a third embodiment. [Figure 12] FIG. 11 is an explanatory diagram showing an overview of a third learning model. [Figure 13] 13 is a flowchart illustrating an example of a processing procedure executed by an information processing apparatus according to a third embodiment. [Figure 14] FIG. 13 is a schematic diagram showing an example of a screen showing reliability. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The present disclosure will be specifically described with reference to the drawings showing the embodiments thereof.

[0012] (First embodiment) 1 is a schematic diagram of a diagnosis support system 100. The diagnosis support system 100 includes an information processing device 1 and a terminal device 2. The information processing device 1 and the terminal device 2 are communicatively connected via a network N such as the Internet. The number of terminal devices 2 may be one or three or more.

[0013] The information processing device 1 is a device capable of various information processing and information transmission and reception, such as a server computer, a personal computer, a quantum computer, etc. The terminal device 2 is an information terminal device used by a doctor, which is an example of a user, such as a personal computer, a smartphone, a tablet terminal, etc.

[0014] The information processing device 1 generates diagnostic support information according to the symptoms of the patient received from the terminal device 2 using a first learning model and a second learning model described below, and outputs the generated diagnostic support information to the terminal device 2. The diagnostic support information of this embodiment includes an ordered list of multiple possible (candidate) diseases, and information on training data showing the basis of the list.

[0015] When diagnosing a patient, doctors use their own experience and knowledge to synthesize the patient's symptoms, narrow down the possible diseases (differential diagnoses), and then proceed to determine the definitive disease. In order to properly diagnose a definitive disease, it is important to extract a wide range of possible differential diagnoses. By using this system as a disease recall tool, doctors can identify diseases that they themselves would not have been able to remember, which can lead to preventing oversights and misdiagnoses. In addition to the estimated disease results, this system provides information on training data that shows the basis for the estimation, thereby more appropriately supporting doctors' diagnoses.

[0016] 2 is a block diagram showing a configuration example of the information processing device 1. The information processing device 1 includes a control unit 11, a storage unit 12, and a communication unit 13. The information processing device 1 may be configured to perform distributed processing by using a plurality of computers, may be realized by a plurality of virtual machines provided in one server, or may be realized by using a cloud server.

[0017] The control unit 11 includes a processor using one or more central processing units (CPUs), graphics processing units (GPUs), etc. The control unit 11 executes processing by controlling each component using built-in memories such as read only memory (ROM) or random access memory (RAM), clocks, counters, etc. Each functional unit of the information processing device 1 may be realized by software, hardware, or a combination of these.

[0018] The storage unit 12 includes a non-volatile memory such as a hard disk, a flash memory, or an SSD (Solid State Drive). The storage unit 12 may be an external storage device connected to the information processing device 1. The storage unit 12 stores various computer programs and data referenced by the control unit 11. The storage unit 12 of this embodiment stores a program 1P for causing a computer to execute processing related to the generation of diagnostic assistance information, a training data DB (Data Base) 121 required for executing the program 1P, a first learning model 122, and a second learning model 123. The first learning model 122 and the second learning model 123 are machine learning models based on training data. The first learning model 122 and the second learning model 123 are expected to be used as program modules constituting artificial intelligence software.

[0019] A computer program (computer program product) including the program 1P may be provided by a non-transitory recording medium 1A on which the computer program is readably recorded. The storage unit 12 stores the computer program read from the recording medium 1A by a reading device (not shown). The recording medium 1A is, for example, a magnetic disk, an optical disk, a semiconductor memory, or the like. The program may also be downloaded from an external server connected to a communication network and stored in the storage unit 12. The program 1P may be a single computer program or may be composed of multiple computer programs, and may be executed on a single computer or multiple computers interconnected by a communication network.

[0020] The communication unit 13 includes a communication module for performing processing related to communication. The control unit 11 transmits and receives data to and from the terminal device 2 via the communication unit 13.

[0021] The configuration of the information processing device 1 is not limited to the above example, and may include, for example, an operation unit for accepting user operations, a display unit for displaying various information, and the like.

[0022] 3 is a block diagram showing an example of the configuration of the terminal device 2. The terminal device 2 is a computer, and includes a control unit 21, a storage unit 22, a communication unit 23, a display unit 24, and an operation unit 25.

[0023] The control unit 21 includes a processor using one or more CPUs, GPUs, etc. The control unit 21 executes processing by controlling each component using built-in memory such as ROM or RAM, a clock, a counter, etc. Each functional unit of the terminal device 2 may be realized by software, hardware, or a combination of these.

[0024] The storage unit 22 includes a non-volatile memory such as a hard disk, a flash memory, an SSD, etc. The storage unit 22 stores various computer programs and data referenced by the control unit 21. The storage unit 22 stores a program 2P for causing a computer to execute processing related to acquisition of diagnostic assistance information.

[0025] A computer program (computer program product) including the program 2P may be provided by a non-transitory recording medium 2A on which the computer program is readably recorded. The storage unit 22 stores the computer program read from the recording medium 2A by a reading device (not shown). The recording medium 2A is, for example, a magnetic disk, an optical disk, a semiconductor memory, or the like. The program may also be downloaded from an external server connected to a communication network and stored in the storage unit 22. The program 2P may be a single computer program or may be composed of multiple computer programs, and may be executed on a single computer or multiple computers interconnected by a communication network.

[0026] The communication unit 23 is a communication module for performing processes related to communication. The control unit 21 transmits and receives data to and from the information processing device 1 via the communication unit 23.

[0027] The display unit 24 includes a display device such as a liquid crystal display, an organic EL (Electro Luminescence) display, etc. The display unit 24 outputs various information including diagnostic assistance information according to instructions from the control unit 21.

[0028] The operation unit 25 is an interface that accepts user operations. The operation unit 25 includes, for example, a keyboard, a mouse, a touch panel device with a built-in display, a speaker, a microphone, etc. The operation unit 25 accepts operation input from the user and sends a control signal to the control unit 21 according to the operation content.

[0029] 4 is a diagram showing an example of the contents of information stored in the training data DB121. The training data DB121 is a database that stores case data including information on symptoms and diseases corresponding to a plurality of cases. A case is represented by symptoms and diseases. One case is composed of one symptom group including multiple symptoms and one disease group including multiple diseases. The information stored in the training data DB121 corresponds to training data used for learning the first learning model 122 and the second learning model 123.

[0030] The training data DB 121 stores records (instances) in which information such as symptom information related to symptoms and disease information related to diseases is linked with a case ID for identifying a case as a key.

[0031] The symptom information includes a symptom ID for identifying the symptom, and the symptom. The symptom includes, for example, the subjective symptoms, objective symptoms, physical findings, clinical test results, and image test results of the patient. The symptom column includes information representing those symptoms.

[0032] The disease information includes a disease ID for identifying a disease, a disease name, a score, and a ranking. The score is the score of a disease. The score is indicated, for example, from 0 to 100, and the higher the number, the more likely it is that the disease is the disease. Among multiple diseases associated with the same case ID, the highest score is assigned to a disease that corresponds to a confirmed disease. A disease that is assigned a score of zero indicates that it is not a symptom corresponding to a case, i.e., that it is not related to the symptoms. The ranking is the priority order of the disease. The ranking is determined by a relative evaluation based on the scores of each disease associated with the same case ID, and the disease with the highest score is ranked first and the disease with the lowest score is ranked last.

[0033] The information in the training data DB121 is preferably collected based on medical literature such as papers, specialist books, case reports, etc. For example, a predetermined text mining method is used to extract information on symptoms, disease names, confirmed diseases, differential diagnoses, etc., related to a specific case from medical literature. Training data can be generated by assigning a score to each disease based on the extracted information and taking into account the judgment of an expert, and ranking the diseases. It is preferable that training data related to one case is generated based on a description in one medical literature, since the basis of the data is clear.

[0034] The training data is associated with literature information on medical literature corresponding to the training data. Fig. 5 is a diagram showing an example of the contents of the literature information. The literature information includes information on medical literature used to generate the training data related to the cases.

[0035] The literature information includes, for example, bibliographic information of medical literature (e.g., author, title, page number of reference, URL, summary, etc.), excerpted information excerpted from medical literature, etc. The excerpted information includes, for example, excerpts of descriptions regarding confirmed diseases, differential diagnoses, symptoms and diseases that require attention, tests that are recommended to be performed, etc. Symptoms and diseases that require attention may include symptoms and diseases that should not be overlooked, symptoms and diseases that should be ruled out, etc. The literature information may be generated for each case ID.

[0036] The information processing device 1 acquires training data and literature information related to a large number of cases, and stores the acquired training data and literature information linked to the training data in the training data DB 121 or the storage unit 12. Hereinafter, for convenience of explanation, each piece of training data stored in the training data DB 121, i.e., training data corresponding to one case ID, will also be referred to as case data.

[0037] 6 is an explanatory diagram for explaining a method of estimation processing using the first learning model 122 and the second learning model 123. The first learning model 122 and the second learning model 123 receive a plurality of symptoms as input, and output a plurality of diseases corresponding to the plurality of symptoms and the priority order of each disease. In this embodiment, the first learning model 122 is used as a model for disease estimation, and the second learning model 123 is used as a model for acquiring estimation grounds.

[0038] The first learning model 122 is a neural network model generated by deep learning, and is constructed by, for example, ranking learning (LTR: learning to rank). Ranking learning is a method of learning the order of data to be output from input data. The first learning model 122 has, for example, an input layer to which a plurality of symptoms, which are input data, are input, an intermediate layer to extract features of the input data using various parameters, and an output layer to output a list of a plurality of diseases, which are output data, arranged in order of priority. The first learning model 122 ranks a plurality of diseases in descending order of score for the input symptom group, and outputs a list of a plurality of diseases arranged in order of priority. Hereinafter, the output data output from the first learning model 122 is also referred to as the first output data.

[0039] The first learning model 122 can be generated by preparing training data in which multiple symptoms are associated with multiple diseases and the scores of each disease, and using the training data to train an untrained neural network by machine learning.

[0040] The information processing device 1 performs learning using, for example, a known list-wise method. The information processing device 1 inputs a plurality of symptoms included in the training data as input data to the first learning model 122, and obtains a list of a plurality of diseases in order of priority output from the first learning model 122. The information processing device 1 optimizes various parameters using, for example, a gradient descent method so as to optimize the error between the output list of diseases and the list of diseases that is the correct answer value. Through the above-mentioned process, it is possible to construct the first learning model 122 that has been trained to be able to appropriately estimate a plurality of diseases and the priority order of each disease for a plurality of symptoms.

[0041] The second learning model 123 is constructed by the k-nearest neighbor method. In the k-nearest neighbor method, a set of case data (known data) is plotted on a vector space, and when input data (new data) is obtained, k pieces of case data are selected in order of proximity from the input data, and estimation is performed. The distance between data plotted on the vector space may be calculated as, for example, Euclidean distance or Manhattan distance. The distance between the input data and the case data corresponds to the similarity between the input data and the case data.

[0042] The second learning model 123 uses a symptom group including multiple symptoms as input data and acquires a disease group corresponding to k symptom groups present in the vicinity of the input data. The average value of the scores of each disease in each acquired disease group is calculated for each disease type to estimate the score of each disease in the disease group as output data. For example, when k=3, the average value of the three scores is calculated for each disease based on the scores of each disease in the disease group corresponding to each of the three nearest case data in the set of case data, thereby obtaining the score of each disease for the input data to be estimated. The average value may be a weighted average value taking into account the distance from the input data. The priority order of each disease is obtained by sorting the multiple diseases in descending order of the obtained scores and assigning priorities. Hereinafter, the output data output from the second learning model 123 is also referred to as second output data.

[0043] In the above-mentioned process, the information processing device 1 may determine parameters in the second learning model 123 so that the second output data output from the second learning model 123 approximates the first output data output from the first learning model 122. In the case of the second learning model 123 using the k-nearest neighbor method, the above-mentioned parameter may be the number of neighbors, i.e., the value of k.

[0044] The information processing device 1 acquires second output data from the second learning model 123 corresponding to each number of neighbors by changing the number of neighbors within a preset range (for example, k = 1 to 10). The information processing device 1 identifies the second output data that is most similar to the first output data based on a comparison between the first output data and each second output data. The method of identifying the second output data that is most similar to the first output data is not limited, but the information processing device 1, for example, evaluates the error between the first output data and each second output data using a predetermined evaluation function and identifies the second output data with the smallest error. The information processing device 1 may obtain a correlation coefficient indicating the correlation between the first output data and each second output data using a predetermined formula, and identify the second output data with the highest correlation coefficient. The information processing device 1 applies the number of neighbors from which the identified second output data was obtained as the number of neighbors of the second learning model 123. The number of neighbors of the second learning model 123 may be changed each time estimation is performed.

[0045] The information processing device 1 acquires information about case data including a symptom group similar to the input data based on the output result of the second learning model 123. In the case of the second learning model 123 using the k-nearest neighbor method, the similarity between the input data and the case data can be expressed by the distance between the input data and the case data. Therefore, the case data similar to the input data is case data existing in the vicinity of the input data, and can be particularly the k nearest case data. The nearest case data is case data that is the basis for estimating that the disease corresponding to the symptom group of the input data is the disease group of the second output data. In this embodiment, by adopting the second learning model 123 that outputs the second output data that is most similar to the first output data, the second learning model 123 can be used to identify case data that is likely to be the basis for estimating the first output data.

[0046] The information on the case data includes, for example, the case data and literature information corresponding to the case data. For example, when k=2, the information processing device 1 refers to the training data DB 121 to acquire the two nearest case data and the literature information including the bibliographic items, excerpt information, etc. of the medical literature associated with each case data.

[0047] The information processing device 1 outputs, as an estimation result, to the terminal device 2, a list showing a plurality of diseases in order of priority based on the first output data, and information on the case data based on the second output data.

[0048] As described above, the information processing device 1 of this embodiment uses the same training data DB 121 to construct two models, the first learning model 122 for disease estimation and the second learning model 123 for obtaining estimation grounds. Then, the information processing device 1 estimates diseases for the same input data using each of the first learning model 122 and the second learning model 123. A list of diseases estimated by the first learning model 122 and information serving as the basis for estimation by the second learning model 123 are acquired.

[0049] The first learning model 122 using a neural network such as ranking learning can accurately estimate multiple diseases in order of priority according to symptoms, but the recognition process of the model is a black box. The second learning model 123 using the k-nearest neighbor method may be inferior to the first learning model 122 in estimation accuracy, especially in the estimation accuracy of disease priority, but can provide the basis for estimation and is excellent in explainability. By using these two models in combination, improvement of estimation accuracy and improvement of explainability are achieved.

[0050] In the above, an example has been described in which the first learning model 122 is a model based on ranking learning, and the second learning model 123 is a model based on the k-nearest neighbor method, but the configurations of the first learning model 122 and the second learning model 123 are not limited to the above example. The first learning model 122 may be a model capable of relatively accurately estimating a plurality of diseases in order of priority for symptoms, and the second learning model 123 may be a model capable of recognizing the correlation between input / output data and case data.

[0051] 7 is a flowchart showing an example of a processing procedure executed by the diagnosis support system 100. The following processing is executed by the control unit 11 in accordance with a program 1P stored in the storage unit 12 of the information processing device 1, and is also executed by the control unit 21 in accordance with a program 2P stored in the storage unit 22 of the terminal device 2.

[0052] The control unit 21 of the terminal device 2 displays a reception screen for receiving symptoms of the patient on the display unit 24 (step S11). The control unit 21 uses the reception screen to receive a plurality of symptoms by the doctor operating the operation unit 25 (step S12). On the reception screen, for example, a plurality of symptoms set in advance are displayed in a selectable list. The doctor selects a symptom corresponding to the patient's symptom through the operation unit 25. The control unit 21 transmits the received plurality of symptoms to the information processing device 1 (step S13).

[0053] The control unit 11 of the information processing device 1 receives a plurality of symptoms (step S14). The control unit 11 inputs the acquired plurality of symptoms to the first learning model 122 (step S15). The control unit 11 acquires a plurality of diseases and the priority of each disease as first output data output from the first learning model 122 (step S16).

[0054] The control unit 11 inputs the acquired multiple symptoms to the second learning model 123 (step S17). The control unit 11 acquires multiple diseases and priorities according to the scores of each disease as second output data output from the second learning model 123 (step S18). In steps S17 and S18, the control unit 11 acquires second output data from the second learning model 123 corresponding to each of the multiple numbers of neighbors by changing the number of neighbors in the second learning model 123 using the k-nearest neighbor method within a preset range.

[0055] The control unit 11 determines the number of neighbors of the second learning model 123 so that the second output data approximates the first output data based on a comparison between the first output data by the first learning model 122 and the second output data by the second learning model 123 for each number of neighbors (step S19). The control unit 11 identifies the second output data that is most approximate to the first output data based on, for example, an evaluation value by a predetermined evaluation function between the first output data and each of the second output data corresponding to each number of neighbors or a correlation coefficient by a predetermined formula. The control unit 11 sets the number of neighbors from which the identified second output data was obtained as the number of neighbors of the second learning model 123.

[0056] The control unit 11 acquires case data similar to the input data, i.e., case data that is the basis for estimation by the second learning model 123 and literature information corresponding to the case data, based on the second output data by the second learning model 123 to which the number of neighbors determined in step S19 is applied (step S20). The control unit 11 extracts k nearest case data in the second learning model 123 based on the second output data and information stored in the training data DB 121, and extracts literature information associated with each of the extracted k nearest case data.

[0057] The control unit 11 generates a result screen showing a disease list showing a plurality of diseases and the priority order of each disease as the acquired first output data, and information on the case data based on the second output data (step S21). The control unit 11 transmits the generated result screen to the terminal device 2 (step S22).

[0058] The control unit 21 of the terminal device 2 receives the result screen (step S23). The control unit 21 displays the received result screen on the display unit 24 (step S24), and ends the series of processes. The diagnosis support system 100 may execute a loop process that returns the process to step S11, and execute an estimation process for a new symptom.

[0059] 8 is a schematic diagram showing an example of a result screen 240 displayed on the display unit 24 of the terminal device 2. The result screen 240 includes a symptom display section 241 that displays symptoms to be estimated, a disease display section 242 that displays diseases that are estimation results in order of priority, and a literature display section 243 that displays case data and literature information.

[0060] When the information processing device 1 receives a plurality of symptoms from a doctor via the terminal device 2, it displays the received symptoms on the symptom display unit 241. When the information processing device 1 acquires first output data by the first learning model 122 according to the received disease, it displays a plurality of diseases and priorities on the disease display unit 242 in a list format in which the diseases are arranged in order of priority in the first output data. Note that the information processing device 1 may extract a predetermined number (e.g., 50) of diseases in descending order of priority from the diseases output from the first learning model 122 and display them on the disease display unit 242.

[0061] The information processing device 1 also refers to the training data DB 121 to acquire the nearest case data and literature information of the case data, and displays the acquired information on the literature display unit 243. The literature display unit 243 displays, for example, a plurality of symptoms and a plurality of disease names in the nearest case data, bibliographic items of medical literature on which the case data is based, confirmed diseases extracted from medical literature, differential diagnoses, notable symptoms and findings, recommended tests, and the like.

[0062] According to the present embodiment, it is possible to rank and present multiple possible diseases according to the patient's symptoms, and also to present the basis for the estimation. By presenting the case data that is the basis for the estimation in addition to the estimation result, the basis for the estimation can be clearly understood, which makes it easier for doctors to refer to when making a diagnosis.

[0063] By combining the first learning model 122 with excellent estimation accuracy and the second learning model 123 with excellent explanatory power, it is possible to provide highly accurate estimation results and clear estimation grounds. By constructing the second learning model 123 so as to approximate the output of the first learning model 122, it is possible to increase the reliability of the estimation grounds.

[0064] By generating training data based on information obtained from medical literature, it is possible to provide information about the medical literature as the basis for estimation of the learning model. By showing medical literature as a citation source, the value of the evidence is increased compared to using past test data, etc. By presenting literature information along with the training data, more information can be obtained and user convenience is improved.

[0065] Second embodiment In the second embodiment, a configuration for outputting recommended information indicating symptoms or diseases recommended for confirmation will be described. In the following embodiments, differences from the first embodiment will be mainly described, and configurations common to the first embodiment will be denoted by the same reference numerals and detailed descriptions thereof will be omitted.

[0066] The recommendation information includes information regarding at least one of a symptom that is desirable to be identified and a disease that is desirable to be identified.

[0067] The information processing device 1 identifies recommended information based on the input data or the first output data in the first learning model 122 and the training data that serves as the estimation basis for the second learning model 123.

[0068] Specifically, the information processing device 1 identifies a symptom for which confirmation is recommended based on a comparison between a plurality of symptoms included in input data to the first learning model 122 and a plurality of symptoms included in case data similar to the input data. The case data similar to the input data may be k nearest case data based on the second output data of the second learning model 123 in which parameters are set to approximate the first learning model 122. For example, the information processing device 1 identifies a symptom that is included in a predetermined number or more of the k case data and is not included in the input data as a symptom for which confirmation is recommended. When counting the number of case data, weighting may be performed so that the weight of case data that is close to the input data is increased. Note that the information processing device 1 may identify a symptom that is included in any one of the k case data and is not included in the input data as a symptom for which confirmation is recommended.

[0069] The information processing device 1 also identifies a disease recommended for confirmation based on a comparison between a plurality of diseases included in the first output data from the first learning model 122 and a plurality of diseases included in case data similar to the input data. For example, the information processing device 1 identifies a disease that is included in a predetermined number or more of the k case data and is not included in the first output data as a disease recommended for confirmation. The disease recommended for confirmation may be a disease that is included in a predetermined number or more of the k case data and is not included in a predetermined order in the first output data, or a disease that is included in any one of the k case data and is not included in the first output data. The disease recommended for confirmation may be identified by taking into account the second output data from the second learning model 123. In this case, the information processing device 1 identifies a disease that is included in a predetermined number or more of the k case data and the second output data and is not included in the first output data as a disease recommended for confirmation.

[0070] Fig. 9 is a flowchart showing an example of a processing procedure executed by the information processing device 1 of the second embodiment. The processing of the flowchart of Fig. 9 is performed after, for example, step S20 of the first embodiment is completed.

[0071] The control unit 11 of the information processing device 1 identifies symptoms and diseases recommended for confirmation based on a comparison between the input data and the first output data in the first learning model 122 and the nearest case data corresponding to the second output data of the second learning model 123 (step S31). The control unit 11 transmits recommendation information indicating the identified symptoms and diseases recommended for confirmation to the terminal device 2 (step S32).

[0072] Fig. 10 is a schematic diagram showing an example of a screen 245 showing recommended information. The control unit 21 of the terminal device 2 displays the screen 245 shown in Fig. 10 on the display unit 24 based on the recommended information received from the information processing device 1. The screen 245 includes a recommended information display unit 246 that displays a list of symptoms and diseases recommended for confirmation. The screen 245 is displayed on the display unit 24 in response to selection of a display request button for recommended information included in the result screen 240, for example.

[0073] When the information processing device 1 receives a display request for recommended information, it identifies multiple symptoms or diseases recommended for confirmation, and displays a list of the identified symptoms or diseases in the recommended information display unit 246. The information processing device 1 may sort the case data including the identified symptoms or diseases in descending order and display them in the recommended information display unit 246. The information processing device 1 may display case data including the identified symptoms or diseases or literature information corresponding to the case data, in association with the identified symptoms or diseases. The doctor can clearly grasp the symptoms or diseases that should be confirmed through the screen 245.

[0074] According to this embodiment, it is possible to provide symptoms or diseases recommended for confirmation based on the estimation results by the first learning model 122 and the second learning model 123. It is possible to efficiently and accurately obtain recommended information using accumulated training data.

[0075] Third embodiment In the third embodiment, a configuration for outputting the reliability of the first output data will be described.

[0076] Fig. 11 is a block diagram showing a configuration example of an information processing device 1 according to the third embodiment. As shown in Fig. 11, the storage unit 12 of the information processing device 1 according to the third embodiment stores a third learning model 124 in addition to a first learning model 122 and a second learning model 123. The third learning model 124 is used as a model for estimating reliability.

[0077] 12 is an explanatory diagram showing an overview of the third learning model 124. The third learning model 124 is a machine learning model that receives as input information on the first output data output from the first learning model 122, information on the second output data output from the second learning model 123, and information on case data similar to the first output data (nearest case data), and outputs a reliability for the first output data. The reliability is expressed, for example, by a numerical value equal to or greater than zero, and a larger numerical value indicates a higher reliability.

[0078] The input data input to the third learning model 124 may include, for example, a predicted score for the first output data, a predicted score for the second output data, the number of similar case data, the distance between the similar case data and the input data, an evaluation value between the first output data and the second output data using an evaluation function, an evaluation value between the first output data and similar case data using an evaluation function, an evaluation value between the second output data and similar case data using an evaluation function, etc.

[0079] The configuration of the third learning model 124 is not particularly limited, and may be constructed using a known learning algorithm such as a neural network, a support vector machine (SVM), a Bayesian network, a random forest, a linear regression, or a multiple regression. The third learning model 124 may estimate the reliability based on the first output data, information on the second output data, and case data similar to the first output data. The third learning model 124 may derive the reliability by a rule-based method.

[0080] Fig. 13 is a flowchart showing an example of a processing procedure executed by the information processing device 1 of the third embodiment. The processing of the flowchart of Fig. 13 is performed after, for example, step S20 of the first embodiment is completed.

[0081] The control unit 11 of the information processing device 1 acquires information on the first output data output from the first learning model 122, information on the second output data output from the second learning model 123, and information on case data similar to the first output data, and inputs the acquired information to the third learning model 124 (step S41). The control unit 11 acquires a reliability for the first output data output from the third learning model 124 (step S42). The control unit 11 transmits the acquired reliability for the first output data to the terminal device 2 (step S43).

[0082] Fig. 14 is a schematic diagram showing an example of a screen showing the reliability. The control unit 21 of the terminal device 2 displays the screen shown in Fig. 14 on the display unit 24 based on the recommended information received from the information processing device 1. In the example shown in Fig. 14, the reliability is displayed through the disease display section 242 of the result screen 240. The information processing device 1 derives the reliability for the first output data, and displays the derived reliability value in a predetermined area of ​​the disease display section 242. A doctor can clearly grasp the reliability together with the estimated disease through the result screen 240.

[0083] According to this embodiment, the reliability of the first output data can be further output. By using the third learning model 124 that estimates the reliability by considering the first output data, information on the second output data, and case data similar to the first output data, it is possible to take into account various parameters and improve the certainty of the reliability compared to the case where the reliability of the prediction is obtained only from the first learning model 122 or the second learning model 123.

[0084] The following supplementary notes are further disclosed regarding the above embodiment. (Appendix 1) Acquiring multiple symptoms Using a first learning model and a second learning model trained to output a plurality of diseases and a priority order of each disease when a plurality of symptoms are input based on training data, first output data including a plurality of diseases and a priority order of each disease output from the first learning model for the acquired plurality of symptoms, and second output data including a plurality of diseases and a priority order of each disease output from the second learning model are acquired; Acquire information about training data similar to the acquired plurality of symptoms among the training data based on the acquired second output data; The acquired first output data is output in association with information on training data similar to the plurality of symptoms. A computer program that causes a computer to carry out processing. (Appendix 2) The first learning model and the second learning model are learned based on the training data in which a plurality of symptoms are associated with a plurality of diseases and a priority order of each disease based on medical literature; Outputting information about medical literature corresponding to training data similar to the plurality of symptoms. 2. The computer program of claim 1. (Appendix 3) determining parameters in the second learning model such that the second output data output from the second learning model approximates the first output data output from the first learning model; 10. A computer program according to claim 1 or 2. (Appendix 4) The second learning model is a k-nearest neighbor model. 4. A computer program according to any one of claims 1 to 3. (Appendix 5) Acquire information about training data in the vicinity of the plurality of symptoms as training data similar to the plurality of symptoms. 5. A computer program according to any one of claims 1 to 4. (Appendix 6) The information regarding the medical literature includes at least one of the following: a name of the medical literature, an outline of the medical literature, a definite disease indicated by the medical literature, a differential diagnosis, a disease to be noted, a symptom to be noted, and a recommended test. 6. A computer program according to any one of claims 2 to 5. (Appendix 7) Identifying a symptom or disease recommended for confirmation based on the plurality of symptoms input to the first learning model or the first output data output from the first learning model and training data similar to the plurality of symptoms. 7. A computer program according to any one of claims 1 to 6. (Appendix 8) Identifying a symptom other than the symptom input to the first learning model among a plurality of symptoms in the training data similar to the plurality of symptoms as a symptom for which confirmation is recommended, or identifying a disease other than the disease in the first output data output from the first learning model among a plurality of diseases in the training data similar to the plurality of symptoms as a disease for which confirmation is recommended. 8. The computer program of claim 7. (Appendix 9) Deriving a reliability for the first output data based on the first output data, the second output data, and training data similar to the plurality of symptoms. 9. A computer program according to any one of claims 1 to 8. (Appendix 10) A third learning model is trained to output a reliability for the first output data when information on the first output data, information on the second output data, and information on training data similar to the plurality of symptoms are input, and the reliability for the first output data is acquired by inputting information on the first output data, information on the second output data, and information on training data similar to the plurality of symptoms into the third learning model. 11. A computer program according to any one of claims 1 to 10.

[0085] The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The technical features described in each embodiment can be combined with each other, and the scope of the present invention is intended to include all modifications within the scope of the claims and the scope equivalent to the claims. The sequences shown in each embodiment are not limited, and the order of each process may be changed and multiple processes may be executed in parallel, as long as there is no contradiction. The subject of each process is not limited, and the process of each device may be executed by another device, as long as there is no contradiction.

[0086] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations regardless of the citation format. Furthermore, the claims use a format in which a claim cites two or more other claims (multiple claim format), but this is not limited to this. A multiple claim (multi-multi claim) that cites at least one multiple claim may also be used. [Explanation of symbols]

[0087] 100 Diagnostic Support System 1. Information processing device 11 Control section 12 Storage section 13. Communications Department 1P Program 121 Training Data DB 122 First Learning Model 123 Second Learning Model 124 Third Learning Model 1A Recording medium 2 Terminal Equipment 21 Control section 22 Memory section 23 Communications Department 24 Display section 25 Control section 2P Program 2A Recording Media

Claims

1. Acquiring multiple symptoms Using a first learning model and a second learning model trained to output a plurality of diseases and a priority order of each disease when a plurality of symptoms are input based on training data, first output data including a plurality of diseases and a priority order of each disease output from the first learning model for the acquired plurality of symptoms, and second output data including a plurality of diseases and a priority order of each disease output from the second learning model are acquired; Acquire information on training data similar to the acquired plurality of symptoms among the training data based on the acquired second output data; The acquired first output data is output in association with information on training data similar to the plurality of symptoms. A computer program that causes a computer to carry out processing.

2. The first learning model and the second learning model are learned based on the training data in which a plurality of symptoms are associated with a plurality of diseases and a priority order of each disease based on medical literature; Outputting information about medical literature corresponding to training data similar to the plurality of symptoms.

2. The computer program product of claim 1.

3. determining parameters in the second learning model such that the second output data output from the second learning model approximates the first output data output from the first learning model; 3. A computer program according to claim 1 or 2.

4. The second learning model is a k-nearest neighbor model.

3. A computer program according to claim 1 or 2.

5. Acquire information about training data in the vicinity of the plurality of symptoms as training data similar to the plurality of symptoms.

5. A computer program according to claim 4.

6. The information on the medical literature includes at least one of the following: a name of the medical literature, an outline of the medical literature, a definite disease indicated by the medical literature, a differential diagnosis, a disease to be noted, a symptom to be noted, and a recommended test.

3. A computer program according to claim 2.

7. Identifying a symptom or disease recommended for confirmation based on the plurality of symptoms input to the first learning model or the first output data output from the first learning model and training data similar to the plurality of symptoms.

3. A computer program according to claim 1 or 2.

8. Identifying a symptom other than the symptom input to the first learning model among a plurality of symptoms in the training data similar to the plurality of symptoms as a symptom for which confirmation is recommended, or identifying a disease other than the disease of the first output data output from the first learning model among a plurality of diseases in the training data similar to the plurality of symptoms as a disease for which confirmation is recommended.

8. A computer program according to claim 7.

9. Deriving a reliability for the first output data based on the first output data, the second output data, and training data similar to the plurality of symptoms.

3. A computer program according to claim 1 or 2.

10. A third learning model is trained to output a reliability level for the first output data when information on the first output data, information on the second output data, and information on training data similar to the plurality of symptoms are input, and the reliability level for the first output data is acquired by inputting information on the first output data, information on the second output data, and information on training data similar to the plurality of symptoms into the third learning model.

3. A computer program according to claim 1 or 2.

11. Acquiring multiple symptoms Using a first learning model and a second learning model trained to output a plurality of diseases and a priority order of each disease when a plurality of symptoms are input based on training data, first output data including a plurality of diseases and a priority order of each disease output from the first learning model for the acquired plurality of symptoms, and second output data including a plurality of diseases and a priority order of each disease output from the second learning model are acquired; Acquire information on training data similar to the acquired plurality of symptoms among the training data based on the acquired second output data; The acquired first output data is output in association with information on training data similar to the plurality of symptoms. An information processing method in which processing is performed by a computer.

12. Acquiring multiple symptoms Using a first learning model and a second learning model trained to output a plurality of diseases and a priority order of each disease when a plurality of symptoms are input based on training data, first output data including a plurality of diseases and a priority order of each disease output from the first learning model for the acquired plurality of symptoms, and second output data including a plurality of diseases and a priority order of each disease output from the second learning model are acquired; Acquire information on training data similar to the acquired plurality of symptoms among the training data based on the acquired second output data; The acquired first output data is output in association with information on training data similar to the plurality of symptoms. Equipped with a control unit that executes processing Information processing device.

Images