Medical data processing device and method

Abstract

To enable provision of information for assisting in disease determination in a form that can be understood by users such as a doctor trying to determine a disease.SOLUTION: A medical data processing device according to an embodiment comprises a reception unit, determination unit, and output unit. The reception unit receives medical data as an input. The determination unit makes a medical determination based on the input data. An output unit outputs information regarding a determination process together with a result of the determination by the determination unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The embodiments disclosed in this specification and the drawings relate to medical data processing apparatus and methods. [Background technology]

[0002] There is a technology that uses MR images and MR spectra to provide medical classification. The output of this technology is a "grade," and the classification (discrimination) is performed in a black box. This causes a problem that users, such as doctors who distinguish diseases, cannot interpret the results according to the guidelines. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-41978 [Patent Document 2] JP 2017-026482 A [Patent Document 3] JP 2023-57036 A [Non-patent literature]

[0004] [Non-Patent Document 1] Hidenori Takeshima, Deep Learning and Its Application to Function Approximation for MR in Medicine: An Overview, Magn Reson Med Sci, doi:10.2463 / mrms.rev.2021-004, Published Online: September 17, 2021 Summary of the Invention [Problem to be solved by the invention]

[0005] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to present information that assists in the diagnosis of a disease in a form that can be interpreted by a user who is discriminating between diseases. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problems. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0006] A medical data processing device according to an embodiment includes a receiving unit, a discrimination unit, and an output unit. The receiving unit receives medical data as input. The discrimination unit performs medical discrimination based on the input data. The output unit outputs information related to the discrimination process together with a discrimination result by the discrimination unit. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a medical data processing system and a medical data processing device according to the first embodiment. [Diagram 2] FIG. 2 is a diagram for explaining medical classification executed by the discrimination function of the medical data processing device according to the first embodiment. [Diagram 3] FIG. 3 is a diagram for explaining details of an example of a trained model according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a process executed by the visualization function according to the first embodiment. [Diagram 5] FIG. 5 is a flowchart showing an example of the flow of processing executed by the medical data processing apparatus according to the first embodiment. [Figure 6] FIG. 6 is a diagram for explaining an example of processing executed by a medical data processing device according to a modification of the first embodiment. [Figure 7] FIG. 7 is a diagram showing an example of the configuration of a medical data processing system and a medical data processing device according to the second embodiment. [Figure 8A]FIG. 8A is a diagram showing an example of display content displayed on a display according to the second embodiment. [Figure 8B] FIG. 8B is a diagram showing an example of display content displayed on a display according to the second embodiment. [Figure 8C] FIG. 8C is a diagram showing an example of display content displayed on a display according to the second embodiment. [Figure 8D] FIG. 8D is a diagram showing an example of display content displayed on a display according to the second embodiment. [Figure 9A] FIG. 9A is a diagram showing an example of display content displayed on a display according to the second embodiment. [Figure 9B] FIG. 9B is a diagram showing an example of display content displayed on a display according to the second embodiment. [Figure 9C] FIG. 9C is a diagram showing an example of display content displayed on a display according to the second embodiment. [Figure 9D] FIG. 9D is a diagram showing an example of display content displayed on a display according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, the embodiments and modifications of the medical data processing device and method will be described in detail with reference to the drawings. Note that the medical data processing device and method according to the present application are not limited to the embodiments and modifications shown below. In addition, the embodiments can be combined with other embodiments, modifications, or conventional techniques as long as no inconsistencies arise in the processing content. Similarly, the modifications can be combined with the embodiments, other modifications, or conventional techniques as long as no inconsistencies arise in the processing content.

[0009] (First embodiment) FIG. 1 is a diagram showing an example of the configuration of a medical data processing system 100 and a medical data processing device 150 according to the first embodiment.

[0010] 1, the medical data processing system 100 according to the present embodiment includes a medical image diagnostic apparatus group 110, a medical data storage apparatus 120, a terminal 130, and a medical data processing apparatus 150. Here, each apparatus is communicably connected to each other via a network 160. The medical data processing system 100 and the medical data processing apparatus 150 according to the present embodiment are installed in medical facilities such as hospitals and clinics, for example, and support diagnoses by users such as doctors.

[0011] The medical image diagnostic device group 110 is a collection of a plurality of medical image diagnostic devices. Examples of such medical image diagnostic devices include various medical image diagnostic devices such as a Magnetic Resonance Imaging (MRI) device, an X-ray CT (Computed Tomography) diagnostic device, an ultrasonic diagnostic device, an X-ray diagnostic device, a PET (Positron Emission Tomography) device, and a SPECT (Single Photon Emission Computed Tomography) device.

[0012] An MRI device generates MR images and MR spectra. An X-ray CT device generates CT images. An ultrasound diagnostic device generates ultrasound images. An X-ray diagnostic device generates X-ray images and angio images. A PET device generates PET images. A SPECT device generates SPECT images. MR images, MR spectra, CT images, ultrasound images, X-ray images, angio images, PET images, and SPECT images are all medical data, and are data obtained by medical imaging diagnostic devices. In addition, MR images, CT images, ultrasound images, X-ray images, angio images, PET images, and SPECT images are also medical images.

[0013] The medical data storage device 120 stores various types of medical data. Specifically, the medical data storage device 120 acquires medical data from the medical image diagnostic apparatus group 110 via the network 160, and stores the acquired medical data in a memory circuit within the medical data storage device 120. For example, the medical data storage device 120 is realized by a PACS (Picture Archiving and Communication System) or the like, and stores the medical data in a format conforming to DICOM. The medical data storage device 120 is realized by computer equipment such as a server or a workstation.

[0014] The terminal 130 is a terminal used by a user, and displays various medical data. For example, the terminal 130 acquires various medical data such as medical images, MR spectra, and medical classification results from the medical data storage device 120 and the medical data processing device 150 via the network 160. Then, the terminal 130 displays the acquired medical data on a display provided in the terminal 130. The display provided in the terminal 130 is an example of a display unit. For example, the terminal 130 is realized by a computer device such as a workstation, a personal computer, or a tablet terminal.

[0015] The medical data processing device 150 performs medical discrimination including various medical classifications based on the medical data. That is, the medical data processing device 150 performs medical classification and medical discrimination based on the medical data. In the following description, the medical data processing device 150 performing medical classification and medical discrimination may be collectively referred to as "performing medical discrimination". For example, the medical data processing device 150 acquires medical data from the medical image diagnostic device group 110 or the medical data storage device 120 via the network 160. Then, the medical data processing device 150 performs various medical discriminations based on the acquired medical data. For example, the medical data processing device 150 is realized by a computer device such as a server or a workstation. In addition, in the following description, a case where one medical data processing device 150 executes various processes described below will be described, but multiple medical data processing devices 150 may execute various processes described below in a distributed manner.

[0016] As shown in FIG. 1, the medical data processing device 150 includes a communication interface 151, a memory circuitry 152, an input interface 153, a display 154, and a processing circuitry 155.

[0017] The communication interface 151 controls transmission and communication of various information and various data transmitted and received between the medical data processing device 150 and other devices connected to the medical data processing device 150 via a network 160. The communication interface 151 is connected to the processing circuit 155. The communication interface 151 receives data transmitted by other devices. In this case, the communication interface 151 transmits the received data to the processing circuit 155. The communication interface 151 also receives data transmitted by the processing circuit 155. In this case, the communication interface 151 transmits the received data to other devices. For example, the communication interface 151 is realized by a network card, a network adapter, a NIC (Network Interface Controller), or the like.

[0018] The memory circuitry 152 stores various data and various programs. The memory circuitry 152 is connected to the processing circuitry 155. The memory circuitry 152 stores data transmitted by the processing circuitry 155 under the control of the processing circuitry 155. The data stored in the memory circuitry 152 is read by the processing circuitry 155. For example, the memory circuitry 152 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, or the like.

[0019] The memory circuitry 152 stores a trained model 172 (see FIG. 2) and a decision tree 173 (see FIG. 2) described later. The memory circuitry 152 is, for example, an example of a memory unit.

[0020] The input interface 153 accepts input operations of various instructions and various information from a user such as a doctor. The input interface 153 is connected to the processing circuit 155. The input interface 153 converts the operation accepted from the user into an electrical signal and transmits it to the processing circuit 155. For example, the input interface 153 is realized by a trackball, a switch button, a mouse, a keyboard, a touch pad that accepts operations by touching the operation surface, a touch screen in which a display screen and a touch pad are integrated, a non-contact input interface using an optical sensor, a voice input interface, and the like. Note that in this specification, the input interface 153 is not limited to only those that have physical operation parts such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and transmits this electrical signal to the processing circuit 155 is also included as an example of the input interface 153.

[0021] The display 154 displays various information and data. The display 154 is connected to the processing circuit 155. The display 154 is controlled by the processing circuit 155 and displays various information and data transmitted by the processing circuit 155. For example, the display 154 is realized by a liquid crystal display, a CRT (Cathode Ray Tube) display, a touch panel, or the like. The display 154 is an example of a display unit.

[0022] The processing circuitry 155 controls the entire medical data processing device 150. For example, the processing circuitry 155 performs various medical classifications in response to an operation received from a user via the input interface 153. In addition, for example, the processing circuitry 155 acquires medical data transmitted by the medical image diagnostic device group 110 or the medical data storage device 120 via the network 160. Then, the processing circuitry 155 stores the acquired medical data in the memory circuitry 152.

[0023] As shown in FIG. 1, the processing circuit 155 has a receiving function 155a, a determining function 155b, and a visualizing function 155c.

[0024] The processing circuitry 155 is realized by, for example, a processor. In this case, each of the above-mentioned processing functions is stored in the storage circuitry 152 in the form of a program (medical data processing program) executable by a computer. Then, the processing circuitry 155 reads each program stored in the storage circuitry 152 and executes each read program to realize each processing function corresponding to each program. In other words, the processing circuitry 155 has each processing function shown in FIG. 1 in a state in which each program is read.

[0025] The processing circuitry 155 may be configured by combining a plurality of independent processors, and each processor may execute a program to realize each processing function. Each processing function of the processing circuitry 155 may be realized by being distributed or integrated into a single or a plurality of processing circuits. Each processing function of the processing circuitry 155 may be realized by a combination of hardware such as a circuit and software. Here, an example in which each program corresponding to each processing function is stored in a single storage circuitry 152 has been described, but each program may be stored in a distributed manner in a plurality of storage circuits. For example, each program corresponding to each processing function may be stored in a distributed manner in a plurality of storage circuits, and the processing circuitry 155 may read and execute each program from each storage circuit.

[0026] An example of the configuration of the medical data processing system 100 and the medical data processing device 150 according to the present embodiment has been described above. As described below, the medical data processing system 100 and the medical data processing device 150 according to the present embodiment are configured to present information that assists in the diagnosis in a form that can be interpreted by a user such as a doctor who discriminates a disease.

[0027] The receiving function 155a acquires medical data from the medical image diagnostic apparatus group 110 or the medical data storage apparatus 120 via the network 160. Then, the receiving function 155a receives the medical data as an input of the trained model 172. The receiving function 155a is, for example, an example of a receiving unit.

[0028] Fig. 2 is a diagram for explaining medical classification executed by the discrimination function 155b of the medical data processing device 150 according to the first embodiment. As shown in Fig. 2, the discrimination function 155b performs medical discrimination by applying a medical discrimination model (medical classification model) including a trained model 172 and a decision tree 173 to medical data 171. That is, the discrimination function 155b performs medical discrimination based on the medical data 171. The medical data 171 includes tissue of a subject to be processed.

[0029] The discrimination function 155b according to this embodiment performs medical discrimination based on one or more medical data 171. More specifically, the discrimination function 155b applies one or more medical data 171 to a medical discrimination model including a trained model 172 and a decision tree 173 to output a medical discrimination result (medical classification result) for the subject. The medical discrimination result is information indicating the result of medical discrimination for the subject of the medical data. As the medical discrimination result, for example, the grade of the disease with which the subject is suffering is output. The discrimination function 155b is, for example, an example of a discrimination unit.

[0030] The medical data 171 applied to the medical discrimination model including the trained model 172 and the decision tree 173 may be various types of data. For example, the medical data 171 may be data obtained by the various medical image diagnostic devices described above. The medical data 171 may also be the results of various tests such as genetic tests, blood tests, pathological tests, and molecular tests. The medical data 171 may also be the results of tests performed by various image diagnostic devices.

[0031] The medical data 171 may be information on the form of the subject to be processed (morphological information), information on the composition of the subject to be processed (compositional information), or information on the physical properties of the subject to be processed (physical property information). The medical data 171 may be physiological information of the subject to be processed.

[0032] Below, we will explain the case where the medical data processing device 150 determines the grade of glioma using a decision tree 173 based on the World Health Organization (WHO) brain tumor classification, which is a medical guideline for classifying the grade of glioma, a type of brain tumor, as a medical determination, but the medical determination performed by the medical data processing device 150 is not limited to this.

[0033] As shown in FIG. 2, the trained model 172 is a machine learning model in which parameters are trained to output one or more intermediate discrimination items using one or more medical data 171 as input. The medical data 171 includes information on brain glial cells as tissue to be processed of the subject. When multiple medical data 171 are input to the trained model 172, multiple medical data 171 including information on brain glial cells of the same subject are input to the trained model 172. As the trained model or machine learning model according to the first embodiment, a classifier or discriminator such as a neural network, a deep neural network, a support vector machine, or a random forest is used. Specifically, the intermediate discrimination items are tissue information, genetic information, and molecular information on the subject, as shown in FIG. 2.

[0034] The tissue information includes information (tumor presence / absence information) indicating whether a tumor (brain tumor, glioma) has occurred in the neuroglial cells of the brain, which is the tissue to be processed of the subject, and information (grade information) provisionally indicating the grade of the tumor. For example, if it is estimated that a tumor has occurred in the neuroglial cells, the tumor presence / absence information indicates "1". On the other hand, if it is estimated that a tumor has not occurred in the neuroglial cells, the tumor presence / absence information indicates "0". For example, the trained model 172 calculates the probability that a tumor has occurred in the neuroglial cells. This probability is a value within a range of 0 to 1, and the higher the probability of a tumor occurring in the neuroglial cells, the closer it is to "1", and the lower the probability of a tumor occurring in the neuroglial cells, the closer it is to "0". The probability is a reliability indicating the likelihood that a tumor has occurred. Then, the trained model 172 compares the probability with a predetermined threshold (reference discrimination value), and generates tumor presence / absence information indicating "1" if the probability is equal to or greater than the reference discrimination value. Furthermore, when the probability is less than the reference discrimination value, the trained model 172 generates tumor presence / absence information indicating "0."

[0035] In addition, when the tumor presence / absence information indicates "1" (i.e., when a tumor has developed), the trained model 172 roughly estimates the grade of the tumor and generates grade information indicating the estimated grade.

[0036] Then, when tumor presence / absence information and grade information are generated, the trained model 172 outputs tissue information including the tumor presence / absence information and grade information.

[0037] The genetic information is information indicating whether or not a mutation has occurred in the gene encoding the IDH of the neuroglial cell. For example, if it is estimated that a mutation has occurred in the gene encoding the IDH of the neuroglial cell, the genetic information indicates "1". On the other hand, if it is estimated that no mutation has occurred in the gene encoding the IDH of the neuroglial cell, the genetic information indicates "0". For example, the trained model 172 calculates the probability that a mutation has occurred in the gene encoding the IDH of the neuroglial cell. This probability is a value within a range of 0 to 1, and the higher the probability that a mutation has occurred in the gene encoding the IDH of the neuroglial cell, the closer the value is to "1", and the lower the probability that a mutation has occurred in the gene encoding the IDH of the neuroglial cell, the closer the value is to "0". The probability is a reliability indicating the likelihood that a mutation has occurred in the gene encoding the IDH of the neuroglial cell. Then, the trained model 172 compares the probability with a standard discrimination value, and if the probability is equal to or greater than the standard discrimination value, outputs genetic information indicating "1". Furthermore, if the probability is less than the standard discrimination value, the trained model 172 outputs genetic information indicating "0".

[0038] The molecular information is information indicating whether or not both the short arm of chromosome 1 (1q) and the long arm of chromosome 19 (19q) are deleted in a neuroglial cell. For example, when it is estimated that both the short arm of chromosome 1 and the long arm of chromosome 19 are deleted in a neuroglial cell, the molecular information indicates "1". On the other hand, when it is estimated that at least one of the short arm of chromosome 1 and the long arm of chromosome 19 is not deleted in a neuroglial cell, the molecular information indicates "0". For example, the trained model 172 calculates the probability that both the short arm of chromosome 1 and the long arm of chromosome 19 are deleted in a neuroglial cell. This probability is a value within a range of 0 to 1, and the higher the probability that both the short arm of chromosome 1 and the long arm of chromosome 19 are deleted in a neuroglial cell, the closer the value is to "1", and the lower the probability that both the short arm of chromosome 1 and the long arm of chromosome 19 are deleted in a neuroglial cell, the closer the value is to "0". The probability is a confidence level indicating the likelihood that both the short arm of chromosome 1 and the long arm of chromosome 19 are deleted. Then, the trained model 172 compares the probability with a standard discrimination value, and if the probability is equal to or greater than the standard discrimination value, outputs molecular information indicating "1." Furthermore, if the probability is less than the standard discrimination value, the trained model 172 outputs molecular information indicating "0."

[0039] An example of a method for generating the trained model 172 will be described. The trained model 172 is a trained machine learning model obtained by making a machine learning model perform machine learning according to a model learning program based on teacher data in which medical data including information on the glial cells of a subject is used as input data, and information indicating whether a tumor has occurred in the glial cells, information indicating the grade of the tumor when a tumor has occurred in the glial cells, information indicating whether a mutation has occurred in the gene encoding the IDH of the glial cells, and information indicating whether both the short arm of chromosome 1 and the long arm of chromosome 19 are deleted in the glial cells are correct data. The trained model 172 is generated by a learning device. For example, the learning device is connected to the medical data processing device 150 via a network 160. The trained model 172 generated by the learning device is provided to the medical data processing device 150 via the network 160.

[0040] The learning device includes a machine learning model such as a convolutional neural network (CNN). The learning device generates a trained model 172 by performing learning (supervised learning) based on teacher data related to glial cells of the same subject. The trained model 172 is provided with a function to output tissue information, genetic information, and molecular information (output data) corresponding to correct answer data when medical data corresponding to input data is input during inference. Note that the medical data processing device 150 may have a function similar to that of the learning device, and the medical data processing device 150 may generate the trained model 172 instead of the learning device.

[0041] A case where the machine learning model is CNN will be described. In this case, in the learning device, input data is input to the CNN, which is the machine learning model. The learning device applies CNN to the input data to generate output data. Then, the output data is output from the CNN. In the learning device, the output data is input to the evaluation function. In the learning device, the above-mentioned correct answer data is also input to the evaluation function. The learning device evaluates the output data generated by the CNN based on the input data and the correct answer data by the evaluation function. For example, the evaluation function compares the generated output data with the correct answer data and corrects the CNN coefficients (network parameters such as weights and biases) by the error backpropagation method. The evaluation by the evaluation function is fed back to the CNN. The learning device repeats such a series of supervised learning based on the teacher data until, for example, the error between the output data and the correct answer data becomes equal to or less than a predetermined threshold. The learning device can output the learned machine learning model as a trained model 172.

[0042] The decision tree 173 is a machine learning model trained to output the grade of glioma using tissue information, genetic information, and molecular information as input. For example, the decision tree 173 is a trained machine learning model obtained by making the machine learning model perform machine learning according to a model learning program using a combination of tissue information, genetic information, and molecular information, and a grade of glioma obtained as a result of performing the World Health Organization's brain tumor classification, which is a medical guideline for discriminating (classifying) the grade of glioma, which is a brain tumor, for this combination. That is, the above-mentioned medical guideline is used when the decision tree 173 is generated. As shown in FIG. 2, when the tissue information, genetic information, and molecular information output from the trained model 172 are input, the decision tree 173 outputs the grade of glioma as a final discrimination result.

[0043] In the decision tree 173, various conditions are defined for various pieces of information, such as tissue information, gene information, and molecular information, and the various conditions are defined as "Yes" or "No." Therefore, the decision tree 173 is a model (interpretable model) that allows a user, such as a doctor who discriminates diseases, to interpret the discrimination process (classification process, inference process). Note that, instead of the decision tree 173, a model having a function similar to that of the decision tree 173 may be used. For example, a random forest, a naive Bayes tree, a decision list, a decision graph, a Bayesian network, a gradient boosting tree, or the like may be used. Like the decision tree 173, the random forest, the naive Bayes tree, the decision list, the decision graph, the Bayesian network, and the gradient boosting tree are models that allow a user, such as a doctor who discriminates diseases, to interpret the discrimination process (classification process).

[0044] FIG. 3 is a diagram for explaining details of an example of the trained model 172 according to the first embodiment. The trained model 172 shown in FIG. 3 receives three pieces of medical data 171a, 171b, and 171c as input, and outputs tissue information, genetic information, and molecular information. In the example of FIG. 3, the three pieces of medical data 171a, 171b, and 171c are MR data. Although details will be described later, the medical data 171a is an MR image, and the medical data 171b and 171c are MR spectra. For this reason, in the following description, the medical data 171a may be referred to as the MR image 171a, and the medical data 171b and 171c may be referred to as the MR spectra 171b and 171c.

[0045] The trained model 172 shown in Fig. 3 is, for example, a neural network, and includes an input layer, an intermediate layer, and an output layer. Note that the input layer and the output layer are omitted in Fig. 3. In the example of Fig. 3, the intermediate layer includes a CNN layer, a fully connected layer, and a pooling layer (not shown), etc.

[0046] In the example of FIG. 3, the CNN layer includes three nodes 172a, 172b, and 172c. An MR image 171a is input to the node 172a as medical data 171 via the input layer. The MR image 171a is a morphological image showing the morphology of glial cells of a subject. For example, the MR image 171a is a T1 weighted image, a T2 weighted image, or a FLAIR (Fluid Attenuated Inversion Recovery) image. When the MR image 171a is a two-dimensional image, the two-dimensional node 172a may be used, and when the MR image 171a is a three-dimensional image, the three-dimensional node 172a may be used. The node 172a receives the MR image 171a as an input and outputs a feature amount a.

[0047] The node 172b receives an MR spectrum 171b as the medical data 171 through the input layer. The node 172c receives an MR spectrum 171c as the medical data 171 through the input layer. Here, the MR spectrum 171b is an MR spectrum collected by the MRI device using the PRESS method, whereas the MR spectrum 171c is an MR spectrum collected by the MRI device using the MEGA-PRESS method. Since the MR spectrum 171b and the MR spectrum 171c are one-dimensional spectra, one-dimensional nodes 172b and 172c may be used. The node 172b receives the MR spectrum 171b as an input and outputs a feature amount b. The node 172c receives the MR spectrum 171c as an input and outputs a feature amount c.

[0048] In the example of Fig. 3, the fully connected layer includes three nodes 172d, 172e, and 172f. Node 172d receives feature a and feature b as input, and outputs tissue information via the output layer. Node 172e receives feature a, feature b, and feature c as input, and outputs gene information via the output layer. Node 172f receives feature a, feature b, and feature c as input, and outputs molecular information via the output layer.

[0049] 3, the MR image 171a, which is a morphological image showing the morphology of the glial cells of the subject, is used as the medical data 171 input to the node 172a. However, other images, such as a CT image, an ultrasound image, or an X-ray image, which is a morphological image showing the morphology of the glial cells of the subject, may be used as the medical data 171 input to the node 172a.

[0050] In addition, the medical data 171 applied to the trained model 172 is not limited to data obtained by a medical image diagnostic device or the like, and may be artificially generated artificial data. For example, medical data obtained by simulation, deep learning, or the like may be applied to the trained model 172 as artificial data. Specifically, MR data of MR images and MR spectra may be generated by physical calculations called Bloch simulators or MR simulators, and the generated MR data may be applied to the trained model 172. In addition, data indicating the behavior of molecules obtained by molecular dynamics simulation may be applied to the trained model 172. In addition, values ​​obtained by converting actual measurements obtained by a medical image diagnostic device or the like through physical calculations may be applied to the trained model 172.

[0051] The visualization function 155c visualizes the discrimination process (classification process) of the medical discrimination (medical classification) performed by the discrimination function 155b. Fig. 4 is a diagram for explaining an example of a process executed by the visualization function 155c according to the first embodiment. Fig. 4 shows an example of the display content displayed on the display 154 by the visualization function 155c.

[0052] For example, the visualization function 155c displays the contents 181 of the decision tree 173 on the display 154 as shown in FIG. 4. In the contents 181 of the decision tree 173, each branch condition and the discrimination result of the leaf node connected by an arrow thicker than the other arrows are information indicating the discrimination process by the decision tree 173, and are information about the discrimination process. Specifically, in FIG. 3, the part of "branch condition of the root node "Tumor?" (Yes) → branch condition of the node "Grade=4?" (No) → branch condition of the node "IDH mutant?" (Yes) → branch condition of the node "1p19q Co-deletion?" (No) → discrimination result of the leaf node "LGG IDH mutant non co-deletion final grade WHO2"" is highlighted, and is information indicating the discrimination process by the decision tree 173, and is information about the discrimination process. In addition, the discrimination result of the leaf node "LGG IDH mutant non co-deletion final grade WHO2" is information indicating the discrimination result, and is information about the discrimination result.

[0053] Here, an example of a method for generating information on the discrimination process will be described. For example, the visualization function 155c identifies the discrimination process in the decision tree 173 by determining whether each branch condition of the decision tree 173 is "Yes" or "No" using the tissue information, gene information, and molecular information output from the trained model 172. Then, the visualization function 155c generates information indicating the identified discrimination process.

[0054] In this way, the visualization function 155c and the display 154 visualize the discrimination process that can be interpreted by a user such as a doctor. That is, the visualization function 155c and the display 154 output (display) the discrimination process that can be interpreted by a user such as a doctor together with the discrimination result. More specifically, the visualization function 155c and the display 154 output the contents 181 of the decision tree 173 showing the inference process from the root node (branching condition "Tumor?") to the leaf node showing the discrimination result ("LGG IDH mutant non co-deletion final grade WHO2"). Therefore, according to the medical data processing device 150 according to the first embodiment, information that assists in the discrimination can be presented in a form that can be interpreted by a user such as a doctor who discriminates a disease. The visualization function 155c and the display 154 are, for example, an example of an output unit.

[0055] In addition, the visualization function 155c displays on the display 154, as shown in FIG. 4, the medical data 171 (171a, 171b, 171c) input to the trained model 172, and a table 182 in which the contents of the tissue information, genetic information, and molecular information are described, together with the contents 181 of the decision tree 173. Here, "tumor present" in the tissue information shown in FIG. 4 corresponds to "1" indicated by the tumor presence / absence information in the tissue information output from the trained model 172. In addition, "mutation present" in the genetic information shown in FIG. 4 corresponds to "1" indicated by the genetic information output from the trained model 172. In addition, "no deletion" in the molecular information shown in FIG. 4 corresponds to "0" indicated by the molecular information output from the trained model 172. In addition, when the tumor presence / absence information in the tissue information output from the trained model 172 indicates "0", "no tumor" is described in the table 182. In addition, when the genetic information output from the trained model 172 indicates "0", "no mutation" is described in the table 182. Furthermore, when the molecular information output from the trained model 172 indicates "1", "deletion present" is entered in the table 182. That is, the table 182 is a table showing numerical values ​​used when the discrimination function 155b makes a medical discrimination. In this way, the visualization function 155c and the display 154 visualize the medical data 171 input to the trained model 172, as well as the intermediate discrimination items of tissue information, genetic information, and molecular information, in addition to the discrimination result and discrimination process.

[0056] FIG. 5 is a flowchart showing an example of the flow of processing executed by the medical data processing apparatus 150 according to the first embodiment.

[0057] As shown in FIG. 5, the receiving function 155a acquires medical data from the medical image diagnostic apparatus group 110 or the medical data storage apparatus 120 via the network 160, and receives the medical data as input to the trained model 172 (step S101).

[0058] Then, the discrimination function 155b performs medical discrimination by applying the medical discrimination model including the trained model 172 and the decision tree 173 to the medical data 171 (step S102).

[0059] Then, the visualization function 155c and the display 154 visualize the discrimination process that can be interpreted by a user such as a doctor together with the discrimination result (step S103).

[0060] The above describes the medical data processing device 150 and the medical data processing system 100 according to the first embodiment. According to the first embodiment, it is possible to present information that assists in the diagnosis in a form that can be interpreted by a user such as a doctor who discriminates a disease.

[0061] (Modification of the first embodiment) Next, the medical data processing device 150 according to a modified example of the first embodiment will be described. In the table 182, for example, there are cases where there is no consistency between a plurality of numerical values ​​used when the discrimination function 155b performs medical discrimination. For example, even though "no tumor" is described in the table 182, "mutation" and "deletion" may be described. That is, even though the table 182 shows that no tumor has occurred, there are cases where it shows that a mutation has occurred in a gene encoding the IDH of a neuroglial cell, or that both the short arm of chromosome 1 and the long arm of chromosome 19 are deleted in a neuroglial cell.

[0062] FIG. 6 is a diagram for explaining an example of a process executed by the medical data processing device 150 according to the modification of the first embodiment. In the above-mentioned case, since the multiple pieces of information listed in the table 182 are an unexpected combination, the visualization function 155c of the medical data processing device 150 according to the modification of the first embodiment displays a message "Unexpected combination, so reliability is low" 186 indicating that the reliability of the multiple pieces of information (numerical values) and the reliability of the medical discrimination result are low, as shown in FIG. 6. That is, the visualization function 155c and the display 154 output the message 186 indicating that the reliability of the multiple pieces of information and the reliability of the medical discrimination result are low, that is, that the medical discrimination result is unreliable.

[0063] This allows the user to understand that the presented medical discrimination result is unreliable. Therefore, according to the modification of the first embodiment, it is possible to prevent the user from recognizing an erroneous medical discrimination result as a correct medical discrimination result.

[0064] Second embodiment Next, a medical data processing device and a medical data processing system according to a second embodiment will be described. In the first embodiment, as shown in FIG. 4, in addition to the contents 181 of the decision tree 173, a table 182 is displayed. For this reason, a large amount of information is displayed at once, and it may be difficult for the user to easily grasp the displayed information. Therefore, the medical data processing device and the medical data processing system according to the second embodiment are configured to allow the user to easily grasp the displayed information. In the following description of the second embodiment, the same reference numerals may be used to designate the same configuration as in the first embodiment, and the description may be omitted. In addition, in the description of the second embodiment, the differences from the first embodiment will be mainly described.

[0065] 7 is a diagram showing an example of the configuration of a medical data processing system 100a and a medical data processing device 150a according to the second embodiment. The medical data processing system 100a according to the second embodiment is different from the medical data processing system 100 according to the first embodiment in that it includes a medical data processing device 150a instead of the medical data processing device 150. The medical data processing device 150a according to the second embodiment is different from the medical data processing device 150 according to the first embodiment in that it includes a processing circuit 156a instead of the processing circuit 155. The processing circuit 156a according to the second embodiment is different from the processing circuit 155 according to the first embodiment in that it includes a layout change function 155d.

[0066] In the second embodiment, the receiving function 155a receives a layout change command input by a user via the input interface 153. Here, the layout change command is a command for changing the layout of the display contents displayed on the display 154. When the receiving function 155a receives the layout change command, the layout change function 155d controls the visualization function 155c so as to change the layout of the display contents displayed on the display 154. Therefore, when the receiving function 155a receives the layout change command, the visualization function 155c performs various processes described below under the control of the layout change function 155d.

[0067] Here, in the second embodiment, the visualization function 155c does not simultaneously display the contents 181 and the table 182 of the decision tree 173 on the display 154. Specifically, when the contents 181 of the decision tree 173 are displayed on the display 154, the visualization function 155c does not display the table 182 on the display 154. In addition, when the table 182 is displayed on the display 154, the visualization function 155c does not display the contents 181 of the decision tree 173 on the display 154.

[0068] More specifically, when a layout change command is received by the receiving function 155a while the content 181 of the decision tree 173 is being displayed on the display 154, the visualization function 155c causes the display 154 to display a table 182. Also, when a layout change command is received by the receiving function 155a while the table 182 is being displayed on the display 154, the visualization function 155c causes the display 154 to display the content 181 of the decision tree 173. In other words, when a layout change command is received by the receiving function 155a, the display 154 and the visualization function 155c switch the display content to either the content 181 of the decision tree 173 or the table 182.

[0069] Therefore, according to the second embodiment, the contents 181 and table 182 of the decision tree 173 are not displayed simultaneously, but are displayed at different time periods, allowing the user to easily understand the contents 181 and table 182 of the decision tree 173.

[0070] In the second embodiment, the visualization function 155c may not simultaneously display the contents 181 of the decision tree 173 and the medical data 171 on the display 154. Specifically, when the contents 181 of the decision tree 173 are displayed on the display 154, the visualization function 155c does not display the medical data 171 on the display 154. In addition, when the medical data 171 is displayed on the display 154, the visualization function 155c does not display the contents 181 of the decision tree 173 on the display 154.

[0071] More specifically, when a layout change command is received by the receiving function 155a while the content 181 of the decision tree 173 is being displayed on the display 154, the visualization function 155c causes the display 154 to display the medical data 171. Also, when a layout change command is received by the receiving function 155a while the medical data 171 is being displayed on the display 154, the visualization function 155c causes the display 154 to display the content 181 of the decision tree 173. In other words, when a layout change command is received by the receiving function 155a, the display 154 and the visualization function 155c switch the display content to either the content 181 of the decision tree 173 or the medical data 171.

[0072] Therefore, since the contents 181 of the decision tree 173 and the medical data 171 are not displayed simultaneously but at different time periods, the user can easily understand the contents 181 of the decision tree 173 and the medical data 171.

[0073] Hereinafter, an example of switching of other display contents in the second embodiment will be described. Figures 8A, 8B, 8C, 8D, 9A, 9B, 9C, and 9D are diagrams showing an example of various display contents displayed on the display 154 according to the second embodiment.

[0074] 8A, the visualization function 155c displays the medical data 171, the contents 181 of the decision tree 173, and a table 185 on the display 154. In the table 185, the above-mentioned probability (the reliability indicating the likelihood of the intermediate discrimination item) is described for each intermediate discrimination item (tissue information, gene information, and molecular information).

[0075] When a layout change command is accepted by the accepting function 155a while the display content shown in Fig. 8A is being displayed on the display 154, the visualization function 155c causes the display 154 to display the display content shown in Fig. 8B. For example, as shown in Fig. 8B, the visualization function 155c causes the display 154 to display the medical data 171, the contents 181 of the decision tree 173, and a table 187. In the table 187, the above-mentioned reference discrimination value to be compared with the probability is described for each intermediate discrimination item.

[0076] Furthermore, when a layout change command is accepted by the accepting function 155a while the display content shown in FIG. 8B is being displayed on the display 154, the visualizing function 155c causes the display 154 to display the display content shown in FIG. 8A.

[0077] That is, when the receiving function 155a receives a layout change command, the display 154 and the visualization function 155c switch the display content to either the probability indicating the likelihood of the intermediate discrimination item or the discrimination reference value.

[0078] Therefore, according to the second embodiment, the probability indicating the certainty of the intermediate discrimination item and the discrimination criterion value are not displayed simultaneously but are displayed at different time periods, so that the user can easily grasp the probability indicating the certainty of the intermediate discrimination item and the discrimination criterion value.

[0079] The visualization function 155c may display the display content shown in FIG. 8C or the display content shown in FIG. 8D on the display 154 instead of the display content shown in FIG. 8A. For example, the visualization function 155c may display the medical data 171, the contents 181 of the decision tree 173, and the table 185a on the display 154 as shown in FIG. 8C. The table 185a describes the sensitivity (True Positive Rate) for each intermediate discrimination item. The visualization function 155c acquires the sensitivity for each intermediate discrimination item calculated when the trained model 172 is trained from the above-mentioned learning device, and generates the table 185a using the acquired sensitivity.

[0080] 8D, the visualization function 155c may display the medical data 171, the contents 181 of the decision tree 173, and the table 185b on the display 154. The table 185b describes a specificity (True Negative Rate) for each intermediate discrimination item. The visualization function 155c acquires the specificity for each intermediate discrimination item calculated when the trained model 172 is trained from the learning device, and generates the table 185b using the acquired specificity.

[0081] At the time of inference of the trained model 172, by presenting evaluation indices such as sensitivity and specificity to the user, it is possible to present the user with reliability information of the trained model 172. As a result, the user can grasp the performance of the trained model 172.

[0082] Also, for example, the visualization function 155c causes the display 154 to display the contents 181 of the decision tree 173 as shown in Fig. 9A. Then, when a layout change command is accepted by the acceptance function 155a while the display content shown in Fig. 9A is being displayed on the display 154, the visualization function 155c causes the display 154 to display the display content shown in Fig. 9B. For example, the visualization function 155c causes the display 154 to display the contents of the guideline 188 as shown in Fig. 9B. The guideline 188 is the above-mentioned medical guideline used when the decision tree 173 was generated.

[0083] Furthermore, when a layout change command is accepted by the accepting function 155a while the display content shown in FIG. 9B is being displayed on the display 154, the visualizing function 155c causes the display 154 to display the display content shown in FIG. 9A.

[0084] That is, when the layout change command is accepted by the acceptance function 155a, the display 154 and the visualization function 155c switch the display contents to either the contents 181 of the decision tree 173 or the contents of the guideline 188.

[0085] Therefore, according to the second embodiment, the contents 181 of the decision tree 173 and the contents of the guideline 188 are not displayed simultaneously, but are displayed at different time periods, so that the user can easily understand the contents 181 of the decision tree 173 and the contents of the guideline 188.

[0086] The visualization function 155c may switch the version of the guideline 188 whose contents are displayed. For example, the visualization function 155c includes a selection function. When the selection function receives a command to switch the version of the guideline to be complied with via the input interface 153 while the contents of the guideline 188 are displayed on the display 154, the content of the guideline whose version has been switched may be displayed on the display 154. For example, the guidelines are updated once every few years. Therefore, the selection function switches the version of the guideline whose contents are displayed, for example, from version "WHO2016" to the latest version "WHO2021".

[0087] Also, for example, when a layout change command is accepted by the acceptance function 155a while the display content shown in Fig. 9A is being displayed on the display 154, the visualization function 155c causes the display 154 to display the display content shown in Fig. 9C. For example, as shown in Fig. 9C, the visualization function 155c causes the display 154 to display only the inference process 181a from the root node (branching condition "Tumor?") to the leaf node showing the discrimination result ("LGG IDH mutant non co-deletion final grade WHO2") of the decision tree 173.

[0088] Furthermore, when a layout change command is accepted by the accepting function 155a while the display content shown in FIG. 9C is being displayed on the display 154, the visualizing function 155c causes the display 154 to display the display content shown in FIG. 9A.

[0089] That is, when the acceptance function 155a accepts a layout change command, the display 154 and the visualization function 155c switch the display contents to either the contents 181 of the decision tree 173 or the inference process 181a.

[0090] Therefore, according to the second embodiment, the contents 181 of the decision tree 173 and the contents of the inference process 181a are not displayed simultaneously, but are displayed at different time periods, so that the user can easily understand the contents 181 of the decision tree 173 and the contents of the inference process 181a.

[0091] Furthermore, for example, when a layout change command is accepted by the accepting function 155a while the display content shown in Fig. 9A is being displayed on the display 154, the visualization function 155c may cause the display 154 to display the display content shown in Fig. 9D. For example, as shown in Fig. 9D, the visualization function 155c may cause the display 154 to display only data in which determination basis information 171d, 171e indicating the basis for determining the intermediate discrimination items is superimposed on the medical data 171b, 171c. The visualization function 155c can generate the basis for determining the intermediate discrimination items by an existing method such as Grad-CAM.

[0092] In the example of FIG. 9D, the evidence portions in the medical data 171b and 171c are indicated by shading, but the evidence portions may be indicated by coloring, or the evidence portions may be indicated by arrows.

[0093] Furthermore, when a layout change command is accepted by the accepting function 155a while the display content shown in FIG. 9D is being displayed on the display 154, the visualizing function 155c causes the display 154 to display the display content shown in FIG. 9A.

[0094] That is, when a layout change command is accepted by the accepting function 155a, the display 154 and the visualization function 155c switch the display content to either the content 181 of the decision tree 173 or the data in which the decision basis information 171d, 171e is superimposed on the medical data 171b, 171c.

[0095] Therefore, according to the second embodiment, the contents 181 of the decision tree 173 and the data in which the decision basis information 171d, 171e is superimposed on the medical data 171b, 171c are not displayed simultaneously but at different time periods, so that the user can easily understand the contents 181 of the decision tree 173 and the data in which the decision basis information 171d, 171e is superimposed on the medical data 171b, 171c.

[0096] The term "processor" used in the description of the above-mentioned embodiments means a circuit such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). Here, instead of storing a program in the memory circuit 152, the program may be directly embedded in the circuit of the processor. In this case, the processor realizes its function by reading and executing the program embedded in the circuit. In addition, each processor in each embodiment is not limited to being configured as a single circuit for each processor, and may be configured as one processor by combining multiple independent circuits to realize its function.

[0097] Here, the program executed by the processor is provided in advance in a ROM (Read Only Memory) or a storage circuit. The program may be provided by being recorded in a computer-readable non-transitory storage medium such as a CD (Compact Disk)-ROM, a FD (Flexible Disk), a CD-R (Recordable), or a DVD (Digital Versatile Disk) in a format that can be installed in these devices or in a format that can be executed. The program may also be provided or distributed by being stored on a computer connected to a network such as the Internet and downloaded via the network. For example, the program is composed of modules including each of the above-mentioned processing functions. As actual hardware, a CPU reads out the program from a storage medium such as a ROM and executes it, so that each module is loaded onto a main storage device and generated on the main storage device.

[0098] According to at least one of the embodiments described above, it is possible to present information that assists a user, such as a doctor, in diagnosing a disease in a form that can be interpreted by the user.

[0099] Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0100] 100,100a Medical Data Processing System 150, 150a Medical data processing device 155 Processing Circuit 155a Acceptance function 155b Discrimination function 155c Visualization function 155d Layout change function

Claims

1. a receiving unit that receives medical data as input; A discrimination unit that performs medical discrimination based on the medical data; an output unit that outputs information about the discrimination process together with the discrimination result by the discrimination unit; A medical data processing device comprising:

2. A storage unit that stores a medical discrimination model for performing the medical discrimination, the medical discrimination model including an interpretable model in which the discrimination process can be interpreted; The medical data processing device according to claim 1 , wherein the discrimination unit performs the medical discrimination by applying the medical discrimination model to the medical data.

3. The medical data is MR data, The discrimination unit performs medical discrimination including medical classification based on the MR data, The medical data processing device according to claim 1 , wherein the output unit outputs information about a discrimination process together with the discrimination result including the medical classification.

4. The output unit displays information about the discrimination process together with the discrimination result, The receiving unit receives a layout change command for changing a layout of the display content displayed on the output unit, the output unit changes a layout of the display content when the layout change command is accepted by the acceptance unit.

4. The medical data processing device according to claim 1.

5. The medical data processing device of claim 4, wherein when the layout change command is accepted by the accepting unit, the output unit switches the display content as information regarding the discrimination process to either a decision tree which is an interpretable model in which the discrimination process can be interpreted, showing an inference process from a root node to a leaf node indicating the discrimination result, or a table showing numerical values ​​used when the discrimination unit makes the medical discrimination.

6. The medical data processing device according to claim 5 , wherein the output unit outputs a message indicating that the discrimination result is unreliable when consistency cannot be achieved between the plurality of numerical values ​​in the table.

7. 5. The medical data processing device of claim 4, wherein when the layout change command is accepted by the accepting unit, the output unit switches the display content to either one of a reliability indicating the likelihood of an intermediate discrimination item calculated in a medical discrimination model that performs the medical discrimination when the discrimination unit obtains the discrimination result, and a discrimination standard value used in the medical discrimination model when the discrimination unit obtains the discrimination result and compared with the reliability, as information regarding the discrimination process.

8. 5. The medical data processing device according to claim 4, wherein when the layout change command is accepted by the accepting unit, the output unit switches the display content as information about the discrimination process to either a decision tree that is an interpretable model in which the discrimination process can be interpreted, showing an inference process from a root node to a leaf node indicating the discrimination result, or medical guidelines used when the decision tree was generated.

9. 5. The medical data processing device according to claim 4, wherein when the layout change command is accepted by the accepting unit, the output unit switches the display content as information regarding the discrimination process to either a decision tree which is an interpretable model in which the discrimination process can be interpreted, showing an inference process from a root node to a leaf node indicating the discrimination result, or only the inference process of the decision tree from the root node to a leaf node indicating the discrimination result.

10. 5. The medical data processing device according to claim 4, wherein when the layout change command is accepted by the accepting unit, the output unit switches the display content as information about the discrimination process to either a decision tree that is an interpretable model in which the discrimination process can be interpreted, showing an inference process from a root node to a leaf node indicating the discrimination result, or only the medical data.

11. 5. The medical data processing device according to claim 4, wherein when the layout change command is accepted by the accepting unit, the output unit switches the display content as information regarding the discrimination process to either a decision tree which is an interpretable model in which the discrimination process can be interpreted, showing an inference process from a root node to a leaf node indicating the discrimination result, or only data in which judgment basis information is superimposed on the medical data.

12. The medical data processing device according to claim 8 , wherein the output unit further comprises a selection function for switching between versions of medical guidelines to be complied with.

13. Accepting medical data as input; making a medical judgment based on the medical data; outputting information about the medical discrimination process together with the medical discrimination result; method.

Images