Medical information processing device and medical information processing method

Abstract

To improve the correctness of medical judgement.SOLUTION: A medical information processing device has an acquisition unit, a mapping unit, and an estimation unit. The acquisition unit acquires medical care information on a target patient's medical care event. The mapping unit maps the medical care information to a first graph having nodes corresponding to the medical care event and edges representing the relationship between the nodes to create a second graph related to the target patient. The estimation unit estimates medical judgement information for the target patient based on the second graph related to the target patient.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] Embodiments disclosed herein and in the drawings are medical information processing systems. reference Regarding medical information processing methods. [Background technology]

[0002] By systematizing disease classifications and relationships as medical knowledge and constructing databases, it is expected that this can be used to understand complex diseases, discover research hypotheses, and support clinical practice. For example, in the medical field, medical ontologs such as ICD-10, a disease classification system, and SNOMED-CT, which describes the semantic relationships between medical terms, have been developed. In recent years, data analysis methods using graph structures that can represent the relationships between multiple medical events have also attracted attention.

[0003] Diseases are subject to fluctuations. In other words, there isn't always a single, clear criterion for a disease; even the same disease can have various pathological conditions, its definition may change over time, and there may be inconsistencies in judgment even among different doctors making the diagnosis. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2018-60529 [Non-patent literature]

[0005] [Non-Patent Document 1] Y Li et al., “Graph Neural Network-Based Diagnosis Prediction”, Big Data, 8(5):379-390, 2020. [Overview of the project] [Problems that the invention aims to solve]

[0006] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve the accuracy of medical judgment. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problems. The problems corresponding to the effects of each configuration shown in the embodiments described later can also be regarded as other problems.

Means for Solving the Problems

[0007] The medical information processing device according to the embodiment includes an acquisition unit, a mapping unit, and an estimation unit. The acquisition unit acquires medical information regarding the medical events of the target patient. The mapping unit maps the medical information to a first graph having nodes corresponding to the medical events and edges representing the relationships between the nodes, and generates a second graph regarding the target patient. The estimation unit estimates medical judgment information regarding the target patient based on the second graph regarding the target patient.

Brief Description of the Drawings

[0008] [Figure 1] FIG. 1 is a diagram showing a configuration example of the medical information system according to the present embodiment. [Figure 2] FIG. 2 is a conceptual diagram of a medical knowledge graph. [Figure 3] FIG. 3 is a diagram showing a configuration example of the medical information processing device shown in FIG. 1. [Figure 4] FIG. 4 is a diagram showing the flow of medical information processing by the medical information processing device shown in FIG. 3. [Figure 5] FIG. 5 is a diagram showing an example of the mapping process according to step SA2 in FIG. 4. [Figure 6] FIG. 6 is a diagram showing an example of the estimation process according to step SA3 in FIG. 4. [Figure 7] FIG. 7 is a diagram showing an example of the display screen displayed in step SA4 in FIG. 4. [Figure 8] FIG. 8 is a diagram showing an example of the patient-specific circumstances DB used in step SA5 in FIG. 4. [Figure 9]Figure 9 shows an example of the configuration of the medical information learning device shown in Figure 1. [Figure 10] Figure 10 shows the flow of machine learning processing by the medical information learning device shown in Figure 9. [Figure 11] Figure 11 is a schematic diagram showing the update process related to step SB3 in Figure 10. [Figure 12] Figure 12 shows an example of the configuration of the medical information display device shown in Figure 1. [Figure 13] Figure 13 shows the flow of medical knowledge graph display processing using the medical information display device shown in Figure 12. [Figure 14] Figure 14 shows an example of the initial screen displayed in step SC1 of Figure 13. [Figure 15] Figure 15 shows an example of the display screen shown in step SC4 of Figure 13. [Figure 16] Figure 16 shows an example of a visualization graph displaying information for the entire patient population. [Figure 17] Figure 17 shows an example of a visualization graph displaying information about the disease as a whole. [Figure 18] Figure 18 is a conceptual diagram of the assignment of graph features (occurrence order, occurrence frequency, and occurrence severity) related to Application Example 1. [Figure 19] Figure 19 schematically shows the input and output of two machine learning models related to Application Example 2. [Figure 20] Figure 20 is a schematic diagram illustrating the convolution process performed by a graph convolutional layer in Application Example 3. [Figure 21] Figure 21 shows an example of the input and output of a linear coupling layer related to Application Example 4. [Figure 22] Figure 22 is a schematic diagram illustrating the machine learning model related to Application Example 5. [Figure 23] Figure 23 is a diagram showing an overview of the graph features related to Application Example 6. [Figure 24] Figure 24 is a diagram showing an overview of the graph features related to Application Example 7. [Figure 25]Figure 25 shows an example of the estimation process for medical judgment information related to Application Example 7. [Figure 26] Figure 26 schematically illustrates the relationship between the patient graph and spatial information related to Application Example 9. [Figure 27] Figure 27 shows the concept of a patient graph network related to application example 9. [Figure 28] Figure 28 is a diagram showing an overview of the graph features related to application example 10. [Modes for carrying out the invention]

[0009] The embodiments of the medical information processing device, medical information learning device, medical information display device, and medical information processing method will be described in detail below with reference to the drawings.

[0010] Figure 1 shows an example configuration of the medical information system 100 according to this embodiment. The medical information system 100 is a computer network system having a medical information storage device 1, a medical knowledge graph storage device 2, a medical information processing device 3, a medical information learning device 4, and a medical information display device 5. The medical information storage device 1, the medical knowledge graph storage device 2, the medical information processing device 3, the medical information learning device 4, and the medical information display device 5 are connected to the network so that they can communicate with each other.

[0011] Medical Information Storage Device 1 is a computer with a storage device for storing medical information of multiple patients. Medical Knowledge Graph Storage Device 2 is a computer with a storage device for storing medical knowledge graphs. A medical knowledge graph is represented as a graph structure generated based on medical knowledge such as medical information and medical ontologities of multiple patients. Hereafter, it is assumed that the medical knowledge graph is a graph. Medical Information Processing Device 3 is a computer that estimates medical judgment information for a target patient using the target patient's medical information and medical knowledge graph. Medical Information Processing Device 3 can also store various kinds of medical knowledge in the medical knowledge graph. Medical Information Learning Device 4 is a computer that trains machine learning models used to estimate medical judgment information. Medical Information Display Device 5 is a computer that extracts and displays desired medical knowledge from the medical knowledge graph.

[0012] Figure 2 is a conceptual diagram of the Medical Knowledge Graph 20. The Medical Knowledge Graph 20 is a graph that combines medical knowledge with the medical information of multiple patients. Medical information is information generated through the medical treatment of a particular patient. Medical information is classified into information related to medical events (hereinafter referred to as medical event information) and information related to diseases (hereinafter referred to as disease information). Medical events are specific matters related to medical treatment. The types of medical events can be divided into four categories, for example: symptoms, findings, treatment, and response. Symptoms are physical and mental changes caused by a disease. Specifically, types of medical events related to symptoms include, for example, edema, palpitations, and shortness of breath. Findings are the doctor's judgment regarding the symptoms. Findings may be further divided into categories of physical findings and laboratory findings. Treatment is a medical act to cure or alleviate symptoms. Types of medical events related to treatment include, for example, administration of cardiac stimulants and CRT implantation surgery. Responses are the physical and mental reactions of the patient to the treatment. The responses may be further divided into categories of therapeutic responses and side effects. Examples of clinical events related to responses include, for example, diuresis of 40 mL / h or more or electrolyte abnormalities in response to diuretic administration. Disease information refers to information about the name or symbol of the disease that the patient has been diagnosed with.

[0013] Medical information is collected through various hospital information systems, such as the Hospital Information System (HIS), the Radiology Information System (RIS), and the Picture Archiving and Communication System (PACS). Each medical event and disease information included in the medical information is associated with the date the information occurred.

[0014] As shown in Figure 2, the medical knowledge graph 20 is a graph constructed from multiple nodes 21 and multiple edges 22. Nodes 21 correspond to medical events. Edges 22 indicate relationships between nodes 21. That is, edges 22 indicate relationships between medical events corresponding to the connected nodes 21. Relationships between medical events refer to causal relationships and correlations between medical events. Causal relationships refer to cause-and-effect relationships that exist between medical events, while correlations refer to relationships that do not involve causality. That is, edges 22 representing causal relationships are directed, and edges 22 representing correlations are undirected. Edges 22 can also connect medical events belonging to different categories. The medical events used in the medical knowledge graph 20 are selected from medical information stored in the medical information storage device 1 based on medical knowledge, medical ontology, etc. The relationship between one medical event and another medical event is analyzed based on medical knowledge, medical ontology, etc., and if a relationship is recognized, two nodes 21 corresponding to those two medical events are connected by an edge 22.

[0015] Even within the same disease, the presence or strength of relationships between medical events varies depending on the individual circumstances of each patient. Estimating a patient's disease without considering their individual circumstances does not necessarily yield accurate results.

[0016] The medical knowledge graph 20 does not have nodes corresponding to clinical events such as diseases. A specific disease is extracted from a set of clinical events, including symptoms, findings, treatment, and response, that occurred in the target patient. In other words, the medical knowledge graph 20 represents individual diseases as a chain of relationships between a series of clinical events: symptoms, findings, treatment, and response. The medical knowledge graph 20 can also be said to be a graph representing the disease concept, describing a chain of relationships between a series of clinical events: symptoms, findings, treatment, and response. The medical information system 100 constructs a medical knowledge graph (patient graph) adapted to the context of the individual patient's circumstances to make accurate medical judgments, such as disease estimation.

[0017] Figure 3 shows an example of the configuration of a medical information processing device 3 according to this embodiment. As shown in Figure 3, the medical information processing device 3 has a processing circuit 31, a memory 32, an input interface 33, a communication interface 34, and a display 35.

[0018] The processing circuit 31 includes processors such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processing circuit 31 executes a medical information processing program to estimate and present medical judgment information, thereby realizing functions such as acquisition 311, mapping 312, estimation 313, visualization graph generation 314, storage 315, and display control 316. Note that each of the functions 311 to 316 is not limited to being realized by a single processing circuit. A processing circuit may be configured by combining multiple independent processors, and each processor may realize each of the functions 311 to 316 by executing a program. Furthermore, each of the functions 311 to 316 may be a modularized program that constitutes the medical information processing program, or it may be an individual program. These programs are stored in memory 32.

[0019] By implementing the acquisition function 311, the processing circuit 31 acquires various types of information. For example, the processing circuit 31 acquires medical event information, which is medical information related to the medical events of the target patient, from the medical information storage device 1. The processing circuit 31 also acquires a medical knowledge graph from the medical knowledge graph storage device 2.

[0020] By implementing the mapping function 312, the processing circuit 31 generates a patient graph for the target patient, which maps the patient's medical event information onto a medical knowledge graph having nodes corresponding to medical events and edges representing the relationships between those nodes.

[0021] By implementing the estimation function 313, the processing circuit 31 estimates medical judgment information for the target patient based on a patient graph related to the target patient. The medical judgment information is information relating to at least one medical judgment from among disease classification information, prognosis prediction information, and severity classification information.

[0022] By implementing the visualization graph generation function 314, the processing circuit 31 generates a patient graph for display (hereinafter referred to as the visualization graph) for the target patient based on the medical knowledge graph and the patient's medical information.

[0023] With the implementation of the storage function 315, the processing circuit 31 stores individual patient information in the medical knowledge graph. Individual patient information will be described later.

[0024] With the implementation of the display control function 316, the processing circuit 31 displays various information on the display 35. For example, the processing circuit 31 displays medical judgment information for the target patient and visualization graphs of the target patient.

[0025] Memory 32 is a storage device such as ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), or integrated circuit memory that stores various types of information. In addition to the above storage devices, memory 32 may also be a drive device that reads and writes various types of information to portable storage media such as CDs (Compact Discs), DVDs (Digital Versatile Discs), flash memory, or semiconductor memory elements. Furthermore, memory 32 may be located in another computer connected to the medical information processing device 3 via a network. For example, memory 32 stores a medical knowledge graph acquired by the acquisition function 311.

[0026] The input interface 33 receives various input operations from the operator, converts the received input operations into electrical signals, and outputs them to the processing circuit 31. Specifically, the input interface 33 is connected to input devices such as a mouse, keyboard, trackball, switch, button, joystick, touchpad, and touch panel display. The input interface 33 outputs electrical signals to the processing circuit 31 corresponding to the input operations to the input device. Furthermore, the input device connected to the input interface 33 may be an input device provided on another computer connected via a network or the like.

[0027] The communication interface 34 is an interface for sending and receiving various types of information between the medical information system 100 and other computers, such as the medical information storage device 1, the medical knowledge graph storage device 2, the medical information learning device 4, and the medical information display device 5.

[0028] The display 35 displays various information according to the display control function 316 of the processing circuit 31. As the display 35, for example, a liquid crystal display (LCD), a cathode ray tube (CRT) display, an organic electroluminescent display (OELD), a plasma display, or any other display can be used as appropriate. Alternatively, a projector may be provided instead of or in conjunction with the display 35.

[0029] Next, the medical information processing performed by the processing circuit 31 according to the medical information processing program will be described. In the following embodiment, the medical judgment information is assumed to be disease classification information.

[0030] Figure 4 is a diagram illustrating the flow of medical information processing. As shown in Figure 4, the processing circuit 31 acquires the patient's medical event information and medical knowledge graph by realizing the acquisition function 311 (step SA1). The medical knowledge graph is acquired from the medical knowledge graph storage device 2. The medical event information is information that has a combination of the type of medical event and a value indicating the degree to which the patient is relevant to that medical event. The degree of relevance is typically represented by two values, either applicable or not applicable, but it may also be defined by three or more discrete or continuous values ​​indicating the degree of relevance. In the following embodiment, the degree of relevance in the medical event information acquired in step SA1 is represented by a binary value. The medical event information is acquired, for example, as follows.

[0031] First, the processing circuit 31 obtains the medical information history of the target patient from the medical information storage device 1. Next, the processing circuit 31 obtains medical event information from the medical information history. For example, the processing circuit 31 applies information processing such as document search using medical events registered in the medical knowledge graph as search keys to the medical information history of the target patient, and determines whether or not the medical events registered in the medical knowledge graph are included in the medical information history. Then, the processing circuit 31 generates medical event information by assigning a value to each medical event, i.e., a value "1" indicating that it is included in the history, or a value "0" indicating that it is not included in the history. Note that the medical event information may be stored in the medical information storage device 1, and the medical event information may be obtained from the medical information storage device 1 in step SA1. Note that in step SA1, medical event information for the period corresponding to the period to be considered when estimating disease classification information is obtained. Also, the medical event information obtained in step SA1 does not have to include medical event information for all categories, and it is acceptable if medical event information for some categories is not included.

[0032] When step SA1 is performed, the processing circuit 31, by realizing the mapping function 312, maps the medical event information acquired in step SA1 onto a medical knowledge graph to generate a patient graph for the target patient (step SA2).

[0033] Figure 5 shows an example of the mapping process related to step SA2. As shown in Figure 5, the medical event information includes information indicating whether each medical event is applicable or not. Medical events that are indicated as applicable or not in the medical event information 11 correspond to each node that makes up the medical knowledge graph. If the medical event occurs in the target patient, a value of "1" is assigned to indicate that it is "applicable," and if the medical event does not occur in the target patient, a value of "0" is assigned to indicate that it is "not applicable." For example, the medical event information 11 is given in the format that the medical event "cardiac dilation" is applicable "1," the medical event "edema" is applicable "1," and the medical event "palpitations" is not applicable "0."

[0034] As described above, the medical knowledge graph 20A is composed of multiple nodes 21 and edges 22 connecting the nodes 21. Nodes 21 and edges 22 are also called graph components. Nodes 21 correspond to predefined medical events. Medical events are classified into four categories: symptoms, findings, treatment, and response, and each node 21 is assigned a specific medical event. Edges 22 represent the relationships between medical events. If two medical events are related, the two nodes 21 corresponding to those two medical events are connected by an edge 22. If two medical events are not related, the two nodes 21 corresponding to those two medical events are not connected by an edge 22.

[0035] Edge 22 can be described by the adjacency matrix A shown in equation (1) below. Each row and column corresponds to each node 21. For example, the matrix element at row number "1" and column number "0" is "0", indicating that there is an edge connecting the first node and the second node. Diagonal elements with the same row and column numbers are assigned "0" because they represent edges between the same nodes.

[0036]

number

[0037] In this embodiment, edges are represented by an adjacency matrix, but the embodiment is not limited to this. Other representations such as adjacency lists or kernel matrices (Gaussian kernels, linear kernels) that represent the similarity between nodes as the strength of the edges may also be used. Furthermore, the medical knowledge graph in this embodiment is assumed to be an undirected graph in which there is no direction for the edge arrows, i.e., no causal relationships between medical events. However, it may also be a directed graph in which there is a direction for the edge arrows, i.e., causal relationships between medical events.

[0038] As shown in Figure 5, the processing circuit 31 maps the medical event information 11 onto the medical knowledge graph 20A to generate the patient graph 20B. Specifically, each node is assigned a feature (hereinafter referred to as a graph feature). A graph feature has a binary value, for example, indicating whether the medical event corresponding to that node is applicable or not. For example, as shown in Figure 5, the medical event "pleural effusion" is applicable ("1"), so the graph feature of the node corresponding to the medical event "pleural effusion" is assigned "1". The medical event "syncope" is not applicable ("0"), so the graph feature of the node corresponding to the medical event "syncope" is assigned "0". In Figure 5, among the nodes 21 included in the patient graph 20B, nodes assigned a graph feature "1" are represented by colored circles, and nodes assigned a feature "0" are represented by white circles.

[0039] The graph feature X of each node can be represented as a matrix, as shown in equation (2) below. Each element in the column of the graph feature X corresponds to each node. Each element is assigned a value such as "1" (applicable) or "0" (not applicable). As mentioned above, the values ​​of the graph feature X are not limited to binary values. The graph feature X can also be called the matrix representation of the medical event information 11.

[0040]

number

[0041] As will be discussed later, graph features are not limited to whether or not a medical event is applicable, but may also include values ​​indicating the number of occurrences, order of occurrence, and severity of occurrence of medical events.

[0042] In the above embodiment, it was assumed that the medical knowledge graph is common to multiple patients, but this embodiment is not limited to this. For example, the medical knowledge graph storage device 2 may store multiple medical knowledge graphs with different connection relationships to the edge nodes, that is, with different combinations of numerical values ​​for each element included in the adjacency matrix. In this case, the processing circuit 31 selects the medical knowledge graph to be used for the target patient from the multiple medical knowledge graphs. The selection of the medical knowledge graph may be performed manually by the user via the input interface 33, or it may be selected automatically according to an arbitrary algorithm.

[0043] When step SA2 is performed, the processing circuit 31 estimates disease classification information based on the patient graph generated in step SA2 and the trained model by implementing the estimation function 313 (step SA3). The trained model is a machine learning model whose learning parameters have been trained by the medical information learning device 4. The learning parameters are trainable parameters such as weights and biases.

[0044] Figure 6 shows an example of the estimation process related to step SA3. As shown in Figure 6, the processing circuit 31 applies the patient graph 20B to the trained model 60 to estimate disease classification information 71. The trained model 60 is a machine learning model whose learning parameters have been trained to take the patient graph 20B as input and output disease classification information 71.

[0045] As shown in Figure 6, the trained model 60 has a graph convolutional layer 61, a readout layer 62, and a linear combination layer 63. The graph convolutional layer 61 is a graph convolutional network (GCN) that takes a patient graph 20B before convolution as input and outputs a patient graph 20C after convolution. The graph convolutional layer 61 performs a convolution operation on each node. Specifically, the graph convolutional layer 61 performs a convolution operation based on the graph features of the target node before convolution, the graph features of nodes connected to the target node via edges (hereinafter referred to as adjacent nodes), and the weights for the edges connecting the target node and the adjacent nodes, and calculates the graph features of the target node after convolution. The weights for the edges are weight parameters included in the weight matrix of the graph convolutional layer 61. Hereafter, the weights for the edges will be referred to as GCN weights.

[0046] The graph feature vector X' of the node being processed after convolution is calculated using equation (3) below, consisting of the graph feature vector X before convolution, the adjacency matrix A, and the GCN weight W. C It can be expressed as a function Conv based on the following: GCN weight W C is a learning parameter trained by the medical information learning device 4. The graph convolution layer 61 can have any number of layers, one or more. As will be described later, the graph convolution layer 61 may be divided into filters for each medical event category, each direction of causality, and each strength of causality / correlation. The convolution process shown in equation (3), etc., is repeated for the number of layers. By performing the above convolution operation on each node included in the patient graph 20B before convolution, the patient graph 20C after convolution is generated. Note that I is the identity matrix and Λ is the order matrix of the graph.

[0047]

number

[0048] As shown in Figure 6, the graph feature X' after convolution will not be a binary value of "1" or "0", but rather a continuous value representing the degree, such as "0.2" or "0.4".

[0049] The readout layer 62 is a network layer that converts the convolutional patient graph 20C into a feature vector 20D. The feature vector 20D is a single column vector with the same number of dimensions as the number of nodes in the convolutional patient graph 20C. In other words, the readout layer 62 reads the graph features of each node in the convolutional patient graph 20C and converts them into a feature vector 20D.

[0050] The linearly connected layer 63 is a network layer that converts feature vectors 20D into disease classification information 71. The linearly connected layer 63 is also called a multilayer perceptron (MLP). The linearly connected layer 63 performs a classification task or a regression task. The classification task may be a two-class classification that determines the presence or absence of the relevant disease, or a multi-class classification that identifies one disease from multiple disease candidates. It may also be a multi-label classification that allows multiple labels for the same dataset. The regression task outputs a numerical value indicating the probability of belonging to each of the one or more relevant diseases.

[0051] In this embodiment, the linearly coupled layer 63 has a classifier that performs multi-class classification, classifying the feature vector 20D into multiple classes corresponding to multiple diseases. In multi-class classification, a softmax function is calculated to output the probability that the feature vector 20D belongs to each disease (class). This probability of belonging to each disease is output as disease classification information 71. For example, as shown in Figure 6, the probability of belonging to the disease "heart failure", the probability of belonging to the disease "renal failure", the probability of belonging to the disease "COPD", etc., are output as disease classification information 71. In this case, the disease classification information 71 can be represented in matrix notation Y as shown in equation (4) below.

[0052]

number

[0053] The probability Y of each disease among the 71 disease classification information is calculated using the convolutional graph feature X' and disease weight W, as shown in equation (5) below. L It is calculated based on the following: Disease weight W L These are learning parameters trained by the medical information learning device 4. Disease weight W L This is set for each element (node) of the feature vector 20D, for each disease class. That is, disease weight W L This can also be described as a parameter representing the degree of influence of each node on the disease in question. The probability Y of each disease is the disease weight W of the graph feature X'. L This is obtained by applying the activation function σ to the weighted addition operation Linear. The activation function σ is realized by the softmax function or the like, as described above.

[0054]

number

[0055] When step SA3 is performed, the processing circuit 31 displays the disease classification information estimated in step SA3 by implementing the display control function 316 (step SA4). In step SA4, the processing circuit 31 displays the disease classification information on a display screen with a predetermined layout.

[0056] Figure 7 shows an example of the disease classification information display screen I1. As shown in Figure 7, the display screen I1 has a disease classification information display field I11, a patient selection field I12, a disease selection field I13, a medical event category selection field I14, and a patient graph display field I15. The disease classification information estimated in step SA3 is displayed in the disease classification information display field I11. The patient selection field I12 displays the identifier of the target patient (hereinafter referred to as the patient identifier). The patient identifier can be the patient ID or patient name, etc. The patient identifier displayed in the selection field I12 can be selected by the operator via the input interface 33, etc. Information corresponding to the selected patient identifier is displayed on the display screen I1.

[0057] The disease selection field I13 displays the identifier of the target disease (hereinafter referred to as the disease identifier). The disease identifier may be the name or symbol of the disease. The disease identifier displayed in selection field I13 can be selected by the operator via the input interface 33, etc. The probability of the selected disease identifier being present is displayed in display field I11.

[0058] The medical event category selection field I14 displays a list of medical event categories to be displayed. The displayed medical event categories may be the name or symbol of the medical event category, or they may be simulated images or thumbnail images of patient graphs related to the medical event category.

[0059] The display area I15 of the patient graph displays a visualization graph 81 based on either the patient graph before or after convolution. The generation of the visualization graph 81 by the visualization graph generation function 314 and the display of the visualization graph 81 by the display control function 316 will be explained in detail below. When there is no particular distinction between the patient graph before and after convolution, they will simply be referred to as the patient graph.

[0060] First, the processing circuit 31 identifies the medical event category selected in the selection field I14. If no medical event category is selected in the selection field I14 or when initially displaying the visualization graph 81, all medical event categories are identified as the selected medical event category. Next, the processing circuit 31 extracts from the patient graph a graph (hereinafter referred to as a partial patient graph) consisting of nodes belonging to the identified medical event category and edges connecting between the nodes. For example, in FIG. 7, since the symptom category is selected in the selection field I14, a partial patient graph consisting of nodes belonging to the symptom category and edges connecting between the nodes is extracted from the patient graph.

[0061] Next, for each node belonging to the identified medical event category, the processing circuit 31 determines the patient feature quantity and the disease influence degree simultaneously or sequentially. The patient feature quantity is information obtained by scalarizing the graph feature quantity of the target patient for each node. When the graph feature quantity is a scalar value, the patient feature quantity is determined as the scalar value or a value obtained by correcting the scalar value. When the graph feature quantity is a vector, first, the graph feature quantity vector is converted into a scalar value such as a statistical value of the element values of the vector. As the statistical value, for example, it may be set to the average value, median value, maximum value, minimum value, arbitrary quantile, etc. of the element values of the graph feature quantity vector. The patient feature quantity is determined as a scalar value based on the graph feature quantity vector or a value obtained by correcting the scalar value. A machine learning model for calculating a scalar value from the graph feature quantity vector may be used to obtain the scalar value. Note that the patient feature quantity may be determined based on the graph feature quantity X before convolution.

[0062] The disease influence degree is information obtained by quantifying the influence degree of the target disease on each node. The disease influence degree is determined based on the disease weight W L of each node. The disease weight W L is calculated for each disease class for each element (node) of the feature vector as described above. When the linear combination layer 63 is a single layer, the disease influence degree is a value that matches the disease weight W L or the disease weight W LThe value is determined based on the following. If the linear coupling layer 63 is multilayered, first, the disease weight W of each layer is determined. L The contribution rate of each node is calculated based on this. For example, the contribution rate can be calculated by learning the explanatory contribution rate using a global or local explainable model based on the linearly coupled layer 63. The disease impact is determined to be a value that matches the contribution rate or a value based on the contribution rate.

[0063] Next, the processing circuit 31 determines the display format of the node based on the patient features and the disease impact. An example of the display format is described below. As shown in Figure 7, the display color of the node is determined according to the patient features. The display color is determined so that it changes from light red to dark red as the patient features increase. Auxiliary information I16 is displayed to the operator showing the visual relationship between the patient features and the display color. The display size of the node is determined according to the disease impact. The display size is determined so that it changes from small to large as the disease impact increases. Auxiliary information I17 is displayed to the operator showing the visual relationship between the disease impact and the display size.

[0064] The processing circuit 31 then displays the probability of disease occurrence for the target disease in display field I11, and displays a visualization graph 81, which visualizes a partial patient graph, in display field I15. The visualization graph 81 represents a partial patient graph relating to the target patient specified in selection field I12, the target disease selected in selection field I13, and the medical event belonging to the medical event category selected in selection field I14. For example, in Figure 7, the visualization graph 81 relating to the target patient "0001", the target disease "heart failure", and the medical event category "symptoms" is displayed. The processing circuit 31 displays the nodes 82 included in the visualization graph 81 in a display format corresponding to the patient features assigned to the nodes included in the patient graph. The processing circuit 31 also displays the nodes 82 included in the visualization graph 81 in a display format corresponding to the disease impact assigned to the nodes included in the patient graph. Patient features and disease impact are visualized in the node 82. Edges 83 connecting the two related nodes 82 are drawn. Furthermore, each node 82 should be labeled with the name or symbol of the medical event corresponding to that node 82. This will make it possible to understand the meaning of each node.

[0065] A visualization graph is a display object that visualizes a patient graph, representing a series of medical events that occurred in a target patient using nodes connected by edges according to their relationships. Each node is highlighted with patient characteristics and disease impact specific to the target patient. By observing such a visualization graph, users can visually understand which medical events were involved in the onset of the target disease. For example, a user can understand the medical events of "symptoms" related to the target disease "heart failure" and the relationships between these medical events, as well as the degree of patient characteristics and disease impact of each medical event. Furthermore, by observing the visualization graph, users can grasp the essential characteristics of the disease and improve their understanding of the disease mechanism.

[0066] The user can, at any time, instruct the selection field I14 to change the category of medical events to be displayed. If a change in the medical event category is instructed, the processing circuit 31 reconstructs and displays a visualization graph based on the nodes of medical events belonging to the changed medical event category and the partial patient graphs connecting those nodes, in the same manner as described above. If a change in the target disease is instructed in the selection field I13, the processing circuit 31 displays the disease weight W of the changed target disease in the same manner as described above. L Based on this, the disease impact level is recalculated, and node 82 is displayed in a display format corresponding to that disease impact level.

[0067] When step SA4 is performed, the processing circuit 31 stores the individual circumstances information of the target patient in the medical knowledge graph by realizing the storage function 315 (step SA5). The individual circumstances information includes, for example, the graph features before convolution, the graph features after convolution, disease weights, and disease impact for each node. For example, the individual circumstances information is recorded in a database such as a LUT (Look Up Table). Hereinafter, this database will be referred to as the patient individual circumstances DB.

[0068] Figure 8 shows an example of a patient-specific circumstances database. As shown in Figure 8, the patient-specific circumstances database associates individual circumstances information such as graph features before convolution, graph features after convolution, patient features, disease weight, and disease impact with each node included in the medical knowledge graph in a searchable manner. The patient-specific circumstances database also associates the disease probability with the entire patient-specific information of the target patient. The individual circumstances information is calculated by the processing circuit 31 as described above. Note that disease weight and disease impact are calculated for each node for multiple disease classes. For example, node number "1" has graph feature "X1", graph feature after convolution "X1'", patient feature "F1", and disease weight "W" for the first disease label. L 11", disease weight of the second disease label "W LThe following are recorded: "12", ..., the disease impact level of the first disease label "I11", the disease impact level of the second disease label "I12", ..., etc. In addition, the probability of disease occurrence for the first disease label "Y11", the probability of disease occurrence for the second disease label "Y12", etc., are recorded for the entirety of this individual circumstances information. Note that the disease label represents a numerical value indicating applicability to each disease class.

[0069] A patient-specific circumstances database is created for each patient. The patient-specific circumstances database is stored together with the medical knowledge graph by the medical knowledge graph storage device 2. The individual circumstances information recorded in the patient-specific circumstances database may be only a part of the information described above, or it may include other information as well.

[0070] Once step SA5 is performed, the medical information processing by the processing circuit 31 is completed.

[0071] As described above, the medical information processing device 3 according to this embodiment has a processing circuit 31. The processing circuit 31 acquires medical event information of the target patient. The processing circuit 31 maps the medical event information to a medical knowledge graph having nodes corresponding to the medical event information and edges representing the relationships between those nodes to generate a patient graph about the target patient. Based on the patient graph about the target patient, the processing circuit 31 estimates medical judgment information for the target patient.

[0072] A patient graph is a medical knowledge graph that maps the clinical event information of a target patient, and therefore reflects the individual circumstances of each patient. By using such a patient graph, it is possible to estimate medical judgment information, such as the probability of disease occurrence, while considering the individual circumstances of each patient. Furthermore, it becomes possible to provide robust support for medical judgment that is not influenced by specific symptoms or similar patients.

[0073] Next, the medical information learning device 4 according to this embodiment will be described.

[0074] Figure 9 shows an example of the configuration of the medical information learning device 4. As shown in Figure 9, the medical information learning device 4 has a processing circuit 41, a memory 42, an input interface 43, a communication interface 44, and a display 45.

[0075] The processing circuit 41 has processors such as a CPU and a GPU. The processing circuit 41 implements the acquisition function 411, mapping function 412, learning function 413, and display control function 414 by executing a learning program to generate a trained model used by the medical information processing device 3. Note that each of the functions 411 to 414 is not limited to being implemented by a single processing circuit. Multiple independent processors may be combined to form a processing circuit, and each processor may implement each of the functions 411 to 414 by executing a program. Also, each of the functions 411 to 414 may be a modularized program that constitutes the learning program, or it may be an individual program. These programs are stored in memory 42.

[0076] By realizing the acquisition function 411, the processing circuit 41 acquires various types of information. For example, the processing circuit 41 acquires multiple learning samples for multiple patients from the medical information storage device 1. The learning samples correspond to medical information and include medical event information and medical judgment information. The medical event information is used as input samples for machine learning, and the medical judgment information is used as output samples (teaching samples) for machine learning. Note that the medical judgment information included in the learning samples is not medical judgment information estimated by the medical information processing device 3, but rather information regarding medical judgments made by doctors or other medical professionals for the patient in question. The processing circuit 41 also acquires a medical knowledge graph from the medical knowledge graph storage device 2.

[0077] By implementing the mapping function 412, the processing circuit 41 maps the medical information of the learning sample to a medical knowledge graph having nodes corresponding to medical events and edges representing the relationships between those nodes, thereby generating a patient graph related to the learning sample.

[0078] With the implementation of the learning function 413, the processing circuit 41 trains a machine learning model based on the patient graph and medical judgment information to generate a trained model that estimates medical judgment information from the patient graph.

[0079] With the implementation of the display control function 414, the processing circuit 41 displays various information on the display 45. For example, the processing circuit 41 displays the machine learning settings screen, etc.

[0080] Memory 42 is a storage device such as ROM, RAM, HDD, SSD, or integrated circuit memory that stores various types of information. In addition to the above-mentioned storage devices, memory 42 may also be a drive device that reads and writes various types of information to and from portable storage media such as CDs, DVDs, or flash memory, or semiconductor memory elements. Furthermore, memory 42 may be located in another computer connected to the medical information learning device 4 via a network. For example, memory 42 stores medical knowledge graphs acquired by the acquisition function 411.

[0081] The input interface 43 receives various input operations from the operator, converts the received input operations into electrical signals, and outputs them to the processing circuit 41. Specifically, the input interface 43 is connected to input devices such as a mouse, keyboard, trackball, switch, button, joystick, touchpad, and touch panel display. The input interface 43 outputs electrical signals to the processing circuit 41 corresponding to the input operations to the input device. Furthermore, the input device connected to the input interface 43 may be an input device provided on another computer connected via a network or the like.

[0082] The communication interface 44 is an interface for sending and receiving various types of information between the medical information system 100 and other computers, such as the medical information storage device 1, the medical knowledge graph storage device 2, the medical information processing device 3, and the medical information display device 5.

[0083] The display 45 displays various information according to the display control function 414 of the processing circuit 41. For example, a liquid crystal display, CRT display, organic EL display, plasma display, or any other display can be used as the display 45. A projector may also be provided instead of or in conjunction with the display 45.

[0084] Next, we will describe the machine learning process executed by the processing circuit 41 according to the learning program. In the following embodiment, the medical judgment information is assumed to be disease classification information.

[0085] Figure 10 is a diagram illustrating the flow of machine learning processing. As shown in Figure 10, the processing circuit 41 acquires training samples by realizing the acquisition function 411 (step SB1). The training samples include a combination of medical event information and disease information for a single patient. The medical event information is, as described above, medical information relating to medical events that occurred in the patient. The disease information is information relating to the disease that the patient is suffering from and corresponds to the medical event information for the patient. The disease information has a number of dimensions corresponding to the number of disease classes and has a value indicating applicability or non-applicability for each disease class. The disease information is disease classification information to which values ​​have been artificially assigned, and is a type of disease classification information. The value indicating applicability for each disease class is also called a disease label. For the medical event information for the patient, a disease label for one disease may be assigned, or multiple disease labels for two or more diseases may be assigned. For the medical event information for the patient, two or more disease labels may be assigned according to the hierarchical structure in disease classification. The processing circuit 41 may identify disease information based on at least one of the medical event information and the medical ontology.

[0086] When step SB1 is performed, the processing circuit 41 maps the medical event information acquired in step SB1 onto a medical knowledge graph to generate a patient graph (step SB2). Step SB2 is the same as step SA2.

[0087] When step SB2 is performed, the processing circuit 41 updates the learning parameters of the machine learning model based on the patient graph generated in step SB2 and the disease information acquired in step SB1 (step SB3).

[0088] Figure 11 is a schematic diagram illustrating the update process. As shown in Figure 11, the machine learning model 65 is trained based on supervised learning, using the patient graph 20B as input samples and disease information 72 as teaching samples. The machine learning model 65 has a graph convolutional layer 66, a readout layer 67, and a linear combination layer 68. The graph convolutional layer 66 corresponds to the graph convolutional layer 61 of the trained model 60 and is a network layer that performs convolution on the patient graph 20B. The readout layer 67 corresponds to the readout layer 62 of the trained model 60 and is a network layer that converts the patient graph into feature vectors. The linear combination layer 68 corresponds to the linear combination layer 63 of the trained model 60 and is a network layer that converts the feature vectors into disease classification information. Initially, the learning parameters included in the machine learning model 65 are set to arbitrary initial values. The learning parameters are the GCN weights W of the graph convolutional layer 66. C and disease weight W of the linear coupling layer 68 L This includes.

[0089] In the update process, the processing circuit 41 calculates a loss function. The loss function is a function that evaluates the error between the disease classification information, which is calculated by sequentially propagating the patient graph to the graph convolutional layer 66, the readout layer 67, and the linear combination layer 68, and the disease information, which is the teaching sample. The processing circuit 41 updates the learning parameters of the machine learning model 65 to minimize the loss function according to an arbitrary optimization method. Specifically, the learning parameters are the GCN weights W of the graph convolutional layer 66. C and disease weight W of the linear coupling layer 68 L This is updated. Any optimization method can be used, such as stochastic gradient descent or Adam (adaptive moment estimation).

[0090] When step SB3 is performed, the processing circuit 41 determines whether or not the stop condition is met (step SB4). The stop condition may be set to, for example, that the number of learning parameter updates has reached a predetermined number or that the amount of learning parameter updates is less than a threshold. If it is determined that the stop condition is not met (step SB4: NO), the processing circuit 41 acquires another learning sample (step SB1). The processing circuit 41 then similarly performs the patient graph generation (step SB2), learning parameter update (step SB3), and determination of whether the stop condition is met (step SB4) for each learning sample in the same order. In this way, the processing circuit 41 repeats the patient graph generation (step SB2), learning parameter update (step SB3), and determination of whether the stop condition is met (step SB4) for each learning sample until the stop condition is met.

[0091] Then, if it is determined in step SB4 that the stop condition is met (step SB4: YES), the processing circuit 41 outputs the machine learning model at the time the stop condition was met as the trained model (step SB5). The trained model is transmitted to the medical information processing device 3, etc., and stored.

[0092] Once step SB5 is performed, the machine learning processing by the processing circuit 41 is completed.

[0093] In the learning process shown in Figure 10, the learning parameters of the machine learning model are updated for each training sample. However, this embodiment is not limited to this. For example, mini-batch learning may be performed, in which the learning parameters of the machine learning model are updated for each of multiple training samples. Also, if the stopping condition is that the amount of updated learning parameters falls below a threshold, step SB4 may be executed after step SB2, and if it is determined that the stopping condition is not met, step SB3 may be executed.

[0094] As described above, the medical information learning device 4 according to this embodiment has a processing circuit 41. The processing circuit 41 acquires a medical knowledge graph having nodes corresponding to medical events and edges representing the relationships between those nodes, and medical event information, generates a patient graph by mapping the medical event information onto the medical knowledge graph, and trains a machine learning model for estimating medical judgment information from the patient graph based on the patient graph and medical judgment information corresponding to the medical event information.

[0095] According to the above configuration, it is possible to generate a machine learning model for estimating medical decision information from patient graphs, thereby enabling the estimation of highly accurate medical decision information.

[0096] Next, the medical information display device 5 according to this embodiment will be described.

[0097] Figure 12 shows an example configuration of the medical information display device 5. As shown in Figure 12, the medical information display device 5 has a processing circuit 51, a memory 52, an input interface 53, a communication interface 54, and a display 55.

[0098] The processing circuit 51 has processors such as a CPU and a GPU. The processing circuit 51 implements an acquisition function 511, a selection function 512, a visualization graph generation function 513, and a display control function 514 by executing a display program to display a medical knowledge graph. Note that each of the functions 511 to 514 is not limited to being implemented by a single processing circuit. A processing circuit may be configured by combining multiple independent processors, and each processor may implement each of the functions 511 to 514 by executing a program. Furthermore, each of the functions 511 to 514 may be a modularized program that constitutes the display program, or it may be an individual program. These programs are stored in memory 52.

[0099] By implementing the acquisition function 511, the processing circuit 51 acquires various types of information. For example, the processing circuit 51 acquires the medical knowledge graph and the patient individual circumstances database from the medical knowledge graph storage device 2. As described above, the patient individual circumstances database is a database that associates individual circumstances information such as graph features, patient features, disease weights, and disease impact for each of multiple nodes.

[0100] The selection function 512 enables the processing circuit 51 to select the patient and / or disease to be displayed. The processing circuit 51 may select only the patient, only the disease, or both the patient and the disease. It may also select one patient or multiple patients. Similarly, it may select one disease or multiple diseases.

[0101] With the implementation of the visualization graph generation function 513, the processing circuit 51 generates a visualization graph that visualizes the patient graph according to the patient characteristics and / or disease impact of the patient and / or disease to be displayed, based on the medical knowledge graph.

[0102] With the implementation of the display control function 514, the processing circuit 51 displays various information on the display 55. For example, the processing circuit 51 displays a visualization graph that visualizes the patient graph according to the patient characteristics and / or disease impact of the patient and / or disease being displayed. When a specific patient is selected as the display target, the processing circuit 51 displays the nodes included in the visualization graph in a display format according to the patient characteristics of the specific patient corresponding to that node. When a specific disease is identified as the display target, the processing circuit 51 displays the nodes included in the visualization graph in a display format according to the disease impact of the specific disease corresponding to that node. When a specific combination of patient and disease is identified as the display target, the processing circuit 51 displays the nodes included in the visualization graph in a first display format according to the patient characteristics of the combination corresponding to that node and a second display format according to the disease impact of the combination.

[0103] Memory 52 is a storage device such as ROM, RAM, HDD, SSD, or integrated circuit storage device that stores various types of information. In addition to the above storage devices, memory 52 may also be a drive device that reads and writes various types of information to and from portable storage media such as CDs, DVDs, or flash memory, or semiconductor memory elements. Furthermore, memory 52 may be located in another computer connected to the medical information display device 5 via a network. For example, memory 52 stores a medical knowledge graph acquired by the acquisition function 511 and a patient individual circumstances database.

[0104] The input interface 53 receives various input operations from the operator, converts the received input operations into electrical signals, and outputs them to the processing circuit 51. Specifically, the input interface 53 is connected to input devices such as a mouse, keyboard, trackball, switch, button, joystick, touchpad, and touch panel display. The input interface 53 outputs electrical signals to the processing circuit 51 corresponding to the input operations to the input device. Furthermore, the input device connected to the input interface 53 may be an input device provided on another computer connected via a network or the like.

[0105] The communication interface 54 is an interface for sending and receiving various types of information between the medical information system 100 and other computers, such as the medical information storage device 1, the medical knowledge graph storage device 2, the medical information processing device 3, and the medical information learning device 4.

[0106] The display 55 displays various information according to the display control function 514 of the processing circuit 51. For example, a liquid crystal display, CRT display, organic EL display, plasma display, or any other display can be used as the display 55. A projector may also be provided instead of or in conjunction with the display 55.

[0107] Next, the medical knowledge graph display process, which is executed by the processing circuit 51 according to the display program, will be described. The medical knowledge graph display process is a process for viewing individual circumstances information stored in the medical knowledge graph by the medical information processing device 3 together with the medical knowledge graph.

[0108] Figure 13 shows the flow of the medical knowledge graph display process. At the start of Figure 13, it is assumed that the processing circuit 51 has already acquired the medical knowledge graph and the patient individual circumstances database for multiple patients from the medical knowledge graph storage device 2. The medical knowledge graph and the patient individual circumstances database for multiple patients are stored in memory 52.

[0109] As shown in Figure 13, the processing circuit 51 displays the initial screen by implementing the display control function 514 (step SC1). The initial screen is displayed on the display 55.

[0110] As shown in Figure 14, the initial screen I2 has a patient selection field I12, a disease selection field I13, a medical event category selection field I14, and a patient graph display field I15. In the selection field I12, patient identifiers for multiple patients whose individual circumstances information is stored in the medical knowledge graph are displayed for selection. In addition, patient identifiers representing statistical groups of patients may be displayed for selection in the selection field I12. Various classifications can be used as statistical groups of patients, such as the entire group of patients whose individual circumstances information is stored in the medical knowledge graph, sexual classifications such as male or female among these patients, and age classifications such as 60s or 70s. In Figure 14, "Total" representing the entire group of patients is displayed as an example.

[0111] In selection field I13, disease identifiers for multiple disease types are displayed for selection. Alternatively, selection field I13 may also display disease identifiers representing statistical populations related to disease types. As statistical populations related to disease types, the entire range of diseases that can be classified as disease classification information can be used as appropriate. Furthermore, disease identifiers may be displayed hierarchically. For example, as shown in Figure 14, the upper level "heart failure" and its lower levels "HFrEF" and "HFpEF" are displayed. Since this is the initial screen I2, the patient graph is not displayed in display field I15.

[0112] When step SC1 is performed, the processing circuit 51 selects the patient and / or disease to be displayed by implementing the selection function 512 (step SC2). If a patient is selected, the user selects the patient identifier displayed in the selection field I12 via the input interface 53. If a disease is selected, the user selects the disease identifier displayed in the selection field I13 via the input interface 53. Both a patient and a disease may be selected, or only a patient or only a disease may be selected.

[0113] Furthermore, in step SC2, the processing circuit 51 selects the medical event category to be displayed by implementing the selection function 512. For example, the user can select any medical event category from among the multiple medical event categories displayed in the selection field I14 via the input interface 53. One or more medical event categories may be selected.

[0114] When step SC2 is performed, the processing circuit 51 generates a visualization graph corresponding to the patient and / or disease and medical event category selected in step SC2 by implementing the visualization graph generation function 513 (step SC3). For example, if a patient and disease are selected, the processing circuit 51 reads individual circumstances information from the patient individual circumstances DB corresponding to the selected target patient and assigns it to the medical knowledge graph. Specifically, the processing circuit 51 reads graph features and patient features for each node and assigns them to the node, and also reads disease weights and disease impact levels corresponding to the selected target disease and assigns them to the node. As described above, the graph features and patient features are based on the patient graph before convolution and the patient graph after convolution, and the user can arbitrarily set from among these.

[0115] After assigning individual circumstances information to each node, the processing circuit 51 extracts nodes corresponding to medical events belonging to the selected medical event category and the edges connecting those nodes. If one medical event category is selected, nodes corresponding to medical events belonging to that one medical event category are extracted. If multiple medical event categories are selected, nodes corresponding to medical events belonging to those multiple medical event categories are extracted. Next, the processing circuit 51 extracts a partial patient graph including the extracted nodes and edges, and generates a visualization graph that visualizes the extracted partial patient graph. At this time, the processing circuit 51 sets the display format of each node in the visualization graph according to the individual circumstances information assigned to that node. For example, the display color of a node is set according to patient features or graph features, and the display size is set according to disease weight or disease impact.

[0116] Note that the procedure for generating the visualization graph is not limited to the procedure described above. Alternatively, a partial patient graph corresponding to the medical events belonging to the selected medical event category may be extracted first, and then individual circumstances information may be assigned to the nodes.

[0117] When step SC3 is performed, the processing circuit 51 displays the visualization graph generated in step SC3 by implementing the display control function 514 (step SC4). The visualization graph is displayed on the display 55.

[0118] As shown in Figure 15, the visualization graph generated in step SC3 is displayed in display area I15 of display screen I3. The nodes of the visualization graph are displayed in a display color and size corresponding to the patient characteristics and disease impact of the target patient. Display area I15 may also display area I11, which shows the probability of the target patient having the disease. The probability of the disease having can be read from the target patient's individual circumstances database. As an example, Figure 15 shows a visualization graph for target patient "00001", target disease "HFrEF", and medical event category "Symptoms".

[0119] As described above, the visualization graph is a display object that visualizes a patient graph, representing a series of medical events that occurred in a target patient using nodes connected by edges according to their relationships. Each node is highlighted with patient characteristics and disease impact values ​​unique to the target patient. By observing such a visualization graph, users can visually understand which medical events were involved in the onset of the target disease. Furthermore, by observing the visualization graph, users can grasp the essential characteristics of the disease and promote an understanding of the disease mechanism.

[0120] Once step SC4 is performed, the medical knowledge graph display processing by the processing circuit 51 is completed.

[0121] The above medical knowledge graph display process is merely an example and is not limited to it. For example, in the above embodiment, one patient is selected, but all patients may be selected.

[0122] Figure 16 shows an example of a visualization graph displaying data for all patients. As shown in Figure 16, the display area of ​​screen I4 displays a visualization graph that visualizes data for all patients. An example of the procedure for generating a visualization graph for all patients is as follows.

[0123] Assume that both the entire patient population and specific diseases are selected in step SC2. In this case, the processing circuit 51 reads the patient individual circumstances DB for all patients, calculates statistical values ​​for each node based on the disease impact of the specific disease for all patients, and assigns the calculated statistical values ​​to the node. The processing circuit 51 then displays each node in a display format such as display size according to the assigned statistical values. If the entire patient population is selected, patient features are not assigned to nodes. Therefore, the nodes are displayed in a display format according only to the disease impact. In this case, the auxiliary information I16 showing the relationship between patient features and display color does not need to be displayed.

[0124] Visualization graphs for the entire patient make it possible to understand the overall impact of the disease on the patient in relation to individual medical events. By displaying visualization graphs for the entire patient and visualization graphs for specific patients alternately or in parallel, it is also possible to understand the specificity of the disease impact on each medical event for a particular patient.

[0125] As another option, the entire disease may be selected. Figure 17 shows an example of a visualization graph for the entire disease. As shown in Figure 17, the display area of ​​display screen I5 shows a visualization graph that visualizes the patient graph for the entire disease. An example of the procedure for generating a visualization graph for the entire disease is as follows.

[0126] In step SC2, a specific patient and the disease as a whole are selected. In this case, the processing circuit 51 reads the patient's individual circumstances DB and assigns the patient's characteristics to each node. The processing circuit 51 then displays each node in a display format such as display color according to the assigned patient characteristics. If the disease as a whole is selected, the disease impact is not assigned to the nodes. Therefore, the nodes are displayed in a display format according only to the patient characteristics. If the disease as a whole is selected, it is preferable that the disease probability be displayed alongside each disease identifier in the selection field I13, as shown in Figure 17. For example, it may be displayed as "heart failure 93%", "renal failure 4%", "COPD 3%", etc. The disease probability can be read from the patient's individual circumstances DB.

[0127] Visualization graphs of the entire disease make it possible to understand the patient characteristics of a specific patient across the entire disease in relation to individual clinical events. By displaying visualization graphs of the entire disease and visualization graphs of a specific disease alternately or in parallel, it is also possible to understand the specificity of patient characteristics for each clinical event in that specific disease.

[0128] As described above, the medical information display device 5 according to this embodiment has a processing circuit 51. The processing circuit 51 stores a medical knowledge graph having nodes corresponding to medical events and edges representing the relationships between nodes. Each node in the medical knowledge graph is assigned patient characteristics and / or disease impact for each patient. The processing circuit 51 identifies the patient and / or disease to be displayed. The processing circuit 51 displays a visualization graph that visualizes the medical knowledge graph according to the patient characteristics and / or disease impact for the patient and / or disease to be displayed.

[0129] A visualization graph is a display object that visualizes a patient graph, representing a series of medical events that occurred in a target patient using nodes connected by edges according to their relationships. Each node is highlighted with patient characteristics and disease impact values ​​unique to the target patient. By observing such a visualization graph, users can visually understand which medical events were involved in the onset of the target disease. Furthermore, by observing the visualization graph, users can grasp the essential characteristics of the disease and promote an understanding of the disease mechanism.

[0130] (Examples of application) The following describes various application examples of this embodiment. In the following description, components having substantially the same function as those in this embodiment are denoted by the same reference numerals and described repeatedly only when necessary.

[0131] (Application Example 1) In some of the embodiments described above, each node is assigned a graph feature indicating whether or not a medical event is applicable. However, this embodiment is not limited to this. In Application Example 1, each node is further assigned graph features including, in addition to whether or not a medical event is applicable, the order of occurrence, the number of occurrences, and / or the degree of occurrence. The order of occurrence refers to the order in which the medical event occurred. The number of occurrences is the number of times the medical event occurred. The degree of occurrence is the severity of the medical event. The order of occurrence, the number of occurrences, and / or the degree of occurrence are included in the medical event information. Alternatively, the order of occurrence, the number of occurrences, and / or the degree of occurrence may be calculated by analyzing the medical event information.

[0132] Figure 18 is a conceptual diagram of the assignment of occurrence order, frequency, and severity. As shown in Figure 18, each node in the patient graph is assigned a value as a graph feature that represents the occurrence order, frequency, and severity of a medical event. For example, the occurrence order may be assigned a numerical value that represents the relative occurrence order among multiple medical events. As another example, in order to represent the occurrence order and occurrence interval with a single value, a numerical value may be assigned relative to the dates in between, based on a numerical value indicating the start date (e.g., 0) and a numerical value indicating the end date (e.g., 1) of the period from which the medical events are extracted. For the frequency, for example, a numerical value representing the frequency itself may be assigned. As another example, to account for differences in the frequency of occurrence between medical events, the numerical value of the frequency itself may be normalized using a standard (mean or maximum value) for each medical event. For the severity, for example, if the medical event is a symptom, a numerical value representing the strength of the symptom may be assigned; if the medical event is a test, a numerical value representing the test value may be assigned; if the medical event is drug treatment, a numerical value representing the dosage of the drug may be assigned; if the medical event is a treatment response, a numerical value representing the strength of the treatment response may be assigned. It is not necessary to assign all of the occurrence order, frequency, and severity; one or two of these may be assigned. Although not shown in Figure 18, each node is also assigned a value indicating whether or not a medical event is applicable.

[0133] In Application Example 1, the graph features of each node are given as multidimensional features consisting of whether a medical event is applicable or not, the order of occurrence, the number of occurrences, and / or the degree of occurrence. By training a machine learning model using a patient graph to which such multidimensional graph features are assigned, it becomes possible to generate a trained model that takes into account the order of occurrence, the number of occurrences, and / or the degree of occurrence of medical events. By taking into account the order of occurrence, the number of occurrences, and / or the degree of occurrence of medical events, it is expected that the estimation accuracy of medical judgment information such as the probability of disease applicability will improve.

[0134] (Application Example 2) In the above embodiments, the period of medical event information mapped to the medical knowledge graph was not particularly limited. The processing circuit 41 in Application Example 2 changes the period of medical event information to be mapped according to the type of disease to be predicted. Specifically, multiple machine learning models are prepared according to the type of disease to be predicted. Examples of disease types to be predicted include acute diseases and chronic diseases.

[0135] Figure 19 schematically shows the input and output of two machine learning models 65A and 65B related to Application Example 2. As shown in Figure 19, the machine learning models 65A and 65B related to Application Example 2 include machine learning model 65A for acute diseases and machine learning model 65B for acute diseases. Machine learning model 65A is input with a patient map onto which short-term medical event information 11A is mapped. Short-term medical event information 11 includes medical event information that occurred in a relatively short period of time retrospectively from the disease diagnosis. In Figure 19, medical event categories are represented by the shape of nodes. For example, circles represent symptom categories, triangles represent treatment categories, and squares represent response categories. Machine learning model 65A is trained based on supervised learning using the patient map and disease information for acute diseases. More specifically, the learning parameters of machine learning model 65A are trained based on supervised learning using the patient map as an input sample and disease information for acute diseases as a teaching sample. This generates a trained model that estimates disease classification information for acute illnesses by inputting a patient map onto which short-term medical event information 11A has been mapped.

[0136] Similarly, the machine learning model 65B is input with a patient map onto which long-term clinical event information 11B is mapped. The long-term clinical event information 11B includes clinical event information that occurred over a relatively long period of time, tracing back from the disease diagnosis. The machine learning model 65B is trained based on supervised learning using this patient map and disease information of chronic diseases. More specifically, the learning parameters of the machine learning model 65B are trained based on supervised learning using the patient map as an input sample and disease information of chronic diseases as a teaching sample. This generates a trained model that estimates disease classification information of chronic diseases by inputting a patient map onto which long-term clinical event information 11B is mapped.

[0137] During disease estimation, the processing circuit 31 determines the duration of the medical event information to be mapped according to the type of disease to be classified. For example, if the disease is an acute disease, the processing circuit 31 determines that the duration of the medical event information to be mapped is short, and if the disease is a chronic disease, it determines that the duration of the medical event information to be mapped is long. The processing circuit 31 then extracts medical event information for a duration suitable for each trained model from the history of the patient's medical event information. Specifically, it extracts short-term medical event information from the patient's medical event information history, maps this short-term information to a medical knowledge graph to generate a patient map, and inputs this patient map into a trained model for acute diseases to estimate the disease classification information for acute diseases. Alternatively, the processing circuit 31 extracts long-term medical event information from the patient's medical event information history, maps this long-term information to a medical knowledge graph to generate a patient map, and inputs this patient map into a trained model for chronic diseases to estimate the disease classification information for chronic diseases.

[0138] According to Application Example 2, the time duration of clinical event information mapped onto the medical knowledge graph differs between the machine learning models for acute diseases and those for chronic diseases. This is addressed by generating separate pre-trained models for disease types with different time durations of clinical event information to consider. By using such pre-trained models, it is possible to improve the accuracy of disease classification information estimation.

[0139] (Application Example 3) In some of the above embodiments, the convolution process performed by the graph convolution layer did not distinguish between edge relationship types. The graph convolution layer in Application Example 3 switches parameters such as the GCN weights of the convolution process according to the edge relationship type of the edge to be processed connected to the node to be processed. Specifically, the edge relationship type is the direction of causality, the strength of causality, and / or the strength of correlation between medical events related to the edge to be processed. Alternatively, the edge relationship type may be a combination of the category of medical events of the node to which the edge to be processed is connected and the category of medical events of the adjacent node adjacent to the node to which the edge to be processed is connected. Note that in Application Example 3 shown below, the edge relationship type is assumed to be a combination of medical event categories.

[0140] Figure 20 schematically illustrates the convolution process performed by the graph convolution layer 66 in Application Example 3. As shown in Figure 20, the patient graph 20B contains multiple nodes belonging to multiple medical event categories. The graph convolution layer 66 sequentially switches and executes multiple convolution processes corresponding to each of the multiple medical event categories. This generates the patient graph 20C after convolution. Specifically, the graph convolution layer 66 has multiple filter layers corresponding to each of the multiple convolution processes. Each filter layer has parameters such as GCN weights that are trained for each different medical event category. For example, the first filter layer is trained only for nodes belonging to the findings category, and the second filter layer is trained only for nodes belonging to the symptom category. The calculation results from each filter layer are integrated and output.

[0141] The convolution process that considers edge relationship types is described by the following equation (6) of R-GCN (Relational Graph Convolutional Network). As shown in equation (6), the graph feature X' after convolution of the node being processed is calculated by the sum of the first term and the second term. The first term is the graph feature X of the neighboring node j before convolution. j The weight matrix W before convolution of edge relation type r rIt is the product of the two. More specifically, the first term is the graph feature X, which spans all combinations of edge relation type r and adjacent node j. j and the weight matrix W r and the normalization constant 1 / c i,r It is the sum of the products of . Note that N in equation (6) i r The first term represents the index of the neighboring node of edge relation type r for node i. The second term is the graph feature X of node i before convolution. i It is calculated as the sum of the product of and the weight matrix W0 of the self-loop before convolution. As shown in equation (6), for each edge relation type r, the weight matrix W r The graph feature X is determined. j and the weight matrix W r The product of the two is calculated. Note that the weight matrix W r The weight matrix W0 represents the GCN weights of the graph convolutional layer and is an example of a learning parameter.

[0142]

number

[0143] As described above, a graph convolutional layer having multiple filter layers is designed depending on the edge relationship type. Following the graph convolutional layer, a readout layer and a linear combination layer are sequentially connected, similar to some of the embodiments described above. The multiple filter layers are provided so as to share the subsequent readout layer and linear combination layer. In addition, the multiple filter layers may share a portion of the graph convolutional layer.

[0144] The machine learning model in Application Example 3 is also trained based on supervised learning using patient graphs and disease information, similar to the several examples described above. This makes it possible to generate a trained model with multiple filter layers corresponding to edge relationship types. In addition, the graph convolutional layer in Application Example 3 may have its parameters, such as the GCN weights of the graph convolution process, switched depending on the type of medical judgment information. The types of medical judgment information are disease classification information, prognosis prediction information, and severity classification information, as described above. Furthermore, if the medical judgment information is disease classification information, the parameters, such as the GCN weights of the graph convolutional layer, may be switched depending on the type of disease.

[0145] According to Application Example 3, by dividing the filter layer according to the edge relationship type, it becomes possible to train the learning parameters according to the edge relationship type. This improves the accuracy of graph convolution by the graph convolution layer. Consequently, the accuracy of estimating medical judgment information such as the probability of disease presence is improved.

[0146] (Application Example 4) In some of the embodiments described above, disease classification information was assumed to be estimated as medical judgment information. However, medical judgment information is not limited to disease classification information; it may also be prognostic information such as patient survival time, or severity classification information such as cancer staging. Furthermore, medical judgment information may be a combination of at least two types of information: disease classification information, prognostic information, and severity classification information.

[0147] Figure 21 shows an example of the input and output of linearly coupled layers 68A and 68B related to Application Example 4. As shown in Figure 21, the machine learning model related to Application Example 4 has two linearly coupled layers 68A and 68B. The first linearly coupled layer 68A takes a feature vector 20D output from a preceding readout layer (not shown) as input and outputs disease classification information. The second linearly coupled layer 68B takes the same feature vector 20D as input and outputs prognosis prediction information.

[0148] In the machine learning process, a machine learning model having a graph convolutional layer 66, a readout layer 67, a first linear combination layer 68A, and a second linear combination layer 68B is trained based on multitask learning, with the first linear combination layer 68A receiving disease classification information as teaching samples, the second linear combination layer 68B receiving prognosis prediction information as teaching samples, and a patient graph (not shown) as the input sample. Through multitask learning, the first linear combination layer 68A is trained to output disease classification information by taking a feature vector 20D as input, and the learning parameters such as disease weights are trained to output prognosis prediction information by taking a feature vector 20D as input. Thus, a trained model is generated that takes a patient graph as input and outputs disease classification information and prognosis prediction information.

[0149] According to Application Example 4, a machine learning model is trained through multitasking learning based on a patient graph and two or more pieces of information from disease classification information, prognosis prediction information, and severity classification information. This makes it possible to estimate two or more types of medical judgment information from a single patient graph. Therefore, the usefulness of the trained model is improved.

[0150] (Application Example 5) The individual elements of application examples 1 to 4 above can be combined in any way. The machine learning model related to application example 5 is constructed by combining the individual elements of application examples 1 to 4 above.

[0151] Figure 22 schematically shows a machine learning model related to Application Example 5. As shown in Figure 22, the trained model has a machine learning model for acute diseases and a machine learning model for chronic diseases. Each machine learning model has a different period of medical event information mapped to the input patient graph. The machine learning model for acute diseases is input with a patient graph that maps short-term medical event information. The machine learning model for chronic diseases is input with a patient graph that maps long-term medical event information. The patient graph is assigned graph features such as whether or not the disease is present, as well as the order of occurrence, number of occurrences, and severity of occurrence of medical events. The graph convolutional layer of each machine learning model has different filter layers for each edge relationship type. Each model has two linearly coupled layers. The first linearly coupled layer takes a feature vector as input and outputs disease classification information, and the second linearly coupled layer takes a feature vector as input and outputs prognosis prediction information.

[0152] The machine learning model in Application Example 5 is trained based on multi-task learning, where the training samples for the first linear coupled layer are disease classification information, and the training samples for the second linear coupled layer are prognosis prediction information. This makes it possible to generate a trained model that outputs disease classification information and prognosis prediction information while performing graph convolution according to the edge relationship type for both acute and chronic diseases.

[0153] (Application Example 6) Figure 23 is a diagram illustrating the overview of graph features related to Application Example 6. As shown in Figure 23, each node in the patient graph 20B is assigned clinical event information such as applicable / unapplicable features, temporal features, and / or spatial features as graph features. Applicable / unapplicable features represent the degree to which a clinical event corresponds to or does not correspond to the clinical event corresponding to that node. Temporal features are information about the time when the clinical event occurred. Specifically, temporal features include the date and time of occurrence, the order of occurrence, and the number of occurrences. Spatial features are information about the location where the clinical event occurred. Specifically, spatial information includes the location and site of occurrence of the clinical event. The location of occurrence is sensor information such as GPS (Global Positioning System) that represents the point where the clinical event occurred. The site of occurrence is the address and name of the medical institution, department, home, or hospital where the clinical event was diagnosed.

[0154] In the following example, the graph features related to Application Example 6 shall include applicable / unapplicable features, temporal features, and spatial features.

[0155] The processing circuit 31 in Application Example 6 generates a patient graph 20B for the target patient by mapping medical event information onto a medical knowledge graph through the implementation of a mapping function 312. Here, the medical event information includes applicable / inapplicable features, temporal features, and spatial features for each medical event. As a result, a patient graph 20B is generated in which applicable / inapplicable features, temporal features, and spatial features are assigned to each node.

[0156] Subsequently, the processing circuit 31, through the implementation of the estimation function 313, applies the patient graph 20B, which has nodes assigned with applicable / inapplicable features, temporal features, and spatial features, to the trained model 60 to estimate medical judgment information such as disease classification information 71. According to application example 6, it becomes possible to estimate medical judgment information by considering the applicable / inapplicable features, temporal features, and spatial features of each medical event. Note that each node does not need to be assigned all of the applicable / inapplicable features, temporal features, and spatial features; it may be assigned only two or one of these types.

[0157] (Application Example 7) Figure 24 is a diagram illustrating the overview of graph features related to Application Example 7. As shown in Figure 24, each node in the patient graph 20B is assigned time-series medical event information, consisting of multiple medical event information items with different occurrence times, as a graph feature. The values ​​of the medical event information change over time. For example, whether a medical event is applicable or not changes over time. Occurrence time information is assigned as metadata to the medical event information. Occurrence time information is a timestamp of the medical event information, and for example, the date and time of diagnosis or the date and time of recording of the medical event is assigned. For example, as shown in Figure 24, each node is assigned medical event information at time t, medical event information at time t+1, and medical event information at time t+2 as graph features.

[0158] The processing circuit 31 in Application Example 7 estimates medical judgment information based on time-series graph features.

[0159] Figure 25 shows an example of the estimation process for medical judgment information related to Application Example 7. As shown in Figure 25, the trained model 60A related to Application Example 7 has a graph convolutional layer 61, a readout layer 62, and an RNN (Recurrent Neural Network) layer 64. The RNN layer 64 outputs disease classification information 71 from time-series graph features 20D.

[0160] As shown in Figure 25, the patient graphs to be processed are patient graph 20B0 assigned graph features at time t, patient graph 20B1 assigned graph features at time t+1, and patient graph 20B2 assigned graph features at time t+2. The processing circuit 31 inputs patient graphs 20B0, 20B1, and 20B2 to the graph convolution layer 61 and outputs the convolutional patient graphs corresponding to patient graph 20B0, patient graph 20B1, and patient graph 20B2. Each convolutional patient graph is then input to the readout layer 62 to output the feature vector 20D0 at time t, the feature vector 20D1 at time t+1, and the feature vector 20D2 at time t+2. Feature vector 20D0 is the vector representation of the graph features at time t, feature vector 20D1 is the vector representation of the graph features at time t+1, and feature vector 20D2 is the vector representation of the graph features at time t+2.

[0161] As shown in Figure 25, the processing circuit 31 inputs feature vectors 20D0, 20D1, and 20D2 to the RNN layer 64 and outputs a single disease classification information 71. By utilizing the RNN layer 64, it becomes possible to obtain the probability of various diseases by using sequential medical event information from multiple time points.

[0162] (Application Example 8) In Application Example 7, time-series graph features are assigned to each node. In Application Example 8, the patient graph itself changes over time.

[0163] The processing circuit 41 of the medical information learning device 4 in Application Example 8 continuously acquires learning samples (medical information) with different occurrence times, even after a trained model has been generated. As described above, the learning samples are a combination of medical event information and disease information. Based on the multiple acquired medical information, the processing circuit 41 continuously learns the learning parameters of the machine learning model periodically or irregularly. Continuous learning generates trained models with different occurrence times in time series. The learning parameters include the adjacency matrix and / or GCN weights. The patient graph before convolution at each time point is applied to the trained model at each time point, generating the patient graph after convolution at each time point. This makes it possible to generate a time-series patient graph in which the graph features of each node and the connection relationships between nodes change over time.

[0164] The pre-trained model for each occurrence time that constitutes the time-series pre-trained model is trained according to the processing procedure shown in Figure 10 above, based on the training samples acquired up to that time. In this case, in order to suppress large changes in the characteristics of the patient graph before and after updating the adjacency matrix A and / or GCN weights W, a regularization term may be added to the loss function as shown in equations (7) and (8) below. Equation (7) is the normalization term L related to the update of the adjacency matrix A. reg This is expressed by the Kullback-Leibler divergence between the adjacency matrix A at the processing reference time and the adjacency matrix A' at other time points. Equation (8) is the GCN weight W (l) Normalization term L related to updates reg The GCN weight W is the processing reference time. (l) and the GCN weights W' at other time points (l) It is represented by the Kullback-Leibler divergence between [the two points].

[0165]

number

[0166] (Application Example 9) In some of the above embodiments, spatial information, such as location features, is assigned to each node of the patient graph as a graph feature. In Application Example 9, the concept of space is added to the patient graph itself.

[0167] Figure 26 schematically illustrates the relationship between patient graphs and spatial information in Application Example 9. As shown in Figure 26, spatial information concerning the patient in question is assigned to each patient graph. Spatial information includes both locational and biological information. Locational information may be sensor information such as GPS indicating the patient's current location, or it may be the location of the medical institution the patient visits, the patient's home address, or the hospital room where the patient is hospitalized. Biological information includes the patient's kinship, medical history, and gene sequence information. By embedding spatial information in patient graphs, it is possible to construct a network (hereinafter referred to as a patient graph network) in which multiple patient graphs are arranged according to the presence or degree of spatial information relationships.

[0168] Figure 27 illustrates the concept of the patient graph network 200. As shown in Figure 27, the patient graph network 200 is composed of multiple patient graphs linked together according to the presence or degree of spatial information relationships. Figure 27 illustrates four patient graphs 201, 202, 203, and 204, but the number of patient graphs is not particularly limited as long as there are two or more.

[0169] Patient graphs connected by edges (for example, patient graph 201 and patient graph 202) have a spatial information relationship with each other. Conversely, patient graphs not connected by edges (for example, patient graph 201 and patient graph 204) do not have a spatial information relationship with each other. The distance between patient graphs connected by edges represents the degree of the spatial information relationship. The processing circuit 31 can evaluate the degree of the spatial information relationship based on location information, for example, the home addresses of both individuals or the distance between their hospital rooms. The processing circuit 31 can evaluate the degree of the spatial information relationship based on biological information, for example, the degree of kinship between the two individuals, the medical relationship of their medical histories, and the gene sequence agreement rate. The processing circuit 31 can determine the presence or absence of a spatial information relationship based on a comparison between the degree evaluated by the above method and a threshold. Edges may be formed based on an overall evaluation of the above elements of spatial information, or they may be formed for each element.

[0170] By assigning spatial information to patient graphs, it becomes possible to easily search for patient graphs whose spatial information is close to that of the target patient. For example, the processing circuit 31 can extract patient graphs connected to the target patient's patient graph by edges as patient graphs of patients whose spatial information is close to that of the target patient.

[0171] Furthermore, spatial information is not limited to including both locational information and biological information; it may include only one of them.

[0172] (Application Example 10) Application Example 10 utilizes the spatial information related to Application Example 9. The processing circuit 31 of the medical information processing device 3 related to Application Example 10 convolves the graph features of the patient graph of a patient different from the target patient onto the patient graph of the target patient.

[0173] Figure 28 is a diagram illustrating the overview of graph features related to Application Example 10. As shown in Figure 28, each node in the patient graph 20B is assigned clinical event information such as applicable / inapplicable features, temporal features, spatial features, and / or spatially nearby patient features as graph features. Applicable / inapplicable features, temporal features, and spatial features are as explained in Application Example 6. Spatially nearby patient features are clinical event information of other patients whose spatial information is close to the target patient. For example, the node for the clinical event "headache" will be assigned applicable / inapplicable features, temporal features, and spatial features of the target patient, as well as applicable / inapplicable features, temporal features, and spatial features of the target patient's father and / or mother.

[0174] The processing circuit 31 related to Application Example 10 maps the medical event information related to Application Example 10 onto a medical knowledge graph by implementing the mapping function 312, thereby generating a patient graph 20B for the target patient. The medical event information related to Application Example 10 includes applicable / inapplicable features, temporal features, spatial features, and spatially nearby patient features for each medical event. The applicable / inapplicable features, temporal features, and spatial features are the medical event information for the target patient. The spatially nearby patient features are the medical event information for other patients whose spatial information is close to the target patient. In other words, the processing circuit 31 related to Application Example 10 maps the medical event information of the target patient and the medical event information of other patients as node features to each node by implementing the mapping function 312. By implementing the mapping function 312, a patient graph 20B is generated in which applicable / inapplicable features, temporal features, spatial features, and spatially nearby patient features are assigned to each node.

[0175] Subsequently, the processing circuit 31, through the implementation of the estimation function 313, applies the patient graph 20B, which has nodes assigned with applicable / inapplicable features, temporal features, spatial features, and spatially nearby patient features, to the trained model 60 to estimate medical judgment information such as disease classification information 71. According to application example 10, it becomes possible to estimate medical judgment information by considering the applicable / inapplicable features, temporal features, spatial features, and spatially nearby patient features of each medical event. This makes it possible to estimate medical judgment information that considers not only the medical information (graph features) of the target patient but also the medical information (graph features) of patients whose spatial information is close to the target patient. Note that each node does not need to be assigned all of the applicable / inapplicable features, temporal features, spatial features, and spatially nearby patient features; it may be assigned spatially nearby patient features and any two or one of the applicable / inapplicable features, temporal features, and spatial features.

[0176] (Application Example 11) In some of the embodiments described above, the processing circuit 31 displays a visualization graph based on a partial patient graph, which includes only nodes and edges belonging to some of the medical event categories, including symptoms, physical findings, laboratory findings, treatment, treatment response, and side effects. However, as shown in Figure 2, the processing circuit 31 may also display a visualization graph based on the entire patient graph, which covers all medical event categories. In this case, the processing circuit 31 may display the visualization graph in a way that allows each category to be distinguished by visual effects such as color. This makes it possible to get an overview of the entire patient graph.

[0177] (Other examples) Several further embodiments can be added to or replaced with the above embodiments. For example, transfer learning may be performed on a machine learning model. First, the learning parameters of a machine learning model that estimates disease classification information for a first disease (e.g., acute disease) may be trained, and some of these learning parameters may be set in a machine learning model that estimates disease classification information for a second disease (e.g., chronic disease), and the remaining learning parameters may be trained. This is useful, for example, when there are few training samples for the second disease. The second disease to which transfer learning is performed should preferably have some medical commonality or similarity with the first disease from which the transfer originated.

[0178] Regularization may be considered in machine learning processing. This can be achieved by adding an arbitrary regularization term to the loss function. Suitable regularization terms to be added include, for example, a penalty term that penalizes the magnitude of the norm of the learning parameters, or a penalty term that penalizes an index of the independence of network parameters between diseases.

[0179] In the machine learning process described above, the adjacency matrix that defines the relationships between edges was not treated as a learning parameter. However, this embodiment is not limited to this, and the adjacency matrix may also be trained as a learning parameter in the machine learning process.

[0180] The above machine learning model may be equipped with additional modules such as an attention mechanism. For example, an attention mechanism may be provided in parallel between the output layer of the readout layer and the input layer of the linear combination layer. The attention mechanism outputs an attention mask indicating the degree of emphasis for each node. The linear combination layer estimates medical judgment information based on the attention mask and feature vectors. By providing an attention mechanism, the feature portions of the patient graph that are useful for estimating medical judgment information are emphasized, thereby improving the accuracy of estimating medical judgment information.

[0181] In the above embodiment, graph convolutional layers such as GCN and R-GCN were used as the model for processing the patient graph. However, this embodiment is not limited to this, and Boltzmann machines or their applications using Markov random fields (MRF) or conditional random fields (CRF) may also be used as the model for processing the patient graph.

[0182] The teaching sample was assumed to be disease information obtained from clinical information or disease information obtained from a medical ontology. However, the teaching sample may contain both disease information obtained from clinical information and disease information obtained from a medical ontology. In this case, the machine learning model may output a multi-label classification that outputs disease information obtained from clinical information and disease information obtained from a medical ontology.

[0183] According to at least one embodiment described above, the accuracy of medical judgments can be improved.

[0184] In the above description, the term "processor" refers to circuits such as CPUs, GPUs, Application Specific Integrated Circuits (ASICs), and programmable logic devices (e.g., Simple Programmable Logic Devices (SPLDs), Complex Programmable Logic Devices (CPLDs), and Field Programmable Gate Arrays (FPGAs)). The processor achieves its function by reading and executing a program stored in a memory circuit. Alternatively, instead of storing the program in a memory circuit, the processor may be configured to directly incorporate the program into its circuitry. In this case, the processor achieves its function by reading and executing the program incorporated into the circuitry. Furthermore, instead of executing a program, the processor may achieve the function corresponding to the program through a combination of logic circuits. In this embodiment, each processor is not limited to being configured as a single circuit; multiple independent circuits may be combined to form a single processor and achieve its function. Moreover, multiple components shown in Figures 1, 3, 9, and 12 may be integrated into a single processor to achieve its function.

[0185] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0186] 1 Medical information storage device 2. Medical Knowledge Graph Storage Device 3. Medical Information Processing Device 4. Medical Information Learning Device 5 Medical information display device 31 Processing Circuit 32 memory 33 Input Interfaces 34 Communication Interfaces 35 displays 41 Processing Circuit 42 memory 43 Input Interfaces 44 Communication Interfaces 45 displays 51 Processing Circuit 52 memory 53 Input Interfaces 54 Communication Interfaces 55 displays

Claims

1. An acquisition unit that acquires medical information regarding the medical events of the target patient, A mapping unit generates a second graph relating to the target patient by mapping the medical information onto a first graph having nodes corresponding to medical events and edges representing the relationships between those nodes. An estimation unit that estimates the medical judgment information for the target patient based on the second graph relating to the target patient, using a trained model which is a machine learning model trained to take the second graph as input and output medical judgment information, A medical information processing device equipped with the following features.

2. The medical information processing apparatus according to claim 1, wherein the medical event includes an event belonging to at least one of the following categories: symptoms, physical findings, laboratory findings, treatment, treatment response, and side effects.

3. The medical information processing apparatus according to claim 1, wherein the trained model comprises a graph convolution layer that performs a convolution operation on the second graph to output a third graph, a readout layer that converts the third graph into feature vectors, and a linear combination layer that converts the feature vectors into medical judgment information.

4. The graph convolution layer calculates a post-convolution feature for each of the multiple nodes included in the second graph, based on the pre-convolution feature quantities relating to the target node and its neighboring nodes, the adjacency matrix representing the edges connecting the target node and its neighboring nodes, and the weights for the edges. The readout layer converts the convolutional feature quantities of each of the plurality of nodes into the feature vector. The medical information processing device according to claim 3.

5. The medical information processing apparatus according to claim 3, further comprising a display control unit that displays a visualization graph in which the second graph or the third graph relating to the subject patient is visualized.

6. The medical information processing apparatus according to claim 5, wherein the display control unit displays the nodes included in the visualization graph in a display format corresponding to the patient feature quantities assigned to the nodes included in the second graph or the third graph.

7. The medical information processing apparatus according to claim 5, wherein the display control unit displays the node in a display format corresponding to the degree of disease impact assigned to the node that corresponds to the node and is included in the second graph or the third graph.

8. The medical information processing apparatus according to claim 5, wherein the display control unit extracts a subgraph from the second graph or the third graph that includes nodes to be displayed and edges connecting the nodes to be displayed, and displays a visualization graph that visualizes the subgraph.

9. The aforementioned node is classified into at least two categories from symptoms, physical findings, laboratory findings, treatment, treatment response, and side effects. The display control unit extracts a subgraph from the second graph or the third graph that includes nodes belonging to the category to be displayed and edges connecting those nodes, and displays a visualization graph that visualizes the subgraph. The medical information processing device according to claim 5.

10. The medical information processing apparatus according to claim 5, wherein the display control unit assigns the name or symbol of the medical event corresponding to the node to the node included in the visualization graph.

11. The graph convolution layer switches the parameters of the convolution process according to the edge relationship type of the edge to be processed connected to the node to be processed. The edge relationship type is the direction of causality, the strength of causality, and / or the strength of correlation between medical events relating to the edge to be processed. The medical information processing device according to claim 3.

12. The graph convolution layer switches the parameters of the convolution process according to the edge relationship type of the edge to be processed connected to the node to be processed. The edge relationship type is a combination of the category of medical events of the processing target node to which the processing target edge is connected and the category of medical events of the adjacent node adjacent to the processing target node. The medical information processing device according to claim 3.

13. The medical information processing apparatus according to claim 3, wherein the graph convolution layer switches the parameters of the convolution process according to the type of medical judgment information.

14. The medical information processing apparatus according to claim 3, wherein the graph convolution layer switches the parameters of the convolution process according to the type of disease among the medical judgment information.

15. The trained model comprises a trained model for acute diseases that outputs disease types classified as acute diseases as medical judgment information, and a trained model for chronic diseases that outputs disease types classified as chronic diseases as medical judgment information, The medical information processing apparatus according to claim 1, wherein the mapping unit determines the period of the medical information to be mapped, depending on whether it is using a trained model for acute diseases to estimate disease types classified as acute diseases or using a trained model for chronic diseases to estimate disease types classified as chronic diseases.

16. The medical information includes the sequence of occurrence, the number of occurrences, and / or the severity of the medical events. The mapping unit maps the medical information to the nodes as node features. The medical information processing device according to claim 1.

17. The aforementioned medical information includes spatial and / or temporal information relating to the aforementioned medical event. The mapping unit maps the spatial information and / or temporal information to the node as node features, The aforementioned spatial information is information relating to the location where the medical event occurred. The aforementioned temporal information is information relating to the time when the medical event occurred. The medical information processing device according to claim 1.

18. The estimation unit estimates the medical judgment information for the target patient from the plurality of second graphs using a trained model that includes a graph convolution layer that outputs a second graph after convolution of a plurality of different time periods from a plurality of second graphs to which graph features of a plurality of different time periods are assigned, and a recurrent neural network layer that outputs the medical judgment information from a plurality of different time feature vectors corresponding to each of the second graphs after convolution of a plurality of different time periods. The medical information processing device according to claim 17.

19. The medical information includes the medical information of the target patient and the medical information of other patients whose spatial information is adjacent to the target patient. The spatial information includes the location information and / or biological information of the other patient. The mapping unit maps the medical information of the target patient and the medical information of the other patient to the node as node features. The medical information processing device according to claim 1.

20. The medical information processing device according to claim 1, wherein the estimation unit estimates at least one piece of information corresponding to the second graph relating to the target patient, namely disease classification information, prognosis prediction information, and severity classification information, as the medical judgment information.

21. The system further includes a display control unit that displays the second graph relating to the target patient, The aforementioned medical events include events belonging to at least one of the following categories: symptoms, physical findings, laboratory findings, treatment, treatment response, and adverse reactions. The display control unit displays the second graph in a way that allows the category to be identified. The medical information processing device according to claim 1.

22. The medical information processing apparatus according to claim 1, wherein the first graph is a graph generated based on the medical information or medical ontology of multiple patients.

23. We obtain medical information regarding the medical events of the target patient. A second graph relating to the target patient is generated by mapping the medical information onto a first graph having nodes corresponding to medical events and edges representing the relationships between those nodes. Using a trained model, which is a machine learning model trained to take the second graph as input and output medical judgment information, the medical judgment information for the target patient is estimated based on the second graph relating to the target patient. A computer-implemented medical information processing method that includes the following features.

Images