Information processing device, information processing method, and program
By using a link prediction model to verify the truth of relationships between entities extracted from natural language data, the method addresses inaccuracies in existing knowledge graph construction, resulting in a highly accurate knowledge graph.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- NEC CORP
- Filing Date
- 2024-11-20
- Publication Date
- 2026-06-01
AI Technical Summary
Existing methods for constructing knowledge graphs from natural language data, such as those described in Patent Document 1, often result in inaccuracies due to hallucinations generated by large language models, leading to knowledge graphs that do not accurately reflect the facts in the original data.
A process that includes extracting entities and relationships from natural language data using a link prediction model to determine the truth or falsehood of these relationships, adding only true relationships to the knowledge graph while excluding false ones.
This approach enables the construction of a highly accurate knowledge graph that conforms to the facts in the original data, improving the reliability and accuracy of the knowledge representation.
Smart Images
Figure 2026089316000001_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to a technique for constructing a knowledge graph.
Background Art
[0002] Knowledge graphs for representing knowledge in various fields have been developed and utilized. A knowledge graph is constructed, for example, by extracting entities and relationships between entities from text data. Patent Document 1 describes a method for extracting relationships between entities from electronic medical records and generating a set of entities and links between entities.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] However, even when using the method described in Patent Document 1, it is not always possible to construct a highly accurate knowledge graph.
[0005] One object of this disclosure is to provide an information processing apparatus capable of constructing a highly accurate knowledge graph.
Means for Solving the Problems
[0006] In one aspect of this disclosure, the information processing apparatus includes extraction means for extracting entities and relationships between entities from natural language data, and determination means for determining the truth or falsehood between the entities by predicting the relationships between the entities using a link prediction model. A graph construction means that adds relationships between entities determined to be true by the determination means to the knowledge graph, and does not add relationships between entities determined to be false by the determination means to the knowledge graph, It is equipped with.
[0007] From another perspective of this disclosure, the information processing method is: A method of information processing performed by a computer, An extraction process is performed to extract entities and relationships between entities from natural language data. By predicting the relationships between the entities using a link prediction model, a determination process is performed to determine the truth value of the relationships between the entities. The system performs a graph construction process that adds relationships between entities determined to be true by the aforementioned determination process to the knowledge graph, and does not add relationships between entities determined to be false by the aforementioned determination process to the knowledge graph.
[0008] In yet another aspect of this disclosure, the program is An extraction process that extracts entities and relationships between entities from natural language data, A determination process is performed to determine the truth value of the relationships between the entities by predicting the relationships between the entities using a link prediction model, The computer is instructed to perform a graph construction process that adds the relationships between entities determined to be true by the aforementioned determination process to the knowledge graph, and does not add the relationships between entities determined to be false by the aforementioned determination process to the knowledge graph. [Effects of the Invention]
[0009] This disclosure makes it possible to provide an information processing device capable of constructing a highly accurate knowledge graph. [Brief explanation of the drawing]
[0010] [Figure 1] This is a diagram conceptually illustrating the information processing device related to this disclosure. [Figure 2]It is a block diagram showing the hardware configuration of an information processing apparatus according to the present disclosure. [Figure 3] It is a block diagram showing the functional configuration of an information processing apparatus according to the present disclosure. [Figure 4] It is an example of natural language data. [Figure 5] It is an example of an extracted entity. [Figure 6] It is an example of a prompt by the relationship extraction unit. [Figure 7] It is an example of an answer from the LLM. [Figure 8] It is a flowchart of processing by the information processing apparatus according to the present disclosure. [Figure 9] It is a block diagram showing the functional configuration of another information processing apparatus according to the present disclosure. [Figure 10] It is a flowchart of processing by another information processing apparatus according to the present disclosure.
Embodiments for Carrying Out the Invention
[0011] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the drawings.
[0012] <First Embodiment> [Overall Configuration] In the fields of medicine and drug discovery, there is abundant data written in natural language, such as papers and electronic medical records. By constructing a knowledge graph from such natural language data, the relationships between the data can be expressed and utilized for advanced search and prediction tasks, etc.
[0013] The knowledge graph is constructed, for example, by using a large language model (LLM: Large Language Models). However, in the above method, when the LLM generates hallucinations (i.e., when the LLM generates false information), there is a possibility that a knowledge graph different from the facts described in the original natural language data is constructed.
[0014] Therefore, in this embodiment, when constructing the knowledge graph, a process of verifying the truth or falsehood of the relationships between entities is included. As a result, a highly accurate knowledge graph that conforms to facts can be constructed.
[0015] Figure 1 is a diagram conceptually showing the information processing apparatus of this embodiment. The information processing apparatus 10 constructs a knowledge graph from natural language data such as input papers. First, the information processing apparatus 10 extracts a list of triples (node, edge, node) from the natural language data using an LLM. Nodes represent entities, and edges represent relationships between nodes. Next, the information processing apparatus 10 performs link prediction on each triple using a link prediction model to determine the truth or falsehood of the relationship between nodes. The information processing apparatus 10 adds triples for which the relationship between nodes is determined to be true to the knowledge graph, and does not add and excludes triples for which the relationship between nodes is determined to be false. In this way, the information processing apparatus 10 can construct a highly accurate knowledge graph by performing link prediction on the information (triples) obtained from the LLM and including the process of verifying the truth or falsehood of the relationship between nodes.
[0016] In this embodiment, a knowledge graph in the field of medical and drug discovery is constructed, but the target field is not limited to this, and it can be applied to other fields such as material development and pesticide development.
[0017] [Hardware Configuration] Figure 2 is a block diagram showing the hardware configuration of the information processing apparatus 10 according to the first embodiment. The information processing apparatus 10 is an example of an information processing apparatus. As shown in the figure, the information processing apparatus 10 includes an interface (I / F) 11, a processor 12, a memory 13, a recording medium 14, and a database (DB) 15.
[0018] The I / F 11 performs data input / output with an external device. Specifically, the I / F 11 acquires natural language data used by the information processing apparatus 10 from an external device.
[0019] Processor 12 is a computer such as a CPU (Central Processing Unit) and controls the entire information processing device 10 by executing a pre-prepared program. Processor 12 may be a GPU (Graphics Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Point Number Processing Unit), PPU (Physics Processing Unit), TPU (Tensor Processing Unit), quantum processor, microcontroller, or a combination thereof. Processor 12 executes the knowledge graph construction process described later.
[0020] Memory 13 consists of ROM (Read Only Memory), RAM (Random Access Memory), and other components. Memory 13 is also used as working memory while the processor 12 is executing various processes.
[0021] The recording medium 14 is a non-volatile, non-temporary recording medium such as a disk-shaped recording medium or semiconductor memory, and is configured to be detachable from the information processing device 10. The recording medium 14 stores various programs that the processor 12 executes. When the information processing device 10 executes various processes, the programs stored in the recording medium 14 are loaded into the memory 13 and executed by the processor 12. The DB 15 stores an existing knowledge graph, which will be described later. The DB 15 may also store a link prediction model.
[0022] In addition to the above, the information processing device 10 may also be equipped with a display device such as a liquid crystal display, and an input device such as a keyboard or mouse. These display devices and input devices are used, for example, by the administrator of the information processing device 10 to perform necessary management.
[0023] [Functional Configuration] Figure 3 is a block diagram showing the functional configuration of the information processing device 10 of the first embodiment. Functionally, the information processing device 10 comprises a named entity recognition unit 101, a relationship extraction unit 102, and a link prediction unit 103.
[0024] Natural language data is input to the information processing device 10 via the I / F 11. The natural language data is input to the named entity recognition unit 101 and the relational data extraction unit 102. Figure 4 shows an example of natural language data. The natural language data in Figure 4 is a medical paper, obtained from a medical literature search database, etc.
[0025] The named entity recognition unit 101 extracts named entities from natural language data using a model such as LLM. Named entities include proper nouns and numerical representations such as dates and times. In the fields of medicine and drug discovery, named entities include, for example, disease names, drug names, gene names, and protein names. The named entities extracted by the named entity recognition unit 101 are treated as candidates for entities in the knowledge graph. The named entity recognition unit 101 outputs the extracted named entities (hereinafter also referred to as "entities") to the relational extraction unit 102.
[0026] Figure 5 shows an example of extracted entities. In Figure 5, the named entity recognition unit 101 extracts entities such as cholesterol, DNA, PCSK9, and R3500Q from the natural language data in Figure 4.
[0027] The relationship extraction unit 102 extracts relationships between entities from the natural language data based on the natural language data and the entities. The relationship extraction unit 102 extracts a list of triples as the relationships between entities.
[0028] Specifically, the relationship extraction unit 102 creates a prompt as shown in Figure 6 and inputs the created prompt to the LLM. This prompt instructs the LLM to extract triples from the natural language data. The relationship extraction unit 102 then obtains the response to the prompt (i.e., a list of triples) from the LLM. The relationship extraction unit 102 outputs the LLM's response to the link prediction unit 103.
[0029] Figure 6 shows an example of a prompt from the relationship extraction unit 102. The prompt in Figure 6 includes an instruction sentence 51, a specific example 52, and a context 53. Instruction sentence 51 is an instruction sentence for the LLM. Instruction sentence 51 includes sentences that instruct the LLM to infer relationships between entities from natural language data, and sentences that instruct the LLM to output the answer in triple format. Specific example 52 is an example of a process that we want the LLM to perform. By showing such an example, the accuracy of the LLM's answer can be improved. Context 53 is a source of information for the LLM to generate the answer, and includes entities input from the named entity recognition unit 101 and natural language data.
[0030] Figure 7 shows an example of an LLM response. As shown in Figure 7, the relation extraction unit 102 obtains a list of triples as the LLM response.
[0031] Returning to Figure 3, the link prediction unit 103 uses a pre-prepared link prediction model to perform link prediction for each triple and estimate the score for each triple.
[0032] The link prediction model is a model trained on an existing knowledge graph using machine learning algorithms such as TransE or DistMult. The link prediction model is trained to predict the relationships between unlinked nodes in the existing knowledge graph. The existing knowledge graph is one in the fields of medicine and drug discovery. It is assumed that the existing knowledge graph is constructed from natural language data, such as medical papers and drug discovery papers, and is already partially complete.
[0033] Specifically, the link prediction unit 103 first generates a query by removing the edge from the three elements (node, edge, node) of the triple. For example, the link prediction unit 103 generates the query (DNA, ?, PCSK9) from the triple (DNA, Association, PCSK9). The "?" indicates that the edge is missing.
[0034] Next, the link prediction unit 103 inputs the generated query into the link prediction model. The link prediction model outputs candidate edges for the missing data and their predicted scores as an answer to the query. The predicted scores are expressed in the range of 0 to 1. For example, as an answer to the query (DNA,?,PCSK9), the link prediction model outputs candidate edge "Association" with a predicted score of "0.7", candidate edge "Positive_Correlation" with a predicted score of "0.2", and candidate edge "Negative_Correlation" with a predicted score of "0.1".
[0035] Next, the link prediction unit 103 estimates the triple score based on the link prediction model's response. For example, the link prediction unit 103 extracts the same edge from among several edge candidates as the original triple and uses the predicted score of that edge as the triple score. In the example above, the original triple is "DNA,Association,PCSK9", and the predicted score of the edge candidate "Association" is "0.7". Therefore, the link prediction unit 103 estimates that the score of the triple (DNA,Association,PCSK9) is "0.7".
[0036] The link prediction unit 103 determines the truth value of the relationship between nodes based on the score of each triple. "True" indicates that a relationship (triple) exists between the nodes, while "false" indicates that a relationship (triple) does not exist between the nodes. The link prediction unit 103 constructs a knowledge graph based on the truth value determination result.
[0037] Specifically, the link prediction unit 103 determines that a relationship between nodes is true if the triple's score is above a predetermined threshold, and adds that triple to the knowledge graph. On the other hand, if the triple's score is below a predetermined threshold, the link prediction unit 103 determines that a relationship between nodes is false, and excludes that triple without adding it to the knowledge graph.
[0038] The knowledge graph to which the triples are added is, for example, an existing knowledge graph used to train the link prediction model. The link prediction unit 103 determines the truth value of the relationships between nodes and then adds the new relationships to the existing knowledge graph. This improves the accuracy of the knowledge graph.
[0039] Furthermore, the named entity recognition unit 101 and the relation extraction unit 102 can use OpenAI's GPT (Generative Pre-trained Transformer) as the LLM. The LLMs used by the named entity recognition unit 101 and the relation extraction unit 102 may be the same model or different models. For example, the named entity recognition unit 101 may use a pre-trained language model specific to a domain (in this embodiment, the medical and drug discovery field).
[0040] In the above configuration, the named entity recognition unit 101 and the relationship extraction unit 102 are examples of extraction means, and the link prediction unit 103 is an example of a determination means and a graph construction means.
[0041] [Knowledge graph construction process] Next, we will explain the process of constructing the knowledge graph as described above. Figure 8 is a flowchart of the knowledge graph construction process by the information processing device 10. This process is realized when the processor 12 shown in Figure 2 executes a pre-prepared program and operates as each element shown in Figure 3.
[0042] First, natural language data is input to the information processing device 10 through the I / F 11 (step S101). The natural language data is input to the named entity recognition unit 101 and the relation extraction unit 102.
[0043] Next, the named entity recognition unit 101 extracts entities from the natural language data (step S102). The named entity recognition unit 101 outputs the extracted entities to the relation extraction unit 102. Next, the relation extraction unit 102 extracts a list of triples from the natural language data based on the natural language data and the entities (step S103). The relation extraction unit 102 outputs the list of triples to the link prediction unit 103.
[0044] Next, the link prediction unit 103 uses a pre-prepared link prediction model to perform link prediction for each triple and estimate the score of each triple (step S104). Then, if the score of a triple is above a predetermined threshold, the link prediction unit 103 adds that triple to the knowledge graph. On the other hand, if the score of a triple is below a predetermined threshold, the link prediction unit 103 discards that triple without adding it to the knowledge graph (step S105). Then the process ends.
[0045] [Examples of use] The knowledge graph constructed by the information processing device 10 can be used, for example, in semantic search. Furthermore, by utilizing the constructed knowledge graph in machine learning, it can be applied to various prediction tasks.
[0046] <Second Embodiment> Figure 9 is a block diagram showing the functional configuration of the information processing device of the second embodiment. The information processing device 20 comprises an extraction means 201, a determination means 202, and a graph construction means 203.
[0047] Figure 10 is a flowchart of the processing performed by the information processing device of the second embodiment. Extraction means 201 extracts entities and relationships between entities from natural language data (step S201). Determination means 202 determines the truth value of the relationships between entities by predicting the relationships between entities using a link prediction model (step S202). Graph construction means 203 adds the relationships between entities determined to be true by the determination means to the knowledge graph, and does not add the relationships between entities determined to be false by the determination means to the knowledge graph (step S203).
[0048] According to the information processing device of the second embodiment, it is possible to construct a highly accurate knowledge graph.
[0049] Some or all of the above embodiments may also be described as follows, but are not limited to the following:
[0050] (Note 1) An extraction means for extracting entities and relationships between entities from natural language data, A determination means for determining the truth value between entities by predicting the relationships between entities using a link prediction model, A graph construction means that adds relationships between entities determined to be true by the determination means to the knowledge graph, and does not add relationships between entities determined to be false by the determination means to the knowledge graph, An information processing device equipped with the following features.
[0051] (Note 2) The link prediction model outputs a score for the relationship between the entities. The information processing apparatus according to Appendix 1, wherein the determination means determines that the relationship between the entities is true if the score is equal to or greater than a predetermined threshold, and determines that the relationship between the entities is false if the score is less than a predetermined threshold.
[0052] (Note 3) The extraction means extracts a list of triples as the relationships between the entities, The determination means obtains the score of each triple using the link prediction model, The graph construction means is an information processing device according to Appendix 2, which adds triples with a score equal to or greater than a predetermined threshold to the knowledge graph and does not add triples with a score less than a predetermined threshold to the knowledge graph.
[0053] (Note 4) The aforementioned triple includes three elements: a first node, an edge, and a second node. The information processing device described in Appendix 3, wherein the determination means generates data with missing edges among the three elements as a query, inputs the query to the link prediction model, obtains edge candidates and prediction scores from the link prediction model, and estimates the triple score based on the edge candidates and prediction scores.
[0054] (Note 5) The aforementioned link prediction model is a model trained to predict relationships between entities that are not linked in a known knowledge graph. The known knowledge graph is a known knowledge graph in the same field as the natural language data, and the information processing device described in Appendix 1 includes entities of the natural language data.
[0055] (Note 6) The aforementioned natural language data includes the information processing device described in Appendix 1, which includes research paper data and electronic medical records.
[0056] (Note 7) The extraction means is an information processing device according to Appendix 1, which uses a large-scale language model to extract entities and relationships between entities.
[0057] (Note 8) The extraction means is an information processing device according to Appendix 7, which extracts the entities using a domain-specific, pre-trained, large-scale language model.
[0058] (Note 9) A method of information processing performed by a computer, An extraction process is performed to extract entities and relationships between entities from natural language data. By predicting the relationships between the entities using a link prediction model, a determination process is performed to determine the truth value of the relationships between the entities. An information processing method that performs a graph construction process which adds relationships between entities determined to be true by the aforementioned determination process to the knowledge graph, and does not add relationships between entities determined to be false by the aforementioned determination process to the knowledge graph.
[0059] (Note 10) An extraction process that extracts entities and relationships between entities from natural language data, A determination process is performed to determine the truth value of the relationships between the entities by predicting the relationships between the entities using a link prediction model, A program that causes a computer to perform a graph construction process that adds relationships between entities determined to be true by the aforementioned determination process to the knowledge graph, and does not add relationships between entities determined to be false by the aforementioned determination process to the knowledge graph.
[0060] Although the present disclosure has been described above with reference to embodiments and examples, the present disclosure is not limited to the above embodiments and examples. Various modifications to the structure and details of the present disclosure can be understood by those skilled in the art within the scope of the present disclosure. [Explanation of Symbols]
[0061] 10 Information Processing Devices 101 Named entity extraction unit 102 Relationship Extraction Unit 103 Link Prediction Unit
Claims
1. An extraction means for extracting entities and relationships between entities from natural language data, A determination means for determining the truth value between entities by predicting the relationships between entities using a link prediction model, A graph construction means that adds relationships between entities determined to be true by the determination means to the knowledge graph, and does not add relationships between entities determined to be false by the determination means to the knowledge graph, An information processing device equipped with the following features.
2. The link prediction model outputs a score for the relationship between the entities. The information processing apparatus according to claim 1, wherein the determination means determines that the relationship between the entities is true if the score is equal to or greater than a predetermined threshold, and determines that the relationship between the entities is false if the score is less than a predetermined threshold.
3. The extraction means extracts a list of triples as the relationships between the entities, The determination means obtains the score of each triple using the link prediction model, The information processing apparatus according to claim 2, wherein the graph construction means adds triples whose scores are equal to or greater than a predetermined threshold to the knowledge graph, and does not add triples whose scores are less than a predetermined threshold to the knowledge graph.
4. The aforementioned triple includes three elements: a first node, an edge, and a second node. The information processing apparatus according to claim 3, wherein the determination means generates data with missing edges among the three elements as a query, inputs the query to the link prediction model, obtains edge candidates and prediction scores from the link prediction model, and estimates the triple score based on the edge candidates and prediction scores.
5. The aforementioned link prediction model is a model trained to predict relationships between entities that are not linked in a known knowledge graph. The information processing apparatus according to claim 1, wherein the known knowledge graph is a known knowledge graph in the same field as the natural language data, and includes entities of the natural language data.
6. The information processing apparatus according to claim 1, which includes the aforementioned natural language data such as research paper data and electronic medical records.
7. The information processing apparatus according to claim 1, wherein the extraction means extracts entities and relationships between entities using a large-scale language model.
8. The information processing apparatus according to claim 7, wherein the extraction means extracts the entities using a domain-specific, pre-trained, large-scale language model.
9. A method of information processing performed by a computer, An extraction process is performed to extract entities and relationships between entities from natural language data. By predicting the relationships between the entities using a link prediction model, a determination process is performed to determine the truth value of the relationships between the entities. An information processing method that performs a graph construction process which adds relationships between entities determined to be true by the aforementioned determination process to the knowledge graph, and does not add relationships between entities determined to be false by the aforementioned determination process to the knowledge graph.
10. An extraction process that extracts entities and relationships between entities from natural language data, A determination process is performed to determine the truth value of the relationships between the entities by predicting the relationships between the entities using a link prediction model, A program that causes a computer to perform a graph construction process that adds relationships between entities determined to be true by the aforementioned determination process to the knowledge graph, and does not add relationships between entities determined to be false by the aforementioned determination process to the knowledge graph.