Information processing device, information processing method, and information processing program

Abstract

According to an embodiment, an information processing device (10) has an editing unit (152) and a generation unit (154). The editing unit (152) adds information that indicates deletion to designated subgraphs that are subgraphs of a knowledge graph. The generation unit (154) reports that an answer to an inputted question is not to be generated when an inputted subgraph that is a subgraph of the knowledge graph includes the information that indicates deletion and uses a language model to generate an answer to the question that is an answer that is based on the subgraph when the subgraph does not include the information that indicates deletion.

Description

Information processing device, information processing method, and information processing program

[0001] This invention relates to an information processing device, an information processing method, and an information processing program.

[0002] Retrieval-Augmented Model Editing (RAE) is a known method for editing the knowledge contained in Large Language Models (LLMs). In RAE, the knowledge contained in an LLM is represented by a knowledge graph. The knowledge graph has nodes as knowledge entities and edges as relationships between entities. In RAE, the addition and modification of knowledge are represented by updating the knowledge graph.

[0003] Yucheng Shi, Qiaoyu Tan, Xuansheng Wu, Shaochen Zhong, Kaixiong Zhou, Ninghao Liu, "Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering," [online], [Retrieved November 25, 2024], Internet <URL: https: / / arxiv.org / abs / 2403.19631>

[0004] However, conventional technologies may result in reduced accuracy in LLM responses.

[0005] According to RAE, by searching for graphs related to user questions and inputting the resulting graphs as prompts into the LLM, it is possible to output answers to the LLM that take into account the information changed through knowledge editing.

[0006] On the other hand, RAE does not consider the deletion of knowledge and the nature of the relationships between entities that affect the graph structure (hereinafter referred to as "relationships") when constructing the graph associated with knowledge editing. Therefore, the knowledge graph in RAE is insufficient to represent the edited knowledge and its relationships. As a result, the accuracy of LLM responses decreases.

[0007] Therefore, the objective of this invention is to improve the accuracy of LLM responses.

[0008] To solve the aforementioned problems, the information processing device of the present invention is characterized by comprising: an editing unit that assigns information indicating deletion to a subgraph of a knowledge graph, and a generation unit that, if the input subgraph contains the information indicating deletion, notifies that it will not generate an answer to the input question, and if the subgraph does not contain the information indicating deletion, generates an answer to the question, based on the subgraph, using a language model.

[0009] According to the present invention, the accuracy of LLM responses can be improved.

[0010] Figure 1 shows an example configuration of an information processing device according to the first embodiment. Figure 2 shows an example of template information. Figure 3 shows an example of a knowledge graph. Figure 4 shows the overall algorithm of the editing process. Figure 5 shows the algorithm of the Update function. Figure 6 shows the algorithm of the Check_logical_constraints function. Figure 7 is a diagram illustrating the editing process according to relationships. Figure 8 is a diagram illustrating the deletion process. Figure 9 is a flowchart showing the flow of the editing process. Figure 10 is a flowchart showing the flow of the generation process. Figure 11 is a diagram illustrating an embodiment. Figure 12 shows an example configuration of a computer that executes an information processing program.

[0011] The embodiments for carrying out the present invention will be described below with reference to the drawings. The present invention is not limited to these embodiments.

[0012] [First Embodiment] The configuration of the information processing device will be explained using Figure 1. Figure 1 is a diagram showing an example of the configuration of the information processing device according to the first embodiment.

[0013] The information processing device 10 performs knowledge editing of the large language model (LLM). Also, the information processing device 10 uses the LLM to output an answer to a question. Note that the LLM is an example of a language model.

[0014] An explanation of knowledge editing will be given. The LLM can generate an answer to a question from a user. At this time, when the knowledge possessed by the LLM is insufficient or stale, it may not be able to generate an appropriate answer. For example, if the LLM does not have the knowledge that the president of the United States has changed due to an election, it is conceivable that the LLM will answer the question "Who is the president of the United States?" with the name of the president before the change.

[0015] Knowledge editing is a method of improving the accuracy of answers by dynamically updating and adding the knowledge possessed by the LLM. On the other hand, as described above, since conventional knowledge editing methods do not consider the deletion of knowledge and the relationship between entities that affects the graph structure, there are cases where the knowledge possessed by the LLM cannot be accurately edited.

[0016] In contrast, the information processing device 10 according to the first embodiment accurately edits the knowledge possessed by the LLM by performing knowledge editing that takes into account the deletion of knowledge and the relationship between entities that affects the graph structure.

[0017] Here, the knowledge δ of the LLM is defined as in equation (1).

[0018]

[0019] The knowledge δ includes h, r, and t as elements. h is the head entity. r is the relationship. t is the tail entity. Examples of each element will be described later.

[0020] Also, as shown in equation (2), knowledge editing is defined as replacing the tail entity t with t'. δ' is the knowledge after editing.

[0021]

[0022] [Configuration of the First Embodiment] As shown in FIG. 1, the information processing apparatus 10 includes a communication unit 11, an input unit 12, an output unit 13, a storage unit 14, and a control unit 15.

[0023] The communication unit 11 is a module for performing data communication with other devices. The communication unit 11 is, for example, a NIC (Network Interface Card). The input unit 12 is an interface connected to input devices such as a mouse and a keyboard. The output unit 13 is an interface connected to output devices such as a speaker and a display.

[0024] The storage unit 14 stores data, programs, etc. that are referred to when the control unit 15 executes various processes. The storage unit 14 is realized by a semiconductor memory element such as a RAM (Random Access Memory), a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. The storage unit 14 stores a graph DB 141, template information 142, and generation model information 143.

[0025] The graph DB 141 is information regarding a knowledge graph. The knowledge graph includes one or more pieces of knowledge as subgraphs (partial graphs).

[0026] The template information 142 is information on templates used in editing processes according to relationships. FIG. 2 is a diagram showing an example of template information. As shown in FIG. 2, the template information 142 includes a "No." column, a "Use" column, and a "Template" column. The value in the "No." column is the identifier of each record. The value in the "Use" column specifies the relationship and the processing content. For example, the relationships include symmetry (Symmetric), transitivity (Transitive), commonality (Commonality), and inverse relationship (Inverse Relation). The value in the "Template" column specifies the specific processing content. Details of the processing using the template information 142 will be described later.

[0027] The generative model information 143 consists of the model parameters. For example, if the model is a neural network, the generative model information 143 consists of parameters such as weights and biases. In this embodiment, the model is an LLM (Long Life Model).

[0028] The control unit 15 is responsible for controlling the entire information processing device 10. The functions of the control unit 15 are realized, for example, by the CPU (Central Processing Unit) executing a program stored in the memory unit 14. The control unit 15 includes a determination unit 151, an editing unit 152, an acquisition unit 153, a generation unit 154, and an output control unit 155.

[0029] The determination unit 151 and the editing unit 152 mainly perform editing processing. The acquisition unit 153, the generation unit 154, and the output control unit 155 mainly perform generation processing.

[0030] Here, we will explain the knowledge graph using Figure 3. Figure 3 is a diagram showing an example of a knowledge graph.

[0031] As shown in Figure 3, the knowledge graph 301 has nodes labeled "Misery," "Horror," "Stephen King," and "US." Each node is an entity.

[0032] Additionally, the edges of the Knowledge Graph 301 are labeled with terms such as "genre," "author," and "nationality." These edges represent the relationships between entities.

[0033] A subgraph is a part of the knowledge graph. However, a subgraph may also represent the entire knowledge graph.

[0034] For example, the combination of node "Misery", edge "author", node "Stephen King", and node "US" is a subgraph. Knowledge with "Misery" as the head entity, "author" as the relation, and "Stephen King" as the tail entity is represented by a subgraph and is written as (Misery, author, Stephen King) according to the definition in equation (1).

[0035] Note that the LLM can understand the relationships between entities based on the labels of the edges. For example, based on the aforementioned subgraph, the LLM can obtain the knowledge that "the author of Misery is Stephen King."

[0036] The algorithm for the editing process will be described using FIG. 4. FIG. 4 is a diagram showing the overall algorithm for the editing process. Note that editing includes operations such as update, addition, and deletion.

[0037] As shown in FIG. 4, the editing unit 152 receives the input of the knowledge graph G, the input knowledge Δ ｕｐ for update, the input knowledge Δ ｉｎｓ for addition, and the input knowledge Δ ｄｅｌ for deletion. The knowledge graph G is read from the graph DB 141. Also, the editing unit 152 outputs the edited knowledge graph G'. For example, the editing unit 152 overwrites the graph DB 141 with the edited knowledge graph G'.

[0038] As shown in FIG. 4, the editing unit 152 updates the knowledge graph G using the input knowledge Δ ｕｐ for update (second line). The input knowledge Δ ｕｐ for update includes the knowledge δ before the update and the knowledge δ' after the update. Thereafter, the editing unit 152 executes an editing process according to the relationship (third line).

[0039] Also, as shown in FIG. 4, the editing unit 152 adds knowledge to the knowledge graph G using the input knowledge Δ ｉｎｓ for addition (seventh line). The input knowledge Δ ｉｎｓ for addition includes the knowledge δ' to be added. Thereafter, the editing unit 152 executes an editing process according to the relationship (eighth line).

[0040] Also, as shown in FIG. 4, the editing unit 152 deletes the knowledge of the knowledge graph G using the input knowledge Δ ｄｅｌ for deletion (fourteenth line). The input knowledge Δ ｄｅｌ for deletion includes the knowledge δ to be deleted.

[0041] To perform deletion, the editing unit 152 uses the input knowledge Δ ｄｅｌThe relationship r of the knowledge δ to be deleted is r ｄｅｌ δ replaced ｄｅｌ Create the following. Editorial department 152 will use the knowledge δ of the knowledge graph G as knowledge δ ｄｅｌ Deletion is achieved by updating. In other words, the knowledge after deletion is not erased, but rather related to r ｄｅｌ This will indicate that it has been deleted.

[0042] The Update function (lines 2, 7, and 14 in Figure 4) is used to edit knowledge. Figure 5 illustrates the algorithm of the Update function. Although this function is named "Update," as shown in Figure 4, it is called for any operation, whether updating, adding, or deleting.

[0043] We call δ' the editing knowledge. The editing knowledge δ' is defined as shown in equation (3). Note that "<" and ">" may be replaced with "(" and ")" as shown in equation (1).

[0044]

[0045] δ ｄｅｌ This is called deletion knowledge. Deletion knowledge δ ｄｅｌ It is defined as shown in equation (4).

[0046]

[0047] As shown in Figure 5, if a subgraph δ that shares the same head entity and relationships as the edited knowledge δ' is included in the knowledge graph G (first row), the editorial unit 152 updates the subgraph δ of the knowledge graph G with the edited knowledge δ', as shown in equation (5) (second row). This process enables not only the updating of knowledge but also the deletion of knowledge.

[0048]

[0049] For example, subgraph <A, r, O> shares the same head entity A and relation r as the editorial knowledge <A, r, B>.

[0050] Furthermore, as shown in Figure 5, if the subgraph δ that shares the same head entity and relationships as the edited knowledge δ' is not included in the knowledge graph G (3rd row), the editorial department 152 adds the edited knowledge δ' to the knowledge graph G as shown in equation (6) (4th row). This process enables the addition of knowledge.

[0051]

[0052] `Check_logical_constraints` (lines 3 and 8 in Figure 4) is a function for editing knowledge. Figure 6 is used to explain the algorithm of the `Check_logical_constraints` function. Figure 6 is a diagram showing the algorithm of the `Check_logical_constraints` function.

[0053] The determination unit 151 determines the function f θ The system uses this to determine the relationships between entities related to the edited knowledge δ' in the edited knowledge graph. For example, the determination unit 151 determines whether the relationship is symmetric, transitive, common, or inverse. θ This is a function that takes a prompt as input to a language model and obtains a judgment result.

[0054] The determination unit 151 inputs prompts based on the template information 142 shown in Figure 2 to the LLM and makes a determination based on the obtained response. Based on the knowledge graph G and the edited knowledge δ', the determination unit 151 substitutes specific values ​​for A, B, C, and r of each template in Figure 2.

[0055] (Determination of symmetry (lines 2 to 5 of Figure 6)) Given entities A and B and relation r, r has symmetry if equation (7) holds.

[0056]

[0057] The determination unit 151 uses template T corresponding to the application "symmetry determination" from template information 142. Ｓ The determination unit 151 obtains the template T. Ｓ Substitute the editing knowledge δ' into the prompt and input the resulting prompt into LLM to obtain the judgment result.

[0058] From Figure 2, template T Ｓ The question is: "Determine whether the following statement is true by answering Yes or No: If A and B are r, then B and A are r."

[0059] For example, the relationship between siblings is symmetry. That is, "When A is B's sibling, then B is A's sibling." holds true. In this case, based on the judgment result regarding the editorial knowledge <A, siblings, B>, editorial department 152 can add the knowledge "B is A's sibling" to the knowledge graph in addition to the knowledge "A is B's sibling."

[0060] (Determination of Transitivity (lines 7 to 12 of Figure 6)) Given entities A, B, C and relation r, r is transitive if equation (8) holds.

[0061]

[0062] The determination unit 151 uses template T corresponding to the application "transitivity determination" from template information 142. ｔ The determination unit 151 obtains the template T. ｔ The prompt created by substituting the edit knowledge δ' and entity C is input into LLM to obtain the judgment result. Entity C is an entity that has a relationship with B.

[0063] From Figure 2, template T ｔ The question is: "Determine whether the following statement is true by answering Yes or No: If A and B are r, and B and C are also r, then A and C are r."

[0064] For example, the relationship of colleagues is transitive. That is, "If A is B's colleague, and B and C are also colleagues, then A and C are colleagues." In this case, based on the judgment result regarding the editorial knowledge <A, colleague, B>, editorial department 152 can add the knowledge "A is C's colleague" to the knowledge graph in addition to the knowledge "A is B's colleague."

[0065] (Determination of commonality (lines 14 to 19 of Figure 6)) Given entities A, B, and C and relations r and r', if equation (9) holds, then r and r' have commonality with each other.

[0066]

[0067] The determination unit 151 uses template information 142 to determine the template T corresponding to the application "commonality determination". ｃ The determination unit 151 obtains the template T. ｃ The prompt created by substituting the edit knowledge δ', entity C, and relation r' is input into LLM to obtain the judgment result. Relation r' is the relationship between B and C.

[0068] From Figure 2, template T ｃ The question is: "Determine whether the following statement is true by answering Yes or No: If A and B are r, and B and C are r', then A and C are r'."

[0069] For example, the relationship of being a child has commonalities. That is, "If A and B are siblings, and B is the child of C, then A is the child of C." is true. In this case, based on the judgment result regarding the editorial knowledge <A, siblings, B>, editorial department 152 can add the knowledge "A is the child of C" to the knowledge graph in addition to the knowledge "A is the sibling of B."

[0070] (Determination of inverse relationship (lines 21 to 25 of Figure 6)) For entities A and B and relation r, if equation (10) holds, then r has an inverse relationship. The property of relation r having an inverse relationship is called an inverse relationship. －１ This is the inverse relationship of r.

[0071]

[0072] The determination unit 151 uses template information 142 to determine the template T corresponding to the application "inverse relationship estimation". ｉ１ The determination unit 151 obtains the template T. ｉ１ Substitute the editing knowledge δ' into the prompt and input the resulting prompt into LLM to obtain the estimation result.

[0073] From Figure 2, template T ｉ１ The question is: "Please determine the inverse relationship between the following sentences: <A, r, B> Inverse relationship:

[0074] Furthermore, the determination unit 151 uses template T corresponding to the application "reverse relationship determination" from template information 142. ｉ２ The determination unit 151 obtains the template T. ｉ２ Editing knowledge δ', and the inverse relationship r of the estimation result －１ Substitute the values ​​and input the resulting prompt into LLM to obtain the judgment result.

[0075] From Figure 2, template T ｉ２ The question is: "Determine whether the following statement is true by answering Yes or No: If A and B are r, then B and A are r^{-1}." "r^{-1}" means that "r" has a superscript "-1" attached to it.

[0076] For example, the relationship of child and the relationship of parent are inverse relationships. That is, "If A is B's child, then B is A's parent." is true. In this case, based on the judgment result regarding the editorial knowledge <A, child, B>, editorial department 152 can add the knowledge "B is A's parent" to the knowledge graph in addition to the knowledge "A is B's child."

[0077] In this way, the determination unit 151 uses a language model to determine the nature of a relationship based on conditions corresponding to the nature of the relationship, the relationship indicated by the edge, and the entities indicated by the two nodes. For example, the determination unit 151 also determines whether the nature of the relationship indicated by the nodes of the subgraph is symmetry, transitivity, commonality, or inverse relationship. For example, the conditions are shown in the template of the template information 142.

[0078] The editing process can be described as a process that changes the information of edges in the knowledge graph. For example, the editing unit 152 can attach labels to the edges of the knowledge graph that indicate deletion, or labels that indicate the relationship of the judgment result.

[0079] Figure 7 illustrates the editing process based on relationships. In the knowledge graph 311 of Figure 7, there are no entities whose relationship with the entity "Stephen King" is "siblings".

[0080] Editorial staff 152 adds the entity "Taro," whose relationship with the entity "Stephen King" is "siblings," to the knowledge graph 311, thereby obtaining the knowledge graph 312.

[0081] Furthermore, the determination unit 151 determines that the relationship "siblings" has symmetry and provides the determination result to the knowledge graph 312 to obtain the knowledge graph 313. In the knowledge graph 313, the symmetry of the relationship "siblings" between "Stephen King" and "Taro" is indicated by two arrows pointing in opposite directions. Note that the arrows may be drawn based on labels attached to the edges when visualizing the knowledge graph.

[0082] In this way, the determination unit 151 determines the nature of the relationship indicated by the edge when editing the knowledge graph, based on the subgraph which includes the edge and the two nodes connected by the edge. The editing unit 152 adds information indicating the nature of the relationship determined by the determination unit 151 to the portion of the knowledge graph corresponding to the subgraph. In the example in Figure 7, the label "siblings" is added to the edge, which is a portion of the knowledge graph. On the other hand, labels may also be added to parts other than edges (for example, nodes).

[0083] Figure 8 illustrates the deletion process. The knowledge graph 321 in Figure 8 shows the knowledge that a relationship "author" exists between the entity "Misery" and the entity "Stephen King". The editorial team 152 deletes the knowledge by labeling the edge of the relationship "author" with [DEL].

[0084] In the example in Figure 8, the label [DEL] is assigned to the edge, which is part of the knowledge graph. On the other hand, labels may also be assigned to parts other than edges (for example, nodes).

[0085] As explained above, the editorial unit 152 is a subgraph of the knowledge graph and assigns information indicating deletion to a specified subgraph. The generation unit 154 is a subgraph of the knowledge graph and, if the input subgraph contains information indicating deletion, notifies that it will not generate an answer to the input question. If the subgraph does not contain information indicating deletion, it generates an answer to the question, based on the subgraph, using a language model. For example, the editorial unit 152 assigns a label indicating deletion to a specified subgraph of a knowledge graph that includes nodes representing knowledge entities and edges representing relationships between entities.

[0086] As a result, the information processing device 10 achieves knowledge deletion. Consequently, according to the first embodiment, the accuracy of the LLM's answers is improved.

[0087] Returning to Figure 1, the generation process by the acquisition unit 153, the generation unit 154, and the output control unit 155 will be explained. The generation process is the process of outputting the answer to the input question.

[0088] The acquisition unit 153 acquires subgraphs related to the input question from the graph DB 141. The generation unit 154 inputs the question and subgraphs into the LLM and generates an answer. The output control unit 155 outputs the answer. For example, the output control unit 155 outputs the answer as text.

[0089] However, the output control unit 155 will not output a response if the acquired subgraph contains deleted relationships. This prevents the output control unit 155 from outputting uncertain responses.

[0090] [Processing in the First Embodiment] The editing process flow will be explained using Figure 9. Figure 9 is a flowchart of the editing process flow. As shown in Figure 9, first the editing unit 152 receives input of editing knowledge (step S101).

[0091] The determination unit 151 determines whether or not the edge indicating deletion is included in the edit knowledge (step S102). If the edge indicating deletion is included in the edit knowledge (step S102; Yes), the editing unit 152 deletes the subgraph corresponding to the edit knowledge (step S103). However, the editing unit 152 does not need to erase the subgraph, but rather needs to add information that identifies that the subgraph has been deleted.

[0092] If the edge indicating deletion is not included in the editing knowledge (step S102; No), the determination unit 151 determines whether or not the editing knowledge is included in the graph (knowledge graph of graph DB 141) (step S104).

[0093] If the editorial knowledge is included in the graph (step S104; Yes), the editorial department 152 updates the subgraph based on the editorial knowledge (step S105) and proceeds to step S107. If the editorial knowledge is not included in the graph (step S104; No), the editorial department 152 adds a subgraph based on the editorial knowledge (step S106).

[0094] Next, the determination unit 151 determines the relationships included in the edited knowledge (step S107). Then, the editing unit 152 updates the graph according to the determination result (step S108).

[0095] The flow of the generation process will be explained using Figure 10. Figure 10 is a flowchart showing the flow of the generation process. As shown in Figure 10, the generation unit 154 receives input for the question (step S201).

[0096] Next, the acquisition unit 153 acquires subgraphs related to the question (step S202). For example, the acquisition unit 153 acquires subgraphs from the knowledge graph of graph DB 141 that contain the keywords included in the question as entities.

[0097] The generation unit 154 generates an answer using a prompt that includes a question and a subgraph (step S203). The generation unit 154 can also provide the subgraph to the LLM using RAG (Retrieval-Augmented Generation).

[0098] If the subgraph contains an edge indicating deletion (step S204; Yes), the output control unit 155 outputs information indicating that it cannot be answered (step S205). In this case, the generation unit 154 also notifies the output control unit 155 that it cannot be answered. When the output control unit 155 is notified that it cannot be answered, it outputs information indicating that it cannot be answered (for example, the message "The question cannot be answered.").

[0099] The generation unit 154 may instruct the LLM to output information indicating that an answer is not possible if the subgraph contains an edge indicating deletion. The output control unit 155 outputs the information indicating that an answer is not possible that was output by the LLM.

[0100] If the subgraph does not contain an edge indicating deletion (step S204; No), the output control unit 155 outputs the generated answer (step S206).

[0101] According to the first embodiment, the embodiment shown in Figure 11 is realized. Figure 11 is a diagram illustrating the embodiment.

[0102] As shown in Figure 11, the LLM generates answers to user questions by referring to the knowledge graph stored in the graph database. Furthermore, the graph database accumulates the latest information such as news and articles through the editing process of the first embodiment. As a result, according to the first embodiment, the accuracy of the LLM's answers is improved.

[0103] [Effects of the First Embodiment] As described above, the editorial unit 152 assigns information indicating deletion to a subgraph of the knowledge graph that is specified. The generation unit 154 notifies that it will not generate an answer to the input question if the input subgraph of the knowledge graph contains information indicating deletion, and if the subgraph does not contain information indicating deletion, it generates an answer to the question, an answer based on the subgraph, using a language model. For example, the editorial unit 152 assigns a label indicating deletion to a specified subgraph of a knowledge graph that includes nodes representing knowledge entities and edges representing relationships between entities.

[0104] As a result, the information processing device 10 achieves knowledge deletion. Consequently, according to the first embodiment, the accuracy of the LLM's answers is improved.

[0105] Furthermore, simply pruning the target knowledge from the knowledge graph (for example, deleting the entire subgraph) may cause the LLM to output an answer based on its own internal knowledge, even though the knowledge was supposed to have been deleted. To address this, the generation unit 154 notifies the user that an answer is not possible if the subgraph contains information that indicates deletion, thereby preventing the output of inaccurate information.

[0106] The determination unit 151 determines the nature of the relationship indicated by the edge when the knowledge graph is edited, based on the subgraph which includes the edge and the two nodes connected by the edge. The editing unit 152 adds information indicating the nature of the relationship determined by the determination unit 151 to the portion of the knowledge graph corresponding to the subgraph. For example, the determination unit 151 uses a language model to determine the nature of the relationship based on the conditions corresponding to the nature of the relationship, the relationship indicated by the edge, and the entities indicated by the two nodes. Alternatively, for example, the determination unit 151 determines whether the nature of the relationship indicated by the nodes in the subgraph is symmetry, transitivity, commonality, or inverse relationship.

[0107] This allows the information processing device 10 to extract further information from the relationships between the knowledge contained in the given edited knowledge and add it to the knowledge graph.

[0108] [About Multi-hop Questions] A multi-hop question is a question that requires a chain of multiple pieces of knowledge to answer. For example, the question, "Which city is the capital of the country where the author of Misery held citizenship?" is a multi-hop question.

[0109] For the above Multi-hop Question, the knowledge required to generate the answer is, for example, δ １ , δ ２ , δ ３ Therefore, since it is necessary to trace three pieces of knowledge to generate the answer, this question can be called a 3-hop question. δ １ =<Misery, author, StephenKing> δ ２ =<StephenKing, citizenof, UnitedStates> δ ３ =<UnitedStates, capital, WashingtonD.C>

[0110] Previous research has indicated that it is difficult to perform knowledge editing adapted to multi-hop questions. This is because a single piece of knowledge editing significantly alters the knowledge required to answer a particular multi-hop question.

[0111] Regarding the example given, "Which city is the capital of the country where the author of Misery held citizenship?", knowledge editing was performed to δ １ Updated δ ´ １ In that case, the following knowledge will be necessary to answer the question. δ ´１ =<Misery, author, Richard Dawkins> δ ４ =<RichardDawkins, citizenof, UnitedKingdom> δ ５ =<United Kingdom, capital, London>

[0112] Thus, simply editing one piece of knowledge can significantly alter the subsequent chain of knowledge required. For this reason, methods that do not use knowledge graphs struggle to handle multi-hop questions.

[0113] On the other hand, the knowledge graph accurately represents the chain of knowledge. Therefore, the information processing device 10 can appropriately search the knowledge graph and obtain the corresponding subgraphs to acquire subgraphs that represent the n-hop knowledge necessary for a Multi-hop Question.

[0114] [System Configuration, etc.] Furthermore, the components of each part shown in the diagram are functional concepts and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown in the diagram, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions. In addition, all or any part of the processing functions performed by each device can be realized by a CPU and the program executed on that CPU, or by hardware using wired logic.

[0115] Furthermore, among the processes described in the embodiments described above, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

[0116] [Program] The information processing device 10 described above can be implemented by installing a program (information processing program) as packaged software or online software on a desired computer. For example, by having the computer run the above program, the computer can function as the information processing device 10. The term "computer" here includes mobile communication terminals such as smartphones, mobile phones and PHS (Personal Handyphone System), as well as terminals such as PDA (Personal Digital Assistant).

[0117] Figure 12 shows an example configuration of a computer that executes an information processing program. Computer 1000 has, for example, memory 1010 and CPU 1020. Computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0118] Memory 1010 includes ROM (Read Only Memory) 1011 and RAM (Random Access Memory) 1012. ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.

[0119] The hard disk drive 1090 stores, for example, the OS 1091, application programs 1092, program modules 1093, and program data 1094. That is, the programs that define each process executed by the information processing device 10 are implemented as program modules 1093 in which executable code for a computer is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing processes similar to the functional configuration of the information processing device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0120] Furthermore, the data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, memory 1010 or hard disk drive 1090. The CPU 1020 then reads the program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as needed and executes them.

[0121] Furthermore, the program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090; for example, they may be stored in a removable storage medium and read by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via a network interface 1070.

[0122] 10 Information Processing Device 11 Communication Unit 12 Input Unit 13 Output Unit 14 Storage Unit 15 Control Unit 141 Graph Database 142 Template Information 143 Generative Model Information 151 Judgment Unit 152 Editing Unit 153 Acquisition Unit 154 Generation Unit 155 Output Control Unit

Claims

1. An information processing device comprising: an editorial unit that assigns information indicating deletion to a subgraph of a knowledge graph; and a generation unit that, if the input subgraph of the knowledge graph contains the information indicating deletion, notifies the user that it will not generate an answer to the input question, and if the subgraph does not contain the information indicating deletion, generates an answer to the question based on the subgraph using a language model.

2. The information processing device according to claim 1, characterized in that the editorial department assigns a label indicating deletion to the designated subgraph of the knowledge graph, which includes nodes representing knowledge entities and edges representing relationships between the entities.

3. The information processing device according to claim 1, further comprising a determination unit that determines the nature of the relationship indicated by the edge when the knowledge graph is edited based on the edge and a subgraph including two nodes connected by the edge, wherein the editing unit adds information indicating the nature of the relationship determined by the determination unit to the portion of the knowledge graph corresponding to the subgraph.

4. The information processing apparatus according to claim 3, characterized in that the determination unit determines the nature of a relationship using a language model based on conditions corresponding to the nature of the relationship, the relationship indicated by the edge, and the entities indicated by the two nodes.

5. The information processing apparatus according to claim 3, characterized in that the determination unit determines whether the nature of the relationship shown by the nodes of the subgraph is symmetry, transitivity, commonality, or inverse relationship.

6. An information processing method performed by a computer, comprising: an editing step of assigning information signifying deletion to a subgraph of a knowledge graph, wherein the subgraph of the knowledge graph, if the input subgraph contains the information signifying deletion, notifying that no answer to the input question will be generated, and if the subgraph does not contain the information signifying deletion, generating an answer to the question, based on the subgraph, using a language model.

7. An information processing program characterized by causing a computer to perform the following steps: an editing step of assigning information indicating deletion to a subgraph of a knowledge graph, and a generation step of notifying the computer that if the input subgraph of the knowledge graph contains the information indicating deletion, it will not generate an answer to the input question, and if the subgraph does not contain the information indicating deletion, it generates an answer to the question, based on the subgraph, using a language model.

Images