Knowledge Graph Question Answering Method, Apparatus, System, Electronic Device and Storage Medium

Abstract

The present application discloses a knowledge graph question-answering method, apparatus, system, electronic device, and storage medium. Among them, the method includes: using an analysis agent to extract keywords in a first query statement; using a tool agent to query relevant information corresponding to the keywords; using the analysis agent to determine an equivalent question corresponding to the first query statement based on the relevant information, and using an encoding agent to encode the equivalent question to obtain a second query statement; in the case where the inspection agent determines that the second query statement has no syntax error, using the tool agent to query in the knowledge graph according to the second query statement to obtain an answer result to the question corresponding to the first query statement. The present application solves the technical problems that the knowledge graph question-answering method in the related art often has poor universality, low recall rate, and weak reasoning ability due to limitations such as the length of the prompt information input to the large model and the lack of a verification mechanism.

Description

Technical Field

[0001] This application relates to the technical field of knowledge graphs. Specifically, it relates to a knowledge graph question answering method, apparatus, system, electronic device, and storage medium. Background Art

[0002] Question answering based on a knowledge graph is a typical application of a knowledge graph. It generally uses a pre-constructed knowledge graph to understand and answer natural language questions posed by users. In recent years, with the development of large models, the technical route of converting a question into GQL (Graph Query Language) and then combining it with the constructed graph for question answering has exceeded traditional syntactic parsing, subgraph recall, and other methods in terms of multi-hop capabilities, question answering accuracy, etc. However, in the related art, the knowledge graph question answering method combined with a large model often has problems such as poor universality, low recall rate, and weak reasoning ability due to limitations such as the length of the prompt input to the large model and the lack of a verification mechanism.

[0003] In response to the above problems, no effective solution has been proposed yet. Summary of the Invention

[0004] Embodiments of this application provide a knowledge graph question answering method, apparatus, system, electronic device, and storage medium to at least solve the technical problems that the knowledge graph question answering method in the related art often has poor universality, low recall rate, and weak reasoning ability due to limitations such as the length of the prompt information input to the large model and the lack of a verification mechanism.

[0005] According to one aspect of the embodiments of this application, a knowledge graph question answering method is provided, including: obtaining a first query statement, and using an analysis agent to extract keywords in the first query statement, where the first query statement is a statement in natural language format; using a tool agent to query relevant information corresponding to the keywords, where the relevant information is information for explaining the concept and meaning of the keywords; using the analysis agent to determine an equivalent question corresponding to the first query statement based on the relevant information, and using an encoding agent to encode the equivalent question to obtain a second query statement, where the second query statement is a statement in graph query language format; in the case where an inspection agent determines that the second query statement has no syntax error, using the tool agent to query in the knowledge graph according to the second query statement to obtain an answer result to the question corresponding to the first query statement; where the analysis agent, tool agent, encoding agent, and inspection agent are all models obtained based on a large language model.

[0006] Optionally, the method further includes: obtaining a large language model and setting different roles for the large language model to obtain multiple agents, where the roles include: an analysis role, a coding role, an inspection role, and a tool role, and the agents include: an analysis agent corresponding to the analysis role, a coding agent corresponding to the coding role, an inspection agent corresponding to the inspection role, and a tool agent corresponding to the tool role; configuring the background knowledge, memory information, and actions corresponding to each agent, where the background knowledge is the relevant knowledge information required by the agent when performing actions, the memory information is the historical input information received by each agent during the historical knowledge graph question-answering process and the corresponding historical output information, and the actions are the capabilities of the encapsulated agent to process information.

[0007] Optionally, the background knowledge corresponding to the analysis agent includes: sentence transformation methods, symmetric relationships, and mutually exclusive relationships; the background knowledge corresponding to the coding agent includes: multiple query examples and the graph query languages corresponding to the query examples; the background knowledge corresponding to the inspection agent includes: multiple grammar error example samples; and the background knowledge corresponding to the tool agent includes: the calling methods of multiple tools and the corresponding interface information.

[0008] Optionally, the actions corresponding to the analysis agent include: a question analysis action for extracting keywords in the first query statement and a question reasoning action for determining an equivalent question corresponding to the first query statement; the actions corresponding to the coding agent include: an action for generating a second query statement according to the first query statement, an action for regenerating the second query statement according to the second query statement and grammar error information, and an action for updating the second query statement according to the outline information of the knowledge graph; the actions corresponding to the inspection agent include: an action for checking the grammar errors of the statements in the graph query language format; and the actions corresponding to the tool agent include: a concept query action for querying relevant information corresponding to the keywords, a parsing action for parsing the second query statement, a recall action for recalling the outline information of the knowledge graph, and an execution action for executing the second query statement to obtain an execution result.

[0009] Optionally, after obtaining the second query statement, the method further includes: using the inspection agent to perform a grammar check on the second query statement, and generating grammar error information corresponding to the second query statement when it is determined that the second query statement has grammar errors; using the coding agent to regenerate the second query statement according to the second query statement and the grammar error information, and sending the regenerated second query statement to the inspection agent again for grammar checking.

[0010] Optionally, querying in the knowledge graph according to the second query statement includes: using a tool agent to parse the second query statement to obtain the constituent elements in the second query statement, and recalling the schema information corresponding to the constituent elements in the knowledge graph, where the schema information is used to represent the specific entities, attributes, and relationships included in the knowledge graph, and the constituent elements include at least one of the following: entity elements, attribute elements, relationship elements; using an encoding agent to replace each constituent element in the second query statement with the corresponding entity, attribute, and relationship according to the schema information to obtain an updated second query statement; using a tool agent to query in the knowledge graph according to the updated second query statement to obtain a query result.

[0011] Optionally, after obtaining the query result, the method further includes: determining a natural language format answer template corresponding to the first query statement; generating a natural language format answer result to the question corresponding to the first query statement according to the answer template and the query result, and sending the answer result to the front-end interaction interface for display.

[0012] According to another aspect of the embodiments of the present application, there is also provided a knowledge graph question answering device, including: a query statement analysis module, configured to obtain a first query statement and use an analysis agent to extract keywords in the first query statement, where the first query statement is a statement in natural language format; a relevant information query module, configured to use a tool agent to query relevant information corresponding to the keywords, where the relevant information is information used to explain the concept and meaning of the keywords; a statement encoding and transformation module, configured to use an analysis agent to determine an equivalent question corresponding to the first query statement according to the relevant information, and use an encoding agent to encode the equivalent question to obtain a second query statement, where the second query statement is a statement in graph query language format; a knowledge graph query module, configured to, when the inspection agent determines that the second query statement has no syntax error, use a tool agent to query in the knowledge graph according to the second query statement to obtain an answer result to the question corresponding to the first query statement; where the analysis agent, the tool agent, the encoding agent, and the inspection agent are all models obtained based on a large language model.

[0013] According to another aspect of the embodiment of the present application, a knowledge graph question-answering system is also provided, including: an analysis agent, a tool agent, a coding agent and an inspection agent obtained based on a large language model, wherein the analysis agent is used to extract keywords in a first query statement and send the keywords to the tool agent, wherein the first query statement is a statement in a natural language format; and, based on the relevant information corresponding to the keywords returned by the tool agent, determine an equivalent question corresponding to the first query statement, and send the equivalent question to the coding agent; the coding agent is used to encode the equivalent question sent by the analysis agent to obtain a second query statement, and send the equivalent question to the coding agent. Send to the inspection agent, wherein the second query statement is a statement in the graph query language format; the inspection agent is used to perform a grammatical check on the second query statement sent by the coding agent, and when it is determined that the second query statement does not have a grammatical error, send the second query statement to the tool agent; the tool agent is used to query relevant information corresponding to the keyword and return the relevant information to the analysis agent, wherein the relevant information is information used to explain the concept and meaning of the keyword; and, receive the second query statement sent by the inspection agent, and based on the second query statement, query in the knowledge graph to obtain the answer result of the question corresponding to the first query statement.

[0014] According to another aspect of an embodiment of the present application, an electronic device is also provided, including: a memory and a processor, the processor being used to run a program stored in the memory, wherein the knowledge graph question-answering method is executed when the program is running.

[0015] According to another aspect of an embodiment of the present application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the knowledge graph question and answer method by running the computer program.

[0016] According to another aspect of the embodiments of the present application, a computer program product is also provided, including a computer program, which implements the steps of the knowledge graph question and answer method when the computer program is executed by a processor.

[0017] In the embodiments of the present application, a first query statement is obtained, and an analysis agent is used to extract keywords from the first query statement, where the first query statement is a statement in natural language format; a tool agent is used to query relevant information corresponding to the keywords, where the relevant information is information for explaining the concepts and meanings of the keywords; an analysis agent is used to determine an equivalent question corresponding to the first query statement based on the relevant information, and an encoding agent is used to encode the equivalent question to obtain a second query statement, where the second query statement is a statement in graph query language format; in the case where the inspection agent determines that the second query statement has no syntax error, a tool agent is used to query in the knowledge graph according to the second query statement to obtain the answer result of the question corresponding to the first query statement. In this way, through the graph question-answering method based on multiple agents, while retaining the advantages of the graph question-answering solution based on large models, the universality, recall rate, and reasoning ability of graph question-answering are improved, thereby solving the technical problems of poor universality, low recall rate, and weak reasoning ability in the knowledge graph question-answering method in the related art due to limitations such as the length of the prompt information input to the large model and the lack of a verification mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0019] Figure 1 is a hardware structure block diagram of a computer terminal (or electronic device) for implementing a method for knowledge graph question-answering according to an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of a method flow for knowledge graph question-answering according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of an overall process of a multi-agent-based knowledge graph question-answering method according to an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of an agent structure according to an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of multi-agent collaboration according to an embodiment of the present application;

[0024] Figure 6 is a schematic diagram of a question-answering process implemented based on multi-agent collaboration according to an embodiment of the present application;

[0025] Figure 7It is a schematic diagram of an equivalent reasoning example provided according to an embodiment of the present application;

[0026] Figure 8 It is a schematic diagram of a GQL generation process provided according to an embodiment of the present application;

[0027] Figure 9 It is a schematic diagram of a schema adaptation process provided according to an embodiment of the present application;

[0028] Figure 10 It is a schematic diagram of an answer optimization process provided according to an embodiment of the present application;

[0029] Figure 11 It is a schematic diagram of the structure of a knowledge graph question answering device provided according to an embodiment of the present application;

[0030] Figure 12 It is a schematic diagram of the structure of a knowledge graph question answering system provided according to an embodiment of the present application. Detailed implementation manners

[0031] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order different from those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0033] In order to facilitate those skilled in the art to better understand the embodiments of the present application, some technical terms or noun explanations involved in the embodiments of the present application are as follows:

[0034] Knowledge graph: A structured knowledge base that organizes, stores, and manages entities in the physical world and their interrelationships in symbolic form.

[0035] Large language model: A natural language processing technology based on deep learning. By learning from a large amount of text data, large language models can generate natural language text, answer questions, perform language translation, text summarization and other tasks.

[0036] Agent: On the basis of a large language model, add capabilities such as memory, perception, and planning, so that it can interact with the environment autonomously and make decisions as a software entity.

[0037] Knowledge graph-based question answering is a typical application of knowledge graphs. It generally uses a pre-constructed knowledge graph to understand and answer natural language questions raised by users. In recent years, with the development of large models, the technical route of converting questions into GQL and then combining with the constructed graph for question answering has exceeded traditional syntactic parsing, subgraph recall and other methods in terms of multi-hop capabilities and question answering accuracy. The general idea is to design the prompt of the large model, input the user's question and the schema of the graph into the large model, use the large model to generate the corresponding GQL, and query the corresponding results of the question in the subgraph.

[0038] However, in related technologies, the knowledge graph question answering method combined with large models still has some defects, as follows:

[0039] 1) Poor universality. Limited by the prompt of the large model input, when there are many types of graph nodes and edges, describing the schema of the graph requires a very long prompt, exceeding the input limit of the model. At this time, the method of fine-tuning the model is generally used to avoid inputting the graph schema, but the fine-tuned model loses its universality and cannot be applied to other graphs.

[0040] 2) Low recall rate. When the variables in the GQL statement generated by the large model do not match the types in the graph or there are syntax errors, invalid results are usually returned.

[0041] 3) Weak reasoning ability. Especially when the graph itself is not complete enough, corresponding results cannot be obtained through appropriate reasoning.

[0042] In summary, the graph question answering scheme based on large models in related technologies has obvious advantages in terms of multi-hop capabilities and answer accuracy compared with traditional grammar parsing-based schemes. However, limited by the prompt input length, GQL grammar ability, etc., the graph question answering scheme based on large models still has problems such as poor universality, low recall rate and weak reasoning ability. To solve the above problems, relevant solutions are provided in the embodiments of this application, which have the characteristics of high universality, high recall and strong reasoning, and will be described in detail below.

[0043] According to an embodiment of the present application, a method embodiment for knowledge graph question answering is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0044] The method embodiment provided by the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or electronic device) for implementing the knowledge graph question answering method is shown. As Figure 1 shown, the computer terminal 10 (or electronic device) may include one or more (shown as 102a, 102b,..., 102n in the figure) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may further include more or fewer components than Figure 1 shown, or have a different configuration from Figure 1 shown.

[0045] It should be noted that the above one or more processors 102 and / or other data processing circuits are generally referred to as "data processing circuits" in this article. The data processing circuit may be embodied in software, hardware, firmware or any combination thereof, in whole or in part. In addition, the data processing circuit may be a single independent processing module, or be incorporated in whole or in part into any one of the other elements in the computer terminal 10 (or electronic device). As involved in the embodiment of the present application, the data processing circuit is a processor control (such as the selection of a variable resistance terminal path connected to an interface).

[0046] The memory 104 can be used to store software programs and modules of application software, such as the program instruction / data storage device corresponding to the knowledge graph question-answering method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above-mentioned knowledge graph question-answering method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the computer terminal 10 through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0047] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of the computer terminal 10. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0048] The display can be, for example, a touch-screen liquid crystal display (LCD), and the liquid crystal display enables a user to interact with the user interface of the computer terminal 10 (or an electronic device).

[0049] Under the above operating environment, the embodiments of the present application provide a knowledge graph question-answering method. Figure 2 It is a schematic diagram of the method flow of a knowledge graph question-answering according to the embodiments of the present application, as Figure 2 shown. The method includes the following steps:

[0050] Step S202: Obtain a first query statement, and use an analysis agent to extract keywords in the first query statement, where the first query statement is a statement in natural language format;

[0051] Step S204: Use a tool agent to query relevant information corresponding to the keywords, where the relevant information is information used to explain the concept and meaning of the keywords;

[0052] Step S206: Use an analysis agent to determine an equivalent question corresponding to the first query statement based on the relevant information, and use an encoding agent to encode the equivalent question to obtain a second query statement, where the second query statement is a statement in graph query language format;

[0053] Step S208. When the inspection agent determines that there is no syntax error in the second query statement, the tool agent is used to query in the knowledge graph according to the second query statement to obtain the answer result of the question corresponding to the first query statement.

[0054] Among them, the above analysis agent, tool agent, coding agent, and inspection agent are all models obtained based on the large language model.

[0055] Through the above steps, through the multi-agent-based knowledge graph question-answering method, while retaining the advantages of the large model-based knowledge graph question-answering solution, the purpose of improving the universality, recall rate, and reasoning ability of the knowledge graph question-answering is achieved. Furthermore, the knowledge graph question-answering method in the related technology often has technical problems such as poor universality, low recall rate, and weak reasoning ability due to being limited by the length of the prompt information input by the large model and the lack of a verification mechanism.

[0056] Next, the knowledge graph question-answering method in steps S202 to S208 of the embodiment of the present application will be further introduced.

[0057] Figure 3 It is a schematic diagram of the overall process of a multi-agent-based knowledge graph question-answering method provided by an embodiment of the present application, as Figure 3 shown.

[0058] First, the required agents are constructed. In this embodiment, the agents that need to be constructed specifically include: an analysis agent (Agent-Reasoner), a coding agent (Agent-Coder), an inspection agent (Agent-Inspector), and a tool agent (Agent-Operator). The specific steps for constructing the agents are as follows.

[0059] In some embodiments of the present application, the method further includes the following steps: obtaining a large language model and setting different roles for the large language model to obtain multiple agents, where the roles include: an analysis role, a coding role, an inspection role, and a tool role, and the agents include: an analysis agent corresponding to the analysis role, a coding agent corresponding to the coding role, an inspection agent corresponding to the inspection role, and a tool agent corresponding to the tool role; configuring the background knowledge, memory information, and actions corresponding to each agent, where the background knowledge is the relevant knowledge information required by the agent when performing actions, the memory information is the historical input information received by each agent during the historical knowledge graph question-answering process and the corresponding historical output information, and the actions are the capabilities of the encapsulated agent to process information.

[0060] Specifically, the structure of the constructed agent is asFigure 4 As shown, each agent includes five parts: a large language model, a role definition, background knowledge, memory information, and actions. Among them, the large language model is the core component of the agent, with text processing and decision-making capabilities. The role definition is used to basically define the agent, prevent the model from having too much freedom when answering questions, and restrict the content of the model's answers, which can improve the answering accuracy. The background knowledge is the background knowledge required in the relevant field, which can assist the large model in reasoning with a chain of thought. The memory information is the historical Q&A in the task, and the large model can adjust subsequent strategies based on this. The action is the ability of the large model to encapsulate the processing of external information based on the role definition, background knowledge, and memory content. Compared with directly using the large model, it has higher accuracy and stronger universality.

[0061] In this embodiment, there are four types of agents, namely: analysis agent, coding agent, inspection agent, and tool agent. In this embodiment, the large language model can be respectively defined according to analysis, coding, inspection, and tools through prompts to obtain the above four types of agents; and the memory information of the four types of agents is set in the same json structure, for example, the role + content is recorded in chronological order;

[0062] For the background knowledge, the content of the four types of agents is different, but all can be implemented in the dictionary form of concept + definition. Among them, the background knowledge corresponding to the analysis agent includes: definitions such as sentence transformation methods, symmetric relationships, and mutually exclusive relationships; the background knowledge corresponding to the coding agent includes: various basic query examples and the corresponding graph query languages for the query examples; the background knowledge corresponding to the inspection agent includes: various typical grammar error example samples; the background knowledge corresponding to the tool agent includes: information on the call methods of various tools and the corresponding interfaces (Application Programming Interface, API).

[0063] In addition, the actions corresponding to each agent are also different. In the embodiments of the present application, the action settings can be implemented through templated prompts by combining the role definitions, memory information, and background knowledge within the agent. Specifically, the actions corresponding to the analysis agent include: a question analysis action for extracting keywords in the first query statement, and a question reasoning action for determining an equivalent question corresponding to the first query statement. The actions corresponding to the encoding agent include: an action of generating a second query statement according to the first query statement (generating GQL according to the question), an action of regenerating the second query statement according to the second query statement and the syntax error information (generating GQL according to the GQL and the error information), and an action of updating the second query statement according to the outline information of the knowledge graph (generating GQL according to the schema). The actions corresponding to the patrol agent include: an action of checking the syntax errors of statements in the graph query language format. The actions corresponding to the tool agent include: a concept query action for querying relevant information corresponding to the keywords (querying concept information), a parsing action for parsing the second query statement (parsing GQL), a recall action for recalling the outline information of the knowledge graph (semantically recalling schema information), and an execution action for executing the second query statement to obtain an execution result (executing GQL and merging the results).

[0064] After the agents are constructed, query statements can be adaptively generated by multiple agents, as Figure 5 shown. By generating multiple collaborations among the four agents, the user's question is converted into a GQL statement that can be universally applied to various graphs and has the characteristics of high recall and strong reasoning, and the GQL language is executed on the graph to obtain the result. The collaboration process among the multiple agents will be further introduced below.

[0065] Figure 6 is a schematic diagram of a question-answering process implemented based on multi-agent collaboration provided by an embodiment of the present application, as Figure 6 shown.

[0066] First, the analysis agent uses the question decomposition action to analyze the question, extract the keywords (such as some terms, basic concepts, proper nouns, etc.) in the question (i.e., the first query statement) proposed by the user, and transfer the extracted keywords to the tool agent. Then, the tool agent uses the concept query action to retrieve and interpret the keywords. Specifically, relevant information of the keywords can be queried by calling tools such as a knowledge base and a search engine, and returned to the analysis agent;

[0067] After the keywords and relevant information are clear, the analysis agent combines the relevant information queried to perform equivalent reasoning on the question, obtains multiple equivalent questions, gets at least one question description that can be queried, and sends it to the encoding agent. For example, asFigure 7 As shown, the original question expressed by the first query statement is: "Who is the author of the Harry Potter series of novels", and then using the analysis agent, it can be inferred that its equivalent questions include: "Who created the Harry Potter series of novels".

[0068] For the original question and each inferred equivalent question, the encoding agent can be used for encoding to write its corresponding query GQL (i.e., the second query statement), and after sending it to the inspection agent, the inspection agent can be used to check the syntax of the query GQL (i.e., the second query statement). The specific steps are as follows.

[0069] In some embodiments of the present application, after obtaining the second query statement, the method further includes the following steps: using the inspection agent to check the syntax of the second query statement, and in the case of determining that the second query statement has a syntax error, generating a syntax error message corresponding to the second query statement; using the encoding agent, according to the second query statement and the syntax error message, regenerating the second query statement, and sending the regenerated second query statement to the inspection agent again for syntax checking.

[0070] For example, as Figure 8 shown, the inspection agent checks the syntax of the query GQL statement generated by the encoding agent. If there is an error, the error message, statement code, and question are returned to the encoding agent for secondary encoding. The encoding agent, according to the error message, calls the encoding action according to the error message, rewrites the GQL language and sends it to the inspection agent, and repeats this operation until the inspection agent believes that there is no syntax error.

[0071] The encoded code is checked for syntax by the inspection agent. In the case where the inspection agent determines that the second query statement has no syntax error, the check passes and proceeds to the next step. For the second query statement with passed syntax check, GQL parsing is performed, and the corresponding schema is semantically recalled to adapt the code to the current knowledge graph. The specific steps are as follows.

[0072] In some embodiments of the present application, querying in the knowledge graph according to the second query statement includes the following steps: using the tool agent to parse the second query statement to obtain the constituent elements in the second query statement, and recalling the outline information corresponding to the constituent elements in the knowledge graph, where the outline information is used to represent the specific entities, attributes, and relationships included in the knowledge graph, and the constituent elements include at least one of the following: entity element, attribute element, relationship element; using the encoding agent, according to the outline information, replacing each constituent element in the second query statement with the corresponding entity, attribute, and relationship to obtain an updated second query statement; using the tool agent, according to the updated second query statement, querying in the knowledge graph to obtain a query result.

[0073] Specifically, as Figure 9 shown, the tool agent parses the query GQL that has passed the syntax check, and based on the parsed entity elements, entity attribute elements, relationship elements, etc., semantically retrieves the corresponding schema in the knowledge graph (i.e., the above-mentioned outline information), and sends the schema to the encoding agent. For example, based on the constituent elements parsed from the second query statement: Person and name, the corresponding schema: Person and Name is retrieved in the knowledge graph. Then, the encoding agent regenerates the query GQL based on the retrieved schema, that is, modifies and updates the second query statement and performs a syntax check operation. Finally, the tool agent executes the second query statement and performs result aggregation (basic algorithms such as voting method can be used) to obtain the query result of the knowledge graph.

[0074] Finally, after obtaining the result of the graph query, it is also possible to optimize the answer in combination with the user's question, and use the large language model to expand the query result of the knowledge graph into a complete answer statement, and regenerate a natural, fluent and easy-to-understand answer. The specific steps are as follows.

[0075] As an optional embodiment, after obtaining the query result, the method further includes the following steps: determining a natural language format answer template corresponding to the first query statement; generating a natural language format answer result for the question corresponding to the first query statement based on the answer template and the query result, and sending the answer result to the front-end interaction interface for display.

[0076] For example, as Figure 10 shown, for the query result "Jia Min" corresponding to the first query statement "Who is Lin Daiyu's mother", after optimization, the natural language format answer result of "Lin Daiyu's mother is Jia Min" is finally output.

[0077] The graph question-answering method based on multi-agent in the embodiments of the present application, compared with the graph question-answering method that directly calls the large model in the related technology, in the collaborative process, improves the recall rate of question-answering through cyclic syntax checking, and improves the reasoning ability of question-answering through question reasoning and concept information supplementation, and based on the generation-retrieval-generation route, realizes the adaptation of the graph structure under a short prompt, improves the universality of graph question-answering, and has the characteristics of high universality, high recall rate and strong reasoning ability. In addition, compared with other agent design schemes, the solution of the present application has better generalization ability, and can derive various agents such as analysis agents, encoding agents, inspection agents and tool agents, so as to perform collaborative question-answering generation.

[0078] According to the embodiments of the present application, an embodiment of a knowledge graph question-answering device is also provided. Figure 11The structural schematic diagram of a knowledge graph question-answering device provided according to an embodiment of the present application. As Figure 11 shown, the device includes:

[0079] A query statement analysis module 110, configured to obtain a first query statement and extract keywords in the first query statement by using an analysis agent, where the first query statement is a statement in natural language format;

[0080] A relevant information query module 112, configured to query relevant information corresponding to the keywords by using a tool agent, where the relevant information is information for explaining the concepts and meanings of the keywords;

[0081] A statement encoding conversion module 114, configured to determine an equivalent question corresponding to the first query statement according to the relevant information by using an analysis agent, and encode the equivalent question by using an encoding agent to obtain a second query statement, where the second query statement is a statement in graph query language format;

[0082] A knowledge graph query module 116, configured to, when a patrol agent determines that the second query statement has no syntax error, query in the knowledge graph according to the second query statement by using a tool agent to obtain an answer result to the question corresponding to the first query statement.

[0083] Optionally, the knowledge graph question-answering device is further configured to: obtain a large language model, and set different roles for the large language model to obtain multiple agents, where the roles include: an analysis role, an encoding role, a patrol role, and a tool role, and the agents include: an analysis agent corresponding to the analysis role, an encoding agent corresponding to the encoding role, a patrol agent corresponding to the patrol role, and a tool agent corresponding to the tool role; configure background knowledge, memory information, and actions corresponding to each agent, where the background knowledge is relevant knowledge information required by the agent when performing actions, the memory information is historical input information received by each agent in the process of historical knowledge graph question-answering, and historical output information corresponding thereto, and the action is the ability of the encapsulated agent to execute processing information.

[0084] Optionally, the background knowledge corresponding to the analysis agent includes: sentence transformation methods, symmetric relationships, and mutually exclusive relationships; the background knowledge corresponding to the encoding agent includes: multiple query examples and the graph query language corresponding to the query examples; the background knowledge corresponding to the patrol agent includes: multiple syntax error example samples; and the background knowledge corresponding to the tool agent includes: call methods of multiple tools and corresponding interface information.

[0085] Optionally, the actions corresponding to the analysis agent include: a question analysis action for extracting keywords in the first query statement, and a question reasoning action for determining an equivalent question corresponding to the first query statement. The actions corresponding to the encoding agent include: an action of generating a second query statement according to the first query statement, an action of regenerating the second query statement according to the second query statement and the syntax error information, and an action of updating the second query statement according to the schema information of the knowledge graph. The actions corresponding to the inspection agent include: an action of checking the syntax error of a statement in the graph query language format. The actions corresponding to the tool agent include: a concept query action for querying relevant information corresponding to the keywords, a parsing action for parsing the second query statement, a recall action for recalling the schema information of the knowledge graph, and an execution action for executing the second query statement to obtain an execution result.

[0086] Optionally, after obtaining the second query statement, the knowledge graph question answering device is further configured to: use the inspection agent to perform a syntax check on the second query statement, and generate syntax error information corresponding to the second query statement when it is determined that the second query statement has a syntax error; use the encoding agent to regenerate the second query statement according to the second query statement and the syntax error information, and send the regenerated second query statement to the inspection agent again for syntax check.

[0087] Optionally, querying in the knowledge graph according to the second query statement includes: using the tool agent to parse the second query statement to obtain the constituent elements in the second query statement, and recalling the schema information corresponding to the constituent elements in the knowledge graph, where the schema information is used to represent the specific entities, attributes, and relationships included in the knowledge graph, and the constituent elements include at least one of the following: entity elements, attribute elements, and relationship elements; using the encoding agent to replace each constituent element in the second query statement with the corresponding entity, attribute, and relationship according to the schema information to obtain an updated second query statement; using the tool agent to query in the knowledge graph according to the updated second query statement to obtain a query result.

[0088] Optionally, after obtaining the query result, the knowledge graph question answering device is further configured to: determine a natural language format answer template corresponding to the first query statement; generate a natural language format answer result to the question corresponding to the first query statement according to the answer template and the query result, and send the answer result to the front-end interaction interface for display.

[0089] It should be noted that each module in the above knowledge graph question-answering device can be a program module (for example, a set of program instructions that implement a specific function), or a hardware module. For the latter, it can be presented in the following forms, but not limited to: the manifestation form of each of the above modules is a processor, or the functions of each of the above modules are implemented by a processor.

[0090] It should be noted that the knowledge graph question-answering device provided in this embodiment can be used to execute Figure 2 the knowledge graph question-answering method shown. Therefore, the relevant explanations of the above knowledge graph question-answering method also apply to the embodiments of this application, and will not be repeated here.

[0091] According to the embodiments of this application, an embodiment of a knowledge graph question-answering system is also provided. Figure 12 is a schematic structural diagram of a knowledge graph question-answering system provided according to the embodiments of this application. As Figure 12 shown, the system includes:

[0092] An analysis agent 120, configured to extract keywords in the first query statement and send the keywords to the tool agent, where the first query statement is a statement in natural language format; and, determine an equivalent question corresponding to the first query statement according to the relevant information returned by the tool agent corresponding to the keywords, and send the equivalent question to the encoding agent;

[0093] An encoding agent 122, configured to encode the equivalent question sent by the analysis agent to obtain a second query statement, and send the encoding agent to the inspection agent, where the second query statement is a statement in graph query language format;

[0094] An inspection agent 124, configured to perform a syntax check on the second query statement sent by the encoding agent, and in the case of determining that the second query statement has no syntax error, send the second query statement to the tool agent;

[0095] A tool agent 126, configured to query relevant information corresponding to the keywords and return the relevant information to the analysis agent, where the relevant information is information used to explain the concept and meaning of the keywords; and, receive the second query statement sent by the inspection agent, and query in the knowledge graph according to the second query statement to obtain an answer result to the question corresponding to the first query statement.

[0096] It should be noted that each module in the above knowledge graph question-answering system can be a program module (for example, a set of program instructions for implementing a specific function), or a hardware module. For the latter, it can be presented in the following forms, but not limited to: the manifestation form of each of the above modules is a processor, or the functions of each of the above modules are implemented by a processor.

[0097] It should be noted that the knowledge graph question-answering system provided in this embodiment can be used to execute Figure 2 the knowledge graph question-answering method shown. Therefore, the relevant explanations of the above knowledge graph question-answering method also apply to the embodiments of this application, and will not be repeated here.

[0098] The embodiment of this application also provides a non-volatile storage medium. The non-volatile storage medium includes a stored computer program. Among them, the device where the non-volatile storage medium is located executes the following knowledge graph question-answering method by running the computer program: obtaining a first query statement, and using an analysis agent to extract keywords in the first query statement, where the first query statement is a statement in natural language format; using a tool agent to query relevant information corresponding to the keywords, where the relevant information is information for explaining the concept and meaning of the keywords; using an analysis agent to determine an equivalent question corresponding to the first query statement based on the relevant information, and using an encoding agent to encode the equivalent question to obtain a second query statement, where the second query statement is a statement in graph query language format; in the case where the inspection agent determines that the second query statement has no syntax error, using a tool agent to query in the knowledge graph based on the second query statement to obtain the answer result to the question corresponding to the first query statement.

[0099] The embodiment of this application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of the knowledge graph question-answering method described in each embodiment of this application: obtaining a first query statement, and using an analysis agent to extract keywords in the first query statement, where the first query statement is a statement in natural language format; using a tool agent to query relevant information corresponding to the keywords, where the relevant information is information for explaining the concept and meaning of the keywords; using an analysis agent to determine an equivalent question corresponding to the first query statement based on the relevant information, and using an encoding agent to encode the equivalent question to obtain a second query statement, where the second query statement is a statement in graph query language format; in the case where the inspection agent determines that the second query statement has no syntax error, using a tool agent to query in the knowledge graph based on the second query statement to obtain the answer result to the question corresponding to the first query statement.

[0100] The serial numbers of the embodiments of the present application above are only for description and do not represent the advantages or disadvantages of the embodiments.

[0101] In the above embodiments of the present application, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0102] In the several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units can be a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the units or modules can be in an electrical or other form.

[0103] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0104] In addition, the functional units in the various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0105] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. And the aforementioned storage medium includes: USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks or optical disks and other various media that can store program codes.

[0106] The above are only the preferred embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A knowledge graph question answering method, characterized in that, Including: Obtain a first query statement, and use an analysis agent to extract keywords in the first query statement, where the first query statement is a statement in natural language format; Use a tool agent to query relevant information corresponding to the keywords, where the relevant information is information for explaining the concepts and meanings of the keywords; Use the analysis agent to determine an equivalent question corresponding to the first query statement based on the relevant information, and use an encoding agent to encode the equivalent question to obtain a second query statement, where the second query statement is a statement in graph query language format; In the case where the inspection agent determines that the second query statement has no syntax error, use the tool agent to query in the knowledge graph according to the second query statement to obtain an answer result to the question corresponding to the first query statement; Among them, the analysis agent, the tool agent, the encoding agent, and the inspection agent are all models obtained based on a large language model. The method further includes: configuring background knowledge, memory information, and actions corresponding to each of the agents, where the background knowledge is relevant knowledge information required by the agent when performing the action, the memory information is the historical input information received by each agent in the process of historical knowledge graph question answering and the corresponding historical output information, and the action is the ability of the encapsulated agent to process information. The background knowledge corresponding to the analysis agent includes: sentence transformation methods, symmetric relationships, and mutually exclusive relationships. The background knowledge corresponding to the encoding agent includes: multiple query examples and the graph query language corresponding to the query examples. The background knowledge corresponding to the inspection agent includes: multiple syntax error example samples. The background knowledge corresponding to the tool agent includes: call methods of multiple tools and corresponding interface information.

2. The knowledge graph question answering method according to claim 1, wherein The method further includes: Obtain a large language model, and set different roles for the large language model to obtain multiple agents, where the roles include: an analysis role, an encoding role, an inspection role, and a tool role. The agents include: the analysis agent corresponding to the analysis role, the encoding agent corresponding to the encoding role, the inspection agent corresponding to the inspection role, and the tool agent corresponding to the tool role.

3. The knowledge graph question answering method according to claim 2, wherein The actions corresponding to the analysis agent include: a question analysis action for extracting keywords in the first query statement, and a question reasoning action for determining an equivalent question corresponding to the first query statement. The actions corresponding to the encoding agent include: an action of generating the second query statement according to the first query statement, an action of regenerating the second query statement according to the second query statement and the syntax error information, and an action of updating the second query statement according to the schema information of the knowledge graph. The actions corresponding to the inspection agent include: an action of checking the syntax error of a statement in the graph query language format. The actions corresponding to the tool agent include: a concept query action for querying relevant information corresponding to the keywords, a parsing action for parsing the second query statement, a recall action for recalling the schema information of the knowledge graph, and an execution action for executing the second query statement to obtain an execution result.

4. The knowledge graph question answering method according to claim 1, wherein After obtaining the second query statement, the method further includes: Using the inspection agent to perform a syntax check on the second query statement, and generating syntax error information corresponding to the second query statement when it is determined that the second query statement has a syntax error; Using the encoding agent to regenerate the second query statement according to the second query statement and the syntax error information, and sending the regenerated second query statement to the inspection agent again for syntax check.

5. The knowledge graph question answering method according to claim 4, characterized in that, Querying in the knowledge graph according to the second query statement includes: Using the tool agent to parse the second query statement to obtain the constituent elements in the second query statement, and recalling the schema information corresponding to the constituent elements in the knowledge graph, where the schema information is used to represent the specific entities, attributes, and relationships included in the knowledge graph, and the constituent elements include at least one of the following: entity elements, attribute elements, and relationship elements; Using the encoding agent to replace each of the constituent elements in the second query statement with the corresponding entity, attribute, and relationship according to the schema information to obtain an updated second query statement; Using the tool agent to query in the knowledge graph according to the updated second query statement to obtain a query result.

6. The knowledge graph question answering method according to claim 5, wherein After obtaining the query result, the method further includes: Determining a natural language format answer template corresponding to the first query statement; Generating a natural language format answer result for the question corresponding to the first query statement according to the answer template and the query result, and sending the answer result to the front-end interaction interface for display.

7. A knowledge graph question answering device, characterized in that Including: A query statement analysis module for obtaining a first query statement and using an analysis agent to extract keywords in the first query statement, where the first query statement is a statement in natural language format; A relevant information query module for using a tool agent to query relevant information corresponding to the keywords, where the relevant information is information used to explain the concept and meaning of the keywords; A statement encoding conversion module, which uses the analysis agent to determine an equivalent question corresponding to the first query statement according to the relevant information, and uses an encoding agent to encode the equivalent question to obtain a second query statement, where the second query statement is a statement in the format of a graph query language; A knowledge graph query module, which, when the inspection agent determines that the second query statement has no syntax error, uses the tool agent to query in the knowledge graph according to the second query statement to obtain an answer result to the question corresponding to the first query statement; Among them, the analysis agent, the tool agent, the encoding agent, and the inspection agent are all models obtained based on a large language model. The knowledge graph question-answering device is also used to configure the background knowledge, memory information, and actions corresponding to each of the agents. The background knowledge is the relevant knowledge information required by the agent when performing the action. The memory information is the historical input information received by each agent in the process of historical knowledge graph question answering and the corresponding historical output information. The action is the ability of the encapsulated agent to execute processing information. Among them, the background knowledge corresponding to the analysis agent includes: sentence transformation methods, symmetric relationships, and mutually exclusive relationships. The background knowledge corresponding to the encoding agent includes: multiple query examples and the graph query language corresponding to the query examples. The background knowledge corresponding to the inspection agent includes: multiple syntax error example samples. The background knowledge corresponding to the tool agent includes: call methods of multiple tools and corresponding interface information.

8. A knowledge graph question answering system, characterized in that Including: An analysis agent, a tool agent, an encoding agent, and an inspection agent obtained based on a large language model, where The analysis agent is used to extract keywords in the first query statement and send the keywords to the tool agent, where the first query statement is a statement in natural language format; and, according to the relevant information corresponding to the keywords returned by the tool agent, determine an equivalent question corresponding to the first query statement and send the equivalent question to the encoding agent; The encoding agent is used to encode the equivalent question sent by the analysis agent to obtain a second query statement and send the encoded statement to the inspection agent, where the second query statement is a statement in the format of a graph query language; The inspection agent is used to perform a syntax check on the second query statement sent by the encoding agent, and when it determines that the second query statement has no syntax error, send the second query statement to the tool agent; The tool agent is used to query relevant information corresponding to the keyword and return the relevant information to the analysis agent, where the relevant information is information for explaining the concept and meaning of the keyword; and, receive the second query statement sent by the inspection agent, and query in the knowledge graph according to the second query statement to obtain an answer result to the question corresponding to the first query statement. Among them, the analysis agent, the tool agent, the coding agent, and the inspection agent are all models obtained based on a large language model, and are obtained by setting different roles for the large language model and configuring the background knowledge, memory information, and actions corresponding to each agent. Among them, the background knowledge is the relevant knowledge information required by the agent when performing the action, the memory information is the historical input information received by each agent in the process of answering questions in the historical knowledge graph, and the corresponding historical output information, and the action is the ability of the encapsulated agent to execute processing information. Among them, the background knowledge corresponding to the analysis agent includes: sentence transformation methods, symmetric relationships, and mutually exclusive relationships. The background knowledge corresponding to the coding agent includes: various query examples and the graph query language corresponding to the query examples. The background knowledge corresponding to the inspection agent includes: various grammar error example samples. The background knowledge corresponding to the tool agent includes: call methods of various tools and corresponding interface information.

9. An electronic device, characterized in that, Including: A memory and a processor, where the processor is used to run a program stored in the memory, and where the program, when running, executes the knowledge graph question answering method according to any one of claims 1 to 6.

10. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored computer program, and the device where the non-volatile storage medium is located executes the knowledge graph question answering method according to any one of claims 1 to 6 by running the computer program.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the knowledge graph question answering method according to any one of claims 1 to 6 are implemented.

Images