Answer method based on large language model and knowledge graph

Abstract

The invention provides a method for answering based on a large language model and a knowledge graph, which comprises the following steps of: inputting a skeleton, a subject entity or a required relationship of a logic query statement corresponding to a given question into a statement generation model to generate the logic query statement; executing the logic query statement on the knowledge graph to obtain an answer to the given question; wherein the skeleton generation model is used for generating a skeleton of a logic query statement for a given problem; the theme entity generation model is used for generating a theme of a logic query statement for a given problem; and the relation generation model is used for generating a relation required by a logic query statement for a given problem. According to the method, the logic query statement for querying the answer on the knowledge graph can be generated for the given question, and meanwhile, the logic query statement is fused with the knowledge graph by retrieving related information in the knowledge graph in the generation process, so that the accuracy of generating the logic query statement by the large language model is improved.

Description

Technical Field

[0001] The present invention relates to the fields of natural language processing and knowledge graphs, and particularly to a method for answering questions based on large language models and knowledge graphs. Background Art

[0002] Knowledge graph question answering is a long-standing and crucial problem in the field of natural language processing. Its main goal is to use structured information stored in knowledge graphs (such as Freebase, DBPedia, etc.) to extract accurate answers to natural language questions. Traditional knowledge graph question answering methods can mainly be divided into information retrieval-based methods and semantic parsing-based methods. Information retrieval-based methods first extract a subgraph related to the question (including entities, relationships, or triples) from the knowledge graph, and then perform reasoning on the subgraph, such as sorting entities to obtain the answer; while semantic parsing-based methods solve the problem by converting natural language questions into logical query statements (such as SPARQL or S-Expression) that can be executed on the knowledge graph. With the development of large language models, methods based on large language models have made significant progress in the field of knowledge graph question answering, showing stronger generalization ability and higher accuracy than traditional methods. For example, in the literature By utilizing the few-shot learning ability and in-context learning ability of large language models, the large language model generates a logical query statement for a given question based on examples of (question, logical query statement), and then aligns it to the knowledge graph and executes it to obtain the answer. Similarly, in the literature The process of generating logical query statements is formalized as the process of generating Python code, further leveraging the outstanding code generation ability of large language models. In addition, in the literature Fine-tuning open-source large language models such as LLaMA to generate corresponding logical query statements for given questions has also achieved good results.

[0003] However, the current methods for generating logical query statements based on LLM still cannot generate completely correct logical query statements, especially when faced with complex questions. Common errors include skeleton errors, entity errors, and relationship errors. Specifically, skeleton errors include nesting errors, inconsistency with the knowledge graph schema, or failure to accurately capture the logical relationships in the query; entity errors mean that the entities in the generated logical query statement are entities that do not exist in the knowledge graph or entities required to solve the problem are not generated; relationship errors are that the model misidentifies the relationships between entities or generates relationships that do not exist in the knowledge graph.

[0004] The root causes of these problems lie in the following two aspects:

[0005] (1) Coarse-Grained Task Formulation: Directly generating a complete logical query statement from a natural language question requires the model to perform multiple subtasks, including constructing the skeleton of the logical query statement, identifying the topic entity, generating the correct relationship, etc. This is a complex task for an untrained model.

[0006] (2) Lack of Knowledge Graph Awareness Information in the Generation Process: Large language models are usually trained on unstructured text and lack the ability to directly access knowledge graph information during the generation of logical query statements, making it easy to generate non-existent entities, relationships, or query structures.

[0007] References:

[0008] [1] T. Li, X. Ma, A. Zhuang, Y. Gu, Y. Su, and W. Chen, “Few-shot In-context Learning for Knowledge Base Question Answering,” May 04, 2023, arXiv: arXiv:2305.01750. doi: 10.48550 / arXiv.2305.01750.

[0009] [2] Z. Nie, R. Zhang, Z. Wang, and X. Liu, “Code-Style In-Context Learning for Knowledge-Based Question Answering,” Jan. 05, 2024, arXiv: arXiv:2309.04695. Accessed: Mar. 21, 2024. [Online]. Available: http: / / arxiv.org / abs / 2309.04695.

[0010] [3] H. Luo et al., “ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models,” May 30, 2024, arXiv: arXiv:2310.08975. doi: 10.48550 / arXiv.2310.08975. Summary of the Invention

[0011] To solve the above problems, a method based on large language models and knowledge graphs for answering questions is proposed in the present invention, including:

[0012] Input the skeleton, subject entity, and required relationship input statement of the logical query statement corresponding to the given question into the statement generation model to generate the logical query statement;

[0013] Execute the logical query statement on the knowledge graph to obtain the answer to the given question; where,

[0014] The skeleton generation model is used to generate the skeleton of the logical query statement for the given question;

[0015] The subject entity generation model is used to generate the subject of the logical query statement for the given question;

[0016] The relationship generation model is used to generate the required relationship of the logical query statement for the given question.

[0017] Furthermore, the skeleton generation model, the subject entity generation model, the relationship generation model, and the statement generation model are large language models.

[0018] Furthermore, the subject entity generation model is formed by performing LoRa fine-tuning on the skeleton generation model, and the fine-tuning method includes:

[0019] Use the entities appearing in the logical query statement corresponding to each question in the training set as subject entities to generate training data, and each piece of training data includes the given question and the subject entity corresponding to the question;

[0020] Perform LoRa fine-tuning on the skeleton generation model through this training data to form the subject entity generation model.

[0021] Furthermore, the relationship generation model is formed by fine-tuning the skeleton generation model or the subject entity generation model, and the fine-tuning method includes:

[0022] S11. Use a vector encoder to encode the question and the relationships in the knowledge graph, and then calculate the vector representation of the question and the vector representation of the relationship the cosine similarity between them, and select the top most relevant relationships to form a candidate relationship set ;

[0023] S12. Use the questions, subject entities, and candidate relationship sets in the training set as the input of the relationship generation model, output the relationships required to solve the question, and use the golden relationship corresponding to the question as a comparison to train the relationship generation model.

[0024] Furthermore, fine-tune the vector encoder so that the problem and the relationships required to solve the problem are closer in the vector space of the vector encoder; the fine-tuning method includes:

[0025] S21. For the problems in the training set, calculate the highly relevant relationships from the knowledge graph as the candidate relationship set ;

[0026] S22. Construct positive and negative samples according to the problem, the candidate relationship set and the relationships in the logical query statement corresponding to the problem;

[0027] S23. Use the contrastive learning algorithm to fine-tune the vector encoder.

[0028] Furthermore, in S22, the method for obtaining positive and negative samples includes: for each problem, use the training data corresponding to the relationships in the candidate relationship set but not included in the golden relationship set as hard negative samples, and the training data corresponding to the relationships in the golden relationship set as positive samples; the golden relationship is the relationship that appears in the logical query statement corresponding to the given problem.

[0029] Furthermore, the statement generation model is formed by performing LoRa fine-tuning on the relationship generation model.

[0030] Furthermore, the method for obtaining answers by executing the logical query statement on the knowledge graph includes:

[0031] Align the entities in the logical query statement with the entities in the knowledge graph:

[0032] Align the relationships in the logical query statement with the relationships in the knowledge graph:

[0033] Generate combinations of entities and relationships, and select the first combinations as candidates for the logical query statement; sequentially execute the candidate logical query statements on the knowledge graph. During the execution process, once the returned answer set is non-empty, stop searching and return the result as the final answer; if no non-empty answer set is found, return an empty set.

[0034] Furthermore, the method for entity alignment includes:

[0035] Extract the entities in the logical query statement;

[0036] Check whether there are exactly matching entities in the knowledge graph; if there is a match, mark the entity as and select the top most matching entities as candidate entities; if there are no exactly matching entities, select the top most similar entities from the knowledge graph as candidate entities.

[0037] Furthermore, the method for relationship alignment includes:

[0038] Extract the relationships in the logical query statement;

[0039] If there is an exactly matching relationship in the knowledge graph, no replacement is made; if there is no matching relationship, vector encoding is performed on the relationships in the knowledge graph and the relationships in the logical query statement, and the top relationships with the highest similarity are selected as candidate relationships according to the vector similarity.

[0040] The present invention has the following beneficial effects:

[0041] (1) By decomposing the task of directly generating logical query statements into multiple subtasks, the present invention separately generates the skeleton, entities, and relevant relationships of the logical query statements, thereby reducing the errors of directly generating these contents by large language models and helping to generate more accurate logical query statements.

[0042] (2) The present invention retrieves relevant relationships from the knowledge graph for a given problem, and at the same time fine-tunes a vector encoder to retrieve relationships more relevant to the problem in the knowledge graph for the given problem, providing more accurate knowledge graph-related information for the large language model, and thus improving the accuracy of the generated logical query statements. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0044] Figure 1 FIG.

[0045] Figure 2 is a schematic flowchart of an answering method according to an embodiment of the present invention;

[0046] Figure 3 FIG.

[0047] Figure 4 is a schematic flowchart of generating relevant relationships according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will, with reference to the accompanying drawings in the embodiments of the present invention, provide a clear and complete description of the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0049] The present invention provides a method for answering questions based on a large language model and a knowledge graph, aiming to fine-tune the large language model with LoRa to generate accurate logical query statements for a given question and obtain accurate answers in the knowledge graph. LoRa fine-tuning is an efficient fine-tuning method for large language models. Its core idea is to update parameters by introducing a low-rank matrix on the basis of a pre-trained model instead of modifying all the weights of the model. This method can significantly reduce the number of parameters to be updated, thereby reducing the training and storage overhead. As Figure 1 shown, the method includes the following steps:

[0050] Input the skeleton, topic entity, and required relationships of the logical query statement corresponding to the given question into the statement generation model to generate a logical query statement;

[0051] Execute the logical query statement on the knowledge graph to obtain the answer to the given question; where

[0052] The skeleton generation model is used to generate the skeleton of the logical query statement for a given question;

[0053] The topic entity generation model is used to generate the topic of the logical query statement for a given question;

[0054] The relationship generation model is used to generate the required relationships of the logical query statement for a given question.

[0055] In the following embodiments, the skeleton generation model, the topic entity generation model, the relationship generation model, and the statement generation model can all be large language models. The large language model can be any open-source large-scale language model, such as models like Llama2, ChatGLM, QWen, etc.

[0056] A skeleton generation model is used to generate the skeleton of a logical query statement for solving a problem based on the problem. The skeleton is a logical query statement in which entities and relationships are replaced by placeholders. By abstracting the logical query statement into a skeleton, the details of specific entities and relationships are eliminated, enabling the model to focus on the logical structure. For example, for the problem "What did Nicolas Cage name his son?", its corresponding logical query statement is: "(AND (JOIN (Rpeople.person.children) Nicolas Cage) (JOIN people.person.gender Male))", and correspondingly, its skeleton is: "(AND (JOIN (R [REL]) [ENT]) (JOIN [REL] [ENT]))". The skeleton generation model can first use a large language model to generate a logical query statement for the problem and then replace the entities and relationships therein; the skeleton generation model can also train the large language model to directly output the skeleton of the logical query statement corresponding to the problem.

[0057] During training, in the training dataset, each natural language problem has a corresponding logical query statement, and the answer can be found from the knowledge graph for the problem.

[0058] In one embodiment, the large language model is trained to output the skeleton. During the training process, the relationships in the logical query statement corresponding to each problem are replaced by [REL], and the entities are replaced by [ENT], thereby constructing the training data from the problem to the skeleton. Subsequently, the large language model is fine-tuned using LoRa with instructions to form a skeleton generation model. The skeleton generation model can generate the skeleton of the corresponding logical query statement for the input problem, thus laying a foundation for subsequent logical reasoning tasks.

[0059] The topic entity generation model can be a large language model or can be formed by fine-tuning the skeleton generation model using LoRa.

[0060] In an application, the topic entities should not only include the entities explicitly mentioned in the question, but also include the potential entities that are not directly mentioned in the question but are implicit. However, large language models do not have the function of identifying potential entities. Therefore, in order to enable large language models to generate more accurately the entities required to solve the problem and capture the entities hidden in the question rather than directly presented, the present invention fine-tunes the skeleton generation model to form a topic entity generation model to solve this problem and generate all topic entities for a given question. Specifically, for each question in the training set, the entities appearing in its corresponding logical query statement are used as topic entities. For example, for the question "What did Nicolas Cage name his son?", its corresponding logical query statement is: "(AND (JOIN (R people.person.children) Nicolas Cage)(JOIN people.person.gender Male))", then the corresponding topic entities are "Nicolas Cage" and "Male". These topic entities will be used to generate training data, that is, each piece of training data includes the given question and the topic entities corresponding to the question. Subsequently, the skeleton generation model is further fine-tuned by LoRa using this training data to form a topic entity generation model (the specific fine-tuning process is prior art and will not be elaborated). In the training set, each question has a corresponding logical query statement, and the entities in the logical query statement include such potential entities. Therefore, the present invention constructs training data using the question and the corresponding logical query statement, thereby training the topic entity generation model to accurately identify such potential entities, and further improving the robustness of the model.

[0061] As Figure 2 shown, the relationship generation model is used to generate the relationships required to solve the problem, which can be a large language model, or can be formed by fine-tuning the skeleton generation model or the topic entity generation model. Here, the relationships required to solve the problem refer to the relationships in the logical query statement corresponding to the question. For example, for the question "What did Nicolas Cage name his son?", its corresponding logical query statement is: "(AND (JOIN (R people.person.children) Nicolas Cage) (JOIN people.person.gender Male))", then the corresponding relationships required to solve the problem are "people.person.children" and "people.person.gender". Specifically, it includes:

[0062] S11. Use a vector encoder, such as BAAI / bge-large-en-v1.5, to encode the question and the relationships in the knowledge graph respectively. Then calculate the vector representation of the question and the vector representation of the relationships and select the top most relevant relationships to form a candidate relationship set .

[0063] S12. Take the question, the topic entity, and the candidate relationship set as the input of the relationship generation model, output the relationships required to solve the question, and use the golden relationship corresponding to the question as a comparison to train the initial relationship generation model (i.e., the skeleton generation model or the topic entity generation model) to form the final relationship generation model.

[0064] In a preferred embodiment, fine-tune the vector encoder so that the question and the relationships required to solve the question are closer in the vector space of the vector encoder. The fine-tuning methods include:[[]]

[0065] S21. For the questions in the training set, calculate the most relevant relationships from the knowledge graph to form a candidate relationship set ;

[0066] S22. Construct positive and negative samples according to the relationships in the question, the candidate relationship set, and the logical query statement corresponding to the question;

[0067] S23. Use the contrastive learning algorithm to fine-tune the vector encoder.

[0068] In S21, encode each question and each relationship in the knowledge graph, then calculate the cosine similarity between each question vector and relationship vector, and select the top most relevant relationships to form a preliminary candidate relationship set for each question .

[0069] In S22, the method for obtaining positive and negative samples includes: for each question in the training set, consider the training data corresponding to the relationships that are in but not in as hard negative samples, that is, relationships that are similar to the relationships required to solve the question but are irrelevant to the question, denoted as ; while the training data corresponding to the relationships in is regarded as positive samples, denoted as is the golden relationship corresponding to each question, that is, the relationship that appears in the logical query statement corresponding to the question.

[0070] In S23, optimizing contrastive learning is a commonly used method for learning data representations by contrasting positive and negative samples. In the present invention, the optimization objective is to distinguish positive and negative samples for the problem, and the loss function is defined as:

[0071]

[0072] where represents the cosine similarity between the problem vector and the relation vector . Through this fine-tuning process, the ability of the vector encoder is significantly enhanced, enabling it to better distinguish relevant and irrelevant relationships. For a given problem, the accuracy of the calculated relationships has been greatly improved.

[0073] In one embodiment, although step S11 can effectively retrieve the relationships highly relevant to the problem , the retrieval results may still contain some noise (i.e., irrelevant relationships). Therefore, a large language model can be used to filter out the truly relevant relationships from these noisy candidate relationships or generate new relationships when necessary.

[0074] Specifically, for each problem, the problem , the topic entity, and the set of candidate relationships are used as the input to the topic entity generation model, and the golden relationship corresponding to this problem is used as the output for LoRa fine-tuning, thereby forming a relationship generation model. Since the golden relationship corresponding to the problem does not always appear in the candidate relationship set , the model after LoRa fine-tuning can not only select the relationships in the candidate relationship set but also generate new relationships to make up for the deficiencies in the candidate relationship set. The relationship generation model can generate the relationships required to solve the problem, i.e., the golden relationships, thereby effectively addressing the problem of noise or irrelevant relationships and accurately generating the relationships that meet the requirements of the problem, thus improving the quality of the finally generated logical query statements.

[0075] In one embodiment, fine-tuning the skeleton generation model to generate a relationship generation model has a stronger effect than existing technical solutions but is weaker than the relationship generation model formed by fine-tuning using the topic entity generation model.

[0076] As Figure 3 shown, the statement generation model is formed by performing LoRa fine-tuning on the relationship generation model and is used to generate the final logical query statement.

[0077] Specific methods for fine-tuning the relation generation model include: inputting questions, topic entities, and relations generated by the relation generation model into the relation generation model to generate logical query statements corresponding to solving the questions, and performing fine-tuning through LoRa to form a statement generation model. After fine-tuning, the statement generation model can generate corresponding logical query statements for a given question.

[0078] In one embodiment, fine-tuning a large language model, a skeleton generation model, or a topic entity generation model to generate a statement generation model has a stronger effect than existing technical solutions but is weaker than the statement generation model formed by fine-tuning using the relation generation model.

[0079] In one embodiment, the statement generation model generates logical query statements through the beam search algorithm. The inputs to the statement generation model include questions, topic entities generated by the model, and relations generated by the model. The beam search algorithm is a heuristic search algorithm widely used in sequence generation tasks (such as machine translation, text generation, etc.). It gradually expands the most likely sequences by maintaining a set of candidate sequences (beams) instead of only focusing on the current optimal candidate, thereby finding a better output in the search space.

[0080] As Figure 4 shown, after generating the logical query statements, the possible mismatch problems between entities and relations must be solved. These entities and relations may not exactly match the entities and relations in the knowledge graph. Therefore, the entities in the logical query statements must be mapped to the entities in the knowledge graph before execution, and the relations must be ensured to be aligned with the relations in the knowledge graph. This process includes the following steps:

[0081] S31. Entity alignment. In the entity alignment stage, first extract the entities in the generated logical query statements. Subsequently, check whether there are exactly matching entities in the knowledge graph. If a match exists, we mark this entity as , and select the top most matching entities as candidate entities; if there are no exactly matching entities, select the top most similar entities (e.g., using the BM25 algorithm) from the knowledge graph as candidate entities. When selecting, the selection can be made according to the popularity score of the entities (such as the FACC1 score).

[0082] S32. Relation alignment. In the relation alignment stage, first extract the relations in the logical query statements. For each relation , if there is an exactly matching relation in the knowledge graph, no replacement is made. If there is no matching relation, select the top most similar relations to as candidate relations, for example, using the SimCSE model to separately compare the relations in the knowledge graph and Perform vector encoding and make selections based on vector similarity.

[0083] S33. Execute answer acquisition. After aligning entities and relationships, generate combinations of entities and relationships based on the generated topic entities and candidate relationships, and select the top several combinations as candidates for the logical query statement. Subsequently, execute these logical query statements on the knowledge graph in sequence. During the execution process, once the returned answer set is non-empty, stop the search and return this result as the final answer. If no non-empty answer set is found, return an empty set.

[0084] Experimental verification:

[0085] In one embodiment, the datasets used are the WebQSP dataset and the ComplexWebQSP (CWQ) dataset, both of which are widely used datasets based on the Freebase knowledge graph.

[0086] The present invention effectively enhances the performance of large language models such as Llama2 on the WebQSP dataset and the CWQ dataset. Specifically, the present invention uses Llama2-7B as the base model for experiments, that is, respectively perform S1. Fine-tune the model to generate the skeleton of the logical query statement for the given question; S2. Fine-tune the model to generate the topic entities required to solve the problem for the given question; S3. Fine-tune the model to generate the relationships required to solve the problem for the given question and topic entities; S4. Fine-tune the model to generate the logical query statement for the given question, topic entities, and relevant relationships. Subsequently, align the entities and relationships in the generated logical query statement in the knowledge graph to obtain candidate logical query statements, and execute such query statements on the knowledge graph until a non-empty query answer is obtained.

[0087] The present invention conducts experiments on the WebQSP dataset and the CWQ dataset, and uses RnG-KBQA, DecAF, ChatGPT, KV-Mem, ReasoningLM, ChatKBQA, RGR-KBQA as baseline models to compare on three metrics: F1, Hits@1, and Accuracy. The experimental results are shown in Table 1 and Table 2 respectively.

[0088] KV-Mem proposes a subgraph retrieval enhanced model for KBQA, which designs a trainable subgraph retriever (SR) decoupled from the subsequent reasoner. The SR is designed as an efficient dual-encoder, which can expand paths to induce subgraphs and can automatically stop expanding. Then, any subgraph-oriented reasoner can be used to cleverly derive answers from the subgraph. This separable retrieval and reasoning ensures that reasoning is only performed on the final whole rather than intermediate partial subgraphs, which enables a plug-and-play framework to enhance any subgraph-oriented reasoner. RNG-KBQA combines a ranker and a generator to solve the coverage problem in rank-based methods while still having generalization ability: First, use the ranker to select a set of relevant logics from the candidate logic pool obtained by searching the graph. The selected logics do not need to cover the correct form, but are semantically coherent and Figure 1 consistent with the latent meaning in the question. Then, a generator is introduced, which uses both the question and the top-k candidates to compose the final logical form. DecAF better adapts to different knowledge bases by linearizing the knowledge graph, retrieving relevant knowledge, reading, then jointly generating logical query statements and answers and combining the answers. ChatGPT is a closed-source large language model based on the GPT architecture, and we use the API service of OpenAI's gpt-3.5-turbo model. ReasoningLM proposes a subgraph-aware self-attention mechanism. This mechanism mimics the working mode of GNN and performs structured reasoning by attending to adjacent nodes on the knowledge graph. ChatKBQA proposes a generation-retrieval knowledge graph question-answering framework ChatKBQA based on large models. By using generation then retrieval instead of retrieval then generation, it solves the pain points such as low retrieval efficiency, retrieval misleading generation, and complex KBQA task solutions. RGR-KBQA adopts a Retrieve-Generate-Retrieve framework to retrieve factual knowledge from the knowledge graph to enhance the semantic understanding ability of the LLM and improve the accuracy of generating logical forms. Subsequently, a fine-tuned model is used to generate the logical form of the question. Finally, unsupervised relation and entity retrieval are performed to further improve the generation accuracy. RGR-KBQA and ChatKBQA are the previous state-of-the-art methods on WebQSP. As shown in Tables 1 and 2, the best results are in bold, and the second-best results are underlined.

[0089] In addition, the present invention conducts ablation experiments on the WebQSP dataset, that is, the effectiveness of the proposed task decomposition is verified by removing any one of the subtasks. Specifically, the effectiveness of three schemes of removing the skeleton generation module, removing the topic entity generation module, and removing the relevant relationship generation module is verified respectively, and the results are shown in Table 1. It can be seen from the table that removing any one of the modules will lead to a decline in effectiveness, but the performance will be better than that of some existing methods, proving the beneficial effects of the task decomposition proposed by the present invention.

[0090] Table 1 Results on the WebQSP dataset

[0091]

[0092] Table 2 Results on the CWQ dataset

[0093]

[0094] Those of ordinary skill in the art can understand that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope defined by the claims of the present invention.

Claims

1. A method for answering questions based on a large language model and a knowledge graph, characterized in that: include: Inputting the skeleton of the logical query statement corresponding to the given question, the subject entity and the required relationship into the statement generation model to generate the logical query statement; Execute logical query statements on the knowledge graph to obtain answers to given questions; The skeleton generation model is used to generate the skeleton of the logical query statement for a given question; Topic entity generation model, used to generate topics for logical query statements for a given question; The relation generation model is used to generate the relations required for the logical query statements for a given question.

2. The method according to claim 1, characterized in that: The skeleton generation model, topic entity generation model, relationship generation model, and sentence generation model are large language models.

3. The method according to claim 1, characterized in that The subject entity generation model is formed by fine-tuning the skeleton generation model through LoRa. The fine-tuning method includes: The entities appearing in the logical query statements corresponding to each question in the training set are used as subject entities to generate training data. Each piece of training data includes a given question and the subject entity corresponding to the question. The skeleton generation model is fine-tuned through the training data to form a subject entity generation model.

4. The method according to claim 1, characterized in that: The relation generation model is formed by fine-tuning the skeleton generation model or the subject entity generation model. The fine-tuning method includes: S11. Use the vector encoder to encode the relationship between the question and the knowledge graph, and then calculate the vector representation of the question Vector representation of the relationship The cosine similarity between The most relevant relations constitute the candidate relation set ; S12. Set the questions, subject entities and candidate relations in the training set As the input of the relationship generation model, the relationship required to solve the problem is output, and the golden relationship corresponding to the problem is used as a comparison to train the relationship generation model.

5. The method according to claim 4, characterized in that Fine-tune the vector encoder to make the problem and the relationship required to solve the problem closer in the vector space of the vector encoder; the fine-tuning method includes: S21. For the problems in the training set, calculate the relationships with high correlation from the knowledge graph as the candidate relationship set ; S22. Based on the question, candidate relationship set And the relationship between the logical query statements corresponding to the question to construct positive and negative samples; S23. Fine-tune the vector encoder using a contrastive learning algorithm.

6. The method according to claim 5, characterized in that In S22, the method for obtaining positive and negative samples includes: for each question, using the training data corresponding to the relationship in the candidate relationship set but not included in the golden relationship set as a hard negative sample, and the training data corresponding to the relationship in the golden relationship set as a positive sample; the golden relationship is the relationship that appears in the logical query statement corresponding to the given question.

7. The method according to claim 1, characterized in that The sentence generation model is formed by fine-tuning the relationship generation model with LoRa.

8. The method according to claim 1, characterized in that The method of executing logical query statements on the knowledge graph to obtain answers includes: Align entities in logical query statements with entities in the knowledge graph: Align the relations in the logical query statement with the relations in the knowledge graph: Generates a combination of entities and relationships and selects the combinations as candidates for logical query statements; execute the candidate logical query statements in sequence on the knowledge graph. During the execution process, once the returned answer set is not empty, stop searching and return the result as the final answer; if no non-empty answer set is found, return an empty set.

9. The method according to claim 8, characterized in that Methods for entity alignment include: Extract entities from logical query statements; Check if there is an exact match in the knowledge graph; if there is a match, mark the entity as , and select the best matching front entities as candidate entities; if there is no exact matching entity, select The most similar entities are selected as candidate entities.

10. The method according to claim 8, characterized in that Methods of relationship alignment include: Extract the relationships in the logical query statements; If there is a completely matching relationship in the knowledge graph, no replacement is made; if there is no matching relationship, the relationship in the knowledge graph and the relationship in the logical query statement are vector-encoded, and the most similar predecessor is selected based on the vector similarity. relations as candidate relations.

Images