Large language model knowledge graph question answering method and system combined with semantic correction

Abstract

The invention provides a big language model knowledge graph question answering method and system combined with semantic correction, and the method comprises the steps: generating a logic form according to an input question through a fine-tuned open source big language model, and executing a query statement corresponding to the logic form to obtain an answer; if the answer cannot be obtained by the query statement corresponding to the execution logic form, extracting a main line entity and a relationship from the non-executable logic form, and performing semantic similarity comparison on the extracted relationship and a relationship in the knowledge graph through an unsupervised dense searcher to generate a candidate relationship set; based on the main line entity and the candidate relation set, iteratively retrieving triples in a knowledge graph to construct a reasoning path; and screening the constructed reasoning path by using a closed source large language model, or selecting a tail entity in the multi-answer path, and outputting a final answer. According to the method, the non-executable logic form is corrected, the semantic analysis effect based on the large language model is optimized, and efficient knowledge graph question answering is achieved.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph question answering technology, and specifically to a large language model knowledge graph question answering method and system combined with semantic correction. Background Art

[0002] Knowledge graphs are a technology that uses semantic networks to describe relationships between entities. Nodes represent entities, and edges represent relationships between entities. They efficiently organize and store knowledge and are widely used in fields such as finance and healthcare. Knowledge graph question answering is a task based on this structure, answering factual questions by parsing the question and querying the graph information.

[0003] Semantic parsing, as an efficient method for answering questions on knowledge graphs, primarily converts natural language questions into concise and structured logical forms, generating and executing query statements to obtain answers. Currently, some research aims to improve the accuracy and efficiency of question-answering systems by fine-tuning large language models to incorporate knowledge graph-related knowledge and subgraph information corresponding to the question. However, this direct approach to generating logical forms can easily lead to errors within and outside the reasoning path, resulting in expressions with similar semantics but grammatical problems. This can lead to queries failing to execute, resulting in an inability to obtain correct answers, ultimately impacting the accuracy and reliability of the question-answering system.

[0004] Patent application document CN119271784A discloses a method and system for enhancing the accuracy of large-scale model answers based on a knowledge graph. The method includes: obtaining a text dataset, preprocessing the text dataset, and extracting entities and relationships in the text dataset; establishing an entity connection graph based on the entities and relationships in the text dataset, fusing the entity nodes in the entity connection graph, and creating a knowledge graph based on the fused entity nodes and relationships; obtaining a user's query question, determining matching nodes for the user's query question based on the user's query question; identifying the user's question intent based on the user's query question, determining a prompt template based on the user's question intent and the matching nodes corresponding to the user's query question, and inputting the prompt template into the large-scale model to obtain a credible answer to the user's query question. However, this patent cannot completely solve the existing technical problems and cannot meet the requirements of the present invention. Summary of the Invention

[0005] In response to the deficiencies in the prior art, the present invention aims to provide a large language model knowledge graph question answering method and system combined with semantic correction.

[0006] The large language model knowledge graph question answering method combined with semantic correction provided by the present invention includes:

[0007] Step 1: Use the fine-tuned open-source large language model to generate a logical form based on the input question and execute the query statement corresponding to the logical form to obtain the answer;

[0008] Step 2: If the query statement corresponding to the logical form cannot obtain an answer, the mainline entities and relations are extracted from the unexecutable logical form. The extracted relations are compared with the relations in the knowledge graph for semantic similarity using an unsupervised dense search engine to generate a set of candidate relations. Based on the mainline entities and candidate relations, the triples in the knowledge graph are iteratively retrieved to construct an inference path.

[0009] Step 3: Use a closed-source large language model to filter the constructed reasoning path, or select the tail entity in the multi-answer path, and output the final answer.

[0010] Preferably, step 1 comprises: constructing an instruction fine-tuning dataset based on an open source dataset, wherein the input is a prompt word containing an instruction and a question, and the output is a logical form; the logical form converts the original SPARQL query statement into an expression containing the following information through the function convert(·): describing the relationship and direction of each jump from the question entity through a JOIN statement, indicating the number of jumps in the reasoning path through nested JOIN, setting conditional restrictions through ARGMAX and LESS THAN operators, and describing the intersection of reasoning paths of different question entities through an AND operator;

[0011] The fine-tuning process is:

[0012]

[0013] Among them, p is the prompt word containing instructions and questions, L is the logical form corresponding to the question, t is the index of the dataset entry, L t Represents the t-th logical form, L ＜t represents the historical logical form before the t-th logical form, G is the knowledge graph, D={(p i ,L i )|i∈[1,n]} represents the constructed data set, (p i ,L i ) represent the prompt word-logical form pairs in the dataset, n is the number of questions in the dataset, Φ0 is the pre-training weight parameter of the open source large language model, θ is the parameter involved in LoRA fine-tuning, and ΔΦ(θ) is the parameter change after LoRA training on a specific task. The probability of generating logical forms based on question and knowledge graph information after integrating LoRA training parameters into the open source large language model.

[0014] Preferably, a beam search is performed using a large language model with knowledge to generate a series of candidate logical forms, which are executed in the knowledge graph and expressed as:

[0015] A ex =execute(convert -1 (LLM(p))|G)

[0016] Among them, A ex To execute the answer to the query statement, LLM(·) represents the process of generating logical forms using the fine-tuned open source large language model, convert -1 (·) represents the process of converting the generated logical form into SPARQL statements, and execute(·|G) represents the operation of executing SPARQL statements in G.

[0017] Preferably, the step 2 includes:

[0018] From the unexecutable logical form L non The mainline entities and relations are extracted from the source, where the mainline entity e0 is the last entity to appear in each logical form. The extracted relations are arranged in order from back to front and used as the relations for each hop retrieval. The semantic similarity of the extracted relations with all relations in the knowledge graph is compared using an unsupervised dense searcher, which is expressed as:

[0019] s=sim(r,r′)=E(r) T E(r′)

[0020] Where r is a relation extracted from the non-executable logical form, r′∈R represents a relation in the knowledge graph, and R represents all relations in the knowledge graph. The encoder E(·) in the unsupervised dense searcher is used to encode relations r and r′, and their similarity score s is obtained by calculating the dot product of the r and r′ vectors.

[0021] For each r, retain the k candidate relations with the highest similarity in R;

[0022] Assume that h relations are extracted in a logical form, then the comparison result is h·k candidate relations. Based on the known entities and the relations obtained by comparison, the triples of each hop are retrieved iteratively. The single retrieval process is expressed as:

[0023] E i,j =retrieve(e i-1 ,r ij |G)

[0024] Among them, i∈[1,h],j∈[1,k],e i-1 Represents a header entity of the i-th hop, r ij Represents the j candidate relations of the i-th hop in the comparison result, retrieve(·|G) describes the relationship between the entity e in the knowledge graph. i-1 and relationship rij Retrieve the tail entity set E i,j operation; for the subsequent i+1 hop, e i The entity is used as the head entity and the search continues at the i+1th hop. The initial head entity of the entire iterative process is the extracted main line entity e0.

[0025] The retrieved triples of each hop are connected into an inference path. If the i+1th hop head entity exists in the i-th hop tail entity, it is connected into an inference path from the i-th hop to the i+1th hop, and the complete path from the main line entity to the answer entity is obtained.

[0026] Preferably, the step 3 includes:

[0027] If there are multiple reasoning paths consisting of different relationships, the closed-source large language model API is called to filter the path sequence number and extract the corresponding tail entity as the answer;

[0028] If there are multiple tail entities in a single-hop path or multiple tail entities are generated due to different intermediate entities in a multi-hop path, the closed-source large language model API is called to filter some tail entities as answer A. non ;

[0029] The output logic of the final answer is:

[0030] The large language model knowledge graph question answering system combined with semantic correction provided by the present invention includes:

[0031] Logical Form Generation Module: This module uses a fine-tuned open-source large language model to generate a logical form based on the input question and executes the query statement corresponding to the logical form to obtain the answer.

[0032] Semantic Revision Module: If the query statement cannot obtain an answer, the mainline entities and relations are extracted from the unexecutable logical form. The extracted relations are compared with the relations in the knowledge graph for semantic similarity using an unsupervised dense search engine to generate a set of candidate relations. Based on the mainline entities and candidate relations, the triples in the knowledge graph are iteratively retrieved to construct an inference path.

[0033] Answer selection module: Uses a closed-source large language model to filter the constructed reasoning path, or selects the tail entity in the multi-answer path, and outputs the final answer.

[0034] Preferably, the logical form generation module comprises: constructing an instruction fine-tuning dataset based on an open source dataset, wherein the input is a prompt word containing instructions and questions, and the output is a logical form; the logical form converts the original SPARQL query statement into an expression containing the following information through the function convert(·): describing the relationship and direction of each jump from the question entity through a JOIN statement, indicating the number of jumps in the reasoning path through nested JOIN, setting conditional restrictions through ARGMAX and LESS THAN operators, and describing the intersection of reasoning paths of different question entities through the AND operator;

[0035] The fine-tuning process is:

[0036]

[0037] Among them, p is the prompt word containing instructions and questions, L is the logical form corresponding to the question, t is the index of the dataset entry, L t Represents the t-th logical form, L ＜t represents the historical logical form before the t-th logical form, G is the knowledge graph, D={(p i ,L i )|i∈[1,n]} represents the constructed data set, (p i ,L i ) represent the prompt word-logical form pairs in the dataset, n is the number of questions in the dataset, Φ0 is the pre-training weight parameter of the open source large language model, θ is the parameter involved in LoRA fine-tuning, and ΔΦ(θ) is the parameter change after LoRA training on a specific task. The probability of generating logical forms based on question and knowledge graph information after integrating LoRA training parameters into the open source large language model.

[0038] Preferably, a beam search is performed using a large language model with knowledge to generate a series of candidate logical forms, which are executed in the knowledge graph and expressed as:

[0039] A ex =execute(convert -1 (LLM(p))|G)

[0040] Among them, A ex To execute the answer to the query statement, LLM(·) represents the process of generating logical forms using the fine-tuned open source large language model, convert -1 (·) represents the process of converting the generated logical form into SPARQL statements, and execute(·|G) represents the operation of executing SPARQL statements in G.

[0041] Preferably, the semantic correction module includes:

[0042] From the unexecutable logical form L non The mainline entities and relations are extracted from the source, where the mainline entity e0 is the last entity to appear in each logical form. The extracted relations are arranged in order from back to front and used as the relations for each hop retrieval. The semantic similarity of the extracted relations with all relations in the knowledge graph is compared using an unsupervised dense searcher, which is expressed as:

[0043] s=sim(r,r′)=E(r) T E(r′)

[0044] Where r is a relation extracted from the non-executable logical form, r′∈R represents a relation in the knowledge graph, and R represents all relations in the knowledge graph. The encoder E(·) in the unsupervised dense searcher is used to encode relations r and r′, and their similarity score s is obtained by calculating the dot product of the r and r′ vectors.

[0045] For each r, retain the k candidate relations with the highest similarity in R;

[0046] Assume that h relations are extracted in a logical form, then the comparison result is h·k candidate relations. Based on the known entities and the relations obtained by comparison, the triples of each hop are retrieved iteratively. The single retrieval process is expressed as:

[0047] E i,j =retrieve(e i-1 ,r ij |G)

[0048] Among them, i∈[1,h],j∈[1,k],e i-1 Represents a header entity of the i-th hop, r ij Represents the j candidate relations of the i-th hop in the comparison result, retrieve(·|G) describes the relationship between the entity e in the knowledge graph. i-1 and relationship r ij Retrieve the tail entity set E i,j operation; for the subsequent i+1 hop, e i The entity is used as the head entity and the search continues at the i+1th hop. The initial head entity of the entire iterative process is the extracted main line entity e0.

[0049] The retrieved triples of each hop are connected into an inference path. If the i+1th hop head entity exists in the i-th hop tail entity, it is connected into an inference path from the i-th hop to the i+1th hop, and the complete path from the main line entity to the answer entity is obtained.

[0050] Preferably, the answer selection module includes:

[0051] If there are multiple reasoning paths consisting of different relationships, the closed-source large language model API is called to filter the path sequence number and extract the corresponding tail entity as the answer;

[0052] If there are multiple tail entities in a single-hop path or multiple tail entities are generated due to different intermediate entities in a multi-hop path, the closed-source large language model API is called to filter some tail entities as answer A. non ;

[0053] The output logic of the final answer is:

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] The present invention proposes a large language model knowledge graph question-answering method combined with semantic correction. First, a fine-tuned open source large language model is used to directly generate the corresponding logical form according to the input question, and the query statement corresponding to the logical form is executed to obtain the answer; then, for the logical forms that cannot obtain the answer, the relationship therein is extracted and semantic correction is performed, and it is converted into semantic content that can be searched in the knowledge graph and reconstructed into a valid query path; finally, based on the constructed path, the closed source large language model is used to perform path selection and answer extraction, thereby obtaining the final answer of the unexecutable logical form; by correcting the unexecutable logical form, the present invention optimizes the semantic parsing effect based on the large language model and realizes efficient knowledge graph question-answering. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0057] Figure 1 This is a schematic diagram of the framework of the large language model knowledge graph question answering system combined with semantic correction of the present invention;

[0058] Figure 2 Generate a module schematic for the logical form of the present invention;

[0059] Figure 3 Schematic diagram of the semantic correction module of the present invention;

[0060] Figure 4 This is a schematic diagram of the answer selection module of the present invention;

[0061] Figure 5 This is a flow chart of the large language model knowledge graph question answering method combined with semantic correction of the present invention. DETAILED DESCRIPTION

[0062] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0063] Example 1

[0064] This paper proposes a large language model knowledge graph question answering system combined with semantic correction, which adopts the knowledge graph question answering framework LFG-KBQA based on the large language model and consists of three basic modules: Figure 1 shown.

[0065] i) Logical Form Generation Module: This module includes fine-tuning the open source large language model to directly generate the logical form corresponding to the question, execute the query, and determine whether the generated result is executable.

[0066] ii) Semantic Revision Module: This module includes an unsupervised dense searcher that retrieves knowledge graph relations with semantically similar non-executable logical forms. These relations are combined with entities to reconstruct reasoning paths.

[0067] iii) Answer selection module: This module includes a closed-source large language model, which is used to filter paths in the case of multiple paths and to refine answers in the case of multiple answers.

[0068] The large language model knowledge graph question answering method combined with semantic correction of the present invention is as follows:

[0069] 1. Logical form generation module

[0070] like Figure 2 As shown in Figure 2, this module contains an open-source large language model for generating logical forms directly from questions. We use instructional fine-tuning to enhance the model's capabilities for this specific task and combine it with the LoRA method to reduce training overhead.

[0071] To this end, we constructed new datasets based on the open-source datasets WebQuestionSP (WebQSP) and ComplexWebQuestions (CWQ). Each data entry takes a prompt consisting of an instruction and a question as input and outputs a logical form. The logical form is obtained by converting the SPARQL query statements in the original dataset using the convert(·) function. This function uses JOIN statements to describe the hop relationships and their directions from the question entity, nested JOINs to indicate the number of hops in the reasoning path, operators such as ARGMAX and LESS THAN to set conditional constraints, and the AND operator to describe information about the intersection of reasoning paths between different question entities.

[0072] This task can be formulated as an optimization problem, which aims to maximize the probability of generating logical forms based on the question and knowledge graph information. Combined with LoRA fine-tuning, the objective function can be expressed as:

[0073]

[0074] Among them, p is a prompt word containing instructions and questions, L is the logical form corresponding to the question, t is the index of a data set entry, L t Represents the t-th logical form, L ＜t Denotes the historical logical form before the t-th logical form, D={(p i ,L i )|i∈[1,n]} represents the constructed data set, (p i ,L i ) represent a prompt word-logical form pair in the dataset, n is the number of questions in the dataset, Φ0 is the pre-training weight parameter of the open source large language model, θ is a small number of parameters involved in LoRA fine-tuning, and ΔΦ(θ) is the parameter change after LoRA training on a specific task. The probability of generating logical forms based on question and knowledge graph information after integrating LoRA training parameters into the open source large language model.

[0075] After fine-tuning, we use the large language model with knowledge to perform beam search to generate a series of candidate logical forms, which are executed in the knowledge graph and expressed as:

[0076] A ex =execute(convert -1 (LLM(p))|G)

[0077] Among them, p is the prompt word containing instructions and questions, G is the knowledge graph, and A exTo execute the answer to the query statement, LLM(·) represents the process of generating logical forms using the fine-tuned open source large language model, convert -1 (·) represents the process of converting the generated logical form into SPARQL statements, and execute(·|G) represents the operation of executing SPARQL statements in G.

[0078] if That is, there is an executable logical form in a certain problem, that is, using A ex As the final answer. That is, if all logical forms generated in the problem are not executable, all the above-mentioned non-executable logical forms are input into the semantic correction module for subsequent processing.

[0079] 2. Semantic Correction Module

[0080] like Figure 3 As shown, we start from the unexecutable logical form L non The mainline entities and relations are extracted from the JOIN table. The mainline entity specifically refers to the last entity e0 in each logical form. Due to the characteristics of the nested JOIN statement, the extracted relations are arranged in a back-to-front order and used as the relations for each hop retrieval. Next, an unsupervised dense searcher is used to compare the semantic similarity of the extracted relations with all relations in the knowledge graph. This process can be expressed as:

[0081] s=sim(r,r′)=E(r) T E(r′)

[0082] Here, r is the relation extracted from the non-executable logical form, r′ (r′∈R) represents the relation in the knowledge graph, and R represents all relations in the knowledge graph. We use the encoder E(·) in the dense retriever to encode relations r and r′, and calculate the dot product of these two vectors to obtain their similarity score s. For each relation, we calculate its similarity with all relations in R one by one and sort all the scores. Ultimately, we retain the k relations with the highest scores, thus obtaining the k candidate relations in the knowledge graph that are semantically closest to the relation. Assuming that h relations are extracted from a logical form, the comparison result is h·k candidate relations.

[0083] Based on the known entities and the relationships obtained by comparison, iterative search is performed to obtain the (head entity, relationship, tail entity) triples of each hop. The single search process can be expressed as:

[0084] E i,j =retrieve(e i-1 ,r ij |G)

[0085] Among them, i∈[1,h],j∈[1,k],e i-1 Represents a header entity of the i-th hop, r ij Represents the j candidate relations of the i-th hop in the comparison result, retrieve(·|G) describes the relationship between the entity e in the knowledge graph. i-1 and relationship r ij Retrieve the tail entity set E i,j For the subsequent (i+1)th jump, e i (e i ∈E ij ) entity as the head entity and continue the (i+1)th hop search. The initial head entity of the entire iterative process is the extracted main line entity e0.

[0086] Connect the retrieved triples of each hop into an inference path. If the (i+1)th hop head entity exists in the i-th hop tail entity, it can be connected to form an inference path from the i-th hop to the (i+1)-th hop. Similarly, the complete path from the main line entity to the answer entity can be obtained.

[0087] 3. Answer selection module

[0088] like Figure 4 As shown, if the semantic correction module generates multiple complete paths consisting of different relationships, these paths are filtered through the closed-source large language model, and the final entity is obtained by extracting the tail entity of the path. Specifically, the candidate paths and their corresponding serial numbers are listed in the prompt word, the closed-source large language model API is called, the corresponding path serial number is selected, and the tail entity corresponding to the serial number is extracted as the answer through text segmentation.

[0089] If a complete path with multiple answers composed of the same relationship is generated, specifically a single-hop path with multiple tail entities, or a multi-hop path with multiple single tail entities due to different intermediate entities, then these answers can be filtered through the closed-source large language model to obtain the final answer A. non Specifically, all candidate tail entities in the prompt word are merged into one path, and the closed-source large language model API is called to select some of the tail entities as the answer.

[0090] Through the above two answer selection methods, the noise introduced when comparing the k closest candidate relations for each hop can be reduced, that is, it is necessary to filter out those paths that can be connected but have inaccurate relations.

[0091] Combining the application of the above three modules, the final answer can be expressed as:

[0092]

[0093] It's worth noting that the framework above only provides a method for optimizing knowledge graph question answering tasks based on semantic parsing using large language models. The open-source and closed-source large language models involved are not limited to a specific type. For example, open-source large language models include common models such as LlaMA, ChatGLM, and Baichuan, while closed-source large language models include GPT-3.5 and GPT-4.

[0094] Using Hits@1 as the performance metric, LFG-KBQA, when applied to LlaMA-2-7B and GPT-4, achieved improvements of 0.78%, 12.78%, 9.28%, and 12.18%, respectively, on the simple WebQSP dataset, compared to existing methods RoG, KD-CoT, UniKGQA, and NSM. On the complex CWQ dataset, these improvements were 8.68%, 20.78%, 20.08%, and 22.48%, respectively. This improvement demonstrates the effectiveness of LFG-KBQA's optimized semantic parsing and its broad applicability to both simple and complex questions.

[0095] In summary, the present invention proposes a large language model knowledge graph question and answer system combined with semantic correction. By correcting unexecutable logical forms, it optimizes the semantic parsing effect based on the large language model and realizes efficient knowledge graph question and answer.

[0096] Example 2

[0097] like Figure 5 The present invention provides a large language model knowledge graph question answering method combined with semantic correction, including: step 1: using a fine-tuned open source large language model to generate a logical form according to an input question, and executing a query statement corresponding to the logical form to obtain an answer; step 2: if the query statement corresponding to the logical form cannot obtain an answer, extracting mainline entities and relationships from the unexecutable logical form, and using an unsupervised dense searcher to compare the semantic similarity of the extracted relationships with the relationships in the knowledge graph to generate a set of candidate relationships; based on the mainline entities and the set of candidate relationships, iteratively retrieve triples in the knowledge graph to construct an inference path; step 3: using a closed source large language model to screen the constructed inference path, or select the tail entity in the multi-answer path, and output the final answer.

[0098] The step 1 includes: constructing an instruction fine-tuning dataset based on an open source dataset, wherein the input is a prompt word containing an instruction and a question, and the output is a logical form; the logical form converts the original SPARQL query statement into an expression containing the following information through the function convert(·): describing the relationship and direction of each jump from the question entity through a JOIN statement, indicating the number of jumps in the reasoning path through nested JOIN, setting conditional restrictions through ARGMAX and LESS THAN operators, and describing the intersection of reasoning paths of different question entities through the AND operator;

[0099] The fine-tuning process is:

[0100]

[0101] Among them, p is the prompt word containing instructions and questions, L is the logical form corresponding to the question, t is the index of the dataset entry, L t Represents the t-th logical form, L <t represents the historical logical form before the t-th logical form, G is the knowledge graph, D={(p i ,L i )|i∈[1,n]} represents the constructed data set, (p i ,L i ) represent the prompt word-logical form pairs in the dataset, n is the number of questions in the dataset, Φ0 is the pre-training weight parameter of the open source large language model, θ is the parameter involved in LoRA fine-tuning, and ΔΦ(θ) is the parameter change after LoRA training on a specific task. The probability of generating logical forms based on question and knowledge graph information after integrating LoRA training parameters into the open source large language model.

[0102] Use a large language model with knowledge to perform beam search to generate a series of candidate logical forms, which are executed in the knowledge graph and expressed as:

[0103] A ex =execute(convert -1 (LLM(p))|G)

[0104] Among them, A ex To execute the answer to the query statement, LLM(·) represents the process of generating logical forms using the fine-tuned open source large language model, convert -1 (·) represents the process of converting the generated logical form into SPARQL statements, and execute(·|G) represents the operation of executing SPARQL statements in G.

[0105] The step 2 includes: from the unexecutable logical form L nonThe mainline entities and relations are extracted from the source, where the mainline entity e0 is the last entity to appear in each logical form. The extracted relations are arranged in order from back to front and used as the relations for each hop retrieval. The semantic similarity of the extracted relations with all relations in the knowledge graph is compared using an unsupervised dense searcher, which is expressed as:

[0106] s=sim(r,r′)=E(r) T E(r′)

[0107] Where r is a relation extracted from the non-executable logical form, r′∈R represents a relation in the knowledge graph, and R represents all relations in the knowledge graph. The encoder E(·) in the unsupervised dense searcher is used to encode relations r and r′, and their similarity score s is obtained by calculating the dot product of the r and r′ vectors.

[0108] For each r, retain the k candidate relations with the highest similarity in R;

[0109] Assume that h relations are extracted in a logical form, then the comparison result is h·k candidate relations. Based on the known entities and the relations obtained by comparison, the triples of each hop are retrieved iteratively. The single retrieval process is expressed as:

[0110] E i,j =retrieve(e i-1 ,r ij |G)

[0111] Among them, i∈[1,h],j∈[1,k],e i-1 Represents a header entity of the i-th hop, r ij Represents the j candidate relations of the i-th hop in the comparison result, retrieve(·|G) describes the relationship between the entity e in the knowledge graph. i-1 and relationship r ij Retrieve the tail entity set E i,j operation; for the subsequent i+1 hop, e i The entity is used as the head entity and the search continues at the i+1th hop. The initial head entity of the entire iterative process is the extracted main line entity e0.

[0112] The retrieved triples of each hop are connected into an inference path. If the i+1th hop head entity exists in the i-th hop tail entity, it is connected into an inference path from the i-th hop to the i+1th hop, and the complete path from the main line entity to the answer entity is obtained.

[0113] The step 3 includes: if there are multiple reasoning paths composed of different relationships, calling the closed-source large language model API to filter the path sequence number and extract the corresponding tail entity as the answer; if there are multiple tail entities in a single-hop path or multiple tail entities in a multi-hop path due to different intermediate entities, calling the closed-source large language model API to filter some tail entities as the answer A non ;

[0114] The output logic of the final answer is:

[0115] Those skilled in the art will appreciate that, in addition to implementing the system, device, and various modules provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, and the like by logically programming the method steps. Therefore, the system, device, and various modules provided by the present invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; the modules for implementing various functions can also be considered both software programs for implementing the method and structures within the hardware component.

[0116] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A large language model knowledge graph question answering method combined with semantic correction, characterized by: include: Step 1: Use the fine-tuned open-source large language model to generate a logical form based on the input question and execute the query statement corresponding to the logical form to obtain the answer; Step 2: If the query statement corresponding to the logical form cannot obtain an answer, the main line entities and relations are extracted from the unexecutable logical form. The extracted relations are compared with the relations in the knowledge graph for semantic similarity through an unsupervised dense search engine to generate a set of candidate relations. Based on the mainline entity and the candidate relationship set, iteratively retrieve triples in the knowledge graph to construct a reasoning path; Step 3: Use a closed-source large language model to filter the constructed reasoning path, or select the tail entity in the multi-answer path, and output the final answer.

2. The large language model knowledge graph question answering method combined with semantic correction according to claim 1 is characterized in that The step 1 includes: constructing an instruction fine-tuning dataset based on an open source dataset, wherein the input is a prompt word containing an instruction and a question, and the output is a logical form; the logical form converts the original SPARQL query statement into an expression containing the following information through the function convert(·): describing the relationship and direction of each jump from the question entity through a JOIN statement, indicating the number of jumps in the reasoning path through nested JOIN, setting conditional restrictions through ARGMAX and LESS THAN operators, and describing the intersection of reasoning paths of different question entities through the AND operator; The fine-tuning process is: Among them, p is the prompt word containing instructions and questions, L is the logical form corresponding to the question, t is the index of the dataset entry, L t Represents the t-th logical form, L ＜t represents the historical logical form before the t-th logical form, G is the knowledge graph, D={(p i ,L i )|i∈[1,n]} represents the constructed data set, (p i ,L i ) represent the prompt word-logical form pairs in the dataset, n is the number of questions in the dataset, Φ0 is the pre-training weight parameter of the open source large language model, θ is the parameter involved in LoRA fine-tuning, ΔΦ(θ) is the parameter change after LoRA training on a specific task, P Φ0+ΔΦ(θ) The probability of generating logical forms based on question and knowledge graph information after integrating LoRA training parameters into the open source large language model.

3. The large language model knowledge graph question answering method combined with semantic correction according to claim 2 is characterized in that: Use a large language model with knowledge to perform beam search to generate a series of candidate logical forms, which are executed in the knowledge graph and expressed as: A ex =execute(convert -1 (LLM(p))|G) Among them, A ex To execute the answer to the query statement, LLM(·) represents the process of generating logical forms using the fine-tuned open source large language model, convert -1 (·) represents the process of converting the generated logical form into SPARQL statements, and execute(·|G) represents the operation of executing SPARQL statements in G.

4. The large language model knowledge graph question answering method combined with semantic correction according to claim 1 is characterized in that The step 2 includes: From the unexecutable logical form L non The mainline entities and relations are extracted from the source, where the mainline entity e0 is the last entity to appear in each logical form. The extracted relations are arranged in order from back to front and used as the relations for each hop retrieval. The semantic similarity of the extracted relations with all relations in the knowledge graph is compared using an unsupervised dense searcher, which is expressed as: s=sim(r,r′)=E(r) T E(r′) Where r is a relation extracted from the non-executable logical form, r′∈R represents a relation in the knowledge graph, and R represents all relations in the knowledge graph. The encoder E(·) in the unsupervised dense searcher is used to encode relations r and r′, and their similarity score s is obtained by calculating the dot product of the r and r′ vectors. For each r, retain the k candidate relations with the highest similarity in R; Assume that h relations are extracted in a logical form, then the comparison result is h·k candidate relations. Based on the known entities and the relations obtained by comparison, the triples of each hop are retrieved iteratively. The single retrieval process is expressed as: E i,j =retrieve(e i-1 ,r ij |G) Among them, i∈[1,h],j∈[1,k],e i-1 Represents a header entity of the i-th hop, r ij Represents the j candidate relations of the i-th hop in the comparison result, retrieve(·|G) describes the relationship between the entity e in the knowledge graph. i-1 and relationship r ij Retrieve the tail entity set E i,j operation; for the subsequent i+1 hop, e i The entity is used as the head entity and the search continues at the i+1th hop. The initial head entity of the entire iterative process is the extracted main line entity e0. The retrieved triples of each hop are connected into an inference path. If the i+1th hop head entity exists in the i-th hop tail entity, it is connected into an inference path from the i-th hop to the i+1th hop, and the complete path from the main line entity to the answer entity is obtained.

5. The large language model knowledge graph question answering method combined with semantic correction according to claim 1 is characterized in that The step 3 comprises: If there are multiple reasoning paths consisting of different relationships, the closed-source large language model API is called to filter the path sequence number and extract the corresponding tail entity as the answer; If there are multiple tail entities in a single-hop path or multiple tail entities are generated due to different intermediate entities in a multi-hop path, the closed-source large language model API is called to filter some tail entities as answer A. non ; The output logic of the final answer is:

6. A large language model knowledge graph question answering system combined with semantic correction, characterized by: include: Logical Form Generation Module: This module uses a fine-tuned open-source large language model to generate a logical form based on the input question and executes the query statement corresponding to the logical form to obtain the answer. Semantic Correction Module: If the query statement cannot obtain an answer, the main line entities and relations are extracted from the unexecutable logical form, and the extracted relations are compared with the relations in the knowledge graph for semantic similarity through an unsupervised dense search engine to generate a set of candidate relations; Based on the mainline entity and the candidate relationship set, iteratively retrieve triples in the knowledge graph to construct a reasoning path; Answer selection module: Uses a closed-source large language model to filter the constructed reasoning path, or selects the tail entity in the multi-answer path, and outputs the final answer.

7. The large language model knowledge graph question answering system combined with semantic correction according to claim 6 is characterized in that: The logical form generation module includes: constructing an instruction fine-tuning dataset based on an open source dataset, wherein the input is a prompt word containing instructions and questions, and the output is a logical form; the logical form converts the original SPARQL query statement into an expression containing the following information through the function convert(·): describing the relationship and direction of each hop from the question entity through a JOIN statement, indicating the number of hops in the reasoning path through nested JOINs, setting conditional restrictions through the ARGMAX and LESS THAN operators, and describing the intersection of reasoning paths of different question entities through the AND operator; The fine-tuning process is: Among them, p is the prompt word containing instructions and questions, L is the logical form corresponding to the question, t is the index of the dataset entry, L t Represents the t-th logical form, L ＜t represents the historical logical form before the t-th logical form, G is the knowledge graph, D={(p i ,L i )|i∈[1,n]} represents the constructed data set, (p i ,L i ) represent the prompt word-logical form pairs in the dataset, n is the number of questions in the dataset, Φ0 is the pre-training weight parameter of the open source large language model, θ is the parameter involved in LoRA fine-tuning, ΔΦ(θ) is the parameter change after LoRA training on a specific task, P Φ0+ΔΦ(θ) The probability of generating logical forms based on question and knowledge graph information after integrating LoRA training parameters into the open source large language model.

8. The large language model knowledge graph question answering system combined with semantic correction according to claim 7 is characterized in that: Use a large language model with knowledge to perform beam search to generate a series of candidate logical forms, which are executed in the knowledge graph and expressed as: A ex =execute(convert -1 (LLM(p))|G) Among them, A ex To execute the answer to the query statement, LLM(·) represents the process of generating logical forms using the fine-tuned open source large language model, convert -1 (·) represents the process of converting the generated logical form into SPARQL statements, and execute(·|G) represents the operation of executing SPARQL statements in G.

9. The large language model knowledge graph question answering system combined with semantic correction according to claim 6 is characterized in that The semantic correction module includes: From the unexecutable logical form L non The mainline entities and relations are extracted from the source, where the mainline entity e0 is the last entity to appear in each logical form. The extracted relations are arranged in order from back to front and used as the relations for each hop retrieval. The semantic similarity of the extracted relations with all relations in the knowledge graph is compared using an unsupervised dense searcher, which is expressed as: s=sim(r,r′)=E(r) T E(r′) Where r is a relation extracted from the non-executable logical form, r′∈R represents a relation in the knowledge graph, and R represents all relations in the knowledge graph. The encoder E(·) in the unsupervised dense searcher is used to encode relations r and r′, and their similarity score s is obtained by calculating the dot product of the r and r′ vectors. For each r, retain the k candidate relations with the highest similarity in R; Assume that h relations are extracted in a logical form, then the comparison result is h·k candidate relations. Based on the known entities and the relations obtained by comparison, the triples of each hop are retrieved iteratively. The single retrieval process is expressed as: E i,j =retrieve(e i-1 ,r ij |G) Among them, i∈[1,h],j∈[1,k],e i-1 Represents a header entity of the i-th hop, r ij Represents the j candidate relations of the i-th hop in the comparison result, retrieve(·|G) describes the relationship between the entity e in the knowledge graph. i-1 and relationship r ij Retrieve the tail entity set E i,j operation; for the subsequent i+1 hop, e i The entity is used as the head entity and the search continues at the i+1th hop. The initial head entity of the entire iterative process is the extracted main line entity e0. The retrieved triples of each hop are connected into an inference path. If the i+1th hop head entity exists in the i-th hop tail entity, it is connected into an inference path from the i-th hop to the i+1th hop, and the complete path from the main line entity to the answer entity is obtained.

10. The large language model knowledge graph question answering system combined with semantic correction according to claim 6 is characterized in that: The answer selection module includes: If there are multiple reasoning paths consisting of different relationships, the closed-source large language model API is called to filter the path sequence number and extract the corresponding tail entity as the answer; If there are multiple tail entities in a single-hop path or multiple tail entities are generated due to different intermediate entities in a multi-hop path, the closed-source large language model API is called to filter some tail entities as answer A. non ; The output logic of the final answer is:

Images