Large language model knowledge question-answering method based on semantic analysis correction

Abstract

The invention discloses a big language model knowledge question-answering method based on semantic analysis correction, and belongs to the field of natural language reasoning, the method comprises the following steps: extracting a key entity from a natural language question, matching a target entity in a knowledge graph, and mining a plurality of knowledge paths in the knowledge graph for model training; adopting a prompt instruction and a knowledge path to supervise and train the large language model, so that the large language model has the capability of generating a multi-hop reasoning path conforming to a knowledge graph structure; performing knowledge path decoding based on the trained model, converting a natural language problem into a plurality of candidate reasoning paths, and converting the candidate reasoning paths into logic query statements to be executed in a knowledge graph, thereby completing fact verification and completion of the paths; and inputting the high-quality path subjected to graph verification and the natural language question into a large language model to obtain an accurate and explainable answer. According to the method, deep fusion of structured knowledge and a language model is realized, and the reasoning ability, the answer accuracy and the interpretability in a complex question and answer task are improved.

Description

Technical Field

[0001] The present invention belongs to the field of natural language reasoning, and more specifically, relates to a large language model knowledge question answering method based on semantic parsing correction. Background Art

[0002] With the rapid development of large-scale pre-trained language models, they have demonstrated strong general capabilities in natural language processing tasks such as open-domain question answering, logical reasoning, and complex task understanding. However, current mainstream large language models still face significant challenges when dealing with scenarios involving knowledge reasoning, especially multi-hop question answering. The main problems include: on the one hand, the knowledge stored in the parameters of large language models is static and time-limited, making it difficult to cover the dynamically growing professional domain knowledge system; on the other hand, in the absence of structured constraints, the model reasoning process is prone to "hallucination" problems, that is, generating logical errors or answers that are inconsistent with the facts, affecting the credibility and controllability of the model in practical applications.

[0003] To this end, researchers have attempted to introduce external knowledge sources, particularly knowledge graphs with clear structures and explicit relationships, to assist models in complex reasoning tasks. Knowledge graphs, with their explicit entity and relationship structures, can effectively support tasks such as entity linking and path reasoning, and are particularly well-suited for modeling multi-hop reasoning problems.

[0004] However, using only knowledge graphs for retrieval-based question answering or logical reasoning often makes it difficult to fully understand the semantic information of natural language questions and lacks flexible language generation capabilities. On the other hand, if knowledge graph information is directly embedded into prompts to guide the reasoning of large language models, improper path selection or redundant path information may still lead to unstable model performance. This is specifically reflected in the following aspects: 1) There is a lack of an effective interactive interface between the knowledge graph and the large language model, and the path generation is separated from the natural language understanding process; 2) The graph reasoning process often has difficulty in structurally decomposing complex problems and lacks the ability to fine-grained modeling of reasoning subtasks; 3) Existing joint models lack a systematic evaluation and backtracking mechanism for the credibility of knowledge paths, and are prone to introducing misleading information; 4) The thought chain prompt fragments generated by some methods lack explainability and consistency in the reasoning path, making it difficult to achieve controllable optimization of model behavior. Summary of the Invention

[0005] In response to the above-mentioned defects or improvement needs of the prior art, the present invention provides a large language model knowledge question answering method based on semantic parsing correction, which aims to improve the reasoning efficiency and accuracy of the large language model combined with the knowledge graph.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a large language model knowledge question answering method based on semantic parsing correction is provided, comprising:

[0007] Training phase:

[0008] S1, pre-processing the initial sample set to obtain a fine-tuning sample set;

[0009] The initial sample set includes natural language questions, entities related to the questions, and a knowledge graph for solving the natural language questions. The pre-processing includes identifying key entities of each natural language question as a starting point, locating target entities in the knowledge graph according to standard answers corresponding to each natural language question as an end point, searching all possible paths from the starting point to the end point in the knowledge graph and saving them to the fine-tuning sample set. Each possible path is represented by the entity nodes and the relationship types between adjacent entities.

[0010] S2, taking each natural language question, entity related to the question, and corresponding path in the fine-tuning sample set in the initial sample set as a pair of fine-tuning instructions, and saving them to a fine-tuning instruction set. Construct a task prompt instruction and store it in each pair of fine-tuning instructions in the fine-tuning instruction set.

[0011] S3, using the fine-tuning instruction set obtained by S2 to supervise the training of a large language model;

[0012] Application phase:

[0013] Input the natural language question to be answered into the trained large language model to obtain a candidate path, convert each candidate path into a logical query statement, and convert the logical query statement into a knowledge graph query statement and query in the knowledge graph to obtain a query result.

[0014] The logical query statement is composed of a starting point and a relationship sequence in the candidate path, and the entities in the candidate path other than the starting point are empty. The query result is not empty only when the empty entity in the candidate path is queried in the knowledge graph.

[0015] The query result is concatenated with the natural language question to be answered and then input into the trained large language model again to obtain the final answer to the natural language question to be answered.

[0016] According to the second aspect of the present application, an electronic device is provided, comprising: a computer readable storage medium and a processor;

[0017] The computer readable storage medium is used to store executable instructions;

[0018] The processor is used to read the executable instructions stored in the computer readable storage medium and execute the method according to the first aspect.

[0019] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the method according to the first aspect.

[0020] According to a fourth aspect of the present invention, there is provided a computer program product comprising a computer program or instructions, which implement the method according to the first aspect when executed by a processor.

[0021] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0022] The large language model knowledge question answering method based on semantic parsing correction provided by the present invention first converts natural language questions, involved entities and their accessible knowledge paths into structured training data by constructing a large language model fine-tuning sample set based on knowledge graph semantic parsing, forming training samples covering multiple reasoning logics; then designs a structured instruction template to guide the large language model to perform the path generation task starting from the question entity and reaching the target entity, and completes the deep injection of structured knowledge and reasoning ability training through supervised fine-tuning; then, after the user inputs a natural language question, the system calls the fine-tuned large language model to generate multiple multi-hop candidate knowledge paths, and converts these paths into formal logical query statements, and performs verification and completion operations in the knowledge graph based on the logical query statements, eliminates candidate paths containing hallucinations or inconsistent with the facts, and extracts executable and highly reliable knowledge paths; finally, the question and the corrected knowledge path are integrated to form a structured prompt input to the large language model, and the final answer and natural language interpretation are generated by combining structured knowledge and language context information.

[0023] The method provided by the present invention not only improves the accuracy and robustness of the language model in multi-hop reasoning tasks, but also significantly enhances the interpretability and controllability of the generated answers; realizes the deep integration of structured knowledge and language models, and improves the reasoning ability, answer accuracy and interpretability in complex question-answering tasks; injects structured knowledge into the model based on fine-tuning of the large language model, and constructs a large language model that can generate knowledge paths to achieve deep injection of structured knowledge and enhance semantic parsing capabilities, so that the large language model has the ability to infer the corresponding knowledge graph path from natural language questions; by effectively guiding the large language model into a controlled generation mode, the generation space is limited to a constrained subset with the path as the target, thereby improving training efficiency and generation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flow chart of a large language model knowledge question answering method based on semantic parsing and correction provided by an embodiment of the present invention;

[0025] Figure 2 A flowchart of injecting structured knowledge into a large language model provided by an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of an embodiment of the present invention for converting logical query statements corresponding to candidate knowledge paths generated by a large language model into knowledge graph query statements for knowledge graph query. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0028] An embodiment of the present invention provides a large language model knowledge question answering method based on semantic parsing correction, comprising:

[0029] Training phase:

[0030] S1, preprocess the initial sample set to obtain the fine-tuning sample set;

[0031] The initial sample set includes: natural language questions, entities involved in the questions, and a knowledge graph used to solve the natural language questions; the preprocessing includes: identifying the key entity of each natural language question and using it as the starting point, locating the target entity in the knowledge graph based on the standard answer corresponding to each natural language question and using it as the end point, searching the knowledge graph for all possible paths from the starting point to the end point and saving them to the fine-tuning sample set; each possible path is characterized by the entity nodes it passes through and the relationship types between adjacent entities;

[0032] Specifically, step S1 includes S11 and S12.

[0033] S11, construct an initial sample set, including: natural language questions, entities involved in the questions, and knowledge graphs that can be used to solve the current problem;

[0034] S12, the initial sample set is preprocessed as follows to obtain a fine-tuning sample set. The preprocessing includes: first, using a named entity recognition tool, extracting key entities with a semantic core role from the natural language question and using them as the starting point of the path search; then, based on the standard answer corresponding to the question, directly matching the target entity in the knowledge graph to determine the end point of the path; further, starting from the question entity, a multi-hop path search is performed based on the existing triple information in the knowledge graph, exploring the jump paths between entities layer by layer, and recording the intermediate relationships and entity sequences; when a path is successfully connected to the target answer entity, the path is determined as a valid knowledge path and retained; if there are multiple reachable paths, all will be retained to cover a variety of possible reasoning logics, providing richer training samples for subsequent model fine-tuning, i.e., the fine-tuning sample set.

[0035] Specifically, the processing includes the following steps: 1) For each natural language question input, key entity nodes are identified from each natural language question using named entity recognition tools. These entities generally appear at the core of the question and are identified as the starting point of the path. 2) Based on the standard answer corresponding to each question, the target entity is directly located in the knowledge graph as the end point of the path. 3) Starting from the question entity, the system expands outward layer by layer, searching for all possible entity jump paths. At each jump, the system searches for the next entity connected to the current entity based on the existing triple information in the knowledge graph and records the relationship types and entity nodes traversed in between. 4) Once a path is found whose end point matches the standard answer entity, the system considers the path valid and retains it in its entirety as a candidate knowledge path for the current question. If multiple different paths can reach the standard answer, all paths are retained to form a fine-tuning sample set used to train the model to more comprehensively understand various reasoning methods.

[0036] In order to avoid irrelevant or redundant information from being introduced by an overly long reasoning path, preferably, the path lengths in the fine-tuning sample set are all less than a length threshold, that is, the lengths are within a controllable range.

[0037] In the fine-tuning sample set, for each natural language question q, the entity e involved in q q , locate the target entity e in the knowledge graph according to the standard answer of q a , the path obtained by searching in the knowledge graph is Each element in the path sequence is connected by arrow symbols to form a linear sequence, which makes it easier for the model to recognize the path structure while retaining the entity-relationship connection semantics in the original graph. q ), ends at the target answer entity (denoted as e a ), through several jump relations r1, r2, ..., r nand intermediate entities e1,e2,...,e n-1 Series connection.

[0038] S2, using each natural language question in the initial sample set, the entities involved in the question, and the corresponding path in the fine-tuning sample set as a pair of fine-tuning instructions, and saving them to the fine-tuning instruction set; constructing a task prompt instruction, and storing it in each pair of fine-tuning instructions in the fine-tuning instruction set;

[0039] S3 uses the fine-tuning instruction set obtained in S2 to perform supervised training on the large language model.

[0040] Specifically, taking questions q and e q , and the corresponding path P k Build a pair of fine-tuning instructions and put them into the fine-tuning instruction set; if a problem contains n paths, build n pairs of fine-tuning instructions and put them into the fine-tuning instruction set.

[0041] Constructing task prompts: This ensures the model accurately understands the task objective and focuses on the subtask of "generating a knowledge path from the starting entity" during training. This involves clarifying the task intent through natural language templates, thereby controlling the language model's behavior and guiding the model to execute the path generation task, reducing ambiguity in the training objective. The prompt is: "Please generate a knowledge path from the question entity to the answer entity via a multi-hop path. This path must conform to the format requirements of the knowledge graph." This template effectively guides the language model to identify the current task as a "knowledge path generation task" rather than a "direct answer generation task," and encourages it to perform structured reasoning based on the known entities involved in the question.

[0042] The task prompt instructions are also included in the fine-tuning instruction set, and the large language model is trained using the fine-tuning instruction set. The fine-tuning instruction set ultimately includes four parts: natural language questions, entities involved in the question, task prompt instructions, and path P k .

[0043] The most fine-tuned instruction set is used as the training sample of the large language model. Combined with the supervised learning mechanism, the parameters of the large language model are fine-tuned to enable it to have the ability to reproduce knowledge paths based on natural language problems.

[0044] Specifically, first deploy the open source large language model locally on the server. The model architecture can adopt the general large language model and fine-tune it based on the original pre-trained parameters. After the deployment is completed, the training set of the model is the fine-tuning instruction set. First, set the input of the large language model as the natural language question, the entity involved in the question, and the task prompt instruction, and set the output of the large language model as the path P k, set the fine-tuning batch size to 4, the learning rate to 2*10-5, the warm-up ratio to 0.03, the learning rate scheduling method to cosine scheduling, and the total number of training rounds to 3.

[0045] Application stage:

[0046] Input the natural language question to be answered into the trained large language model to obtain candidate paths, convert each candidate path into a logical query statement, and convert the logical query statement into a knowledge graph query statement and perform a query in the knowledge graph to obtain the query result;

[0047] The logical query statement is composed of the starting point and relationship sequence in the candidate path. All entities in the candidate path (i.e., intermediate entities) other than the starting point are vacant. The query result is not empty if and only if the vacant entity in the candidate path is found in the knowledge graph.

[0048] The query result is spliced ​​with the natural language question to be answered and then input into the trained large language model again to obtain the final answer to the natural language question to be answered.

[0049] Specifically, after injecting structured knowledge into the large language model, the fine-tuned large language model is used to start serving users. Decoding and prediction are performed based on the multi-path knowledge of the large language model, including:

[0050] (1) For natural language questions input by users, a large language model is used to decode multi-hop candidate paths.

[0051] Using a fine-tuned large language model, we can generate multiple candidate knowledge paths from the natural language questions input by the user. Each represents a possible reasoning route, starting from the entity involved in the question and gradually connecting to the potential answer entity through multiple relationships. Among them, q represents the natural language question, e q Indicates the entity involved in the problem, e c Represents the candidate answer entity obtained by reasoning, P pred Represents the predicted path.

[0052] Preferably, by setting the decoding strategy of the large language model to beam search, the large language model can generate multi-hop candidate paths at once, thereby improving the answer coverage of the large language model for multi-answer questions. For example, a question may correspond to multiple legal answers. Using the decoding strategy of beam search can generate multiple reasoning paths (i.e., candidate paths), each path leading to a different candidate answer.

[0053] (2) Convert each candidate path into a logical query statement oriented to the knowledge graph.

[0054] Extract logical query statements from each candidate path for subsequent knowledge graph query: Each candidate path generated by the large language model is further converted into a logical query statement that can be executed on the knowledge graph. The logical query statement consists of a sequence of relations in the path. The entities involved in the path are left blank and can be replaced by wildcards, such as the wildcard "?". Accordingly, the logical query statement It's understandable that converting knowledge paths into logical queries is intended to unify the query interface and simplify subsequent knowledge graph path verification operations. Logical queries can efficiently check whether a path exists in the knowledge graph and search for more real paths to correct or complete the generated results.

[0055] (3) Use logical query statements to verify and complete the knowledge paths generated by the large language model to further improve the accuracy and completeness of the reasoning path.

[0056] After step (2) above, the candidate paths output by the large language model have been converted into logical query statements, that is, a formalized "relationship chain + placeholder entity" structure. At this point, the logical query statement needs to be converted into a knowledge graph query statement that can be directly executed on the knowledge graph.

[0057] Logical query statements It is an abstract path description format. Knowledge graph query is a query language for knowledge graphs that supports triple pattern matching. The knowledge graph query statement used can be any existing knowledge graph query statement, such as SPARQL statement, SQL statement, etc.

[0058] Taking SPARQL statements as an example, the process of converting logical query statements into knowledge graph query statements includes:

[0059] 1) Entity mapping, e q Mapped to entity number in the knowledge graph;

[0060] 2) Expand the relationship chain and Expand to nested triple query;

[0061] 3) Variable definition: Use variables such as ?x1, ?x2 to represent intermediate or target entities;

[0062] 4) Statement concatenation: combine the above entity numbers, triple queries, and variables to form a complete knowledge graph query statement.

[0063] For example, e qTaking Angelina Jolie as an example, where r1 and r2 are directed and starred respectively, the knowledge graph query statement converted from the logical query statement LQ = Angelina Jolie → directed → ? → starred → ? is:

[0064] "SELECT ?x1 ?x2 WHERE {

[0065] <http: / / kg.org / entity / Angelina_Jolie><http: / / kg.org / relation / directed> ?x1.?x1<http: / / kg.org / relation / starred> ? x2.}”

[0066] After the above conversion, the model's "reasoning path hypothesis" is converted into an executable knowledge graph query language expression.

[0067] (4) Execute the knowledge graph query and return the knowledge path.

[0068] Use logical query statements to verify and complete the knowledge paths generated by the large language model to further improve the accuracy and completeness of the reasoning path: execute the above knowledge graph query statement on the actual knowledge graph (KG) and extract reliable, multi-hop candidate paths as correction results. Specifically, it includes:

[0069] 1) Submit the knowledge graph query statement to the knowledge graph query interface;

[0070] 2) Get the query results returned by the knowledge graph: the complete knowledge path including the intermediate entities Among them, the query result is not empty only when all the intermediate entities on the candidate path are found in the knowledge graph; if the query result is empty, it means that the corresponding logical query (that is, the candidate path output by the large language model) has problems such as "hallucination" or "factual error", and the current logical query is discarded, that is, the candidate path corresponding to the current logical query is regarded as an invalid path. If the query result is empty, it is determined that the knowledge graph does not contain any knowledge path that is helpful for the current problem. Based on the principle of not introducing irrelevant or redundant information, P ans Vacant.

[0071] (5) Fusion Knowledge Path P ans Together with the natural language questions input by the user, the answer prediction is completed.

[0072] P ansAs an additional display input, it is spliced ​​with the natural language question entered by the user and then input into the large language model again for joint reasoning to generate the answer to the natural language question entered by the user and its natural language explanation.

[0073] An embodiment of the present invention provides a large language model knowledge question answering system based on semantic parsing correction, comprising: a computer-readable storage medium and a processor;

[0074] The computer-readable storage medium is used to store executable instructions;

[0075] The processor is configured to read the executable instructions stored in the computer-readable storage medium and execute the method described in any one of the above embodiments.

[0076] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the method described in any of the above embodiments.

[0077] An embodiment of the present invention provides a computer program product, including a computer program or instructions, which implements the method described in any of the above embodiments when executed by a processor.

[0078] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A large language model knowledge question answering method based on semantic parsing correction, characterized by: include: Training phase: S1, preprocess the initial sample set to obtain the fine-tuning sample set; The initial sample set includes: natural language questions, entities involved in the questions, and a knowledge graph used to solve the natural language questions; the preprocessing includes: identifying the key entity of each natural language question and using it as the starting point, locating the target entity in the knowledge graph based on the standard answer corresponding to each natural language question and using it as the end point, searching the knowledge graph for all possible paths from the starting point to the end point and saving them to the fine-tuning sample set; wherein each possible path is characterized by the entity nodes it passes through and the relationship types between adjacent entities; S2, using each natural language question in the initial sample set, the entities involved in the question, and the corresponding path in the fine-tuning sample set as a pair of fine-tuning instructions, and saving them to the fine-tuning instruction set; constructing a task prompt instruction, and storing it in each pair of fine-tuning instructions in the fine-tuning instruction set; S3, uses the fine-tuning instruction set obtained in S2 to supervise the training of the large language model; Application stage: Input the natural language question to be answered into the trained large language model to obtain candidate paths, convert each candidate path into a logical query statement, and convert the logical query statement into a knowledge graph query statement and perform a query in the knowledge graph to obtain the query result; The logical query statement is composed of the starting point and relationship sequence in the candidate path, and all entities in the candidate path except the starting point are vacant; the query result is not empty if and only if the vacant entity in the candidate path is found in the knowledge graph; The query result is spliced ​​with the natural language question to be answered and then input into the trained large language model again to obtain the final answer to the natural language question to be answered.

2. The method according to claim 1, wherein The task prompt instruction is: "Please generate a knowledge path from the question entity to the answer entity through a multi-hop path. This path needs to comply with the format requirements of the knowledge graph." 3. The method according to claim 1 or 2, wherein: The logical query statement is a SPARQL statement, and converting the logical query statement into a knowledge graph query statement includes: Entity mapping: Map the starting point to the entity number in the knowledge graph; Relationship chain expansion: expand the relationship sequence into nested triple queries; Variable definition: Use variables to represent entities other than the starting point; Statement concatenation: Combine entity numbers, nested triple queries, and variables to form a complete knowledge graph query statement.

4. The method according to claim 1, wherein The path lengths in the fine-tuning sample set are all less than a length threshold.

5. The method according to claim 1, wherein The decoding strategy of the large language model is beam search.

6. A large language model knowledge question answering system based on semantic parsing and correction, characterized by: include: Computer-readable storage medium and processor; The computer-readable storage medium is used to store executable instructions; The processor is configured to read the executable instructions stored in the computer-readable storage medium and execute the method according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the method according to any one of claims 1 to 5.

8. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images